You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Pete Wyckoff (JIRA)" <ji...@apache.org> on 2008/09/08 06:02:44 UTC
[jira] Created: (HADOOP-4108) FileSystem support for POSIX access
method
FileSystem support for POSIX access method
------------------------------------------
Key: HADOOP-4108
URL: https://issues.apache.org/jira/browse/HADOOP-4108
Project: Hadoop Core
Issue Type: New Feature
Components: fs
Reporter: Pete Wyckoff
>From man access:
{code}
int access(const char *pathname, int mode);
{code}
DESCRIPTION
access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
easily determine the invoking userâs authority.
Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
RETURN VALUE
On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629988#action_12629988 ]
Owen O'Malley commented on HADOOP-4108:
---------------------------------------
What goes wrong with a utility function that goes like:
{code}
public static boolean access(FileSystem fs, Path p, FsAction perm) throws IOException {
UserGroupInformation user = UserGroupInformation.getCurrentUGI();
FileStatus stat = fs.listStatus(p)[0];
FsAction filePerm = null;
if (user.getUserName().equals(stat.getOwner())) {
filePerm = stat.getPermission().getUserAction();
} else {
for(String group: user.getGroups()) {
if (group.equals(state.getGroup())) {
filePerm = stat.getPermission().getGroupAction();
break;
}
if (filePerm == null) {
filePerm = stat.getPermission().getOtherAction();
}
return filePerm.implies(perm);
}
{code}
What goes wrong?
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631583#action_12631583 ]
Raghu Angadi commented on HADOOP-4108:
--------------------------------------
> I'm not sure I understand, are you proposing something else or not to implement this
I guess I meant we don't need to change FileStatus to implement the API you proposed. I doubt if extra RPC is a big deal for this.
Not that it matters, API wise, I would have just had access(Path), but does not matter now.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630277#action_12630277 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
this is something like:
{code}
public class FileStatus {
+ private FsAction userAccess;
+ public fsAction getUserAccess() { return userAccess; }
+ public void setUserAccess(FsAction userAccess) { this.userAccess = userAccess; }
{code}
Right? And the filesystem API for getFileStatus will set it? +1 I am assuming there are no performance implications of creating this on every getFileStatus which there shouldn't be I wouldn't think.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631575#action_12631575 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
bq. Is a variable superuser name really a universal concept that we want to add to every FileStatus?
This design accommodates both of these points. If we would like we can have the default implementation not include the notion of a superuser and have hdfs override that.
So, then is there agreement on:
{code}
FsAction FileSystem.access(FileStatus f) throws IOException;
{code}
With the default implementation being Owen's code and then we would subclass FileStatus for HDFS to include the superuser?
bq. Looks like not so useful hack to me and might not even work with kerberos etc. Any way I guess there has been enough discussion on this... I could not see need for changing FileStatus for this api alone.
I'm not sure I understand, are you proposing something else or not to implement this?
thanks, pete
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-4108) FileSystem support for
POSIX access method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631553#action_12631553 ]
wyckoff edited comment on HADOOP-4108 at 9/16/08 2:15 PM:
---------------------------------------------------------------
Dhruba and I discussed this offline and here's what we would like to propose:
1. Add a private field superuser to FileStatus and ensure it is set in all the FileStatus constructors (and FileStatus still doesn't have any logic pertaining to a specific user - ie its still really a stat object)
2. Add the utility API: {code} FsAction access(FileStatus f) throws IOException;{code} to FileSystem using the code Owen added to the JIRA.
FileSystems can override the access method and no need for a RPC as you are passed in the FileStatus. Since this is computed on the client-side, the user is known to the FS.
I think this incorporates all the discussed points.
was (Author: wyckoff):
Dhruba and I discussed this offline and here's what we would like to propose:
1. Add a private field superuser to FileStatus and ensure it is set in all the FileStatus constructors (and FileStatus still doesn't have any logic pertaining to a specific user - ie its still really a stat object)
2. Add the utility API: {code} FsAction access(FileStatus f) throws IOException;{code} to FileSystem using the code Owen added to the JIRA.
FileSystems can override the access method and no need for a RPC as you are passed in the FileStatus. Since this is computed on the client-side, the user is known to the FS.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629649#action_12629649 ]
Owen O'Malley commented on HADOOP-4108:
---------------------------------------
Ok, that is a very different patch. If you just want to create a utility function in FileUtil that looks like:
{code}
boolean access(Path path, FsAction perm) throws IOException;
{code}
It might makes sense, but it is probably a one liner. I'm still -1 to implementing the jira as spec'ed.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630937#action_12630937 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
hi doug,
is the FileStatus object implementation dependent on the FS implementation? If not, how would individual filesystems implement this?
Wondering about the design of Path and FileStatus in that combined they provide the functionality of a java io file object.
thanks, pete
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631553#action_12631553 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
Dhruba and I discussed this offline and here's what we would like to propose:
1. Add a private field superuser to FileStatus and ensure it is set in all the FileStatus constructors (and FileStatus still doesn't have any logic pertaining to a specific user - ie its still really a stat object)
2. Add the utility API: {code} FsAction access(FileStatus f) throws IOException;{code} to FileSystem using the code Owen added to the JIRA.
FileSystems can override the access method and no need for a RPC as you are passed in the FileStatus. Since this is computed on the client-side, the user is known to the FS.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629640#action_12629640 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
I'm thinking as more of a utility method since otherwise, callers have to implement the logic to figure this out. so, for example, to implement this in fuse, I would actually have to write logic in c to decide if a uid/gid combination are allowed to access a file. Similarly, fuse-j-hdfs would have to implement this in java.
and others too would have to do it.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630361#action_12630361 ]
Raghu Angadi commented on HADOOP-4108:
--------------------------------------
> Clearly when run by the super-user, they will always have read/write access.
hmm.. thats a fundamental change in definition of FileStatus. We had brief discussion of this when permissions were added.
FileStatus is for the property of a file not related to a user. But this jira seems to change that to _properties that effectively apply to the user requesting it_. I am not sure if that is a good direction.
This jira seems to add more functionality to FileStatus (very commonly accessed everywhere) just to support access() (relatively rarely used).
-1 from me.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-4108) FileSystem support for
POSIX access method
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630304#action_12630304 ]
rangadi edited comment on HADOOP-4108 at 9/11/08 11:57 AM:
----------------------------------------------------------------
How does this handle the superuser case Nicholas and Dhruba mentioned above?
To me {{FileSystem.access(path, mode)}} seems more intuitive.
Edit : I might have missed some context. The above can be ignored.
was (Author: rangadi):
How does this handle the superuser case Nicholas and Dhruba mentioned above?
To me {{FileSystem.access(path, mode)}} seems more intuitive.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630272#action_12630272 ]
dhruba borthakur commented on HADOOP-4108:
------------------------------------------
+1 to Owen's proposal.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631579#action_12631579 ]
Doug Cutting commented on HADOOP-4108:
--------------------------------------
> are you proposing something else or not to implement this?
I prefer a FileStatus method to a FileSystem method, but that's not a huge deal.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630002#action_12630002 ]
Doug Cutting commented on HADOOP-4108:
--------------------------------------
> public static boolean access(FileSystem fs, Path p, FsAction perm)
This doesn't need to be static, but could rather be:
public boolean access(Path p, FsAction perm) ...
with the same default definition you provide. Then a FileSystem impl could override it if needed.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629653#action_12629653 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
IGNORE last comment!
yes, boolean is needed.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630386#action_12630386 ]
Owen O'Malley commented on HADOOP-4108:
---------------------------------------
On a side note, I suspect that once we have real authentication, we will also have something like:
{code}
public abstract class FileSystem {
...
public abstract UserGroupInformation getUserInformation() throws IOException;
}
{code}
because the user for each file system can be different. If we do that, it would make sense to include whether it is a super user in the UGI and the previous suggestion of doing it on the client side would work again. *smile* So I'd be ok with:
{code}
public abstract class FileSystem {
...
public FsAction access(Path p) throws IOException { ... }
}
{code}
so to check read access it would be:
{code}
fs.access(path).implies(FsAction.READ);
{code}
The advantage over a single boolean would be if you wanted to detect read-write vs read-only vs nothing.
{code}
FsAction perm = fs.access(p);
if (perm.implies(FsAction.READ_WRITE)) {
// handle read/write case
} else if (perm.implies(FsAction.READ)) {
// handle read only case
} else {
// handle no access
}
{code}
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630382#action_12630382 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
To Owen's comment about APIs, the File API has canRead and canWrite and it seems the FileStatus object provides a lot of this functionality so in that sense, owen's proposal is consistent with Java io.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629651#action_12629651 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
bq. Ok, that is a very different patch. If you just want to create a utility function in FileUtil that looks like:
yes, that's correct, but instead of boolean, i need to have multiple booleans to indicate read/write/delete/...
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629650#action_12629650 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
I'm not sure I did a good job describing the functionality. It should do a getFileStatus but then look at the current user/groups and have the logic to decide if the user is allowed to access the file. Really api request should have been
{code}
// given the permissions/groups/owner in FileStatus and the mode, output what the passed in user is allowed to do to the file
int access(FileStatus f, int mode, String uid, String groups[]);
{code}
but, as you point out, the client side can implement this itself. But, it does have to loop through the groups and such know the Hadoop permission logic.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630378#action_12630378 ]
Raghu Angadi commented on HADOOP-4108:
--------------------------------------
+1 for the default implementation, of course. I don't think adding more fields to FileStatus for this reason is a good idea.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630045#action_12630045 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4108:
------------------------------------------------
I think the access(..) method for HDFS cannot be implemented in client side since we cannot determine whether the current user is a superuser.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley resolved HADOOP-4108.
-----------------------------------
Resolution: Won't Fix
I don't see any value over FileSystem.listStatus. It would just be a redundant method.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630281#action_12630281 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
one minor issue is that getFileStatus is abstract, so we need to implement it for both RawLocalFileSystem and DFS. I think it's just those 2. I assume for now we want to give the same semantics to both.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631590#action_12631590 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
bq. Not that it matters, API wise, I would have just had access(Path), but does not matter now.
FileStatus so we don't have to do an RPC, but if we went with FileStauts implementing it, we would also have to do the RPC.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629726#action_12629726 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
Thinking about this more, {code}access{code} is filesystem specific (hdfs, s3, kosmix, ext3 may all have different rules). a filesystem may or may not respect setuid bits, need to access the entire Path hierarchy and in addition, need to follow symlinks which is not possible to do efficiently from the client side.
Of course https://issues.apache.org/jira/browse/HADOOP-4044 talk about needing the client to sometimes resolve an external link complicates this.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630369#action_12630369 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
bq. But it is more important to write a good Java API that is internally consistent than slavishly follow a C API that was written 40 years ago.
Yes, I agree - meant all things being equal. But I do think we need to take some consideration that we're able to efficiently implement posix APIs, which this proposal clearly does :)
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630353#action_12630353 ]
Owen O'Malley commented on HADOOP-4108:
---------------------------------------
{quote}
How does this handle the superuser case Nicholas and Dhruba mentioned above?
{quote}
Clearly when run by the super-user, they will always have read/write access.
{quote}
I think it would be nice whenever possible to not invent new api/abstractions, so the POSIX model is nice to keep.
{quote}
But it is more important to write a good Java API that is internally consistent than slavishly follow a C API that was written 40 years ago.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631599#action_12631599 ]
Doug Cutting commented on HADOOP-4108:
--------------------------------------
> if we went with FileStauts implementing it, we would also have to do the RPC.
No, that RPC has already been done, the one that returned the FileStatus.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630405#action_12630405 ]
Doug Cutting commented on HADOOP-4108:
--------------------------------------
> public FsAction access(Path p) throws IOException { ... }
This requires an RPC per file when processing directories. I think it's better to hang this method off of FileStatus, so that requests to the remote FileSystem can be batched for directories.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630391#action_12630391 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
I am also ok with the optimization of not including the perms in every FileStatus object. It's less preferable on my side, but is still a good API.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley reopened HADOOP-4108:
-----------------------------------
Assignee: Pete Wyckoff
Maybe it could be added to the FileStatus:
{code}
public class FileStatus {
...
public FsAction getUserAccess();
}
{code}
Thoughts?
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631561#action_12631561 ]
Doug Cutting commented on HADOOP-4108:
--------------------------------------
> Add a private field superuser to FileStatus [ ... ]
Is a variable superuser name really a universal concept that we want to add to every FileStatus?
I'd still prefer that we add a FileStatus method:
- FsAction getAccess() throws IOException;
The default definition would be like Owen's above, but HDFS would override it to take account of the superuser. HDFS would return a subclass of FileStatus that included the superuser's name. What's the problem with that?
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630709#action_12630709 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
how does one know whethere the filesystem is enforcing ACLs? should access always return true when it is not ?
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630101#action_12630101 ]
dhruba borthakur commented on HADOOP-4108:
------------------------------------------
Hi Nicholas, Yes your point is absolutely correct. I think the access() method can be implemented only inside the file system. Each file system can/will implement access checks differently.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630372#action_12630372 ]
Doug Cutting commented on HADOOP-4108:
--------------------------------------
The FileStatus already contains all of the information to compute this, with the exception of the superuser name. So in HDFS, if we add the superuser's name to HDFS's FileStatus implementation, then access can be computed entirely on the client from the FileStatus, with no added cost on the server. The default implementation on FileStatus could be like Owen's utility method above. HDFS would override this for its FileStatus implementation to check if the current user is superuser and otherwise return super.getAccess().
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630304#action_12630304 ]
Raghu Angadi commented on HADOOP-4108:
--------------------------------------
How does this handle the superuser case Nicholas and Dhruba mentioned above?
To me {{FileSystem.access(path, mode)}} seems more intuitive.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pete Wyckoff updated HADOOP-4108:
---------------------------------
Comment: was deleted
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631093#action_12631093 ]
Doug Cutting commented on HADOOP-4108:
--------------------------------------
> is the FileStatus object implementation dependent on the FS implementation?
It certainly can be. A FileStatus instance should always be created by a FileSystem implementation, so it is free to return a subclass of FileStatus. I don't think any FileSystems currently take advantage of this.
> Wondering about the design of Path and FileStatus in that combined they provide the functionality of a java io file object.
The path is just the file name, while the status contains data about that name. One could cache status data for a path to minimize RPCs. HDFS used to do this. MapReduce got written assuming that certain things were cached (like isDirectory(), getLength(), etc.). But not all FileSystem implementations cached these. In particular S3 did not. So MapReduce performance on S3 bad. Also, once HDFS supported append, the cache would get out of date and it was never invalidated. So we moved to a model where the status is explicitly fetched, with no internal caching of path->status. Thus the API better models the underlying RPCs of both HDFS and S3, and there are fewer performance surprises.
Note that this is somewhat analogous with the Unix 'stat' system call. I would guess that 'ls -l' does not call 'stat' for each column, but once for each file. In Unix you want to minimize system calls, and in Hadoop you want to minimize RPCs.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630389#action_12630389 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
bq. The advantage over a single boolean would be if you wanted to detect read-write vs read-only vs nothing.
+1 didn't mean the exact File API as that would definitely be very inconvenient. But the general idea that you could implement most of the informational functionality of the File API with a FileStatus object is in my opinion a nice property.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630312#action_12630312 ]
Pete Wyckoff commented on HADOOP-4108:
--------------------------------------
bq. To me FileSystem.access(path, mode) seems more intuitive.
I think it would be nice whenever possible to not invent new api/abstractions, so the POSIX model is nice to keep.
Turning my performance note on its head, when one wants to just get access, doing everything else for constructing a FileStatus object is a waste. And also when doing a readdir on a directory with 30,000 files, do I really need the access permissions of every file?
In practice because fuse (the first use case) has to support ls --color=auto, it has to cache stat objects and support access being called for every file in a directory after doing a readdir, so having the access result in the stat object is helpful.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631562#action_12631562 ]
Raghu Angadi commented on HADOOP-4108:
--------------------------------------
> 1. Add a private field superuser to FileStatus and ensure it is set in all the FileStatus constructors
Looks like not so useful hack to me and might not even work with kerberos etc. Any way I guess there has been enough discussion on this... I could not see need for changing FileStatus for this api alone.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4108) FileSystem support for POSIX access
method
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629962#action_12629962 ]
dhruba borthakur commented on HADOOP-4108:
------------------------------------------
I think this is a valid JIRA.
A client side program should not have to re-implement filesystem logic of determining whether a particular user is allowed to access a path. The alternative of doing a stat and looking at the permission bits is not a general-purpose solution, because different file systems can implement access checks differently (irrespective of the mode bits on the file). . The LocalFileSystem may enforce seteuid whereas HDFS might not care. Moreover, when a file system uses kerberos authentication, the mode bits do not make sense any more.
An analogy is that all UNIX file system VFS API has an access() method solely for this reason. The UNIX-OS does not interpret the mode bits, it allows the underlying file system to determine access.
> FileSystem support for POSIX access method
> ------------------------------------------
>
> Key: HADOOP-4108
> URL: https://issues.apache.org/jira/browse/HADOOP-4108
> Project: Hadoop Core
> Issue Type: New Feature
> Components: fs
> Reporter: Pete Wyckoff
>
> From man access:
> {code}
> int access(const char *pathname, int mode);
> {code}
> DESCRIPTION
> access checks whether the process would be allowed to read, write or test for existence of the file (or other file system object) whose name is pathname. If pathname is a symbolic link permissions of the file referred to by this symbolic link are tested.
> mode is a mask consisting of one or more of R_OK, W_OK, X_OK and F_OK.
> R_OK, W_OK and X_OK request checking whether the file exists and has read, write and execute permissions, respectively. F_OK just requests checking for the existence of the file.
> The tests depend on the permissions of the directories occurring in the path to the file, as given in pathname, and on the permissions of directories and files referred to by symbolic links encountered on the way.
> The check is done with the processâs real uid and gid, rather than with the effective ids as is done when actually attempting an operation. This is to allow set-UID programs to
> easily determine the invoking userâs authority.
> Only access bits are checked, not the file type or contents. Therefore, if a directory is found to be "writable," it probably means that files can be created in the directory,
> and not that the directory can be written as a file. Similarly, a DOS file may be found to be "executable," but the execve(2) call will still fail.
> If the process has appropriate privileges, an implementation may indicate success for X_OK even if none of the execute file permission bits are set.
> RETURN VALUE
> On success (all requested permissions granted), zero is returned. On error (at least one bit in mode asked for a permission that is denied, or some other error occurred), -1 is
> returned, and errno is set appropriately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.