[jira] Updated: (HADOOP-1298) adding user info to file

["Christophe Taton (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Christophe Taton updated HADOOP-1298:
-------------------------------------

    Attachment: hdfs-access-control.patch.gz

Hi all,

Here is a patch that brings user info and file access permissions to HDFS. It provides a simple file access checking system that should prevent a user to accidentally destroy others data, and does not target real protection of data against malicious actions for now.

Here are some details:
 - The Namenode has been extended to allow checks on inodes (no changes to Datanode as no checks are done on blocks).
 - The FileSystem interface has been extended to provide.
   * user information management: addUser, removeUser, addUserToGroup, removeUserFromGroup, groups(user), id(user);
   * file ownership and access rights (mode): chown, chgrp, chmod.
 - The FsShell is updated to provide access to the management functions
 - Most FileSystem operations now integrate a ClientContext parameter that holds the principal under which operations are executed.
 - For now, a new configuration parameter (fs.client.username) has been introduced define the user that issues operations to HDFS, but there is no authentication mechanism.
 - The FileStatus now has a FileAccessRights that contains user and group ownership info and file mode.
 - The FSImage is updated to store the list of users and groups and to store the file access rights.

Access permissions:
 - The permissions are designed to look like standard posix permissions, except that we ignore and turn off the execution permission.
 - The standard execution rights of a directory is mapped in HDFS on the read permissions of the directory: access to a file requests read permissions on all intermediate directories from root to the file.
 - File or directory creation requires read and write permissions on the most imbricated existing directory of the file to be created.
 - There always is a root:root user who always has read and write permissions. A nobody:nogroup user is also created.
 - User management operations are allowed to root user only.

The URI should now be fully expressed, as in hdfs://user@hostname:port as it may confuse the FileSystem cache system when trying to access the same hdfs with different users in the same JVM process. The fs.default.name property should then also probably be a full URI.

The JUnit tests pass (all in my eclipse, but one fails outside: org.apache.hadoop.streaming.TestSymLink... currently trying to fix this). I also added some tests, but there should be a lot and I would like to have some feedback on the interface and on the permission semantic.
In particular, I wonder whether a file access error should throw an exception or whether the operation should simply return false (e.g. for mkdirs, delete...). Moreover file creation or deletion currently requests read and write access to the parent directory. This fills the need for protection against accidental operations.

I am also not so comfortable with this current implementation of file access control and I am still trying to find a better way to integrate it into hdfs: the checks for one operation are currently processed as a separate INode computation prior to the effective operation.

I am looking forward to your comments. Thanks in advance,
Christophe


> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch10, hadoop-user-munncha.patch11, hadoop-user-munncha.patch12, hadoop-user-munncha.patch13, hadoop-user-munncha.patch14, hadoop-user-munncha.patch15, hadoop-user-munncha.patch16, hadoop-user-munncha.patch17, hadoop-user-munncha.patch4, hadoop-user-munncha.patch5, hadoop-user-munncha.patch6, hadoop-user-munncha.patch7, hadoop-user-munncha.patch8, hadoop-user-munncha.patch9, hdfs-access-control.patch.gz
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info. 
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical. 
> I wasn't sure what release, i'm working off trunk. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.