[Pig Wiki] Update of "Howl/HowlAuthorizationProposal" by AshutoshChauhan

[Apache Wiki <wi...@apache.org>]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.

The "Howl/HowlAuthorizationProposal" page has been changed by AshutoshChauhan.
http://wiki.apache.org/pig/Howl/HowlAuthorizationProposal?action=diff&rev1=3&rev2=4

--------------------------------------------------

  <<TableOfContents>>
  
  <<Anchor(Permissions)>>
+ 
  == Permissions ==
  The initial idea for authorization in Howl is to use the hdfs permissions to authorize metadata operations. To be able to do this, we would like to extend createTable() to add the ability to record a different group from the user's primary group and to record the complete unix permissions on the table directory. Also, we would like to have a way for partition directories to inherit permissions and group information based on the table directory. To keep the metastore backward compatible for use with hive. The following conf variables can be used to achieve these objectives:
+ 
-    * table.group.name : value will indicate the name of the unix group for the table directory. This will be used by createTable() to perform a chgrp to the value provided. This property will provide the user the ability to choose from one of the many unix groups he is part of to associate with the table.
+  * table.group.name : value will indicate the name of the unix group for the table directory. This will be used by createTable() to perform a chgrp to the value provided. This property will provide the user the ability to choose from one of the many unix groups he is part of to associate with the table.
-    * table.permissions : value will be of the form rwxrwxrwx to indicate read-write-execute permissions on the table directory. This will be used by createTable() to perform a chmod to the value provided. This will let the user decide what permissions he wants on the table.
+  * table.permissions : value will be of the form rwxrwxrwx to indicate read-write-execute permissions on the table directory. This will be used by createTable() to perform a chmod to the value provided. This will let the user decide what permissions he wants on the table.
-    * partitions.inherit.permissions : a value of true will indicate that partitions inherit the group name and permissions of the table level directory. This will be used by addPartition() to perform a chgrp and chmod to the values as on the table directory.
+  * partitions.inherit.permissions : a value of true will indicate that partitions inherit the group name and permissions of the table level directory. This will be used by addPartition() to perform a chgrp and chmod to the values as on the table directory.
  
  Conf properties are preferable over API changes since the complete authorization design for hive is not finalized yet. These properties can be deprecated/removed when that is in place. These properties would also be useful to some installation of vanilla hive since at least DFS level authorization can now be achieved by hive without the user having to manually perform chgrp and chmod operations on DFS.
  
  <<Anchor(Reading_data(Select)/Writing_data_(Insert))>>
+ 
  == Reading data(Select)/Writing data (Insert) ==
  This will simply be governed by the dfs permission at the time of the read and will result in runtime errors if the user does not have permissions.
  
  <<Anchor(Create_table)>>
+ 
  == Create table ==
+ <<Anchor(Internal/External_table_without_location_specified)>>
  
- <<Anchor(Internal/External_table_without_location_specified)>>
  === Internal/External table without location specified ===
- If the user has permissions to the directory pointed by hive.metastore.warehouse.dir then he can create the table. 
+ If the user has permissions to the directory pointed by hive.metastore.warehouse.dir then he can create the table.
  
  <<Anchor(Internal/External_table_with_location_specified)>>
+ 
  === Internal/External table with location specified ===
  If the user has permissions to the location specified then he can create the table.
  
  <<Anchor(Drop_Table)>>
+ 
  == Drop Table ==
- A user can drop a table (internal or external) only if he has write permissions to the table directory. A user could have write permission either by virtue of him being the owner of the table or through the group he belongs
+ A user can drop a table (internal or external) only if he has write permissions to the table directory. A user could have write permission either by virtue of him being the owner of the table or through the group he belongs to. So if the permissions on the table directory allow him to write to it, he can drop the table.
- to. So if the permissions on the table directory allow him to write to it, he can drop the table.
  
  <<Anchor(Partition_permissions)>>
+ 
  == Partition permissions ==
  Partition directories will inherit the permissions/(owner,group) of the table directory.
  
  <<Anchor(Alter_table)>>
+ 
  == Alter table ==
  A user can "alter" table if he has write permissions on the table directory. So any of the following alter table commands are allowed only if the user has write permissions on the table directory:
+ 
-    * ALTER TABLE table_name ADD partition_spec [ LOCATION 'location1' ] partition_spec [ LOCATION 'location2' ] ...
+  * ALTER TABLE table_name ADD partition_spec [ LOCATION 'location1' ] partition_spec [ LOCATION 'location2' ] ...
-    * ALTER TABLE table_name DROP partition_spec, partition_spec,...
+  * ALTER TABLE table_name DROP partition_spec, partition_spec,...
-    * ALTER TABLE table_name RENAME TO new_table_name
+  * ALTER TABLE table_name RENAME TO new_table_name
-    * ALTER TABLE table_name CHANGE [COLUMN] col_old_name col_new_name column_type [COMMENT col_comment] [FIRST||AFTER column_name]
+  * ALTER TABLE table_name CHANGE [COLUMN] col_old_name col_new_name column_type [COMMENT col_comment] [FIRST||AFTER column_name]
-    * ALTER TABLE table_name ADD||REPLACE COLUMNS (col_name data_type [COMMENT col_comment], ...)
+  * ALTER TABLE table_name ADD||REPLACE COLUMNS (col_name data_type [COMMENT col_comment], ...)
-    * ALTER TABLE table_name SET TBLPROPERTIES table_properties
+  * ALTER TABLE table_name SET TBLPROPERTIES table_properties
-    * ALTER TABLE table_name SET SERDE serde_class_name [WITH SERDEPROPERTIES serde_properties]
+  * ALTER TABLE table_name SET SERDE serde_class_name [WITH SERDEPROPERTIES serde_properties]
-    * ALTER TABLE table_name SET SERDEPROPERTIES serde_properties
+  * ALTER TABLE table_name SET SERDEPROPERTIES serde_properties
-    * ALTER TABLE table_name SET FILEFORMAT file_format
+  * ALTER TABLE table_name SET FILEFORMAT file_format
-    * ALTER TABLE table_name CLUSTERED BY (col_name, col_name, ...) [SORTED BY (col_name, ...)] INTO num_buckets BUCKETS
+  * ALTER TABLE table_name CLUSTERED BY (col_name, col_name, ...) [SORTED BY (col_name, ...)] INTO num_buckets BUCKETS
-    * ALTER TABLE table_name TOUCH;
+  * ALTER TABLE table_name TOUCH;
-    * ALTER TABLE table_name TOUCH PARTITION partition_spec;
+  * ALTER TABLE table_name TOUCH PARTITION partition_spec;
  
  <<Anchor(Show_tables)>>
+ 
  == Show tables ==
- Since the top level warehouse dir will have read/write permissions for everyone, show tables will show all tables to all users 
+ Since the top level warehouse dir will have read/write permissions for everyone, show tables will show all tables (of default database) to all users.
  
  <<Anchor(Show_Table/Partitions_Extended)>>
+ 
  == Show Table/Partitions Extended ==
  A user can issue "show table/partitions extended" on a table only if he has read permissions on the table directory. This query is of the form:
+ 
-    * SHOW TABLE EXTENDED [IN||FROM database_name] LIKE identifier_with_wildcards [PARTITION(partition_desc)]
+  * SHOW TABLE EXTENDED [IN||FROM database_name] LIKE identifier_with_wildcards [PARTITION(partition_desc)]
  
  <<Anchor(Show_partitions)>>
+ 
  == Show partitions ==
  A user can issue "show partitions" on a table only if he has read permissions on the table directory.
  
  <<Anchor(Describe_table/column/partition)>>
+ 
  == Describe table/column/partition ==
  A user can issue "describe table/column/partition" on a table only if he has read permissions on the table directory.
  
  <<Anchor(Database_related_operations)>>
+ 
  == Database related operations ==
+ <<Anchor(create_db)>>
  
- <<Anchor(create_db)>>
  === create db ===
- Just like create table, create db will have db.group.name and db.permissions properties which will dictate the group and permissions of the db directory. This will be set up by the howl cli and the database directory will need to be updated with the appropriate chgroup and chmod operations. There will be '''NO''' inheritance of permissions from db directory to table directory. The table directory can have potentially different group/perms from the db directory.
+ Just like create table, create db will have db.group.name and db.permissions properties which will dictate the group and permissions of the db directory. This will be set up by the howl cli and the database directory will need to be updated with the appropriate chgroup and chmod operations. There will be '''NO''' inheritance of permissions from db directory to table directory. The table directory can have potentially different group/perms from the db directory. User can provide the group to which table should belong to, in create table statement. In case he chose not to, then group of table will be same as database group.
  
  <<Anchor(use_db)>>
+ 
  === use db ===
  use db will be permitted only if the user has read permission on the db directory. So subsequent operation like create table will still be authorized based on the rules laid above once the "use db" call has been authorized. So the user would need write permission on the db directory to be able create the table directory under it.
  
@@ -83, +98 @@

  
  == Comments ==
  This model is built around using group permissions to share table access.  A few implementation notes concerning this.
+ 
-  1. Conceptually it is possible for someone to create a table in directory {{{/x/y/z}}} where they own `z` but not `x` or `y`.  If the owner of `x` or `y` then changes their permissions so that the table creator (or others in the group the table is owned by) cannot access direcotry `z` then the table creator (or others in the group) will not be able to access the table.  For tables under the default directory, this will not be an issue.  For tables outside this directory, it will be the responsibility of the user not to put their files in a place where this is likely to happen (note that this is true regardless of the authorization model we adopt).  
+  1. Conceptually it is possible for someone to create a table in directory {{{/x/y/z}}} where they own `z` but not `x` or `y`.  If the owner of `x` or `y` then changes their permissions so that the table creator (or others in the group the table is owned by) cannot access direcotry `z` then the table creator (or others in the group) will not be able to access the table.  For tables under the default directory, this will not be an issue.  For tables outside this directory, it will be the responsibility of the user not to put their files in a place where this is likely to happen (note that this is true regardless of the authorization model we adopt).
   1. What permissions will the top level warehouse directory have?  We could choose to let them have 777, so that anyone can create tables there.  We could also define a `howl` group and declare that all users who wish to use Howl must use be in this group.  The top level directory could then be 770, locking out any users who were not authorized to use Howl.  Given that we expect in many installation all or most all data to be managed by Howl, it is not clear how useful this would be.
   1. There is a security hole in who can drop tables.  Since the ability to delete a file is based on write permission in the directory that contains it and not write permission on the file itself, once a table is created but before it has any files anyone with permission to create tables can drop that table.  (It is no longer a problem once files or directories have been created in the table directory, because removing those will require write permission on the table directory.)  In the case of the default warehouse that means everyone has permission to drop empty tables.  In the case of other databases it means everyone in the database group has these permissions.  One way around that would be to create an empty file under the table directory, `_empty`.
   1. The above vulnerability also applies to database directories and could be resolved in the same way.
   1. In the POSIX permissions model the narrowist applicable category is used in testing whether a specific user has access to a file.  Thus if the owner of a file does not have permission to read the file, but he is in a group that does, he still will not be able to read the file.  To avoid strange side affects of this I propose that it be an error if the owner has less permission than the group.  So `rwxr-x---` would be accepted, but `r-xrwx---` would return an error.
   1. Execute bits give a user permission to open a directory, which is required to open files for either reading or writing.  Thus for all users classes it should be an error for the read or write bit to be set but not the execute bit.  Thus `r-x`, `-wx`, `rwx` would all be valid combinations, but `r--`, `-w-`, and `rw-` would be generate errors.
-  1. Do we ever want to allow a user to set a table as world writable?
  
  === Implementation Notes ===
  http://wiki.apache.org/pig/Howl/AuthorizationImplNotes
-