[jira] [Created] (YARN-3796) Support User level Quota for space and Name (count)

["nijel (JIRA)" <ji...@apache.org>]

nijel created YARN-3796:
---------------------------

             Summary: Support User level Quota for space and Name (count)
                 Key: YARN-3796
                 URL: https://issues.apache.org/jira/browse/YARN-3796
             Project: Hadoop YARN
          Issue Type: New Feature
            Reporter: nijel
            Assignee: nijel


I would like to have one feature in HDFS to have quota management at user level. 

Background :
When the customer uses a multi tenant solution it will have many Hadoop eco system components like HIVE, HBASE, yarn etc. The base folder of these components are different like /hive - Hive , /hbase -HBase. 
Now if a user creates some file  or table these will be under the folder specific to component. If the user name is taken into account it looks like
{code}
                                /hive/user1/table1
                                /hive/user2/table1
                                /hbase/user1/Htable1
                                /hbase/user2/Htable1
 
Same for yarn/map-reduce data and logs
{code}
 
In this case restricting the user to use a certain amount of disk/file is very difficult since the current quota management is at folder level.
 
Requirement: User level Quota for space and Name (count). Say user1 can have 100G irrespective of the folder or location used.
 
Here the idea to consider the file owner ad the key and attribute the quota to it.  So the current quota system can have a initial check for the user quota if defined, before validating the folder quota.

Note:
This need a change in fsimage to store the user and quota information


Please have a look on this scenario. If it sounds good, i will create the tasks and the update the design and prototype.

Thanks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)