[jira] Commented: (MAPREDUCE-1664) Job Acls affect Queue Acls

["Ravi Gummadi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901782#action_12901782 ] 

Ravi Gummadi commented on MAPREDUCE-1664:
-----------------------------------------

Already attached 4 patches form the combined patch for Yahoo security branch 20.1xx.

At a higher level, the above 4 patches did the following:

(1) Removed the flag mapreduce.cluster.job-authorization-enabled and let the flag mapred.acls.enabled drive both queue acls checks and job acls checks. All users will be able to do queue level and job level operations if mapred.acls.enabled is set to false. If mapred.acls.enabled is set to true, then the UNION of users part of queue ACLs and job ACLs will be able to do operation if that operation is driven by both queue ACLs and job ACLs --- like killJob, killTask and setJobPriority.

(2) JobHistory is changed to have "*" as value for the job acls' keys VIEW_JOB and MODIFY_JOB in the line of job submission in history file when mapred.acls.enabled is false. This would allow all users to view job-history of this job even when we restart the cluster with mapred.acls.enabled set to true.

(3) JobHistory is changed to have job queue name in the job submission line in history file. This is used to allow queue admins of that queue to view history details when serving job history based queries.

(4) Created new enum "Operation" that contains the actual operations like SUBMIT_JOB, KILL_TASK, FAIL_TASK, KILL_JOB,SET_JOB_PRIORITY, VIEW_TASK_LOGS, VIEW_JOB_DETAILS, VIEW_JOB_COUNTERS.

(5) Added new class ACLsManager that is responsible for authorization of all queue level and job level operations.
ACLsManager internally maintains MROwner(user who started mapreduce cluster), cluster administrators(configured by MROwner via mapreduce.cluster.permissions.supergroup). ACLsManager uses QueueManager and JobACLsManager for checking queue acls and job acls respectively for authorizing users to do various operations.

(6) Operation SUBMIT_JOB is solely driven by the queue acl acl-submit-job. The operations VIEW_JOB_DETAILS, VIEW_JOB_COUNTERS, VIEW_TASK_LOGS, FAIL_TASK, KILL_TASK, KILL_JOB, SET_JOB_PRIORITY are driven by both job acls(either acl-view-job OR acl-modify-job based on the operation) and queue acl acl-administer-jobs. If a user is part of job acl that drives the specific operation OR part of the queue acl acl-administer-jobs, then he/she is an authorized user to perform the operation. Irrespective of queue acls and job acls configured, any operation can be done by MROwner and cluster administrators(members of supergroup).

(7) Default values for the queue acls is changed from "*"(all users allowed by default) to " "(no user is allowed by default). This is to make the default values of queue acls consistent with job acls.

(8) Since TaskTracker will not have job acls once the job finishes execution, the job acls are persisted to userlogs/$jobid/job-acls.xml so that single file per job is maintained. This file contains job's acl-view-job, job queue name, job owner name, queue admins acl of the queue to which this job was submitted to. These info will be used by TaskTracker to do authorization of users for accessing task logs(in TaskLogServlet).

(9) Made LinuxTaskController to set permissions for job acls file also when permissions are being set for job userlog dir.

(10) For TaskTrackers to write queue admins of specific job queue to job-acls.xml file, the queue admins acl is written to job conf by job client. Made JobClient to get queue admins(of the queue to which the job is being submitted to) from JobTracker and set this ACL in job conf. Tasktracker reads this queue admins ACL from job conf and writes to job-acls.xml file at the time of job localization.

(11) TaskLogServlet checks the queue admins ACL explicitly(as there is no Queuemanager on TaskTracker) before checking for other things like jobOwner, cluster admins and view-job-acl. The queue admins ACL and view-job-acl are obtained from job-acls.xml file.

(12) Changed readCounters() in CompletedJobStatusStore so that queue name is obtained from job profile and thus allowing job-view of that job for the queue admins of that queue.

(13) Since the newly added method getQueueAdmins() in ClientProtocol returns AccessControlList object directly,  creation of AccessControlList object through Reflection utils is needed. This needed a new constructor for AccessControlList and that to be registered in WritableFactories.

> Job Acls affect Queue Acls
> --------------------------
>
>                 Key: MAPREDUCE-1664
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1664
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 0.22.0
>            Reporter: Ravi Gummadi
>            Assignee: Ravi Gummadi
>             Fix For: 0.22.0
>
>         Attachments: 1664.20S.3.4.patch, 1664.qAdminsJobView.20S.v1.6.patch, M1664y20s-testfix.patch, mr-1664-20-bugfix.patch
>
>
> MAPREDUCE-1307 introduced job ACLs for securing job level operations. So in current trunk, queue ACLs and job ACLs are checked(with AND for both acls) for allowing job level operations. So for doing operations like killJob, killTask and setJobPriority user should be part of both mapred.queue.{queuename}.acl-administer-jobs and in mapreduce.job.acl-modify-job. This needs to change so that users who are part of mapred.queue.{queuename}.acl-administer-jobs will be able to do killJob,killTask,setJobPriority and users part of mapreduce.job.acl-modify-job will be able to do killJob,killTask,setJobPriority.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.