[jira] Commented: (HADOOP-3698) Implement access control for submitting jobs to queues in the JobTracker

["Hemanth Yamijala (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12627443#action_12627443 ] 

Hemanth Yamijala commented on HADOOP-3698:
------------------------------------------

I had an offline discussion with Owen on his review comments. Here's the summary, and explanation of the changes in the patch with respect to these.

bq. The JobClient should use UserGroupInformation.login(conf) rather than UnixUserGroupInformation.login(conf).

The JobClient code was already using {{UnixUserGroupInformation.login(conf, saveToConf)}}. There was no API equivalent to this one in UserGroupInformation. Discussing options, we decided it's best to have a new JIRA to address this issue in UserGroupInformation. I will open the JIRA and link it off here. Until then, because this issue is holding up other JIRAs, we decided to retain the current code as is.

bq. I think we should have an exception class to catch for login failures. So we either need to add LoginException to submitJob in JobClient or we should create a new IOException that is for login failures and put it into org.apache.hadoop.security. I'd lean toward changing the signature of run/submitJob.

Because run/submitJob throwing a new Exception would cause backwards incompatibility, we decided to go with the other option, that is create a sub-class of IOException in org.apache.hadoop.security. Looking at it, I've seen there's already an {{AccessControlException}} in org.apache.hadoop.fs.permission. Since both classes are identical, I think we should leave one (possibly the one being defined here), and deprecate and ultimately replace the other. This could be part of the cleanup JIRA being opened above. Thoughts ?
Also, please note that UserGroupInformation still throws only LoginException. This seems fair because the calls that generate this are no remote calls, and hence don't need to throw any IOException or any sub-class of it.

bq. I'd suggest handling mapred.acls.enabled inside of the QueueManager instead of the JobTracker. The same for giving the user access to their own jobs. That should be done in QueueManager, so that all of the rules & exceptions for allowing actions are contained inside the one class.

Done this. I've defined a new API in QueueManager to handle this.
{code}
boolean hasAccess(String queueName, JobInProgress job, UserGroupInformation user, QueueOperation action) { ... }
{code}

bq. The JobTracker should use the standard method for getting the queue of the job instead of using the attribute name "queue.name".

This was being done in HADOOP-3445. Still, I've defined these changes in the JobConf and JobProfile classes to get the queue name of a job. It defaults to "default" - which is the default queue defined in the queue manager. This change might need this JIRA to be committed before HADOOP-3445.

bq. Why does the TaskScheduler need the QueueManager? That seems really unexpected.

This was done as discussed in HADOOP-3930 [here|https://issues.apache.org/jira/browse/HADOOP-3930?focusedCommentId=12626042#action_12626042]. However, Owen suggested that this be defined as part of the TaskTrackerManager interface, and define a {{getQueueManager}} in the JobTracker. Implemented that change.


> Implement access control for submitting jobs to queues in the JobTracker
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-3698
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3698
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Hemanth Yamijala
>            Assignee: Hemanth Yamijala
>             Fix For: 0.19.0
>
>         Attachments: HADOOP-3698.patch, HADOOP-3698.patch, HADOOP-3698.patch
>
>
> HADOOP-3445 implements multiple queues in the JobTracker as part of the new resource manager for Hadoop (HADOOP-3421). There needs to be a mechanism to control who can submit jobs to a specified queue. This JIRA is for tracking the requirements, approach and implementation for the same.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.