You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Hemanth Yamijala (JIRA)" <ji...@apache.org> on 2009/03/06 06:15:56 UTC
[jira] Created: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Provide a way for users to find out what operations they can do on which M/R queues
-----------------------------------------------------------------------------------
Key: HADOOP-5419
URL: https://issues.apache.org/jira/browse/HADOOP-5419
Project: Hadoop Core
Issue Type: Improvement
Components: mapred
Reporter: Hemanth Yamijala
This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419-4.patch
applying the new patch , previous 1 was malformed
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419.patch
uploading the patch
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12694916#action_12694916 ]
rahul k singh commented on HADOOP-5419:
---------------------------------------
to answer nigel comments:
"bq. " This should be available thru cmd line and web ui.
As of now there is no way of authentication for user on web ui . So we cannot show per user data.
Moreover showing acls for all the user's ,imho, might not be such a good thing.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Karam Singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707311#action_12707311 ]
Karam Singh commented on HADOOP-5419:
-------------------------------------
Applied hadoop-5419-5.patch.
Found that when try to run command -: "hadoop queue -showacls" from a user who does not have access to any queue. It throes NPE : -
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.mapred.JobQueueClient.displayQueueAclsInfoForCurrentUser(JobQueueClient.java:162)
at org.apache.hadoop.mapred.JobQueueClient.run(JobQueueClient.java:99)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.mapred.JobQueueClient.main(JobQueueClient.java:182)
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419-5.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419.patch
uploaded the latest patch with suggestions after discussion with hemanth.
In addition mentioning some points about the fix , which weren't very clear from my update above.
-Added new API in JobSubmissionProtocol getQueueAclsInfo.it takes username and returns QueueAclsInfo class
-Created new QueueAclsInfo class , it is a Writable class.
-QueueManager implements the method and synchronized. The method is synchronized because aclsMap in QueueManger might get (incase of automatic acls refresh feature)refreshed.This if happens will lead to issues.
-In JobClient we are doing {code} ugi = UnixUserGroupInformation.login(job, true);
UnixUserGroupInformation.setCurrentUser(ugi);{code}
This is done so that we get ugi value in JobQueueClient which displaying the acls info , as we are displaying username. Anythoughts?
-We had discussed about the web ui part , we thought that any user seeing other acls might lead to some issues. If at all we allow this we need to define special kind of user permissions (for example : administrator) so that those users can view the acls of all the users.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-5419) Provide a way for users
to find out what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12694916#action_12694916 ]
rahul k singh edited comment on HADOOP-5419 at 4/2/09 1:45 AM:
---------------------------------------------------------------
to answer nigel comments:
bq. This should be available thru cmd line and web ui.
As of now there is no way of authentication for user on web ui . So we cannot show per user data.
Moreover showing acls for all the user's ,imho, might not be such a good thing.
was (Author: rksingh):
to answer nigel comments:
"bq. " This should be available thru cmd line and web ui.
As of now there is no way of authentication for user on web ui . So we cannot show per user data.
Moreover showing acls for all the user's ,imho, might not be such a good thing.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419-1.patch
Adding the new patch.
Incorporated suggestion by eric.
Changed the interface , this fix only returns the acls of the current user.
Rectified the issue raised by karam w.r.t groups.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419-5.patch
Incorporated hemanth's comments
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419-5.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419-6.patch
Solves the NPE issue
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419-5.patch, hadoop-5419-6.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419-2.patch
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699519#action_12699519 ]
Sreekanth Ramakrishnan commented on HADOOP-5419:
------------------------------------------------
* Karam's comment about case where no user has access to any queue comment has not been handled. If user cannot submit to any queues which are configured, the appropriate message should be shown instead of blank line.
* Correct JavaDoc in {{JobSubmissionProtocol}} for {{getCurrentUserQueueAclsInfo()}}
* Error message in {{JobTracker.submitJob(JobID)}} can we change it to what Eric has commented?
* In {{QueueManager}} {{getCurrentUserQueueAclsInfo()}} can we {{operationsAllowed}} create it and keep it instead of trying to create it lazily? would defintely improve readability of the code.
* Minor nit the indendation in the {{getCurrentUserQueueAclsInfo()}} seems to be incorrect.
* Why are we doing checking if user has access or not in {{getCurrentUserQueueAclsInfo()}}? Cant we make the code in the method much simpler? As follows
{noformat}
for each queues:
for each operation:
allowed = hasAccess(queue, operation, ugi)
if(allowed)
add to operation allowed list
create queue acls info object based on operations allowed list.
{noformat}
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419-7.patch
uploading the new patch.
Removed changes for NPE issue, as that was due to hadoop-5419-4.patch.
run test patch result is below:
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
[exec]
[exec]
[exec]
[exec] ======================================================================
[exec] ======================================================================
[exec] Finished build.
[exec] ======================================================================
[exec] ======================================================================
[exec]
[exec]
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419-5.patch, hadoop-5419-6-findbugResolution.patch, hadoop-5419-6.patch, hadoop-5419-7.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419-6-findbugResolution.patch
Unused ArrayWritable in QueueAclsInfo is removed so as to make it pass findbug.
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
[exec]
[exec]
[exec]
[exec] ======================================================================
[exec] ======================================================================
[exec] Finished build.
[exec] ======================================================================
[exec] ======================================================================
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419-5.patch, hadoop-5419-6-findbugResolution.patch, hadoop-5419-6.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707647#action_12707647 ]
Hemanth Yamijala commented on HADOOP-5419:
------------------------------------------
Umm. I didn't understand the fix. If a user does not have access to any queue, an empty queue ACLs info list is passed to the client, and it prints the message saying this user has no ACLs or something similar. Where is the NPE coming from ? Can you explain ?
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419-5.patch, hadoop-5419-6-findbugResolution.patch, hadoop-5419-6.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695073#action_12695073 ]
rahul k singh commented on HADOOP-5419:
---------------------------------------
ran test patch.
output:
====
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
[exec]
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Karam Singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700831#action_12700831 ]
Karam Singh commented on HADOOP-5419:
-------------------------------------
Message "Please run hadoop queue -showAcls to find the queues you have access to", is being printed in JobTracker Log, It should be printed on JobClient Side.
As in trunk hadoop command is being for mapred operations, message shoud be changed to "Please run maped queue -showAcls to find the queues you have access to"
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12706791#action_12706791 ]
Hemanth Yamijala commented on HADOOP-5419:
------------------------------------------
Looks fine overall. Few minor comments and suggestions:
- Indentation incorrect in a few places like displayQueueAclsInfoForCurrentUser.
- Wrap to 80 characters at all places in the code.
- Have a space between the opening brace of a block and the condition.
- Message: "User <username> doesnt have access to any queue ": change doesnt to "does not". Terminate with a stop. So: User <username> does not have access to any queue.
- Why do we need UnixUserGroupInformation.setCurrentUser(ugi); in getUGI(). I think we can use the conf defined in JobQueueClient to retrieve the current UGI information since the getUGI method saves it to this conf.
- getCurrentUserQueueAclsInfo - I think a better name is getQueueAclsForCurrentUser(). This is inline with the other APIs defined.
- versionId documentation in JobSubmissionProtocol has wrong method name.
- JobTracker.submitJob() - The log statement should be included in the IOException as well so it can be printed to the client.
- Javadoc: Gets the Queue ACLs for a user in the @return. Similarly in the comment in JobSubmissionProtocol.
- QueueManager.getCurrentUserQueueAclsInfo - I think at this level it can be generic and make it QueueManager.getQueueAcls(UGI user). So tomorrow if there's a need to give for a specific user then we can use the same API.
- In QueueAclsInfo, rather than use ArrayWritable, at other places in M/R, we write a count of the number of items, and then write each item. The elements are stored in a list that's created along with the object. This seems to be better as it does not need new objects to be created each time as is being done now.
- I think it would be better to print the ACLs in a table format like:
Queue Operations
queuesdjkhdsfksjd1 submit-job,adminster-jobs
queue2 submit-job
and so on. Also, avoid space between the operations. This will help the output to work well with standard Unix like tools.
- Test cases: I think we should add the current user to some of the queues and not set an arbitrary job ugi (like u1). You can look at TestQueueManager to see how we get get the Current user's UGI.
- For qu1, why would we not check that both submit and administer jobs is available. Also, rather than checking for substring, I would verify that the entire string is available.
- The group can be the current group for qu5, similar to current user for other queues
- I think there must be another test in which the current user has no access to any queues and verify that an empty list is returned.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala resolved HADOOP-5419.
--------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
I committed this to trunk. Thanks, Rahul !
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419-5.patch, hadoop-5419-6-findbugResolution.patch, hadoop-5419-6.patch, hadoop-5419-7.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-5419:
------------------------------------
Attachment: hadoop-5419-v20.patch
Attached example for earlier version not to be committed.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Fix For: 0.21.0
>
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419-5.patch, hadoop-5419-6-findbugResolution.patch, hadoop-5419-6.patch, hadoop-5419-7.patch, hadoop-5419-v20.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708080#action_12708080 ]
rahul k singh commented on HADOOP-5419:
---------------------------------------
ran "ant test" tests passed sucessfully.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419-5.patch, hadoop-5419-6-findbugResolution.patch, hadoop-5419-6.patch, hadoop-5419-7.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699552#action_12699552 ]
Sreekanth Ramakrishnan commented on HADOOP-5419:
------------------------------------------------
Patch looks fine to me.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Karam Singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12696052#action_12696052 ]
Karam Singh commented on HADOOP-5419:
-------------------------------------
After applying the latest patch, ran some -: mapred. queue -showacls commands and found one issue and one minor enhancement-:
Issue -:
In mapred-site.xml for queue acls if we specify user gorups instead of user queue -showacls command gives black list in console.
E.g
Have two queue: Q1,Q2
Q1 acl: SUBMIT_JOB=u1,u2 and ADMIN_JOB=U3
Q2 acls SUBMIT_JOB= UG1(where UG1 is user group) and ADMIN_JOB=U3
Now use commad: mapred queue -showacls (with user U5 who is part of UG1 group)
It shows -:
[
Queue acls for user : U5
]
Where U5 can submit job to Q2
Enhancemnet-:
Consider same case and use comamnd : mapred queue -showacls (with user U4 who is not part of UG1 group).
It shows -:
[
Queue acls for user : U5
]
It would be better if we show some message e.g. (User does haves perssions on any queue) instead of showing blank line
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh reassigned HADOOP-5419:
-------------------------------------
Assignee: rahul k singh
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695340#action_12695340 ]
rahul k singh commented on HADOOP-5419:
---------------------------------------
run test patch result
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
[exec]
[exec]
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: commands_manual.pdf
adding the new commands_manual.pdf after changes in xdocs
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419-1.patch
added the comments mentioned above.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419-3.patch
Incorporated hemanth's comment
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "eric baldeschwieler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698321#action_12698321 ]
eric baldeschwieler commented on HADOOP-5419:
---------------------------------------------
When a user fails to submit a job, will the resulting error message give them enough info to determine which queues they can run against?
As least an error message like: "Please run hadoop queue -showAcls to find the queues you have access to"
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699568#action_12699568 ]
rahul k singh commented on HADOOP-5419:
---------------------------------------
test patch results:
[exec] There appear to be 651 release audit warnings before the patch and 651 release audit warnings after applying the patch.
[exec]
[exec]
[exec]
[exec]
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
[exec]
[exec]
[exec]
[exec] ======================================================================
[exec] ======================================================================
[exec] Finished build.
[exec] ======================================================================
[exec] ======================================================================
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-5419:
------------------------------------
Fix Version/s: 0.21.0
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Fix For: 0.21.0
>
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419-5.patch, hadoop-5419-6-findbugResolution.patch, hadoop-5419-6.patch, hadoop-5419-7.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419.patch
uploading the patch.
Regarding TestCli , would check if this testcase can be added at this point or else would open a seperate jira for that as mentioned in comment earlier.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695028#action_12695028 ]
Nigel Daley commented on HADOOP-5419:
-------------------------------------
New command line commands and options should be tested in TestCLI and include valid and invalid inputs. Karthikeyan may be able to provide some help as he worked on HADOOP-5080.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12679498#action_12679498 ]
Hemanth Yamijala commented on HADOOP-5419:
------------------------------------------
If the main requirement is met, we can trivially add support to check if a user has access to a given queue, I think.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707253#action_12707253 ]
Hemanth Yamijala commented on HADOOP-5419:
------------------------------------------
Regarding the documentation:
Can you modify the command description as:
Displays the queue name and associated queue operations allowed for the current user.The list consists of only those queues to which the user has access.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12694960#action_12694960 ]
Sreekanth Ramakrishnan commented on HADOOP-5419:
------------------------------------------------
Looked at the patch uploaded. Following are the comments which I had while looking at the code:
* Bump up the {{JobSubmissionProtocol}} version by one.
* Formatting change, can we change the output format to following:
{noformat}
Queue acls for user : <username>
Queue Name : qu1 Job Operations : submit administer
Queue Name : qu2 Job Operations : submit
{noformat}
* Can't just define a {{toString()}} method in {{QueueAclsInfo}} to directly present the string in the way it has to be printed to Console?
* Why has {{getUGI()}} in {{JobClient}} made Public? Can we figure out other methods by which we can see to that visiblity of the method is not changed to public unless it is a pressing reason.
* Shouldn't this new method {{getQueueAclsInfo}} in {{QueueManager}} be thread-safe? Because [HADOOP-5396|https://issues.apache.org/jira/browse/HADOOP-5396] can refresh queue acls while user is looking thro' ACL maps which QueueManager uses.
* Minor nit enclose if block in {{getQueueAclsInfo}} in {{QueueManager}} in line 337.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12694936#action_12694936 ]
rahul k singh commented on HADOOP-5419:
---------------------------------------
ran the test patch
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] -1 javadoc. The javadoc tool appears to have generated 1 warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] -1 findbugs. The patch appears to introduce 1 new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12679661#action_12679661 ]
Nigel Daley commented on HADOOP-5419:
-------------------------------------
This should be available thru cmd line and web ui.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12694894#action_12694894 ]
rahul k singh commented on HADOOP-5419:
---------------------------------------
Command and O/p
hadoop queue -showacls
{noformat}
Queue acls for user : <username>
Queue Name : qu1 Operations : acl-submit-job acl-administer-jobs
Queue Name : qu3 Operations : acl-submit-job acl-administer-jobs
{noformat}
New class QueueAclsInfo is introduced to encapsulate Queue name and Queue operation data on the client side.
{code:title=QueueAclsInfo.java}
/**
* Class to encapsulate Queue ACLs for a particular
* user.
*
*/
class QueueAclsInfo implements Writable {
private String queueName;
private String[] operations;
/**
* Default constructor for QueueAclsInfo.
*
*/
{code}
Added new method to JobSubmissionProtocol
{code:title=JobSubmissionProtocol.java}
/**
* Gets the Queue ACLs for a user
* @param userName User name
* @return array of QueueAclsInfo object for a user.
* @throws IOException
*/
public QueueAclsInfo[] getQueueAclsInfo(String userName) throws IOException;
{code}
Implementation of this method is provided in QueueManager.java
{code:title=QueueManager.java}
/**
* Generates the array of QueueAclsInfo object. The array consists of only those queues
* for which user has acls
*
* @param username
* @return QueueAclsInfo[]
* @throws java.io.IOException
*/
QueueAclsInfo[] getQueueAclsInfo(String username) throws IOException{
if(username == null || username.equals(""))
username = UserGroupInformation.getCurrentUGI().getUserName();
//List of all QueueAclsInfo objects , this list is returned
ArrayList<QueueAclsInfo> queueAclsInfolist = new ArrayList<QueueAclsInfo>();
Iterator<String> iter = queueNames.iterator();
QueueOperation[] operations = QueueOperation.values();
while(iter.hasNext()){
String queueName = iter.next();
//QueueAclsInfo object for queue queueName, this object is lazily initialized when there is atleast one queue operation
//supported for the current queue
QueueAclsInfo queueAclsInfo = null;
//Initialize operationsAllowed only if atleast 1 operation is supported for user <username>
//for queue <queueName>
ArrayList<String> operationsAllowed = null;
//Check if user has access for particular operations
for(int i = 0;i < operations.length;i++){
AccessControlList acl = aclsMap.get(toFullPropertyName(queueName,operations[i].getAclName()));
if(acl == null){
//No acls for this operation
continue;
}else{
boolean allowed = acl.allAllowed();
if(allowed) { //All users granted access for this operation in queue <queueName>
if(operationsAllowed == null) {
operationsAllowed = new ArrayList<String>();
}
operationsAllowed.add(operations[i].getAclName());
}else { // All users have not been granted access , check if this user <username> is .
if(acl.getUsers().contains(username)) {
if(operationsAllowed == null)
operationsAllowed = new ArrayList<String>();
operationsAllowed.add(operations[i].getAclName());
}
}
}
}
//Check if user username has acls for queue queueName
//if not no need to create QueueAclsInfo object
if(operationsAllowed != null) {
//There is atleast 1 operation supported for queue <queueName>, hence initialize queueAclsInfo
queueAclsInfo = new QueueAclsInfo(queueName,operationsAllowed.toArray(new String[operationsAllowed.size()]));
queueAclsInfolist.add(queueAclsInfo);
}
}
return queueAclsInfolist.toArray(new QueueAclsInfo[queueAclsInfolist.size()]);
}
}
{code}
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419.patch
attaching the patch and handled some of the scenarios mentioned by srikanth.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-5419) Provide a way for users
to find out what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12694894#action_12694894 ]
rahul k singh edited comment on HADOOP-5419 at 4/2/09 1:52 AM:
---------------------------------------------------------------
Command and O/p
hadoop queue -showacls
{noformat}
Queue acls for user : <username>
Queue Name : qu1 Operations : acl-submit-job acl-administer-jobs
Queue Name : qu3 Operations : acl-submit-job acl-administer-jobs
{noformat}
New interface method is introduced in JobSubmissionProtocol to fetch the Acls information. This interface provides list of all the queue acls and operations allowed. List only consists of queue for which user has atleast 1 acl.
New class QueueAclsInfo is introduced to encapsulate Queue name and Queue operation data on the client side.
{code:title=QueueAclsInfo.java}z
/**
* Class to encapsulate Queue ACLs for a particular
* user.
*
*/
class QueueAclsInfo implements Writable {
private String queueName;
private String[] operations;
/**
* Default constructor for QueueAclsInfo.
*
*/
{code}
Added new method to JobSubmissionProtocol
{code:title=JobSubmissionProtocol.java}
/**
* Gets the Queue ACLs for a user
* @param userName User name
* @return array of QueueAclsInfo object for a user.
* @throws IOException
*/
public QueueAclsInfo[] getQueueAclsInfo(String userName) throws IOException;
{code}
Implementation of this method is provided in QueueManager.java
{code:title=QueueManager.java}
/**
* Generates the array of QueueAclsInfo object. The array consists of only those queues
* for which user has acls
*
* @param username
* @return QueueAclsInfo[]
* @throws java.io.IOException
*/
QueueAclsInfo[] getQueueAclsInfo(String username) throws IOException{
if(username == null || username.equals(""))
username = UserGroupInformation.getCurrentUGI().getUserName();
//List of all QueueAclsInfo objects , this list is returned
ArrayList<QueueAclsInfo> queueAclsInfolist = new ArrayList<QueueAclsInfo>();
Iterator<String> iter = queueNames.iterator();
QueueOperation[] operations = QueueOperation.values();
while(iter.hasNext()){
String queueName = iter.next();
//QueueAclsInfo object for queue queueName, this object is lazily initialized when there is atleast one queue operation
//supported for the current queue
QueueAclsInfo queueAclsInfo = null;
//Initialize operationsAllowed only if atleast 1 operation is supported for user <username>
//for queue <queueName>
ArrayList<String> operationsAllowed = null;
//Check if user has access for particular operations
for(int i = 0;i < operations.length;i++){
AccessControlList acl = aclsMap.get(toFullPropertyName(queueName,operations[i].getAclName()));
if(acl == null){
//No acls for this operation
continue;
}else{
boolean allowed = acl.allAllowed();
if(allowed) { //All users granted access for this operation in queue <queueName>
if(operationsAllowed == null) {
operationsAllowed = new ArrayList<String>();
}
operationsAllowed.add(operations[i].getAclName());
}else { // All users have not been granted access , check if this user <username> is .
if(acl.getUsers().contains(username)) {
if(operationsAllowed == null)
operationsAllowed = new ArrayList<String>();
operationsAllowed.add(operations[i].getAclName());
}
}
}
}
//Check if user username has acls for queue queueName
//if not no need to create QueueAclsInfo object
if(operationsAllowed != null) {
//There is atleast 1 operation supported for queue <queueName>, hence initialize queueAclsInfo
queueAclsInfo = new QueueAclsInfo(queueName,operationsAllowed.toArray(new String[operationsAllowed.size()]));
queueAclsInfolist.add(queueAclsInfo);
}
}
return queueAclsInfolist.toArray(new QueueAclsInfo[queueAclsInfolist.size()]);
}
}
{code}
was (Author: rksingh):
Command and O/p
hadoop queue -showacls
{noformat}
Queue acls for user : <username>
Queue Name : qu1 Operations : acl-submit-job acl-administer-jobs
Queue Name : qu3 Operations : acl-submit-job acl-administer-jobs
{noformat}
New class QueueAclsInfo is introduced to encapsulate Queue name and Queue operation data on the client side.
{code:title=QueueAclsInfo.java}
/**
* Class to encapsulate Queue ACLs for a particular
* user.
*
*/
class QueueAclsInfo implements Writable {
private String queueName;
private String[] operations;
/**
* Default constructor for QueueAclsInfo.
*
*/
{code}
Added new method to JobSubmissionProtocol
{code:title=JobSubmissionProtocol.java}
/**
* Gets the Queue ACLs for a user
* @param userName User name
* @return array of QueueAclsInfo object for a user.
* @throws IOException
*/
public QueueAclsInfo[] getQueueAclsInfo(String userName) throws IOException;
{code}
Implementation of this method is provided in QueueManager.java
{code:title=QueueManager.java}
/**
* Generates the array of QueueAclsInfo object. The array consists of only those queues
* for which user has acls
*
* @param username
* @return QueueAclsInfo[]
* @throws java.io.IOException
*/
QueueAclsInfo[] getQueueAclsInfo(String username) throws IOException{
if(username == null || username.equals(""))
username = UserGroupInformation.getCurrentUGI().getUserName();
//List of all QueueAclsInfo objects , this list is returned
ArrayList<QueueAclsInfo> queueAclsInfolist = new ArrayList<QueueAclsInfo>();
Iterator<String> iter = queueNames.iterator();
QueueOperation[] operations = QueueOperation.values();
while(iter.hasNext()){
String queueName = iter.next();
//QueueAclsInfo object for queue queueName, this object is lazily initialized when there is atleast one queue operation
//supported for the current queue
QueueAclsInfo queueAclsInfo = null;
//Initialize operationsAllowed only if atleast 1 operation is supported for user <username>
//for queue <queueName>
ArrayList<String> operationsAllowed = null;
//Check if user has access for particular operations
for(int i = 0;i < operations.length;i++){
AccessControlList acl = aclsMap.get(toFullPropertyName(queueName,operations[i].getAclName()));
if(acl == null){
//No acls for this operation
continue;
}else{
boolean allowed = acl.allAllowed();
if(allowed) { //All users granted access for this operation in queue <queueName>
if(operationsAllowed == null) {
operationsAllowed = new ArrayList<String>();
}
operationsAllowed.add(operations[i].getAclName());
}else { // All users have not been granted access , check if this user <username> is .
if(acl.getUsers().contains(username)) {
if(operationsAllowed == null)
operationsAllowed = new ArrayList<String>();
operationsAllowed.add(operations[i].getAclName());
}
}
}
}
//Check if user username has acls for queue queueName
//if not no need to create QueueAclsInfo object
if(operationsAllowed != null) {
//There is atleast 1 operation supported for queue <queueName>, hence initialize queueAclsInfo
queueAclsInfo = new QueueAclsInfo(queueName,operationsAllowed.toArray(new String[operationsAllowed.size()]));
queueAclsInfolist.add(queueAclsInfo);
}
}
return queueAclsInfolist.toArray(new QueueAclsInfo[queueAclsInfolist.size()]);
}
}
{code}
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated HADOOP-5419:
-------------------------------------
Attachment: hadoop-5419-v20.2.patch
This patch fixes a test case path bug in the previous patch for the 20 branch (hadoop-5419-v20.patch). Not to be applied to the 20 branch though.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Fix For: 0.21.0
>
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419-5.patch, hadoop-5419-6-findbugResolution.patch, hadoop-5419-6.patch, hadoop-5419-7.patch, hadoop-5419-v20.2.patch, hadoop-5419-v20.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12706311#action_12706311 ]
Hemanth Yamijala commented on HADOOP-5419:
------------------------------------------
I think the new patch uploaded is not complete. It seems to be missing QueueAclsInfo class. Also, the size is significantly lesser. Can you please check again ?
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708636#action_12708636 ]
Hudson commented on HADOOP-5419:
--------------------------------
Integrated in Hadoop-trunk #834 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/834/])
. Provide a facility to query the Queue ACLs for the current user. Contributed by Rahul Kumar Singh.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419-5.patch, hadoop-5419-6-findbugResolution.patch, hadoop-5419-6.patch, hadoop-5419-7.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rahul k singh updated HADOOP-5419:
----------------------------------
Attachment: hadoop-5419-2.patch
attaching the proper patch
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707306#action_12707306 ]
Hemanth Yamijala commented on HADOOP-5419:
------------------------------------------
Looks good. +1.
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419-5.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707296#action_12707296 ]
Hemanth Yamijala commented on HADOOP-5419:
------------------------------------------
Close, but a little more work:
- In displayQueueAclsInfoForCurrentUser, when no ACLs are returned, we are using UserGroupInformation.getCurrentUGI().getUserName(). This could be null if the login is not done, right ?
- I would recommend that the JobClient API be changed to getQueueAclsForCurrentUser, and the java doc updated accordingly.
- QueueManager API need not be called getCurrentUserQueueAclsInfo, as we are passing a UGI. Let's just call it getQueueAcls(UGI ugi)
- Javadoc for the QueueManager API is still saying 'current user' though we are passing in the UGI.
- java.util.Iterator is not needed in QueueManager, I think, Can you check
- I think the formatting of the QueueAclsInfo in the toString is not right. Let's keep all formatting in JobQueueClient API and remove the toString
-
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419-1.patch, hadoop-5419-2.patch, hadoop-5419-2.patch, hadoop-5419-3.patch, hadoop-5419-4.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695256#action_12695256 ]
Sreekanth Ramakrishnan commented on HADOOP-5419:
------------------------------------------------
Took a look at the changes, everything is fine, expect few comments:
* Why are we using a {{MiniMRCluster}} for test case? Why cant we directly use {{QueueManager.getQueueAclsInfo(String)}} that would help us reduce run time of the test case.
* Why cant we move the replacing of "acl-" in {{toString()}} method of {{QueueAclsInfo}} so that in your {{JobQueueClient.displayQueueAclsInfoForCurrentUser()}} we can just do following
{noformat}
for(QueueAclsInfo queueinfo: queueAclsInfo){
System.out.println(queueinfo);
}
{noformat}
* Also, we should be adding this to {{TestCLI}}, we should do that as a part of a Seperate JIRA, or HADOOP-5080 if it is not commited.
* Please document the new command in hadoop commands manual
Minor nit:
* Keep line width not more than 80 characters in {{QueueManager.getQueueAclsInfo(String)}}
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5419) Provide a way for users to find out
what operations they can do on which M/R queues
Posted by "rahul k singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699513#action_12699513 ]
rahul k singh commented on HADOOP-5419:
---------------------------------------
ran testpatch
[exec]
[exec]
[exec]
[exec]
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
> Provide a way for users to find out what operations they can do on which M/R queues
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5419
> URL: https://issues.apache.org/jira/browse/HADOOP-5419
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: rahul k singh
> Attachments: commands_manual.pdf, hadoop-5419-1.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch, hadoop-5419.patch
>
>
> This issue is to provide an improvement on the existing M/R framework to let users know which queues they have access to, and for what operations. One use case for this would that currently there is no easy way to know if the user has access to submit jobs to a queue, until it fails with an access control exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.