You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Thomas Graves (Created) (JIRA)" <ji...@apache.org> on 2011/11/16 22:01:51 UTC
[jira] [Created] (MAPREDUCE-3417) job access controls let invalid
user see job info via web ui when they shouldn't be able to
job access controls let invalid user see job info via web ui when they shouldn't be able to
-------------------------------------------------------------------------------------------
Key: MAPREDUCE-3417
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Priority: Critical
tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
The exception shown in the log is:
11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
at org.apache.hadoop.util.Shell.run(Shell.java:188)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200232#comment-13200232 ]
Hadoop QA commented on MAPREDUCE-3417:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12513196/MAPREDUCE-3417.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1770//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1770//console
This message is automatically generated.
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles reassigned MAPREDUCE-3417:
------------------------------------------
Assignee: Jonathan Eagles
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Critical
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198559#comment-13198559 ]
Jonathan Eagles commented on MAPREDUCE-3417:
--------------------------------------------
Initial patch contains fixes for Jobhistory server and for working in simple authorization mode. Next patch will include fixes for AM.
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199491#comment-13199491 ]
Jonathan Eagles commented on MAPREDUCE-3417:
--------------------------------------------
javadoc warnings are due
[WARNING] Javadoc Warnings
[WARNING] javadoc: warning - Error fetching URL: http://java.sun.com/javase/6/docs/api/package-list
which is present in the trunk builds. HDFS-1857 seems to be the same issue.
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199909#comment-13199909 ]
Hadoop QA commented on MAPREDUCE-3417:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12513146/MAPREDUCE-3417.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1763//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1763//console
This message is automatically generated.
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200391#comment-13200391 ]
Hudson commented on MAPREDUCE-3417:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #946 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/946/])
MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240428
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Anupam Seth (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anupam Seth reassigned MAPREDUCE-3417:
--------------------------------------
Assignee: (was: Anupam Seth)
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Priority: Critical
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Anupam Seth (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anupam Seth reassigned MAPREDUCE-3417:
--------------------------------------
Assignee: Anupam Seth
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Anupam Seth
> Priority: Critical
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Attachment: MAPREDUCE-3417.patch
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3417:
-----------------------------------------------
Fix Version/s: 0.23.1
Status: Open (was: Patch Available)
Looks good overall. The fix for the main issue is fine. Haven't tested it myself, relying on your tests. Few minor comments:
- Instead of Hadoop authorization flag, we should instead be looking at YarnConfiguration#YARN_ACL_ENABLE. It is debatable whether we need a separate config item besides hadoop-auth flag, but let us resolve that separately and be consistent here.
- Good job on the test! But we shouldn't be needing to set MRConfig.MR_ACLS_ENABLED as that is a MRV1 config which we don't want to use here at all.
- Am not sure of the expected behavior of the proxy. Can Robert/Thomas pitch in?
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199430#comment-13199430 ]
Hadoop QA commented on MAPREDUCE-3417:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12513067/MAPREDUCE-3417.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 3 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1756//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1756//console
This message is automatically generated.
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Attachment: MAPREDUCE-3417.patch
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Thomas Graves (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151617#comment-13151617 ]
Thomas Graves commented on MAPREDUCE-3417:
------------------------------------------
Also note that I tested with valid users and it still allows any user to view the job details even though the job view acls are set to only the job runner. That was with security on and also used a http filter so that the user id actually shows up.
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Priority: Critical
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200030#comment-13200030 ]
Jonathan Eagles commented on MAPREDUCE-3417:
--------------------------------------------
I have tried to access via the trackinging url both the AM and JHS job and task pages both with authorization off, simple authorization on, and kerberos authorization on. I have tested that the job owner always has access to those pages. I have tested non-owner with view acls (both via allAccess -> * and specific users). I have also tested this with both static http users as well as using a custom filter.
.
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160220#comment-13160220 ]
Jonathan Eagles commented on MAPREDUCE-3417:
--------------------------------------------
One aspect of the issue I notice. There is a difference in hadoop.security.authentication=simple for accessing the tracking url via the proxy and accessing via the AM directly. The proxy address allows access for invalid users while the AM address disallows access for invalid users(proxy not forwarding on remote user for simple security?)
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Priority: Critical
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Status: Patch Available (was: Open)
Latest patch removed the unnecessary WebAppProxy changes and addressed comments from @Vinod.
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Status: Patch Available (was: Open)
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Status: Open (was: Patch Available)
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200132#comment-13200132 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3417:
----------------------------------------------------
Patch looks good now. +1.
Will push this in when Jenkins comes back with its blessings.
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200117#comment-13200117 ]
Jonathan Eagles commented on MAPREDUCE-3417:
--------------------------------------------
Removed the WebAppProxy changes and everything still works! Thanks, Bobby.
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200423#comment-13200423 ]
Hudson commented on MAPREDUCE-3417:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #181 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/181/])
MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles.
svn merge --ignore-ancestry -c 1240428 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240429
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Status: Open (was: Patch Available)
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200445#comment-13200445 ]
Hudson commented on MAPREDUCE-3417:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #979 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/979/])
MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240428
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200095#comment-13200095 ]
Jonathan Eagles commented on MAPREDUCE-3417:
--------------------------------------------
bq. Instead of Hadoop authorization flag, we should instead be looking at YarnConfiguration#YARN_ACL_ENABLE. It is debatable whether we need a separate config item besides hadoop-auth flag, but let us resolve that separately and be consistent here.
After discussing with Vinod, the correct setting for now will be MRConfig.MR_ACLS_ENABLED. Since that is already checked in JobACLsManager, I just deleted the AUTHORIZATION check in JobImpl and CompletedJob
bq. Good job on the test! But we shouldn't be needing to set MRConfig.MR_ACLS_ENABLED as that is a MRV1 config which we don't want to use here at all.
As part of changes with first comment, now setting only MR_ACLS_ENABLED and removed AUTHORIZATION set in test code
bq. Am not sure of the expected behavior of the proxy. Can Robert/Thomas pitch in?
Will ping one of them to take a look at the proxy change
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200001#comment-13200001 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3417:
----------------------------------------------------
I am about to review the patch. Jonathan, in the mean while, can you please outline the cluster tests you did for this? Thanks!
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Attachment: MAPREDUCE-3417.patch
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Status: Patch Available (was: Open)
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3417:
-------------------------------------
Priority: Blocker (was: Critical)
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200289#comment-13200289 ]
Hudson commented on MAPREDUCE-3417:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #497 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/497/])
MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles.
svn merge --ignore-ancestry -c 1240428 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240429
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Attachment: MAPREDUCE-3417.patch
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Status: Patch Available (was: Open)
Kicking the build
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Attachment: MAPREDUCE-3417.patch
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200113#comment-13200113 ]
Robert Joseph Evans commented on MAPREDUCE-3417:
------------------------------------------------
I just looked at the proxy changes and I am a bit confused why they are needed. If security is enabled the proxy will try to warn the user that they may be connecting to something that is unsafe. It should not change anything about sending the user name over to the AM. All the changes seem to be doing is enabling warning people if the authentication is set to anything that does not evaluate to a boolean false, so it would be enabled for both kerberos and simple, iff simple is explicitly set.
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3417:
-----------------------------------------------
Resolution: Fixed
Release Note: Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps.
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
I just committed this to trunk and branch-0.23. Thanks Jonathan!
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Attachment: MAPREDUCE-3417.patch
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200279#comment-13200279 ]
Jonathan Eagles commented on MAPREDUCE-3417:
--------------------------------------------
Thanks for taking a look, Vinod and Bobby!
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199212#comment-13199212 ]
Hadoop QA commented on MAPREDUCE-3417:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12513023/MAPREDUCE-3417.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1750//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1750//console
This message is automatically generated.
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199785#comment-13199785 ]
Jonathan Eagles commented on MAPREDUCE-3417:
--------------------------------------------
moved the javadoc warnings issue to HADOOP-8018
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Status: Open (was: Patch Available)
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Status: Patch Available (was: Open)
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200273#comment-13200273 ]
Hudson commented on MAPREDUCE-3417:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1672 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1672/])
MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240428
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Attachment: MAPREDUCE-3417.patch
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200254#comment-13200254 ]
Hudson commented on MAPREDUCE-3417:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #474 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/474/])
MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles.
svn merge --ignore-ancestry -c 1240428 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240429
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200259#comment-13200259 ]
Hudson commented on MAPREDUCE-3417:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1728 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1728/])
MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240428
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3417:
---------------------------------------
Attachment: MAPREDUCE-3417.patch
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200403#comment-13200403 ]
Hudson commented on MAPREDUCE-3417:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #159 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/159/])
MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles.
svn merge --ignore-ancestry -c 1240428 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240429
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200260#comment-13200260 ]
Hudson commented on MAPREDUCE-3417:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #484 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/484/])
MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles.
svn merge --ignore-ancestry -c 1240428 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240429
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Thomas Graves (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Graves updated MAPREDUCE-3417:
-------------------------------------
Summary: job access controls not working app master and job history UI's (was: job access controls let invalid user see job info via web ui when they shouldn't be able to)
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Priority: Critical
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3417) job access controls not working
app master and job history UI's
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200256#comment-13200256 ]
Hudson commented on MAPREDUCE-3417:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1657 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1657/])
MAPREDUCE-3417. Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. Contributed by Jonathan Eagles.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1240428
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AppController.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java
> job access controls not working app master and job history UI's
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-3417
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3417
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Jonathan Eagles
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch, MAPREDUCE-3417.patch
>
>
> tested with security on, no filters defined for httpserver, job acls set so that only I could view/modify the job. Then went to the web ui to app master and job history server and both allowed me to view the job details. The webui shows the user "webuser". The RM properly rejected my request although it was using user "Dr.Who".
> The exception shown in the log is:
> 11/11/16 18:58:53 INFO mapred.JobACLsManager: job checkAccess user is: webuser
> 11/11/16 18:58:53 WARN security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser
> org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
> at org.apache.hadoop.util.Shell.run(Shell.java:188)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1043)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:221)
> at org.apache.hadoop.mapred.JobACLsManager.checkAccess(JobACLsManager.java:103)
> at org.apache.hadoop.mapreduce.v2.hs.CompletedJob.checkAccess(CompletedJob.java:325)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.checkAccess(AppController.java:292)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.requireJob(AppController.java:313)
> at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.job(AppController.java:97)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira