You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Sho Shimauchi (JIRA)" <ji...@apache.org> on 2012/06/08 12:57:23 UTC
[jira] [Created] (MAPREDUCE-4329)
security.task.umbilical.protocol.acl should not be configurable
Sho Shimauchi created MAPREDUCE-4329:
----------------------------------------
Summary: security.task.umbilical.protocol.acl should not be configurable
Key: MAPREDUCE-4329
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4329
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: security
Affects Versions: 1.0.3
Reporter: Sho Shimauchi
On running MapReduce job, username is changed to jobid and the job fails.
Exception is as follows:
{code}
2012-06-08 19:39:26,555 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user job_201206081934_0002
org.apache.hadoop.util.Shell$ExitCodeException: id: job_201206081934_0002: no such user
at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
at org.apache.hadoop.util.Shell.run(Shell.java:182)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:461)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:444)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:68)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:45)
at org.apache.hadoop.security.Groups.getGroups(Groups.java:79)
at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1026)
at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:141)
at org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:99)
at org.apache.hadoop.ipc.Server.authorize(Server.java:1659)
at org.apache.hadoop.ipc.Server$Connection.authorizeConnection(Server.java:1320)
at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1286)
at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1182)
at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
{code}
This issue can be reproduced by following steps:
1. set hadoop.security.authorization = true in core-site.xml
{code}
<property>
<name>hadoop.security.authorization</name>
<value>true</value>
</property>
{code}
2. set any value except for '*' to security.task.umbilical.protocol.acl in hadoop-policy.xml
{code}
<property>
<name>security.task.umbilical.protocol.acl</name>
<value>sho sho</value>
<description>ACL for TaskUmbilicalProtocol, used by the map and reduce
tasks to communicate with the parent tasktracker.
The ACL is a comma-separated list of user and group names. The user and
group list is separated by a blank. For e.g. "alice,bob users,wheel".
A special value of "*" means all users are allowed.</description>
</property>
{code}
3. run any mapreduce job.
h4. Code Analysis
./src/mapred/org/apache/hadoop/mapred/Child.java:102-118
{code}
UserGroupInformation taskOwner
= UserGroupInformation.createRemoteUser(firstTaskid.getJobID().toString());
taskOwner.addToken(jt);
// Set the credentials
defaultConf.setCredentials(credentials);
final TaskUmbilicalProtocol umbilical =
taskOwner.doAs(new PrivilegedExceptionAction<TaskUmbilicalProtocol>() {
@Override
public TaskUmbilicalProtocol run() throws Exception {
return (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
TaskUmbilicalProtocol.versionID,
address,
defaultConf);
}
});
{code}
This code indicates that TaskUmbilicalProtocol uses jobid as username.
This code came from MAPREDUCE-1457.
https://issues.apache.org/jira/browse/MAPREDUCE-1457
Devaraj said as follows in the JIRA:
{quote}
2) In Child.java, the task authenticates to the TaskTracker using the jobtoken. The username in the jobtoken is jobId. The doAs block done using taskOwner is required so that the username mentioned in the token and the one doing the operation matches.
{quote}
We can't change security.task.umbilical.protocol.acl and should always be '*' .
TaskUmbilicalProtocol should be removed from MapReducePolicyProvider to disable security.task.umbilical.protocol.acl.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4329)
security.task.umbilical.protocol.acl should not be configurable
Posted by "Sho Shimauchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291745#comment-13291745 ]
Sho Shimauchi commented on MAPREDUCE-4329:
------------------------------------------
Removing TaskUmbilicalProtocol from MapReducePolicyProvider doesn't work.
{code}
2012-06-08 21:32:32.527 java[9886:1903] Unable to load realm info from SCDynamicStore
Exception in thread "main" org.apache.hadoop.ipc.RemoteException: Protocol interface org.apache.hadoop.mapred.TaskUmbilicalProtocol is not known.
at org.apache.hadoop.ipc.Client.call(Client.java:1070)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at $Proxy1.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:370)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:420)
at org.apache.hadoop.mapred.Child$1.run(Child.java:113)
at org.apache.hadoop.mapred.Child$1.run(Child.java:110)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
at org.apache.hadoop.mapred.Child.main(Child.java:109)
{code}
> security.task.umbilical.protocol.acl should not be configurable
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4329
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4329
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security
> Affects Versions: 1.0.3
> Reporter: Sho Shimauchi
>
> On running MapReduce job, username is changed to jobid and the job fails.
> Exception is as follows:
> {code}
> 2012-06-08 19:39:26,555 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user job_201206081934_0002
> org.apache.hadoop.util.Shell$ExitCodeException: id: job_201206081934_0002: no such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
> at org.apache.hadoop.util.Shell.run(Shell.java:182)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:461)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:444)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:68)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:45)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:79)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1026)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:141)
> at org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:99)
> at org.apache.hadoop.ipc.Server.authorize(Server.java:1659)
> at org.apache.hadoop.ipc.Server$Connection.authorizeConnection(Server.java:1320)
> at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1286)
> at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1182)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> {code}
> This issue can be reproduced by following steps:
> 1. set hadoop.security.authorization = true in core-site.xml
> {code}
> <property>
> <name>hadoop.security.authorization</name>
> <value>true</value>
> </property>
> {code}
> 2. set any value except for '*' to security.task.umbilical.protocol.acl in hadoop-policy.xml
> {code}
> <property>
> <name>security.task.umbilical.protocol.acl</name>
> <value>sho sho</value>
> <description>ACL for TaskUmbilicalProtocol, used by the map and reduce
> tasks to communicate with the parent tasktracker.
> The ACL is a comma-separated list of user and group names. The user and
> group list is separated by a blank. For e.g. "alice,bob users,wheel".
> A special value of "*" means all users are allowed.</description>
> </property>
> {code}
> 3. run any mapreduce job.
> h4. Code Analysis
> ./src/mapred/org/apache/hadoop/mapred/Child.java:102-118
> {code}
> UserGroupInformation taskOwner
> = UserGroupInformation.createRemoteUser(firstTaskid.getJobID().toString());
> taskOwner.addToken(jt);
>
> // Set the credentials
> defaultConf.setCredentials(credentials);
>
> final TaskUmbilicalProtocol umbilical =
> taskOwner.doAs(new PrivilegedExceptionAction<TaskUmbilicalProtocol>() {
> @Override
> public TaskUmbilicalProtocol run() throws Exception {
> return (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
> TaskUmbilicalProtocol.versionID,
> address,
> defaultConf);
> }
> });
> {code}
> This code indicates that TaskUmbilicalProtocol uses jobid as username.
> This code came from MAPREDUCE-1457.
> https://issues.apache.org/jira/browse/MAPREDUCE-1457
> Devaraj said as follows in the JIRA:
> {quote}
> 2) In Child.java, the task authenticates to the TaskTracker using the jobtoken. The username in the jobtoken is jobId. The doAs block done using taskOwner is required so that the username mentioned in the token and the one doing the operation matches.
> {quote}
> We can't change security.task.umbilical.protocol.acl and should always be '*' .
> TaskUmbilicalProtocol should be removed from MapReducePolicyProvider to disable security.task.umbilical.protocol.acl.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4329)
security.task.umbilical.protocol.acl should not be configurable
Posted by "Sho Shimauchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sho Shimauchi updated MAPREDUCE-4329:
-------------------------------------
Attachment: MAPREDUCE-4329.txt
replace the comment on MapReducePolicyProvider to Harsh's one.
> security.task.umbilical.protocol.acl should not be configurable
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4329
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4329
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security
> Affects Versions: 1.0.3
> Reporter: Sho Shimauchi
> Assignee: Sho Shimauchi
> Attachments: MAPREDUCE-4329.txt, MAPREDUCE-4329.txt
>
>
> On running MapReduce job, username is changed to jobid and the job fails.
> Exception is as follows:
> {code}
> 2012-06-08 19:39:26,555 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user job_201206081934_0002
> org.apache.hadoop.util.Shell$ExitCodeException: id: job_201206081934_0002: no such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
> at org.apache.hadoop.util.Shell.run(Shell.java:182)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:461)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:444)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:68)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:45)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:79)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1026)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:141)
> at org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:99)
> at org.apache.hadoop.ipc.Server.authorize(Server.java:1659)
> at org.apache.hadoop.ipc.Server$Connection.authorizeConnection(Server.java:1320)
> at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1286)
> at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1182)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> {code}
> This issue can be reproduced by following steps:
> 1. set hadoop.security.authorization = true in core-site.xml
> {code}
> <property>
> <name>hadoop.security.authorization</name>
> <value>true</value>
> </property>
> {code}
> 2. set any value except for '*' to security.task.umbilical.protocol.acl in hadoop-policy.xml
> {code}
> <property>
> <name>security.task.umbilical.protocol.acl</name>
> <value>sho sho</value>
> <description>ACL for TaskUmbilicalProtocol, used by the map and reduce
> tasks to communicate with the parent tasktracker.
> The ACL is a comma-separated list of user and group names. The user and
> group list is separated by a blank. For e.g. "alice,bob users,wheel".
> A special value of "*" means all users are allowed.</description>
> </property>
> {code}
> 3. run any mapreduce job.
> h4. Code Analysis
> ./src/mapred/org/apache/hadoop/mapred/Child.java:102-118
> {code}
> UserGroupInformation taskOwner
> = UserGroupInformation.createRemoteUser(firstTaskid.getJobID().toString());
> taskOwner.addToken(jt);
>
> // Set the credentials
> defaultConf.setCredentials(credentials);
>
> final TaskUmbilicalProtocol umbilical =
> taskOwner.doAs(new PrivilegedExceptionAction<TaskUmbilicalProtocol>() {
> @Override
> public TaskUmbilicalProtocol run() throws Exception {
> return (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
> TaskUmbilicalProtocol.versionID,
> address,
> defaultConf);
> }
> });
> {code}
> This code indicates that TaskUmbilicalProtocol uses jobid as username.
> This code came from MAPREDUCE-1457.
> https://issues.apache.org/jira/browse/MAPREDUCE-1457
> Devaraj said as follows in the JIRA:
> {quote}
> 2) In Child.java, the task authenticates to the TaskTracker using the jobtoken. The username in the jobtoken is jobId. The doAs block done using taskOwner is required so that the username mentioned in the token and the one doing the operation matches.
> {quote}
> We can't change security.task.umbilical.protocol.acl and should always be '*' .
> TaskUmbilicalProtocol should be removed from MapReducePolicyProvider to disable security.task.umbilical.protocol.acl.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4329)
security.task.umbilical.protocol.acl should not be configurable
Posted by "Sho Shimauchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393480#comment-13393480 ]
Sho Shimauchi commented on MAPREDUCE-4329:
------------------------------------------
Thanks for reviewing the patch!
Yeah your comment makes makes more sense, so I'll update the patch to replace the old comment to yours.
This property was set as deprecated in MAPREDUCE-2746 so I don't think there is nothing to do for trunk.
{code}
Configuration.addDeprecation("security.task.umbilical.protocol.acl",
new String[] {
MRJobConfig.MR_AM_SECURITY_SERVICE_AUTHORIZATION_TASK_UMBILICAL
});
{code}
> security.task.umbilical.protocol.acl should not be configurable
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4329
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4329
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security
> Affects Versions: 1.0.3
> Reporter: Sho Shimauchi
> Assignee: Sho Shimauchi
> Attachments: MAPREDUCE-4329.txt
>
>
> On running MapReduce job, username is changed to jobid and the job fails.
> Exception is as follows:
> {code}
> 2012-06-08 19:39:26,555 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user job_201206081934_0002
> org.apache.hadoop.util.Shell$ExitCodeException: id: job_201206081934_0002: no such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
> at org.apache.hadoop.util.Shell.run(Shell.java:182)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:461)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:444)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:68)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:45)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:79)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1026)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:141)
> at org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:99)
> at org.apache.hadoop.ipc.Server.authorize(Server.java:1659)
> at org.apache.hadoop.ipc.Server$Connection.authorizeConnection(Server.java:1320)
> at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1286)
> at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1182)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> {code}
> This issue can be reproduced by following steps:
> 1. set hadoop.security.authorization = true in core-site.xml
> {code}
> <property>
> <name>hadoop.security.authorization</name>
> <value>true</value>
> </property>
> {code}
> 2. set any value except for '*' to security.task.umbilical.protocol.acl in hadoop-policy.xml
> {code}
> <property>
> <name>security.task.umbilical.protocol.acl</name>
> <value>sho sho</value>
> <description>ACL for TaskUmbilicalProtocol, used by the map and reduce
> tasks to communicate with the parent tasktracker.
> The ACL is a comma-separated list of user and group names. The user and
> group list is separated by a blank. For e.g. "alice,bob users,wheel".
> A special value of "*" means all users are allowed.</description>
> </property>
> {code}
> 3. run any mapreduce job.
> h4. Code Analysis
> ./src/mapred/org/apache/hadoop/mapred/Child.java:102-118
> {code}
> UserGroupInformation taskOwner
> = UserGroupInformation.createRemoteUser(firstTaskid.getJobID().toString());
> taskOwner.addToken(jt);
>
> // Set the credentials
> defaultConf.setCredentials(credentials);
>
> final TaskUmbilicalProtocol umbilical =
> taskOwner.doAs(new PrivilegedExceptionAction<TaskUmbilicalProtocol>() {
> @Override
> public TaskUmbilicalProtocol run() throws Exception {
> return (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
> TaskUmbilicalProtocol.versionID,
> address,
> defaultConf);
> }
> });
> {code}
> This code indicates that TaskUmbilicalProtocol uses jobid as username.
> This code came from MAPREDUCE-1457.
> https://issues.apache.org/jira/browse/MAPREDUCE-1457
> Devaraj said as follows in the JIRA:
> {quote}
> 2) In Child.java, the task authenticates to the TaskTracker using the jobtoken. The username in the jobtoken is jobId. The doAs block done using taskOwner is required so that the username mentioned in the token and the one doing the operation matches.
> {quote}
> We can't change security.task.umbilical.protocol.acl and should always be '*' .
> TaskUmbilicalProtocol should be removed from MapReducePolicyProvider to disable security.task.umbilical.protocol.acl.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4329)
security.task.umbilical.protocol.acl should not be configurable
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393476#comment-13393476 ]
Harsh J commented on MAPREDUCE-4329:
------------------------------------
Thanks Sho. Patch looks good, just one comment (about the comment):
{code}
+ * Since TaskUmbilicalProtocol always uses job ID as username, setting
+ * specific user/group to security.task.umbilical.protocol.acl doesn't
+ * work. This property was removed from hadoop-policy.xml in
+ * MAPREDUCE-4329 but the property itself still works because we cannot
+ * remove the following code.
{code}
I think the following may be better, what are your thoughts?
{code}
Since TaskUmbilicalProtocol uses the job ID (of the task that uses it, hence dynamic) as its identifier, due to the security implementation, setting specific users/groups in security.task.umbilical.protocol.acl has no effect other than breaking jobs. This should never be configured to anything apart from '*', and hence MAPREDUCE-4329 removes it from the docs but this line remains to not break the protocol provider.
{code}
Also, is this issue present with MR2 ACLs too Sho? Or let me know if you'd like me to investigate that instead. We can do with a supplementary patch (Trunk first, branch-1 afterwards).
> security.task.umbilical.protocol.acl should not be configurable
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4329
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4329
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security
> Affects Versions: 1.0.3
> Reporter: Sho Shimauchi
> Assignee: Sho Shimauchi
> Attachments: MAPREDUCE-4329.txt
>
>
> On running MapReduce job, username is changed to jobid and the job fails.
> Exception is as follows:
> {code}
> 2012-06-08 19:39:26,555 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user job_201206081934_0002
> org.apache.hadoop.util.Shell$ExitCodeException: id: job_201206081934_0002: no such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
> at org.apache.hadoop.util.Shell.run(Shell.java:182)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:461)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:444)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:68)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:45)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:79)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1026)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:141)
> at org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:99)
> at org.apache.hadoop.ipc.Server.authorize(Server.java:1659)
> at org.apache.hadoop.ipc.Server$Connection.authorizeConnection(Server.java:1320)
> at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1286)
> at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1182)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> {code}
> This issue can be reproduced by following steps:
> 1. set hadoop.security.authorization = true in core-site.xml
> {code}
> <property>
> <name>hadoop.security.authorization</name>
> <value>true</value>
> </property>
> {code}
> 2. set any value except for '*' to security.task.umbilical.protocol.acl in hadoop-policy.xml
> {code}
> <property>
> <name>security.task.umbilical.protocol.acl</name>
> <value>sho sho</value>
> <description>ACL for TaskUmbilicalProtocol, used by the map and reduce
> tasks to communicate with the parent tasktracker.
> The ACL is a comma-separated list of user and group names. The user and
> group list is separated by a blank. For e.g. "alice,bob users,wheel".
> A special value of "*" means all users are allowed.</description>
> </property>
> {code}
> 3. run any mapreduce job.
> h4. Code Analysis
> ./src/mapred/org/apache/hadoop/mapred/Child.java:102-118
> {code}
> UserGroupInformation taskOwner
> = UserGroupInformation.createRemoteUser(firstTaskid.getJobID().toString());
> taskOwner.addToken(jt);
>
> // Set the credentials
> defaultConf.setCredentials(credentials);
>
> final TaskUmbilicalProtocol umbilical =
> taskOwner.doAs(new PrivilegedExceptionAction<TaskUmbilicalProtocol>() {
> @Override
> public TaskUmbilicalProtocol run() throws Exception {
> return (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
> TaskUmbilicalProtocol.versionID,
> address,
> defaultConf);
> }
> });
> {code}
> This code indicates that TaskUmbilicalProtocol uses jobid as username.
> This code came from MAPREDUCE-1457.
> https://issues.apache.org/jira/browse/MAPREDUCE-1457
> Devaraj said as follows in the JIRA:
> {quote}
> 2) In Child.java, the task authenticates to the TaskTracker using the jobtoken. The username in the jobtoken is jobId. The doAs block done using taskOwner is required so that the username mentioned in the token and the one doing the operation matches.
> {quote}
> We can't change security.task.umbilical.protocol.acl and should always be '*' .
> TaskUmbilicalProtocol should be removed from MapReducePolicyProvider to disable security.task.umbilical.protocol.acl.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4329)
security.task.umbilical.protocol.acl should not be configurable
Posted by "Sho Shimauchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437476#comment-13437476 ]
Sho Shimauchi commented on MAPREDUCE-4329:
------------------------------------------
Thanks for advice, Harsh.
In trunk, MR_AM_SECURITY_SERVICE_AUTHORIZATION_TASK_UMBILICAL is used only in MRAMPolicyProvider class.
{code:java}
private static final Service[] mapReduceApplicationMasterServices =
new Service[] {
new Service(
MRJobConfig.MR_AM_SECURITY_SERVICE_AUTHORIZATION_TASK_UMBILICAL,
TaskUmbilicalProtocol.class),
new Service(
MRJobConfig.MR_AM_SECURITY_SERVICE_AUTHORIZATION_CLIENT,
MRClientProtocolPB.class)
};
{code}
I found the new property "security.job.task.protocol.acl" is documented in hadoop-policy.xml.
I'll file a new JIRA and do the same way with trunk.
> security.task.umbilical.protocol.acl should not be configurable
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4329
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4329
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security
> Affects Versions: 1.0.3
> Reporter: Sho Shimauchi
> Assignee: Sho Shimauchi
> Attachments: MAPREDUCE-4329.txt, MAPREDUCE-4329.txt
>
>
> On running MapReduce job, username is changed to jobid and the job fails.
> Exception is as follows:
> {code}
> 2012-06-08 19:39:26,555 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user job_201206081934_0002
> org.apache.hadoop.util.Shell$ExitCodeException: id: job_201206081934_0002: no such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
> at org.apache.hadoop.util.Shell.run(Shell.java:182)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:461)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:444)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:68)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:45)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:79)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1026)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:141)
> at org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:99)
> at org.apache.hadoop.ipc.Server.authorize(Server.java:1659)
> at org.apache.hadoop.ipc.Server$Connection.authorizeConnection(Server.java:1320)
> at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1286)
> at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1182)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> {code}
> This issue can be reproduced by following steps:
> 1. set hadoop.security.authorization = true in core-site.xml
> {code}
> <property>
> <name>hadoop.security.authorization</name>
> <value>true</value>
> </property>
> {code}
> 2. set any value except for '*' to security.task.umbilical.protocol.acl in hadoop-policy.xml
> {code}
> <property>
> <name>security.task.umbilical.protocol.acl</name>
> <value>sho sho</value>
> <description>ACL for TaskUmbilicalProtocol, used by the map and reduce
> tasks to communicate with the parent tasktracker.
> The ACL is a comma-separated list of user and group names. The user and
> group list is separated by a blank. For e.g. "alice,bob users,wheel".
> A special value of "*" means all users are allowed.</description>
> </property>
> {code}
> 3. run any mapreduce job.
> h4. Code Analysis
> ./src/mapred/org/apache/hadoop/mapred/Child.java:102-118
> {code}
> UserGroupInformation taskOwner
> = UserGroupInformation.createRemoteUser(firstTaskid.getJobID().toString());
> taskOwner.addToken(jt);
>
> // Set the credentials
> defaultConf.setCredentials(credentials);
>
> final TaskUmbilicalProtocol umbilical =
> taskOwner.doAs(new PrivilegedExceptionAction<TaskUmbilicalProtocol>() {
> @Override
> public TaskUmbilicalProtocol run() throws Exception {
> return (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
> TaskUmbilicalProtocol.versionID,
> address,
> defaultConf);
> }
> });
> {code}
> This code indicates that TaskUmbilicalProtocol uses jobid as username.
> This code came from MAPREDUCE-1457.
> https://issues.apache.org/jira/browse/MAPREDUCE-1457
> Devaraj said as follows in the JIRA:
> {quote}
> 2) In Child.java, the task authenticates to the TaskTracker using the jobtoken. The username in the jobtoken is jobId. The doAs block done using taskOwner is required so that the username mentioned in the token and the one doing the operation matches.
> {quote}
> We can't change security.task.umbilical.protocol.acl and should always be '*' .
> TaskUmbilicalProtocol should be removed from MapReducePolicyProvider to disable security.task.umbilical.protocol.acl.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4329)
security.task.umbilical.protocol.acl should not be configurable
Posted by "Sho Shimauchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sho Shimauchi updated MAPREDUCE-4329:
-------------------------------------
Attachment: MAPREDUCE-4329.txt
This patch removes the acl property from hadoop-policy.xml and docs, but doesn't remove the property itself from code.
I wrote comment in MapReducePolicyProvider which registers TaskUmbilicalProtocol as a Service.
> security.task.umbilical.protocol.acl should not be configurable
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4329
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4329
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security
> Affects Versions: 1.0.3
> Reporter: Sho Shimauchi
> Assignee: Sho Shimauchi
> Attachments: MAPREDUCE-4329.txt
>
>
> On running MapReduce job, username is changed to jobid and the job fails.
> Exception is as follows:
> {code}
> 2012-06-08 19:39:26,555 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user job_201206081934_0002
> org.apache.hadoop.util.Shell$ExitCodeException: id: job_201206081934_0002: no such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
> at org.apache.hadoop.util.Shell.run(Shell.java:182)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:461)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:444)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:68)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:45)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:79)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1026)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:141)
> at org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:99)
> at org.apache.hadoop.ipc.Server.authorize(Server.java:1659)
> at org.apache.hadoop.ipc.Server$Connection.authorizeConnection(Server.java:1320)
> at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1286)
> at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1182)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> {code}
> This issue can be reproduced by following steps:
> 1. set hadoop.security.authorization = true in core-site.xml
> {code}
> <property>
> <name>hadoop.security.authorization</name>
> <value>true</value>
> </property>
> {code}
> 2. set any value except for '*' to security.task.umbilical.protocol.acl in hadoop-policy.xml
> {code}
> <property>
> <name>security.task.umbilical.protocol.acl</name>
> <value>sho sho</value>
> <description>ACL for TaskUmbilicalProtocol, used by the map and reduce
> tasks to communicate with the parent tasktracker.
> The ACL is a comma-separated list of user and group names. The user and
> group list is separated by a blank. For e.g. "alice,bob users,wheel".
> A special value of "*" means all users are allowed.</description>
> </property>
> {code}
> 3. run any mapreduce job.
> h4. Code Analysis
> ./src/mapred/org/apache/hadoop/mapred/Child.java:102-118
> {code}
> UserGroupInformation taskOwner
> = UserGroupInformation.createRemoteUser(firstTaskid.getJobID().toString());
> taskOwner.addToken(jt);
>
> // Set the credentials
> defaultConf.setCredentials(credentials);
>
> final TaskUmbilicalProtocol umbilical =
> taskOwner.doAs(new PrivilegedExceptionAction<TaskUmbilicalProtocol>() {
> @Override
> public TaskUmbilicalProtocol run() throws Exception {
> return (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
> TaskUmbilicalProtocol.versionID,
> address,
> defaultConf);
> }
> });
> {code}
> This code indicates that TaskUmbilicalProtocol uses jobid as username.
> This code came from MAPREDUCE-1457.
> https://issues.apache.org/jira/browse/MAPREDUCE-1457
> Devaraj said as follows in the JIRA:
> {quote}
> 2) In Child.java, the task authenticates to the TaskTracker using the jobtoken. The username in the jobtoken is jobId. The doAs block done using taskOwner is required so that the username mentioned in the token and the one doing the operation matches.
> {quote}
> We can't change security.task.umbilical.protocol.acl and should always be '*' .
> TaskUmbilicalProtocol should be removed from MapReducePolicyProvider to disable security.task.umbilical.protocol.acl.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4329)
security.task.umbilical.protocol.acl should not be configurable
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291781#comment-13291781 ]
Harsh J commented on MAPREDUCE-4329:
------------------------------------
I think removing its config from the docs/default policy files (if that works) or documenting that this should never be changed from '*' should be good enough. We should let it exist in the policy set (as its supposed to be a complete list).
The real problem is that users may change this/care about changing it, which is a wrong thing to do.
> security.task.umbilical.protocol.acl should not be configurable
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4329
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4329
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security
> Affects Versions: 1.0.3
> Reporter: Sho Shimauchi
>
> On running MapReduce job, username is changed to jobid and the job fails.
> Exception is as follows:
> {code}
> 2012-06-08 19:39:26,555 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user job_201206081934_0002
> org.apache.hadoop.util.Shell$ExitCodeException: id: job_201206081934_0002: no such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
> at org.apache.hadoop.util.Shell.run(Shell.java:182)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:461)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:444)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:68)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:45)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:79)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1026)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:141)
> at org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:99)
> at org.apache.hadoop.ipc.Server.authorize(Server.java:1659)
> at org.apache.hadoop.ipc.Server$Connection.authorizeConnection(Server.java:1320)
> at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1286)
> at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1182)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> {code}
> This issue can be reproduced by following steps:
> 1. set hadoop.security.authorization = true in core-site.xml
> {code}
> <property>
> <name>hadoop.security.authorization</name>
> <value>true</value>
> </property>
> {code}
> 2. set any value except for '*' to security.task.umbilical.protocol.acl in hadoop-policy.xml
> {code}
> <property>
> <name>security.task.umbilical.protocol.acl</name>
> <value>sho sho</value>
> <description>ACL for TaskUmbilicalProtocol, used by the map and reduce
> tasks to communicate with the parent tasktracker.
> The ACL is a comma-separated list of user and group names. The user and
> group list is separated by a blank. For e.g. "alice,bob users,wheel".
> A special value of "*" means all users are allowed.</description>
> </property>
> {code}
> 3. run any mapreduce job.
> h4. Code Analysis
> ./src/mapred/org/apache/hadoop/mapred/Child.java:102-118
> {code}
> UserGroupInformation taskOwner
> = UserGroupInformation.createRemoteUser(firstTaskid.getJobID().toString());
> taskOwner.addToken(jt);
>
> // Set the credentials
> defaultConf.setCredentials(credentials);
>
> final TaskUmbilicalProtocol umbilical =
> taskOwner.doAs(new PrivilegedExceptionAction<TaskUmbilicalProtocol>() {
> @Override
> public TaskUmbilicalProtocol run() throws Exception {
> return (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
> TaskUmbilicalProtocol.versionID,
> address,
> defaultConf);
> }
> });
> {code}
> This code indicates that TaskUmbilicalProtocol uses jobid as username.
> This code came from MAPREDUCE-1457.
> https://issues.apache.org/jira/browse/MAPREDUCE-1457
> Devaraj said as follows in the JIRA:
> {quote}
> 2) In Child.java, the task authenticates to the TaskTracker using the jobtoken. The username in the jobtoken is jobId. The doAs block done using taskOwner is required so that the username mentioned in the token and the one doing the operation matches.
> {quote}
> We can't change security.task.umbilical.protocol.acl and should always be '*' .
> TaskUmbilicalProtocol should be removed from MapReducePolicyProvider to disable security.task.umbilical.protocol.acl.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4329)
security.task.umbilical.protocol.acl should not be configurable
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404587#comment-13404587 ]
Harsh J commented on MAPREDUCE-4329:
------------------------------------
Patch looks good for branch-1. +1.
However, before we commit it in, I'd like to double-check trunk/branch-2 states. MAPREDUCE-2746 merely deprecated the old name into a new name, but please grep the new name and if you find it documented anywhere, please remove the same.
If none of the new style AM names aren't documented anywhere, then please file a new JIRA to have them documented for trunk/branch-2 (where AMs exist), and we can ensure there (in the description) to not document MR_AM_SECURITY_SERVICE_AUTHORIZATION_TASK_UMBILICAL specifically, for the same reason. Thereby we cover trunk as well as the 1.x maintenance branch.
Does this make sense Sho? Please let us know what you find! :)
> security.task.umbilical.protocol.acl should not be configurable
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4329
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4329
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security
> Affects Versions: 1.0.3
> Reporter: Sho Shimauchi
> Assignee: Sho Shimauchi
> Attachments: MAPREDUCE-4329.txt, MAPREDUCE-4329.txt
>
>
> On running MapReduce job, username is changed to jobid and the job fails.
> Exception is as follows:
> {code}
> 2012-06-08 19:39:26,555 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user job_201206081934_0002
> org.apache.hadoop.util.Shell$ExitCodeException: id: job_201206081934_0002: no such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
> at org.apache.hadoop.util.Shell.run(Shell.java:182)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:461)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:444)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:68)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:45)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:79)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1026)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:141)
> at org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:99)
> at org.apache.hadoop.ipc.Server.authorize(Server.java:1659)
> at org.apache.hadoop.ipc.Server$Connection.authorizeConnection(Server.java:1320)
> at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1286)
> at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1182)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> {code}
> This issue can be reproduced by following steps:
> 1. set hadoop.security.authorization = true in core-site.xml
> {code}
> <property>
> <name>hadoop.security.authorization</name>
> <value>true</value>
> </property>
> {code}
> 2. set any value except for '*' to security.task.umbilical.protocol.acl in hadoop-policy.xml
> {code}
> <property>
> <name>security.task.umbilical.protocol.acl</name>
> <value>sho sho</value>
> <description>ACL for TaskUmbilicalProtocol, used by the map and reduce
> tasks to communicate with the parent tasktracker.
> The ACL is a comma-separated list of user and group names. The user and
> group list is separated by a blank. For e.g. "alice,bob users,wheel".
> A special value of "*" means all users are allowed.</description>
> </property>
> {code}
> 3. run any mapreduce job.
> h4. Code Analysis
> ./src/mapred/org/apache/hadoop/mapred/Child.java:102-118
> {code}
> UserGroupInformation taskOwner
> = UserGroupInformation.createRemoteUser(firstTaskid.getJobID().toString());
> taskOwner.addToken(jt);
>
> // Set the credentials
> defaultConf.setCredentials(credentials);
>
> final TaskUmbilicalProtocol umbilical =
> taskOwner.doAs(new PrivilegedExceptionAction<TaskUmbilicalProtocol>() {
> @Override
> public TaskUmbilicalProtocol run() throws Exception {
> return (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
> TaskUmbilicalProtocol.versionID,
> address,
> defaultConf);
> }
> });
> {code}
> This code indicates that TaskUmbilicalProtocol uses jobid as username.
> This code came from MAPREDUCE-1457.
> https://issues.apache.org/jira/browse/MAPREDUCE-1457
> Devaraj said as follows in the JIRA:
> {quote}
> 2) In Child.java, the task authenticates to the TaskTracker using the jobtoken. The username in the jobtoken is jobId. The doAs block done using taskOwner is required so that the username mentioned in the token and the one doing the operation matches.
> {quote}
> We can't change security.task.umbilical.protocol.acl and should always be '*' .
> TaskUmbilicalProtocol should be removed from MapReducePolicyProvider to disable security.task.umbilical.protocol.acl.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-4329)
security.task.umbilical.protocol.acl should not be configurable
Posted by "Sho Shimauchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sho Shimauchi reassigned MAPREDUCE-4329:
----------------------------------------
Assignee: Sho Shimauchi
> security.task.umbilical.protocol.acl should not be configurable
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4329
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4329
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security
> Affects Versions: 1.0.3
> Reporter: Sho Shimauchi
> Assignee: Sho Shimauchi
>
> On running MapReduce job, username is changed to jobid and the job fails.
> Exception is as follows:
> {code}
> 2012-06-08 19:39:26,555 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user job_201206081934_0002
> org.apache.hadoop.util.Shell$ExitCodeException: id: job_201206081934_0002: no such user
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
> at org.apache.hadoop.util.Shell.run(Shell.java:182)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:461)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:444)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:68)
> at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:45)
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:79)
> at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1026)
> at org.apache.hadoop.security.authorize.AccessControlList.isUserAllowed(AccessControlList.java:141)
> at org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:99)
> at org.apache.hadoop.ipc.Server.authorize(Server.java:1659)
> at org.apache.hadoop.ipc.Server$Connection.authorizeConnection(Server.java:1320)
> at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1286)
> at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1182)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> {code}
> This issue can be reproduced by following steps:
> 1. set hadoop.security.authorization = true in core-site.xml
> {code}
> <property>
> <name>hadoop.security.authorization</name>
> <value>true</value>
> </property>
> {code}
> 2. set any value except for '*' to security.task.umbilical.protocol.acl in hadoop-policy.xml
> {code}
> <property>
> <name>security.task.umbilical.protocol.acl</name>
> <value>sho sho</value>
> <description>ACL for TaskUmbilicalProtocol, used by the map and reduce
> tasks to communicate with the parent tasktracker.
> The ACL is a comma-separated list of user and group names. The user and
> group list is separated by a blank. For e.g. "alice,bob users,wheel".
> A special value of "*" means all users are allowed.</description>
> </property>
> {code}
> 3. run any mapreduce job.
> h4. Code Analysis
> ./src/mapred/org/apache/hadoop/mapred/Child.java:102-118
> {code}
> UserGroupInformation taskOwner
> = UserGroupInformation.createRemoteUser(firstTaskid.getJobID().toString());
> taskOwner.addToken(jt);
>
> // Set the credentials
> defaultConf.setCredentials(credentials);
>
> final TaskUmbilicalProtocol umbilical =
> taskOwner.doAs(new PrivilegedExceptionAction<TaskUmbilicalProtocol>() {
> @Override
> public TaskUmbilicalProtocol run() throws Exception {
> return (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
> TaskUmbilicalProtocol.versionID,
> address,
> defaultConf);
> }
> });
> {code}
> This code indicates that TaskUmbilicalProtocol uses jobid as username.
> This code came from MAPREDUCE-1457.
> https://issues.apache.org/jira/browse/MAPREDUCE-1457
> Devaraj said as follows in the JIRA:
> {quote}
> 2) In Child.java, the task authenticates to the TaskTracker using the jobtoken. The username in the jobtoken is jobId. The doAs block done using taskOwner is required so that the username mentioned in the token and the one doing the operation matches.
> {quote}
> We can't change security.task.umbilical.protocol.acl and should always be '*' .
> TaskUmbilicalProtocol should be removed from MapReducePolicyProvider to disable security.task.umbilical.protocol.acl.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira