You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Daniel Fritsi (Jira)" <ji...@apache.org> on 2021/01/29 23:30:00 UTC
[jira] [Created] (YARN-10601) The Yarn client should use the UGI
who created the Yarn client for obtaining a delegation token for the remote
log dir
Daniel Fritsi created YARN-10601:
------------------------------------
Summary: The Yarn client should use the UGI who created the Yarn client for obtaining a delegation token for the remote log dir
Key: YARN-10601
URL: https://issues.apache.org/jira/browse/YARN-10601
Project: Hadoop YARN
Issue Type: Bug
Components: log-aggregation
Affects Versions: 3.3.0
Reporter: Daniel Fritsi
It seems there was a bug introduced in YARN-10333 in this section of *{color:#0747A6}{{addLogAggregationDelegationToken}}{color}*:
{code:java}
Path remoteRootLogDir = fileController.getRemoteRootLogDir();
FileSystem fs = remoteRootLogDir.getFileSystem(conf);
final org.apache.hadoop.security.token.Token<?>[] finalTokens =
fs.addDelegationTokens(masterPrincipal, credentials);
{code}
*{color:#0747A6}{{remoteRootLogDir.getFileSystem}}{color}* simply does this:
{code:java}
public FileSystem getFileSystem(Configuration conf) throws IOException {
return FileSystem.get(this.toUri(), conf);
}
{code}
As far as I know it's customary to create a YarnClient instance via *{color:#0747A6}{{YarnClient.createYarnClient()}}{color}* in a UserGroupInformation.doAs block if you would like to use it with a different user then the current one. E.g.:
{code:java}
YarnClient yarnClient = ugi.doAs(new PrivilegedExceptionAction<YarnClient>() {
@Override
public YarnClient run() throws Exception {
YarnClient yarnClient = YarnClient.createYarnClient();
yarnClient.init(conf);
yarnClient.start();
return yarnClient;
}
});
{code}
If this statement is correct then I think YarnClient should save the *{color:#0747A6}{{UserGroupInformation.getCurrentUser()}}{color}* when the YarnClient is being created and the *{color:#0747A6}{{remoteRootLogDir.getFileSystem(conf)}}{color}* call should be made inside an ugi.doAs block with that saved user.
A more concrete example:
{code:java}
public YarnClient createYarnClient(UserGroupInformation ugi, Configuration conf) throws Exception {
return ugi.doAs((PrivilegedExceptionAction<YarnClient>) () -> {
// Her I am the submitterUser (see below)
YarnClient yarnClient = YarnClient.createYarnClient();
yarnClient.init(conf);
yarnClient.start();
return yarnClient;
});
}
public void run() {
// Here I am the serviceUser
// ...
Configuration conf = ...
// ...
UserGroupInformation ugi = getSubmitterUser();
// ...
YarnClient yarnClient = createYarnClient(ugi);
// ...
ApplicationSubmissionContext context = ...
// ...
yarnClient.submitApplication(context);
}
{code}
As you can see *{color:#0747A6}{{submitApplication}}{color}* is not invoked inside an ugi.doAs block and submitApplication is the one who will eventually invoke *{color:#0747A6}{{addLogAggregationDelegationToken}}{color}*. That's why we need to save the UGI during the YarnClient creation and create the FileSystem instance inside an ugi.doAs with that saved user. Otherwise Yarn will try to get a delegation token with an incorrect user (serviceUser) instead of the submitterUser.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org