[jira] [Commented] (YARN-10601) The Yarn client should use the UGI who created the Yarn client for obtaining a delegation token for the remote log dir

["Daniel Fritsi (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275423#comment-17275423 ] 

Daniel Fritsi commented on YARN-10601:
--------------------------------------

cc: [~prabhujoseph], [~weichiu]

> The Yarn client should use the UGI who created the Yarn client for obtaining a delegation token for the remote log dir
> ----------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-10601
>                 URL: https://issues.apache.org/jira/browse/YARN-10601
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: log-aggregation
>    Affects Versions: 3.3.0
>            Reporter: Daniel Fritsi
>            Priority: Critical
>
> It seems there was a bug introduced in YARN-10333 in this section of *{color:#0747A6}{{addLogAggregationDelegationToken}}{color}*:
> {code:java}
> Path remoteRootLogDir = fileController.getRemoteRootLogDir();
> FileSystem fs = remoteRootLogDir.getFileSystem(conf);
> final org.apache.hadoop.security.token.Token<?>[] finalTokens =
>     fs.addDelegationTokens(masterPrincipal, credentials);
> {code}
> *{color:#0747A6}{{remoteRootLogDir.getFileSystem}}{color}* simply does this:
> {code:java}
> public FileSystem getFileSystem(Configuration conf) throws IOException {
>   return FileSystem.get(this.toUri(), conf);
> }
> {code}
> As far as I know it's customary to create a YarnClient instance via *{color:#0747A6}{{YarnClient.createYarnClient()}}{color}* in a UserGroupInformation.doAs block if you would like to use it with a different user then the current one. E.g.:
> {code:java}
> YarnClient yarnClient = ugi.doAs(new PrivilegedExceptionAction<YarnClient>() {
>     @Override
>     public YarnClient run() throws Exception {
>         YarnClient yarnClient = YarnClient.createYarnClient();
>         yarnClient.init(conf);
>         yarnClient.start();
>         return yarnClient;
>     }
> });
> {code}
> If this statement is correct then I think YarnClient should save the *{color:#0747A6}{{UserGroupInformation.getCurrentUser()}}{color}* when the YarnClient is being created and the *{color:#0747A6}{{remoteRootLogDir.getFileSystem(conf)}}{color}* call should be made inside an ugi.doAs block with that saved user.
> A more concrete example:
> {code:java}
> public YarnClient createYarnClient(UserGroupInformation ugi, Configuration conf) throws Exception {
>     return ugi.doAs((PrivilegedExceptionAction<YarnClient>) () -> {
>         // Her I am the submitterUser (see below)
>         YarnClient yarnClient = YarnClient.createYarnClient();
>         yarnClient.init(conf);
>         yarnClient.start();
>         return yarnClient;
>     });
> }
> public void run() {
>     // Here I am the serviceUser
>     // ...
>     Configuration conf = ...
>     // ...
>     UserGroupInformation ugi = getSubmitterUser();
>     // ...
>     YarnClient yarnClient = createYarnClient(ugi);
>     // ...
>     ApplicationSubmissionContext context = ...
>     // ...
>     yarnClient.submitApplication(context);
> }
> {code}
> As you can see *{color:#0747A6}{{submitApplication}}{color}* is not invoked inside an ugi.doAs block and submitApplication is the one who will eventually invoke *{color:#0747A6}{{addLogAggregationDelegationToken}}{color}*. That's why we need to save the UGI during the YarnClient creation and create the FileSystem instance inside an ugi.doAs with that saved user. Otherwise Yarn will try to get a delegation token with an incorrect user (serviceUser) instead of the submitterUser.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org