[jira] [Updated] (YARN-6137) Yarn client implicitly invoke ATS client which accesses HDFS

["Li Lu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-6137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Li Lu updated YARN-6137:
------------------------
    Attachment: YARN-6137-trunk.001.patch

First patch to fix this issue. Note that we always start a timeline client when we start a YarnClientImpl when timeline service is enabled. In ATS v1.5, timeline client will check HDFS access upon service start, this requires the yarn client user to be authenticated when it's started. In fact, users only need this client to renew timeline tokens under secured environment, and it's totally fine to firstly start the client user process, authenticate it, and then renew the delegation token.

So in this patch I'm delaying the start of the timeline client to the first time user needs a delegation token. For secured environments, this allows the parent process (running this client) to finish authentication after service start, then use the timeline client to renew tokens. One thing I'm not sure about is if yarn client itself should be thread safe. If this is the case I can add some synchronization for the time client initialization. 

Another change I made is to remove one unit test to check if YarnClient would catch an Error, and fails the test when we did not catch the Error. To me this does not appear to be a reasonable behavior. Since it blocks testing, I'm removing it. 

> Yarn client implicitly invoke ATS client which accesses HDFS
> ------------------------------------------------------------
>
>                 Key: YARN-6137
>                 URL: https://issues.apache.org/jira/browse/YARN-6137
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Yesha Vora
>            Assignee: Li Lu
>         Attachments: YARN-6137-trunk.001.patch
>
>
> Yarn is implicitly trying to invoke ATS Client even though client does not need it. and ATSClient code is trying to access hdfs. Due to that service is hitting GSS exception. 
> Yarnclient is implicitly creating ats client that tries to access Hdfs.
> All servers that use yarnclient cannot be expected to change to accommodate this behavior.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org