You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Rohith Sharma K S (JIRA)" <ji...@apache.org> on 2019/01/23 10:15:00 UTC
[jira] [Commented] (YARN-9224)
TimelineDelegationTokenIdentifier.Renewer contacts KDC for every
renew/cancel token operation
[ https://issues.apache.org/jira/browse/YARN-9224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749766#comment-16749766 ]
Rohith Sharma K S commented on YARN-9224:
-----------------------------------------
It means that application submission rate per second is very high. When the application is submitted/finished, RM renews/cancel ALL the tokens passed in submissionContext. Since these tokens are maintained by RM, it is responsibility of RM to renew/cancel. The renew is happening for different application every time(Not for same application). It is directly proportional to application submission rate.
> TimelineDelegationTokenIdentifier.Renewer contacts KDC for every renew/cancel token operation
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-9224
> URL: https://issues.apache.org/jira/browse/YARN-9224
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.6.0, 2.7.3
> Reporter: Tarun Parimi
> Priority: Major
>
> In a production cluster, we have observed the active RM principal making excessive requests to the KDC server. Being a service principal, this shouldn't be the case normally.
> On capturing tcpdump for the connections between RM and KDC, we saw that these excessive requests were for the SPNEGO service HTTP/ats-host.example.com .
> The requests were also matching in frequency with the below log entry in RM.
> {code:java}
> 2019-01-09T03:41:56.048-0500 INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl: Timeline service address: http://ats-host.example.com:8188/ws/v1/timeline/
> {code}
> On looking at the code in TimelineDelegationTokenIdentifier.java, it seems this kdc request for SPNEGO is done as we are creating a new timeline client instance every time.
> {code:java}
> @SuppressWarnings("unchecked")
> @Override
> public long renew(Token<?> token, Configuration conf) throws IOException,
> InterruptedException {
> TimelineClient client = TimelineClient.createTimelineClient();
> try {
> client.init(conf);
> client.start();
> return client.renewDelegationToken(
> (Token<TimelineDelegationTokenIdentifier>) token);
> } catch (YarnException e) {
> throw new IOException(e);
> } finally {
> client.stop();
> }
> }
> @SuppressWarnings("unchecked")
> @Override
> public void cancel(Token<?> token, Configuration conf) throws IOException,
> InterruptedException {
> TimelineClient client = TimelineClient.createTimelineClient();
> try {
> client.init(conf);
> client.start();
> client.cancelDelegationToken(
> (Token<TimelineDelegationTokenIdentifier>) token);
> } catch (YarnException e) {
> throw new IOException(e);
> } finally {
> client.stop();
> }
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org