You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "YulongZ (Jira)" <ji...@apache.org> on 2022/06/08 06:17:00 UTC
[jira] [Updated] (YARN-11139) Minor performance improvements in DelegationTokenRenewerPoolTracker
[ https://issues.apache.org/jira/browse/YARN-11139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
YulongZ updated YARN-11139:
---------------------------
Affects Version/s: 3.3.3
> Minor performance improvements in DelegationTokenRenewerPoolTracker
> -------------------------------------------------------------------
>
> Key: YARN-11139
> URL: https://issues.apache.org/jira/browse/YARN-11139
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
> Affects Versions: 3.3.1, 3.3.3
> Environment: Hadoop3.3.1 with Kerberos
> Reporter: YulongZ
> Priority: Minor
> Attachments: CPU cost of ResourceManager with no sleep time.png, CPU cost of ResourceManager with sleep 1.png, Flame diagram of ResourceManager with no sleep time.svg, Flame diagram of ResourceManager with sleep 1.svg
>
>
> The DelegationTokenRenewerPoolTracker thread wastes of CPU resources when no application running，the run method always executes a for loop, but actually does nothing.
> {code:java}
> // code with no change
> @Override
> public void run() {
> while (true) {
> for (Map.Entry<DelegationTokenRenewerEvent, Future<?>> entry : futures
> .entrySet()) {
> DelegationTokenRenewerEvent evt = entry.getKey();
> Future<?> future = entry.getValue();
> try {
> future.get(tokenRenewerThreadTimeout, TimeUnit.MILLISECONDS);
> } catch (TimeoutException e) {
> // Cancel thread and retry the same event in case of timeout
> if (future != null && !future.isDone() && !future.isCancelled()) {
> future.cancel(true);
> futures.remove(evt);
> if (evt.getAttempt() < tokenRenewerThreadRetryMaxAttempts) {
> renewalTimer.schedule(
> getTimerTask((AbstractDelegationTokenRenewerAppEvent) evt),
> tokenRenewerThreadRetryInterval);
> } else {
> LOG.info(
> "Exhausted max retry attempts {} in token renewer "
> + "thread for {}",
> tokenRenewerThreadRetryMaxAttempts, evt.getApplicationId());
> }
> }
> } catch (Exception e) {
> LOG.info("Problem in submitting renew tasks in token renewer "
> + "thread.", e);
> }
> }
> }
> }
> {code}
> Maybe we can add a tiny sleep to avoid waste of CPU resources. For example below，
> {code:java}
> // code with a tiny sleep
> public void run() {
> while (true) {
> try {
> Thread.sleep(1);
> } catch (InterruptedException e) {
> Thread.currentThread().interrupt();
> }
> for (Map.Entry<DelegationTokenRenewerEvent, Future<?>> entry : futures
> .entrySet()) {
> DelegationTokenRenewerEvent evt = entry.getKey();
> ... {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org