[jira] [Created] (YARN-12024) Distributed shell cannot be submitted through the YARN Router: getResourceProfiles hides YARNFeatureNotEnabledException

["Jose Luis López (Jira)" <ji...@apache.org>]

Jose Luis López created YARN-12024:
--------------------------------------

             Summary: Distributed shell cannot be submitted through the YARN Router: getResourceProfiles hides YARNFeatureNotEnabledException
                 Key: YARN-12024
                 URL: https://issues.apache.org/jira/browse/YARN-12024
             Project: Hadoop YARN
          Issue Type: Bug
          Components: federation, router, distributed-shell
    Affects Versions: 3.6.0
            Reporter: Jose Luis López


*Problem*: in a YARN federation, submitting a distributed shell application through the Router fails before the application is created:
{noformat}
ERROR distributedshell.Client: Error running Client
org.apache.hadoop.yarn.exceptions.YarnException: Unable to get resource profiles due to exception.invokeConcurrent Failed = org.apache.hadoop.yarn.exceptions.YarnException: subClusterId sc1 exec getResourceProfiles error Resource profile is not enabled, please enable resource profile feature before using its functions. (by setting yarn.resourcemanager.resource-profiles.enabled to true)
  at org.apache.hadoop.yarn.server.router.RouterServerUtil.logAndThrowException(RouterServerUtil.java:131)
{noformat}
Submitting the same application directly to a sub-cluster's RM works.

*Cause*: resource profiles are off by default, so each RM's {{getResourceProfiles}} throws {{YARNFeatureNotEnabledException}}. The distributed shell {{Client}} expects that and carries on without profiles:
{code:java}
try {
  profiles = yarnClient.getResourceProfiles();
} catch (YARNFeatureNotEnabledException re) {
  profiles = null;
}
{code}
{{FederationClientInterceptor.getResourceProfiles}} calls {{invokeConcurrent}} on every sub-cluster and, on any exception, throws a new plain {{YarnException}} through {{RouterServerUtil.logAndThrowException("Unable to get resource profiles due to exception.", ex)}}. The client no longer sees a {{YARNFeatureNotEnabledException}} and fails. Any client that relies on the same exception (it is part of the {{getResourceProfiles}} contract) breaks the same way through the Router.

*To reproduce*: two sub-clusters with default settings behind a Router ({{FederationClientInterceptor}}), then {{yarn jar hadoop-yarn-applications-distributedshell-*.jar -jar <same jar> -shell_command echo -num_containers 1}} with {{yarn.resourcemanager.address}} pointing at the Router's client RPC address.

*Proposed fix*: when the sub-clusters report that resource profiles are disabled, the Router should report the same, e.g. in {{getResourceProfiles}} (and likewise {{getResourceProfile}}) rethrow a {{YARNFeatureNotEnabledException}} when the cause found in {{invokeConcurrent}} is one, instead of wrapping it. A test in {{TestFederationClientInterceptor}} with profiles disabled can assert the exception type.

Found by an end-to-end test of a federated cluster built from trunk.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org