[jira] [Updated] (YARN-12031) [Federation] Router GET scheduler-conf always fails: forwarded RM response entity is cast to ConfInfo

["Jose Luis López (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-12031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jose Luis López updated YARN-12031:
-----------------------------------
    Attachment: TestSchedConfRepro.java

> [Federation] Router GET scheduler-conf always fails: forwarded RM response entity is cast to ConfInfo
> -----------------------------------------------------------------------------------------------------
>
>                 Key: YARN-12031
>                 URL: https://issues.apache.org/jira/browse/YARN-12031
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: federation, router, webapp
>    Affects Versions: 3.4.0, 3.6.0
>            Reporter: Jose Luis López
>            Priority: Major
>         Attachments: TestSchedConfRepro.java
>
>
> {{FederationInterceptorREST#getSchedulerConfiguration}} fans out to every active sub-cluster through {{DefaultRequestInterceptorREST#getSchedulerConfiguration}}, which forwards with {{RouterWebServiceUtil.genericForward(..., Response.class, ...)}} and returns the Jersey client's inbound {{Response}} unread. For an inbound response {{getEntity()}} is the raw entity stream ({{org.glassfish.jersey.client.internal.HttpUrlConnector$2}}), never a {{ConfInfo}}:
> {code:java}
> } else if (response.getStatus() == Status.BAD_REQUEST.getStatusCode()) {
>   String errorMsg = String.valueOf(response.getEntity());
>   federationConfInfo.getErrorMsgs().add(errorMsg);
> } else if (response.getStatus() == Status.OK.getStatusCode()) {
>   ConfInfo fedConfInfo = (ConfInfo) response.getEntity();
> {code}
> * A sub-cluster that answers 200 (mutable-conf CapacityScheduler) throws {{ClassCastException}}. The {{catch (Exception e)}} turns it into {{400 getSchedulerConfiguration error.}} for the whole federation and does not log the cause.
> * A sub-cluster that answers 400 (no mutable store) adds an object-identity string such as {{org.glassfish.jersey.client.internal.HttpUrlConnector$2@74b97416}} to {{errorMsgs}} instead of the RM's message.
> {{TestFederationInterceptorREST#testGetSchedulerConfiguration}} passes only because {{MockDefaultRequestInterceptorREST#getSchedulerConfiguration}} returns an outbound {{Response.status(OK).entity(new ConfInfo(...))}}, whose {{getEntity()}} is the {{ConfInfo}}. The cast has been wrong since the API was added (YARN-11446, 3.4.0): before HADOOP-15984, {{clientResponseToResponse}} also set {{getEntityInputStream()}} as the entity.
> {{updateSchedulerConfiguration}} is not affected: it passes {{response.getEntity()}} through as the Router's own entity, which Jersey streams to the caller unchanged.
> *Repro*
> A real federation from the {{TestFederationSubCluster}} harness: ZooKeeper, two MiniYARNCluster RMs and the Router, each in its own JVM. SC-1 runs CapacityScheduler with {{yarn.scheduler.configuration.store.class=memory}}, SC-2 a plain CapacityScheduler. The attached {{TestSchedConfRepro.java}} drives it; its javadoc has the setup. trunk 9ede2e95c6f, Ubuntu 24.04, OpenJDK 17:
> {noformat}
> router GET accept=application/xml  status=400 body=getSchedulerConfiguration error.
> router GET accept=application/json status=400 body=getSchedulerConfiguration error.
> {noformat}
> Forwarding to each RM with {{genericForward(..., Response.class, ...)}} and {{Accept: application/xml}}, as {{DefaultRequestInterceptorREST}} does:
> {noformat}
> SC-1 status=200 getEntity().getClass()=org.glassfish.jersey.client.internal.HttpUrlConnector$2
> SC-1 status=200 cast -> java.lang.ClassCastException: class org.glassfish.jersey.client.internal.HttpUrlConnector$2 cannot be cast to class org.apache.hadoop.yarn.webapp.dao.ConfInfo
> SC-1 status=200 readEntity(ConfInfo) items=17
> SC-2 status=400 String.valueOf(getEntity())=org.glassfish.jersey.client.internal.HttpUrlConnector$2@74b97416
> SC-2 status=400 readEntity(String)=This API only supports to retrieve scheduler configuration from a mutable-conf scheduler, underneath scheduler CapacityScheduler is not an instance of MutableConfScheduler
> {noformat}
> *Proposed fix*
> Read the entity instead of casting it ({{readEntity}} also closes the response):
> {code:java}
> String errorMsg = response.readEntity(String.class);
> ...
> ConfInfo fedConfInfo = response.readEntity(ConfInfo.class);
> {code}
> and make {{MockDefaultRequestInterceptorREST#getSchedulerConfiguration}} return a response that can only be read with {{readEntity}}, so the existing test fails without the fix. With that change the Router returns 200 with SC-1's configuration and SC-2's error message, for both XML and JSON callers. The fan-out always asks the RMs for XML ({{clone(hsr)}} pins {{Accept}}), so this does not depend on YARN-12025.
> Statuses other than 200 and 400 (for example 403 for a non-admin caller) are still dropped silently and their response is not closed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org