[jira] [Assigned] (YARN-12025) YARN Router REST forwarding cannot read JSON responses and ignores its configured Jersey client

["Jose Luis López (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-12025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jose Luis López reassigned YARN-12025:
--------------------------------------

    Assignee: Jose Luis López

> YARN Router REST forwarding cannot read JSON responses and ignores its configured Jersey client
> -----------------------------------------------------------------------------------------------
>
>                 Key: YARN-12025
>                 URL: https://issues.apache.org/jira/browse/YARN-12025
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: federation, router, webapp
>    Affects Versions: 3.6.0
>            Reporter: Jose Luis López
>            Assignee: Jose Luis López
>            Priority: Major
>         Attachments: Yarn12025.java
>
>
> *Problem*: requests to the YARN Router's REST API that the Router has to parse fail when the response is JSON. For example {{POST /ws/v1/cluster/apps/new-application}} without an {{Accept}} header (or with {{Accept: application/json}}) answers 503 "No active SubCluster available to submit the request." while the Router logs:
> {noformat}
> ERROR org.apache.hadoop.yarn.server.router.RouterServerUtil: MessageBodyReader not found for media type=application/json;charset=utf-8, type=class org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NewApplication, genericType=class org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NewApplication.
> org.glassfish.jersey.message.internal.MessageBodyProviderNotFoundException: MessageBodyReader not found for media type=application/json;charset=utf-8, type=class org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NewApplication, ...
> {noformat}
> Calls that the Router forwards with a DAO return type work, because {{RouterWebServiceUtil.getMediaTypeFromHttpServletRequest}} asks the RM for XML in that case, and JAXB reads XML without extra providers.
> *Cause*: {{RouterWebServiceUtil.invokeRMWebService}}
> * builds a bare client, {{Client client1 = ClientBuilder.newClient();}}, with no JSON provider registered (the Router's own web app registers {{JettisonFeature}} and {{JAXBContextResolver}}, but this client does not), so a JSON entity cannot be read;
> * ignores the {{Client client}} it is given, built by {{createJerseyClient}} with the configured connect and read timeouts ({{yarn.router.webapp.connect-timeout}}, {{yarn.router.webapp.read-timeout}}, YARN-8529); it only closes that client, and never closes {{client1}}. So the timeouts do not apply to forwarded calls, and every forwarded call leaves a client open.
> *To reproduce*: a Router with {{FederationInterceptorREST}} in front of two sub-clusters, then {{curl -X POST 'http://<router>:8089/ws/v1/cluster/apps/new-application?user.name=<user>'}}.
> *Proposed fix*: in {{invokeRMWebService}}, send the request through the {{client}} parameter instead of a new one, and register the JSON support the Router web app uses ({{JettisonFeature}} and the {{JAXBContextResolver}}) in {{createJerseyClient}}. A test in {{TestRouterWebServicesREST}} (which runs real RMs) for {{createNewApplication}} with JSON would cover it.
> Found during the Jetty 12 (EE8) migration testing (HADOOP-19912), by an end-to-end test of a federated cluster built from trunk. The bug is on trunk itself; the migration does not cause it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org