[jira] [Commented] (SAMZA-1447) YarnCliJobStatusProvider cannot handle a high QPS to the JobsResource REST endpoint

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16200948#comment-16200948 ] 

ASF GitHub Bot commented on SAMZA-1447:
---------------------------------------

Github user asfgit closed the pull request at:

    https://github.com/apache/samza/pull/317


> YarnCliJobStatusProvider cannot handle a high QPS to the JobsResource REST endpoint
> -----------------------------------------------------------------------------------
>
>                 Key: SAMZA-1447
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1447
>             Project: Samza
>          Issue Type: Bug
>            Reporter: Abhishek Shivanna
>            Assignee: Abhishek Shivanna
>
> YarnCliJobStatusProvider shells out to a new process (running org.apache.hadoop.yarn.client.cli.ApplicationCLI) to fetch metadata about applications in the cluster.
> This leads to the host running out of memory when the number of requests being made to the JobsResource endpoint is high.
> The fix should be to use the Resource Manager REST endpoint instead which provides the same data without the new process per call overhead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)