[jira] [Commented] (SLIDER-1168) Avoid or eliminate expensive YARN get all applications call

["Gour Saha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SLIDER-1168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15654923#comment-15654923 ] 

Gour Saha commented on SLIDER-1168:
-----------------------------------

YARN now has the API to filter by application tags - YARN-4491. We should be able to use this to avoid expensive calls.

> Avoid or eliminate expensive YARN get all applications call
> -----------------------------------------------------------
>
>                 Key: SLIDER-1168
>                 URL: https://issues.apache.org/jira/browse/SLIDER-1168
>             Project: Slider
>          Issue Type: Bug
>    Affects Versions: Slider 0.91
>            Reporter: Gour Saha
>
> YARN get all applications call will prove to be very costly in a cluster with hundreds or thousands of Slider apps running. We need to come up with a strategy to avoid or eliminate these expensive calls. This would mean that the application id needs to be stored in Slider's state store in HDFS (or some other better strategy). This is specifically more critical when Slider Client calls are made from a service like the YARN native REST API service (YARN-4793).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)