[jira] [Commented] (YARN-6546) SLS is slow while loading 10k queues

["Yufei Gu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993827#comment-15993827 ] 

Yufei Gu commented on YARN-6546:
--------------------------------

Patch v1 remote the code to create metrics files for all leaf queues. All metrics files are still created for active queues. Based my tests, SLS get much better performance with the patch. 

> SLS is slow while loading 10k queues
> ------------------------------------
>
>                 Key: YARN-6546
>                 URL: https://issues.apache.org/jira/browse/YARN-6546
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: scheduler-load-simulator
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Yufei Gu
>            Assignee: Yufei Gu
>         Attachments: Desktop.png, YARN-6546.001.patch
>
>
> It takes a long time (more than 10 minutes) to load 10k queues in SLS. The problem should be in {{com.codahale.metrics.CsvReporter}} based on the result from profiler. SLS creates 14 .csv files for each leaf queue, and update them constantly during execution. It is not necessary to log information for inactive queues. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org