You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Yufei Gu (JIRA)" <ji...@apache.org> on 2017/02/07 20:07:42 UTC
[jira] [Comment Edited] (YARN-4090) Make Collections.sort() more
efficient in FSParentQueue.java
[ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15856692#comment-15856692 ]
Yufei Gu edited comment on YARN-4090 at 2/7/17 8:06 PM:
--------------------------------------------------------
Hi [~zsl2007], thanks for woking on this. I could misunderstand your patch. To make sure we are in the same page, let me clarify this: we are trying to remove the recursive computation of {{getResourceUsage()}} to improve the performance. Right? Base on that, here are some thoughts:
# FSParentQueue don’t do the recursive {{getResourceUsage()}} in your patch and resource usage updates while allocating, recovering and moving. Does FSParentQueue get updated on resource usage when an application finishes or its tasks finish? Resource usage is a critical metrics of queues, it would be nice to including some unit tests (or modification of existing tests) to make sure we do it right.
# Any reason we don't refactor the {{getResourceUsage()}} in FSLeafQueue?
Some minor nits:
# Use {{fsQueue}} instead of {{queue}} so you don’t need to cast.
# Need one empty line after function move()
was (Author: yufeigu):
Hi [~zsl2007], thanks for woking on this. I could misunderstand your patch. To make sure we are in the same page, let me clarify this: we are trying to remove the recursive computation of {{getResourceUsage()}} to improve the performance. Right? Base on that, here are some thoughts:
1. FSParentQueue don’t do the recursive {{getResourceUsage()}} in your patch and resource usage updates while allocating, recovering and moving. Does FSParentQueue get updated on resource usage when an application finishes or its tasks finish? Resource usage is a critical metrics of queues, it would be nice to including some unit tests (or modification of existing tests) to make sure we do it right.
2. Any reason we don't refactor the {{getResourceUsage()}} in FSLeafQueue?
Some minor nits:
# Use {{fsQueue}} instead of {{queue}} so you don’t need to cast.
# Need one empty line after function move()
> Make Collections.sort() more efficient in FSParentQueue.java
> ------------------------------------------------------------
>
> Key: YARN-4090
> URL: https://issues.apache.org/jira/browse/YARN-4090
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Reporter: Xianyin Xin
> Assignee: zhangshilong
> Attachments: sampling1.jpg, sampling2.jpg, YARN-4090.001.patch, YARN-4090.002.patch, YARN-4090.003.patch, YARN-4090.004.patch, YARN-4090.005.patch, YARN-4090.006.patch, YARN-4090-preview.patch, YARN-4090-TestResult.pdf
>
>
> Collections.sort() consumes too much time in a scheduling round.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org