You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Takanobu Asanuma (Jira)" <ji...@apache.org> on 2020/08/12 07:08:00 UTC
[jira] [Comment Edited] (HADOOP-17165) Implement service-user
feature in DecayRPCScheduler
[ https://issues.apache.org/jira/browse/HADOOP-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176082#comment-17176082 ]
Takanobu Asanuma edited comment on HADOOP-17165 at 8/12/20, 7:07 AM:
---------------------------------------------------------------------
Thanks for your comment, [~csun]. I will create PR and address your comment later.
I did some tests for our workload using Dynamometer with or without this patch. I uploaded the graph of the queue length. (before.png and after.png is a same workload.)
In the test, the queue size is two, high priority and low priority, and the weight ratio is 9:1. Our important service-user submitted jobs heavily in the time range. From the before graph, the RPCs of the service-user are stacked into the low priority queue. From the after graph (applied this patch), you can see they are into the high priority queue. Of course, the service-user command latency is faster with this patch.
was (Author: tasanuma0829):
Thanks for your comment, [~csun]. I will create PR and address your comment later.
I did some tests for our workload using Dynamometer with or without this patch. I uploaded the graph of the queue length.
In the test, the queue size is two, high priority and low priority, and the weight ratio is 9:1. Our important service-user submitted jobs heavily in the time range. From the before graph, the RPCs of the service-user are stacked into the low priority queue. From the after graph (applied this patch), you can see they are into the high priority queue. Of course, the service-user command latency is faster with this patch.
> Implement service-user feature in DecayRPCScheduler
> ---------------------------------------------------
>
> Key: HADOOP-17165
> URL: https://issues.apache.org/jira/browse/HADOOP-17165
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Takanobu Asanuma
> Assignee: Takanobu Asanuma
> Priority: Major
> Attachments: HADOOP-17165.001.patch, HADOOP-17165.002.patch, after.png, before.png
>
>
> In our cluster, we want to use FairCallQueue to limit heavy users, but not want to restrict certain users who are submitting important requests. This jira proposes to implement the service-user feature that the user is always scheduled high-priority queue.
> According to HADOOP-9640, the initial concept of FCQ has this feature, but not implemented finally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org