You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Janus Chow (Jira)" <ji...@apache.org> on 2020/12/11 09:13:00 UTC
[jira] [Updated] (HADOOP-17429) Add reload option to
RefreshCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-17429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Janus Chow updated HADOOP-17429:
--------------------------------
Attachment: HADOOP-17429.001.patch
Assignee: Janus Chow
Status: Patch Available (was: Open)
The usage would be as follows:
* hdfs dfsadmin -refreshCallQueue (original operation, reconstruct scheduler and queues)
* hdfs dfsadmin -refreshCallQueue -reload (reload some configurations)
Currently the reload function won't do anything, with [HADOOP-17421|https://issues.apache.org/jira/projects/HADOOP/issues/HADOOP-17421] we can use the function to reload the static users without affecting NameNode's performance.
> Add reload option to RefreshCallQueue
> -------------------------------------
>
> Key: HADOOP-17429
> URL: https://issues.apache.org/jira/browse/HADOOP-17429
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Janus Chow
> Assignee: Janus Chow
> Priority: Major
> Attachments: HADOOP-17429.001.patch
>
>
> Currently the can use the dfsadmin command of "refreshCallQueue" to refresh all configs of FairCallQueue, but it will cause call queue spike during the refresh on NameNode.
> In [HADOOP-17421|https://issues.apache.org/jira/projects/HADOOP/issues/HADOOP-17421?filter=allopenissues], we added some configurations to specify queues for some static users, with which we can have fine-grained control on the behavior of different users manually, this feature will require us to refresh call queue of NameNodes constantly, which will sure cause bad impacts to NameNode's performance.
> This ticket was to propose adding a reload option for the command of refreshCallQueue, which will only trigger the _callQueueManager_ to reload something, instead of reconstructing new scheduler and new queues.
> The basic design is :
> # Add a method of "reload" in the interface of _Scheduler._
> # Add different _RefreshCallQueueTypes_ to trigger different refresh operations.
> # Add the argument in DFSAdmin for *reload* option.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org