You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Janus Chow (Jira)" <ji...@apache.org> on 2020/12/11 09:13:00 UTC

[jira] [Updated] (HADOOP-17429) Add reload option to RefreshCallQueue

     [ https://issues.apache.org/jira/browse/HADOOP-17429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Janus Chow updated HADOOP-17429:
--------------------------------
    Attachment: HADOOP-17429.001.patch
      Assignee: Janus Chow
        Status: Patch Available  (was: Open)

The usage would be as follows:
 * hdfs dfsadmin -refreshCallQueue (original operation, reconstruct scheduler and queues)
 * hdfs dfsadmin -refreshCallQueue -reload  (reload some configurations)

Currently the reload function won't do anything, with [HADOOP-17421|https://issues.apache.org/jira/projects/HADOOP/issues/HADOOP-17421] we can use the function to reload the static users without affecting NameNode's performance.

> Add reload option to RefreshCallQueue
> -------------------------------------
>
>                 Key: HADOOP-17429
>                 URL: https://issues.apache.org/jira/browse/HADOOP-17429
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Janus Chow
>            Assignee: Janus Chow
>            Priority: Major
>         Attachments: HADOOP-17429.001.patch
>
>
> Currently the can use the dfsadmin command of "refreshCallQueue" to refresh all configs of FairCallQueue, but it will cause call queue spike during the refresh on NameNode.
> In  [HADOOP-17421|https://issues.apache.org/jira/projects/HADOOP/issues/HADOOP-17421?filter=allopenissues], we added some configurations to specify queues for some static users, with which we can have fine-grained control on the behavior of different users manually, this feature will require us to refresh call queue of NameNodes constantly, which will sure cause bad impacts to NameNode's performance.
> This ticket was to propose adding a reload option for the command of refreshCallQueue, which will only trigger the _callQueueManager_ to reload something, instead of reconstructing new scheduler and new queues.
> The basic design is :
>  # Add a method of "reload" in the interface of _Scheduler._
>  # Add different _RefreshCallQueueTypes_ to trigger different refresh operations.
>  # Add the argument in DFSAdmin for *reload* option.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org