[jira] [Work logged] (HDFS-16646) [RBF] Improved isolation for downstream name nodes. {Elastic}

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-16646?focusedWorklogId=786515&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-786515 ]

ASF GitHub Bot logged work on HDFS-16646:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 30/Jun/22 11:05
            Start Date: 30/Jun/22 11:05
    Worklog Time Spent: 10m 
      Work Description: ZanderXu opened a new pull request, #4519:
URL: https://github.com/apache/hadoop/pull/4519

   ### Description of PR
   As we all known, `StaticRouterRpcFairnessPolicyController` is very helpfully for RBF to minimize impact of clients connecting to healthy vs unhealthy nameNodes. 
   But in prod environment, the traffic of clients accessing each NS and the pressure of downstream namenodes are dynamically changed. So if we only have one static permit conf, RBF cannot able to adapt to the changes in traffic to achieve optimal results. 
   
   So here I propose an elastic RouterRpcFairnessPolicyController to help RBF adapt to traffic changes to achieve an optimal result.
   
   The overall idea is:
   - Each name service can configured the exclusive permits like `StaticRouterRpcFairnessPolicyController`
   - TotalPermits is more than sum(NsExclusivePermit) and mark TotalPermits - sum(NsExclusivePermit) as SharedPermits
   - Each name service can properly preempt the SharedPermits after it's own exclusive permits is used up.
   - But the maximum value of SharedPermits preempted by each nameservice should be limited. Such as 20% of SharedPermits.
   
   Suppose we have 200 handlers and 5 name services, and each name services configured different exclusive Permits, like:
   | NS1 | NS2 | NS3 | NS4 | NS5 | Concurrent NS |
   |

Issue Time Tracking
-------------------

            Worklog Id:     (was: 786515)
    Remaining Estimate: 0h
            Time Spent: 10m

> [RBF] Improved isolation for downstream name nodes. {Elastic}
> -------------------------------------------------------------
>
>                 Key: HDFS-16646
>                 URL: https://issues.apache.org/jira/browse/HDFS-16646
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: ZanderXu
>            Assignee: ZanderXu
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org