You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Chengwei Wang (Jira)" <ji...@apache.org> on 2021/10/27 09:55:00 UTC
[jira] [Commented] (HDFS-14815) RBF: Update the quota in MountTable
when calling setQuota on a MountTable src.
[ https://issues.apache.org/jira/browse/HDFS-14815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17434763#comment-17434763 ]
Chengwei Wang commented on HDFS-14815:
--------------------------------------
HI [~LiJinglun] [~ayushtkn], I think this feature may make users a little confused.
Users will get different result when they try to set quota for mount table paths and normal paths by dfsadmin CLI or FileSystem#setQuota interface with _*fs.defaultFS=router*_. especially when call FileSystem#setQuota inside users code. In other word, the _*ClientProtocol#setQuota*_ interface implement inside router is just partial available, this seems illogical and would make users doubt whether the hdfs cluster is healthy.
In my opnions, _*ClientProtocol#setQuota as a public api,*_ the** _**_ requests should be all passed or all rejected to keep behavior consistent. So, I prefer approach 2 and 3, and we can choose consistent strategy (*Strong* or *Eventually*) by configuration.
> RBF: Update the quota in MountTable when calling setQuota on a MountTable src.
> ------------------------------------------------------------------------------
>
> Key: HDFS-14815
> URL: https://issues.apache.org/jira/browse/HDFS-14815
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Jinglun
> Assignee: Jinglun
> Priority: Major
> Fix For: 3.3.0
>
> Attachments: HDFS-14815.001.patch, HDFS-14815.002.patch, HDFS-14815.003.patch, HDFS-14815.004.patch, HDFS-14815.005.patch
>
>
> The method setQuota() can make the remote quota(the quota on real clusters) inconsistent with the MountTable. I think we have 3 ways to fix it:
> # Reject all the setQuota() rpcs if it trys to change the quota of a mount table.
> # Let setQuota() to change the mount table quota. Update the quota on zk first and then update remote quotas.
> # Do nothing. The RouterQuotaUpdateService will finally make all the remote quota right. We can tolerate short-term inconsistencies.
> I'd like option 1 because I want the RouterAdmin to be the only entrance to update the MountTable.
> Option 3 we don't need change anything, but the quota will be inconsistent for a short-term. The remote quota will be effective immediately and auto-changed back after a while. User might be confused about the behavior.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org