You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "zhengchenyu (Jira)" <ji...@apache.org> on 2021/02/22 12:10:00 UTC
[jira] [Updated] (HDFS-15750) RBF: Make sure the multi destination
are consistent after write operation
[ https://issues.apache.org/jira/browse/HDFS-15750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhengchenyu updated HDFS-15750:
-------------------------------
Attachment: HDFS-15750.001.patch
> RBF: Make sure the multi destination are consistent after write operation
> --------------------------------------------------------------------------
>
> Key: HDFS-15750
> URL: https://issues.apache.org/jira/browse/HDFS-15750
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: zhengchenyu
> Assignee: zhengchenyu
> Priority: Major
> Attachments: HDFS-15750.001.patch
>
>
> Nowdays, RBF can't make sure the multi destination are consistent.
> Case 1: RBF can't remove multi destination's file.
> If /user/userA is mountable which mounts two nameservice: ns1, ns2. But if both hdfs://ns1/user/userA/a.log and hdfs://ns2/user/userA/a.log exists. I want to remove hdfs://ns-fed/user/userA/a.log (Note: a.log is file) to trash, then only one nameservice take effect. I think it means inconsistence. Through HDFS-14343 already solve the problem in some level, but not completed.
> Case 2: RBF regard the operation success, through only one of multi destination operations success.
> In other way, if we wanna delete hdfs://ns-fed/user/userA/dirA (Note: dirA is directroy.) If hdfs://ns1/user/userA/dirA's permission is not same with hdfs://ns2/user/userA/dirA's permission, or one of namenode is down.If one destination's result be success, the rbf regard the operation success (invokeConcurrent and invokeAll's logical). We maybe can't rename all location. I think it also means inconsistence.
> I think we need stricter check. If one operation (which shoud success) failed , we should throw exception.
> Note: In fact,If we only use hdfs://ns-fed, I think there is no problem.But when migration data from one ns1 to ns2 (a mountable mouts ns1 and ns2) , and rewrite some hive table's old partition (which mount multi destination), this problem would occure!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org