You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "zhengchenyu (Jira)" <ji...@apache.org> on 2020/12/22 08:52:00 UTC
[jira] [Commented] (HDFS-14343) RBF: Fix renaming folders spread
across multiple subclusters
[ https://issues.apache.org/jira/browse/HDFS-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253343#comment-17253343 ]
zhengchenyu commented on HDFS-14343:
------------------------------------
[~inigoiri][~ayushtkn].
Hi, I have some question, can you give me some suggestion. In this patch, use isMultiDestDirectory to check whether rename all location or not. I think the design of MultipleDestinationMountTableResolver may consider there is no repeated file among multi nameservice.
If /user/userA is mountable which mounts two nameservice: ns1, ns2. But if both hdfs://ns1/user/userA/a.log and hdfs://ns2/user/userA/a.log exists. I want to remove hdfs://ns-fed/user/userA/a.log (Note: a.log is file) to trash, then only one nameservice take effect. I think it means inconsistence.
(Note: In fact, when migration data from one nameservice to nameservice, and rewrite some hive table's old partition, this problem would occure!)
In other way, if we hdfs://ns-fed/user/userA/dirA (Note: dirA is directroy.) If hdfs://ns1/user/userA/dirA's permission is not same with hdfs://ns2/user/userA/dirA's permission. Or one of nameservice is down. We maybe can't rename all location. I think it also means inconsistence. I think we need stricter check. If one operation error, we should throw exception.
> RBF: Fix renaming folders spread across multiple subclusters
> ------------------------------------------------------------
>
> Key: HDFS-14343
> URL: https://issues.apache.org/jira/browse/HDFS-14343
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Íñigo Goiri
> Assignee: Ayush Saxena
> Priority: Major
> Fix For: 3.3.0, HDFS-13891
>
> Attachments: HDFS-14343-HDFS-13891-01.patch, HDFS-14343-HDFS-13891-02.patch, HDFS-14343-HDFS-13891-03.patch, HDFS-14343-HDFS-13891-04.patch, HDFS-14343-HDFS-13891-05.patch
>
>
> The {{RouterClientProtocol#rename()}} function assumes that we are renaming files and only renames one of them (i.e., {{invokeSequential()}}). In the case of folders which are in all subclusters (e.g., HASH_ALL) we should rename all locations (i.e., {{invokeAll()}}).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org