You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Shilun Fan (Jira)" <ji...@apache.org> on 2024/01/28 02:13:00 UTC

[jira] [Updated] (HDFS-17045) File renamed from a snapshottable dir to a non-snapshottable dir cannot be deleted.

     [ https://issues.apache.org/jira/browse/HDFS-17045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shilun Fan updated HDFS-17045:
------------------------------
     Target Version/s: 3.4.0
    Affects Version/s: 3.4.0

> File renamed from a snapshottable dir to a non-snapshottable dir cannot be deleted.
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-17045
>                 URL: https://issues.apache.org/jira/browse/HDFS-17045
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode, snapshots
>    Affects Versions: 3.4.0
>            Reporter: Tsz-wo Sze
>            Assignee: Tsz-wo Sze
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.4.0
>
>
> HDFS-16972 added a [shouldDestroy|https://github.com/szetszwo/hadoop/blob/331e075115b4a35574622318b26f6d4731658d57/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeReference.java#L834-L845] method which caused the following bug.
> h3. Background:
>  - When {{FileSystem.rename(src, dst)}} from a snapshottable dir (src) to a snapshottable dir (dst), dstSnapshotId is set to the latest snapshot at dst. As a result, dst is NOT in dstSnapshotId because dstSnapshotId was already taken before rename.
>  - snapshotToBeDeleted is the snapshot id of the current operation if the operation is {{{}FileSystem.deleteSnapshot{}}}. Otherwise, snapshotToBeDeleted is set to CURRENT_STATE_ID.
>  - If (snapshotToBeDeleted > dstSnapshotId), dst is in snapshotToBeDeleted. The shouldDestroy method returns true to continue deletion.
>  - If (snapshotToBeDeleted <= dstSnapshotId), dst must not be in snapshotToBeDeleted. The shouldDestroy method returns false to stop deletion.
> All the above are correct for renaming within snapshottable directories.
> h3. Bug:
>  - If rename(src, dst) from a snapshottable dir (src) to a non-snapshottable dir (dst), dstSnapshotId becomes CURRENT_STATE_ID.
>  - When {{FileSystem.delete(dst)}} happens, snapshotToBeDeleted is also set to CURRENT_STATE_ID.
>  - In this case, snapshotToBeDeleted == dstSnapshotId, the shouldDestroy method will return false and it incorrectly stops the deletion.
> Not that this bug may cause fsimage corruption and quota miscalculation since some files can be partially deleted.  Fortunately, this bug won't cause data loss.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org