You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Jing Zhao (JIRA)" <ji...@apache.org> on 2014/12/20 01:57:14 UTC

[jira] [Comment Edited] (HDFS-7056) Snapshot support for truncate

    [ https://issues.apache.org/jira/browse/HDFS-7056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14254385#comment-14254385 ] 

Jing Zhao edited comment on HDFS-7056 at 12/20/14 12:57 AM:
------------------------------------------------------------

Some comments so far. I will continue reviewing this weekend.
# I may have missed something here, but does the current patch persist {{FileDiff#blocks}} into FSImage? Not correctly persisting this information can cause data loss.
# We need to have tests for the above case. I suggest we include the {{FileDiff#blocks}} information into the output of
  {{INodeFile#dumpTreeRecursively}}, and add tests similar to {{TestSnapshot#checkFSImage}} for every operation in TestFileTruncate.
# I'm still a little confused about the current searching process in {{findLaterSnapshotBlocks}}. The following code looks like an {{nlog\(n\)}}
  process since {{getPrior}} has {{log\(n\)}} complexity. Why not just doing a linear scan in the diff list after the given snapshot id?
{code}
  public BlockInfo[] findLaterSnapshotBlocks(int snapshotId) {
    assert snapshotId != Snapshot.NO_SNAPSHOT_ID : "Wrong snapshot id";
    if(snapshotId == Snapshot.CURRENT_STATE_ID)
      return null;
    int p = Snapshot.CURRENT_STATE_ID;
    FileDiff diff = null;
    while(snapshotId < p) {
      p = getPrior(p, true);
      if(p < snapshotId) break;
      FileDiff cur = getDiffById(p);
      if(cur.getBlocks() != null)
        diff = cur;
    }
    return (diff == null) ? null : diff.getBlocks();
  }
{code}
# Nit: I still see a lot of places missing "{" and "}" for "if"/"while"/"for" statements.



was (Author: jingzhao):
Some comments so far. I will continue reviewing this weekend.
# I may have missed something here, but does the current patch persist {{FileDiff#blocks}} into FSImage? Not correctly persisting this information can cause data loss.
# We need to have tests for the above case. I suggest we include the {{FileDiff#blocks}} information into the output of
  {{INodeFile#dumpTreeRecursively}}, and add tests similar to {{TestSnapshot#checkFSImage}} for every operation in TestFileTruncate.
# I'm still a little confused about the current searching process in {{findLaterSnapshotBlocks}}. The following code looks like an nlog(n)
  process since {{getPrior}} has log(n) complexity. Why not just doing a linear scan in the diff list after the given snapshot id?
{code}
  public BlockInfo[] findLaterSnapshotBlocks(int snapshotId) {
    assert snapshotId != Snapshot.NO_SNAPSHOT_ID : "Wrong snapshot id";
    if(snapshotId == Snapshot.CURRENT_STATE_ID)
      return null;
    int p = Snapshot.CURRENT_STATE_ID;
    FileDiff diff = null;
    while(snapshotId < p) {
      p = getPrior(p, true);
      if(p < snapshotId) break;
      FileDiff cur = getDiffById(p);
      if(cur.getBlocks() != null)
        diff = cur;
    }
    return (diff == null) ? null : diff.getBlocks();
  }
{code}
# Nit: I still see a lot of places missing "{" and "}" for "if"/"while"/"for" statements.


> Snapshot support for truncate
> -----------------------------
>
>                 Key: HDFS-7056
>                 URL: https://issues.apache.org/jira/browse/HDFS-7056
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: 3.0.0
>            Reporter: Konstantin Shvachko
>            Assignee: Plamen Jeliazkov
>         Attachments: HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFSSnapshotWithTruncateDesign.docx
>
>
> Implementation of truncate in HDFS-3107 does not allow truncating files which are in a snapshot. It is desirable to be able to truncate and still keep the old file state of the file in the snapshot.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)