You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Jing Zhao (JIRA)" <ji...@apache.org> on 2014/12/20 01:57:14 UTC
[jira] [Comment Edited] (HDFS-7056) Snapshot support for truncate
[ https://issues.apache.org/jira/browse/HDFS-7056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14254385#comment-14254385 ]
Jing Zhao edited comment on HDFS-7056 at 12/20/14 12:57 AM:
------------------------------------------------------------
Some comments so far. I will continue reviewing this weekend.
# I may have missed something here, but does the current patch persist {{FileDiff#blocks}} into FSImage? Not correctly persisting this information can cause data loss.
# We need to have tests for the above case. I suggest we include the {{FileDiff#blocks}} information into the output of
{{INodeFile#dumpTreeRecursively}}, and add tests similar to {{TestSnapshot#checkFSImage}} for every operation in TestFileTruncate.
# I'm still a little confused about the current searching process in {{findLaterSnapshotBlocks}}. The following code looks like an {{nlog\(n\)}}
process since {{getPrior}} has {{log\(n\)}} complexity. Why not just doing a linear scan in the diff list after the given snapshot id?
{code}
public BlockInfo[] findLaterSnapshotBlocks(int snapshotId) {
assert snapshotId != Snapshot.NO_SNAPSHOT_ID : "Wrong snapshot id";
if(snapshotId == Snapshot.CURRENT_STATE_ID)
return null;
int p = Snapshot.CURRENT_STATE_ID;
FileDiff diff = null;
while(snapshotId < p) {
p = getPrior(p, true);
if(p < snapshotId) break;
FileDiff cur = getDiffById(p);
if(cur.getBlocks() != null)
diff = cur;
}
return (diff == null) ? null : diff.getBlocks();
}
{code}
# Nit: I still see a lot of places missing "{" and "}" for "if"/"while"/"for" statements.
was (Author: jingzhao):
Some comments so far. I will continue reviewing this weekend.
# I may have missed something here, but does the current patch persist {{FileDiff#blocks}} into FSImage? Not correctly persisting this information can cause data loss.
# We need to have tests for the above case. I suggest we include the {{FileDiff#blocks}} information into the output of
{{INodeFile#dumpTreeRecursively}}, and add tests similar to {{TestSnapshot#checkFSImage}} for every operation in TestFileTruncate.
# I'm still a little confused about the current searching process in {{findLaterSnapshotBlocks}}. The following code looks like an nlog(n)
process since {{getPrior}} has log(n) complexity. Why not just doing a linear scan in the diff list after the given snapshot id?
{code}
public BlockInfo[] findLaterSnapshotBlocks(int snapshotId) {
assert snapshotId != Snapshot.NO_SNAPSHOT_ID : "Wrong snapshot id";
if(snapshotId == Snapshot.CURRENT_STATE_ID)
return null;
int p = Snapshot.CURRENT_STATE_ID;
FileDiff diff = null;
while(snapshotId < p) {
p = getPrior(p, true);
if(p < snapshotId) break;
FileDiff cur = getDiffById(p);
if(cur.getBlocks() != null)
diff = cur;
}
return (diff == null) ? null : diff.getBlocks();
}
{code}
# Nit: I still see a lot of places missing "{" and "}" for "if"/"while"/"for" statements.
> Snapshot support for truncate
> -----------------------------
>
> Key: HDFS-7056
> URL: https://issues.apache.org/jira/browse/HDFS-7056
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Konstantin Shvachko
> Assignee: Plamen Jeliazkov
> Attachments: HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFSSnapshotWithTruncateDesign.docx
>
>
> Implementation of truncate in HDFS-3107 does not allow truncating files which are in a snapshot. It is desirable to be able to truncate and still keep the old file state of the file in the snapshot.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)