You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by juil cho <tj...@kgrid.co.kr> on 2014/08/20 08:22:46 UTC
Why not remove the block on the disk if the snapshot?
hadoop version 2.4.1.
I have tested the snapshot.
1. upload file, /tmp/test.avi
2. create snapshot, /tmp snap1
3. delete file, /tmp/test.avi
4. moved to the Trash, /user/hadoop/.Trash/tmp/test.avi
5. file deleted in trash.
6. but, blocks is not deleted in datanode disk. why??
7. delete snapshot(snap1)
8. blocks is deleted in datanode disk. why???
I do not understand. Why not remove the block on the disk if the snapshot?
RE: Why not remove the block on the disk if the snapshot?
Posted by "Liu, Yi A" <yi...@intel.com>.
Yes, that’s correct behavior.
You remove the file, but snapshot is there and it has ref for that file, so the blocks will not be removed. Only after you delete all snapshots and original file, then blocks are removed.
Keep in mind that blocks in datanodes are not copied for snapshot.
Regards,
Yi Liu
From: juil cho [mailto:tjstory@kgrid.co.kr]
Sent: Wednesday, August 20, 2014 2:23 PM
To: user@hadoop.apache.org
Subject: Why not remove the block on the disk if the snapshot?
hadoop version 2.4.1.
I have tested the snapshot.
1. upload file, /tmp/test.avi
2. create snapshot, /tmp snap1
3. delete file, /tmp/test.avi
4. moved to the Trash, /user/hadoop/.Trash/tmp/test.avi
5. file deleted in trash.
6. but, blocks is not deleted in datanode disk. why??
7. delete snapshot(snap1)
8. blocks is deleted in datanode disk. why???
I do not understand. Why not remove the block on the disk if the snapshot?
[http://mail.naver.com/readReceipt/notify/?img=AlFZ1zFdMrYsbrJR%2BHFGhziCFqvZpo%2B0FrKdMrulFxudK4u%2FKoK9Fq2%2FMrMqp6UZtzFXp6UwFZl5WLl51zlqDBFdp6d5MreRhoR9brkZtzCCpzeTbVlCbzJo1zE5WXiN.gif]
Re: Why not remove the block on the disk if the snapshot?
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
The blocks of a file and the blocks of the snapshot are the same.
(i.e. There is no data copying when creating snapshots)
Therefore the blocks are not removed from datanode disks if a file is
removed.
Thanks,
Akira
(2014/08/20 15:22), juil cho wrote:
> hadoop version 2.4.1.
>
> I have tested the snapshot.
>
> 1. upload file, /tmp/test.avi
>
> 2. create snapshot, /tmp snap1
>
> 3. delete file, /tmp/test.avi
>
> 4. moved to the Trash, /user/hadoop/.Trash/tmp/test.avi
>
> 5. file deleted in trash.
>
> 6. *but, blocks is not deleted in datanode disk. why??*
>
> 7. delete snapshot(snap1)
>
> 8. blocks is deleted in datanode disk. why???
>
> I do not understand. Why not remove the block on the disk if the snapshot?
>
RE: Why not remove the block on the disk if the snapshot?
Posted by "Liu, Yi A" <yi...@intel.com>.
Yes, that’s correct behavior.
You remove the file, but snapshot is there and it has ref for that file, so the blocks will not be removed. Only after you delete all snapshots and original file, then blocks are removed.
Keep in mind that blocks in datanodes are not copied for snapshot.
Regards,
Yi Liu
From: juil cho [mailto:tjstory@kgrid.co.kr]
Sent: Wednesday, August 20, 2014 2:23 PM
To: user@hadoop.apache.org
Subject: Why not remove the block on the disk if the snapshot?
hadoop version 2.4.1.
I have tested the snapshot.
1. upload file, /tmp/test.avi
2. create snapshot, /tmp snap1
3. delete file, /tmp/test.avi
4. moved to the Trash, /user/hadoop/.Trash/tmp/test.avi
5. file deleted in trash.
6. but, blocks is not deleted in datanode disk. why??
7. delete snapshot(snap1)
8. blocks is deleted in datanode disk. why???
I do not understand. Why not remove the block on the disk if the snapshot?
[http://mail.naver.com/readReceipt/notify/?img=AlFZ1zFdMrYsbrJR%2BHFGhziCFqvZpo%2B0FrKdMrulFxudK4u%2FKoK9Fq2%2FMrMqp6UZtzFXp6UwFZl5WLl51zlqDBFdp6d5MreRhoR9brkZtzCCpzeTbVlCbzJo1zE5WXiN.gif]
RE: Why not remove the block on the disk if the snapshot?
Posted by "Liu, Yi A" <yi...@intel.com>.
Yes, that’s correct behavior.
You remove the file, but snapshot is there and it has ref for that file, so the blocks will not be removed. Only after you delete all snapshots and original file, then blocks are removed.
Keep in mind that blocks in datanodes are not copied for snapshot.
Regards,
Yi Liu
From: juil cho [mailto:tjstory@kgrid.co.kr]
Sent: Wednesday, August 20, 2014 2:23 PM
To: user@hadoop.apache.org
Subject: Why not remove the block on the disk if the snapshot?
hadoop version 2.4.1.
I have tested the snapshot.
1. upload file, /tmp/test.avi
2. create snapshot, /tmp snap1
3. delete file, /tmp/test.avi
4. moved to the Trash, /user/hadoop/.Trash/tmp/test.avi
5. file deleted in trash.
6. but, blocks is not deleted in datanode disk. why??
7. delete snapshot(snap1)
8. blocks is deleted in datanode disk. why???
I do not understand. Why not remove the block on the disk if the snapshot?
[http://mail.naver.com/readReceipt/notify/?img=AlFZ1zFdMrYsbrJR%2BHFGhziCFqvZpo%2B0FrKdMrulFxudK4u%2FKoK9Fq2%2FMrMqp6UZtzFXp6UwFZl5WLl51zlqDBFdp6d5MreRhoR9brkZtzCCpzeTbVlCbzJo1zE5WXiN.gif]
RE: Why not remove the block on the disk if the snapshot?
Posted by "Liu, Yi A" <yi...@intel.com>.
Yes, that’s correct behavior.
You remove the file, but snapshot is there and it has ref for that file, so the blocks will not be removed. Only after you delete all snapshots and original file, then blocks are removed.
Keep in mind that blocks in datanodes are not copied for snapshot.
Regards,
Yi Liu
From: juil cho [mailto:tjstory@kgrid.co.kr]
Sent: Wednesday, August 20, 2014 2:23 PM
To: user@hadoop.apache.org
Subject: Why not remove the block on the disk if the snapshot?
hadoop version 2.4.1.
I have tested the snapshot.
1. upload file, /tmp/test.avi
2. create snapshot, /tmp snap1
3. delete file, /tmp/test.avi
4. moved to the Trash, /user/hadoop/.Trash/tmp/test.avi
5. file deleted in trash.
6. but, blocks is not deleted in datanode disk. why??
7. delete snapshot(snap1)
8. blocks is deleted in datanode disk. why???
I do not understand. Why not remove the block on the disk if the snapshot?
[http://mail.naver.com/readReceipt/notify/?img=AlFZ1zFdMrYsbrJR%2BHFGhziCFqvZpo%2B0FrKdMrulFxudK4u%2FKoK9Fq2%2FMrMqp6UZtzFXp6UwFZl5WLl51zlqDBFdp6d5MreRhoR9brkZtzCCpzeTbVlCbzJo1zE5WXiN.gif]
Re: Why not remove the block on the disk if the snapshot?
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
The blocks of a file and the blocks of the snapshot are the same.
(i.e. There is no data copying when creating snapshots)
Therefore the blocks are not removed from datanode disks if a file is
removed.
Thanks,
Akira
(2014/08/20 15:22), juil cho wrote:
> hadoop version 2.4.1.
>
> I have tested the snapshot.
>
> 1. upload file, /tmp/test.avi
>
> 2. create snapshot, /tmp snap1
>
> 3. delete file, /tmp/test.avi
>
> 4. moved to the Trash, /user/hadoop/.Trash/tmp/test.avi
>
> 5. file deleted in trash.
>
> 6. *but, blocks is not deleted in datanode disk. why??*
>
> 7. delete snapshot(snap1)
>
> 8. blocks is deleted in datanode disk. why???
>
> I do not understand. Why not remove the block on the disk if the snapshot?
>
Re: Why not remove the block on the disk if the snapshot?
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
The blocks of a file and the blocks of the snapshot are the same.
(i.e. There is no data copying when creating snapshots)
Therefore the blocks are not removed from datanode disks if a file is
removed.
Thanks,
Akira
(2014/08/20 15:22), juil cho wrote:
> hadoop version 2.4.1.
>
> I have tested the snapshot.
>
> 1. upload file, /tmp/test.avi
>
> 2. create snapshot, /tmp snap1
>
> 3. delete file, /tmp/test.avi
>
> 4. moved to the Trash, /user/hadoop/.Trash/tmp/test.avi
>
> 5. file deleted in trash.
>
> 6. *but, blocks is not deleted in datanode disk. why??*
>
> 7. delete snapshot(snap1)
>
> 8. blocks is deleted in datanode disk. why???
>
> I do not understand. Why not remove the block on the disk if the snapshot?
>
Re: Why not remove the block on the disk if the snapshot?
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
The blocks of a file and the blocks of the snapshot are the same.
(i.e. There is no data copying when creating snapshots)
Therefore the blocks are not removed from datanode disks if a file is
removed.
Thanks,
Akira
(2014/08/20 15:22), juil cho wrote:
> hadoop version 2.4.1.
>
> I have tested the snapshot.
>
> 1. upload file, /tmp/test.avi
>
> 2. create snapshot, /tmp snap1
>
> 3. delete file, /tmp/test.avi
>
> 4. moved to the Trash, /user/hadoop/.Trash/tmp/test.avi
>
> 5. file deleted in trash.
>
> 6. *but, blocks is not deleted in datanode disk. why??*
>
> 7. delete snapshot(snap1)
>
> 8. blocks is deleted in datanode disk. why???
>
> I do not understand. Why not remove the block on the disk if the snapshot?
>