You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by panfei <cn...@gmail.com> on 2017/08/04 03:38:00 UTC
Fwd: How to restore data from HDFS rm -skipTrash
---------- Forwarded message ----------
From: panfei <cn...@gmail.com>
Date: 2017-08-04 11:23 GMT+08:00
Subject: How to restore data from HDFS rm -skipTrash
To: CDH Users <cd...@cloudera.org>
some one mistakenly do a rm -skipTrash operation on the HDFS, but we stop
the namenode and datanodes immediately. (CDH 5.4.5)
I want to know is there any way to stop the deletion process ?
and how ?
thanks very in advance.
Re: Fwd: How to restore data from HDFS rm -skipTrash
Posted by Margus Roo <ma...@roo.ee>.
Hi
Basically it deletes metadata. If you have backup from metadata you
maybe can restore.
Br, Margus
Margus (margusja) Roo
http://margus.roo.ee
skype: margusja
https://www.facebook.com/allan.tuuring
+372 51 48 780
On 04/08/2017 06:38, panfei wrote:
> I want to know is there any way to stop the deletion process ?
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
RE: How to restore data from HDFS rm -skipTrash
Posted by Brahma Reddy Battula <br...@huawei.com>.
You can refer the following link also.
https://community.hortonworks.com/articles/26181/how-to-recover-accidentally-deleted-file-in-hdfs.html
--Brahma Reddy Battula
-----Original Message-----
From: Konstantin Shvachko [mailto:shv.hadoop@gmail.com]
Sent: 05 August 2017 02:30
To: panfei
Cc: hdfs-dev@hadoop.apache.org
Subject: Re: How to restore data from HDFS rm -skipTrash
If you stopped the cluster immediately you may have a chance to restore most of the data.
It's a manual and hacky process.
You can use the Offline Edits Viewer to see the latest edits and cut off the delete transactions that was issues by mistake.
Then you can restart the namenode with the modifies edits log.
Hope this helps.
Thanks,
--Konstantin
On Thu, Aug 3, 2017 at 8:38 PM, panfei <cn...@gmail.com> wrote:
> ---------- Forwarded message ----------
> From: panfei <cn...@gmail.com>
> Date: 2017-08-04 11:23 GMT+08:00
> Subject: How to restore data from HDFS rm -skipTrash
> To: CDH Users <cd...@cloudera.org>
>
>
> some one mistakenly do a rm -skipTrash operation on the HDFS, but we
> stop the namenode and datanodes immediately. (CDH 5.4.5)
>
> I want to know is there any way to stop the deletion process ?
>
> and how ?
>
> thanks very in advance.
>
Re: How to restore data from HDFS rm -skipTrash
Posted by Konstantin Shvachko <sh...@gmail.com>.
If you stopped the cluster immediately you may have a chance to restore
most of the data.
It's a manual and hacky process.
You can use the Offline Edits Viewer to see the latest edits and cut off
the delete transactions that was issues by mistake.
Then you can restart the namenode with the modifies edits log.
Hope this helps.
Thanks,
--Konstantin
On Thu, Aug 3, 2017 at 8:38 PM, panfei <cn...@gmail.com> wrote:
> ---------- Forwarded message ----------
> From: panfei <cn...@gmail.com>
> Date: 2017-08-04 11:23 GMT+08:00
> Subject: How to restore data from HDFS rm -skipTrash
> To: CDH Users <cd...@cloudera.org>
>
>
> some one mistakenly do a rm -skipTrash operation on the HDFS, but we stop
> the namenode and datanodes immediately. (CDH 5.4.5)
>
> I want to know is there any way to stop the deletion process ?
>
> and how ?
>
> thanks very in advance.
>
Re: How to restore data from HDFS rm -skipTrash
Posted by Eric Payne <er...@yahoo.com.INVALID>.
Panfei
> we stop the namenode and datanodes
This is also really hacky, but if all else fails...
It still may be too late, but if you are only running one datanode, you could look at your hdfs-site.xml, find the property named "dfs.data.dir", and go to that directory. Look around under there and see if the blocks still contain your data. Depending on how big your data was and how much other data you have in the filesystem, you may be able to piece your deleted data together.
: Eric Payne
From: Wei-Chiu Chuang <we...@apache.org>
To: panfei <cn...@gmail.com>
Cc: Hdfs-dev <hd...@hadoop.apache.org>
Sent: Friday, August 4, 2017 7:57 AM
Subject: Re: How to restore data from HDFS rm -skipTrash
If the directory has snapshot enabled, the file can be retrieved from the
past snapshots.
Otherwise, the file inodes are removed from namenode metadata, and blocks
are scheduled for deletion.
You might want to play with edit log a bit. Remove the delete entries from
edit logs. But it's hacky and does not guarantee the blocks are still there.
On Thu, Aug 3, 2017 at 8:38 PM, panfei <cn...@gmail.com> wrote:
> ---------- Forwarded message ----------
> From: panfei <cn...@gmail.com>
> Date: 2017-08-04 11:23 GMT+08:00
> Subject: How to restore data from HDFS rm -skipTrash
> To: CDH Users <cd...@cloudera.org>
>
>
> some one mistakenly do a rm -skipTrash operation on the HDFS, but we stop
> the namenode and datanodes immediately. (CDH 5.4.5)
>
> I want to know is there any way to stop the deletion process ?
>
> and how ?
>
> thanks very in advance.
>
--
A very happy Hadoop contributor
Re: How to restore data from HDFS rm -skipTrash
Posted by Wei-Chiu Chuang <we...@apache.org>.
If the directory has snapshot enabled, the file can be retrieved from the
past snapshots.
Otherwise, the file inodes are removed from namenode metadata, and blocks
are scheduled for deletion.
You might want to play with edit log a bit. Remove the delete entries from
edit logs. But it's hacky and does not guarantee the blocks are still there.
On Thu, Aug 3, 2017 at 8:38 PM, panfei <cn...@gmail.com> wrote:
> ---------- Forwarded message ----------
> From: panfei <cn...@gmail.com>
> Date: 2017-08-04 11:23 GMT+08:00
> Subject: How to restore data from HDFS rm -skipTrash
> To: CDH Users <cd...@cloudera.org>
>
>
> some one mistakenly do a rm -skipTrash operation on the HDFS, but we stop
> the namenode and datanodes immediately. (CDH 5.4.5)
>
> I want to know is there any way to stop the deletion process ?
>
> and how ?
>
> thanks very in advance.
>
--
A very happy Hadoop contributor