You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Manoj Samel <ma...@gmail.com> on 2014/04/14 23:14:39 UTC
Applying delta between two HDFS snapshot using hdfs command
Hi,
It seems the only restore from a HDFS snapshot using hdfs command line is
copy snapshot files to a target path.
If the use cases are
0. .... stuff ...
1. Take snapshot s_N
2. Add some files, delete other files
3. Take snapshot s_N+1
then copying s_N+1 to target just copies the newly added file to target, it
does not propagates the deletes between s_N and s_N+1.
How does one apply the full delta between s_N and s_N+1 to target (and not
just additions) ? Is Using Java API the only option or is it possible to
use command line to do it ?
Thanks,
Re: Applying delta between two HDFS snapshot using hdfs command
Posted by Manoj Samel <ma...@gmail.com>.
Thanks Jing,
The Jira is open since Nov 12 but seems a design doc was added just few
days back ...
Would you have any ETA on this ?
Thanks again !
Manoj
On Mon, Apr 14, 2014 at 2:47 PM, Jing Zhao <ji...@hortonworks.com> wrote:
> Hi Manoj,
>
> You're right, right now we do not have a complete snapshot
> rollback/restore functionality in HDFS. Thus users have to manually
> copy/delete files according to the snapshot diff report. There's an open
> jira HDFS-4167 for it. We plan to provide this support soon.
>
> Thanks,
> -Jing
>
>
> On Mon, Apr 14, 2014 at 2:14 PM, Manoj Samel <ma...@gmail.com>wrote:
>
>> Hi,
>>
>> It seems the only restore from a HDFS snapshot using hdfs command line is
>> copy snapshot files to a target path.
>>
>> If the use cases are
>>
>>
>> 0. .... stuff ...
>>
>> 1. Take snapshot s_N
>>
>> 2. Add some files, delete other files
>>
>> 3. Take snapshot s_N+1
>>
>>
>> then copying s_N+1 to target just copies the newly added file to target,
>> it does not propagates the deletes between s_N and s_N+1.
>>
>> How does one apply the full delta between s_N and s_N+1 to target (and
>> not just additions) ? Is Using Java API the only option or is it possible
>> to use command line to do it ?
>>
>> Thanks,
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Applying delta between two HDFS snapshot using hdfs command
Posted by Manoj Samel <ma...@gmail.com>.
Thanks Jing,
The Jira is open since Nov 12 but seems a design doc was added just few
days back ...
Would you have any ETA on this ?
Thanks again !
Manoj
On Mon, Apr 14, 2014 at 2:47 PM, Jing Zhao <ji...@hortonworks.com> wrote:
> Hi Manoj,
>
> You're right, right now we do not have a complete snapshot
> rollback/restore functionality in HDFS. Thus users have to manually
> copy/delete files according to the snapshot diff report. There's an open
> jira HDFS-4167 for it. We plan to provide this support soon.
>
> Thanks,
> -Jing
>
>
> On Mon, Apr 14, 2014 at 2:14 PM, Manoj Samel <ma...@gmail.com>wrote:
>
>> Hi,
>>
>> It seems the only restore from a HDFS snapshot using hdfs command line is
>> copy snapshot files to a target path.
>>
>> If the use cases are
>>
>>
>> 0. .... stuff ...
>>
>> 1. Take snapshot s_N
>>
>> 2. Add some files, delete other files
>>
>> 3. Take snapshot s_N+1
>>
>>
>> then copying s_N+1 to target just copies the newly added file to target,
>> it does not propagates the deletes between s_N and s_N+1.
>>
>> How does one apply the full delta between s_N and s_N+1 to target (and
>> not just additions) ? Is Using Java API the only option or is it possible
>> to use command line to do it ?
>>
>> Thanks,
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Applying delta between two HDFS snapshot using hdfs command
Posted by Manoj Samel <ma...@gmail.com>.
Thanks Jing,
The Jira is open since Nov 12 but seems a design doc was added just few
days back ...
Would you have any ETA on this ?
Thanks again !
Manoj
On Mon, Apr 14, 2014 at 2:47 PM, Jing Zhao <ji...@hortonworks.com> wrote:
> Hi Manoj,
>
> You're right, right now we do not have a complete snapshot
> rollback/restore functionality in HDFS. Thus users have to manually
> copy/delete files according to the snapshot diff report. There's an open
> jira HDFS-4167 for it. We plan to provide this support soon.
>
> Thanks,
> -Jing
>
>
> On Mon, Apr 14, 2014 at 2:14 PM, Manoj Samel <ma...@gmail.com>wrote:
>
>> Hi,
>>
>> It seems the only restore from a HDFS snapshot using hdfs command line is
>> copy snapshot files to a target path.
>>
>> If the use cases are
>>
>>
>> 0. .... stuff ...
>>
>> 1. Take snapshot s_N
>>
>> 2. Add some files, delete other files
>>
>> 3. Take snapshot s_N+1
>>
>>
>> then copying s_N+1 to target just copies the newly added file to target,
>> it does not propagates the deletes between s_N and s_N+1.
>>
>> How does one apply the full delta between s_N and s_N+1 to target (and
>> not just additions) ? Is Using Java API the only option or is it possible
>> to use command line to do it ?
>>
>> Thanks,
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Applying delta between two HDFS snapshot using hdfs command
Posted by Manoj Samel <ma...@gmail.com>.
Thanks Jing,
The Jira is open since Nov 12 but seems a design doc was added just few
days back ...
Would you have any ETA on this ?
Thanks again !
Manoj
On Mon, Apr 14, 2014 at 2:47 PM, Jing Zhao <ji...@hortonworks.com> wrote:
> Hi Manoj,
>
> You're right, right now we do not have a complete snapshot
> rollback/restore functionality in HDFS. Thus users have to manually
> copy/delete files according to the snapshot diff report. There's an open
> jira HDFS-4167 for it. We plan to provide this support soon.
>
> Thanks,
> -Jing
>
>
> On Mon, Apr 14, 2014 at 2:14 PM, Manoj Samel <ma...@gmail.com>wrote:
>
>> Hi,
>>
>> It seems the only restore from a HDFS snapshot using hdfs command line is
>> copy snapshot files to a target path.
>>
>> If the use cases are
>>
>>
>> 0. .... stuff ...
>>
>> 1. Take snapshot s_N
>>
>> 2. Add some files, delete other files
>>
>> 3. Take snapshot s_N+1
>>
>>
>> then copying s_N+1 to target just copies the newly added file to target,
>> it does not propagates the deletes between s_N and s_N+1.
>>
>> How does one apply the full delta between s_N and s_N+1 to target (and
>> not just additions) ? Is Using Java API the only option or is it possible
>> to use command line to do it ?
>>
>> Thanks,
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Applying delta between two HDFS snapshot using hdfs command
Posted by Jing Zhao <ji...@hortonworks.com>.
Hi Manoj,
You're right, right now we do not have a complete snapshot
rollback/restore functionality in HDFS. Thus users have to manually
copy/delete files according to the snapshot diff report. There's an open
jira HDFS-4167 for it. We plan to provide this support soon.
Thanks,
-Jing
On Mon, Apr 14, 2014 at 2:14 PM, Manoj Samel <ma...@gmail.com>wrote:
> Hi,
>
> It seems the only restore from a HDFS snapshot using hdfs command line is
> copy snapshot files to a target path.
>
> If the use cases are
>
>
> 0. .... stuff ...
>
> 1. Take snapshot s_N
>
> 2. Add some files, delete other files
>
> 3. Take snapshot s_N+1
>
>
> then copying s_N+1 to target just copies the newly added file to target,
> it does not propagates the deletes between s_N and s_N+1.
>
> How does one apply the full delta between s_N and s_N+1 to target (and not
> just additions) ? Is Using Java API the only option or is it possible to
> use command line to do it ?
>
> Thanks,
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Applying delta between two HDFS snapshot using hdfs command
Posted by Jing Zhao <ji...@hortonworks.com>.
Hi Manoj,
You're right, right now we do not have a complete snapshot
rollback/restore functionality in HDFS. Thus users have to manually
copy/delete files according to the snapshot diff report. There's an open
jira HDFS-4167 for it. We plan to provide this support soon.
Thanks,
-Jing
On Mon, Apr 14, 2014 at 2:14 PM, Manoj Samel <ma...@gmail.com>wrote:
> Hi,
>
> It seems the only restore from a HDFS snapshot using hdfs command line is
> copy snapshot files to a target path.
>
> If the use cases are
>
>
> 0. .... stuff ...
>
> 1. Take snapshot s_N
>
> 2. Add some files, delete other files
>
> 3. Take snapshot s_N+1
>
>
> then copying s_N+1 to target just copies the newly added file to target,
> it does not propagates the deletes between s_N and s_N+1.
>
> How does one apply the full delta between s_N and s_N+1 to target (and not
> just additions) ? Is Using Java API the only option or is it possible to
> use command line to do it ?
>
> Thanks,
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Applying delta between two HDFS snapshot using hdfs command
Posted by Jing Zhao <ji...@hortonworks.com>.
Hi Manoj,
You're right, right now we do not have a complete snapshot
rollback/restore functionality in HDFS. Thus users have to manually
copy/delete files according to the snapshot diff report. There's an open
jira HDFS-4167 for it. We plan to provide this support soon.
Thanks,
-Jing
On Mon, Apr 14, 2014 at 2:14 PM, Manoj Samel <ma...@gmail.com>wrote:
> Hi,
>
> It seems the only restore from a HDFS snapshot using hdfs command line is
> copy snapshot files to a target path.
>
> If the use cases are
>
>
> 0. .... stuff ...
>
> 1. Take snapshot s_N
>
> 2. Add some files, delete other files
>
> 3. Take snapshot s_N+1
>
>
> then copying s_N+1 to target just copies the newly added file to target,
> it does not propagates the deletes between s_N and s_N+1.
>
> How does one apply the full delta between s_N and s_N+1 to target (and not
> just additions) ? Is Using Java API the only option or is it possible to
> use command line to do it ?
>
> Thanks,
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Applying delta between two HDFS snapshot using hdfs command
Posted by Jing Zhao <ji...@hortonworks.com>.
Hi Manoj,
You're right, right now we do not have a complete snapshot
rollback/restore functionality in HDFS. Thus users have to manually
copy/delete files according to the snapshot diff report. There's an open
jira HDFS-4167 for it. We plan to provide this support soon.
Thanks,
-Jing
On Mon, Apr 14, 2014 at 2:14 PM, Manoj Samel <ma...@gmail.com>wrote:
> Hi,
>
> It seems the only restore from a HDFS snapshot using hdfs command line is
> copy snapshot files to a target path.
>
> If the use cases are
>
>
> 0. .... stuff ...
>
> 1. Take snapshot s_N
>
> 2. Add some files, delete other files
>
> 3. Take snapshot s_N+1
>
>
> then copying s_N+1 to target just copies the newly added file to target,
> it does not propagates the deletes between s_N and s_N+1.
>
> How does one apply the full delta between s_N and s_N+1 to target (and not
> just additions) ? Is Using Java API the only option or is it possible to
> use command line to do it ?
>
> Thanks,
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.