You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by oc tsdb <oc...@gmail.com> on 2013/11/19 11:03:02 UTC
HBase snapshots
Hi,
We are using HBase 0.94.10 version.
We could able to take snapshots for all hbase tables and export into
another cluster successfully.
To take HBase table snapshots we followed procedure given at below apache
link.
http://hbase.apache.org/book/ops.snapshots.html
Purpose of taking snapshots is to take cluster backup.
Can someone please help us on below queries.
1.Is there a way to take incremental snapshots on HBase tables?
2.Can we export snapshots to outside of HDFS(e.g backup server)?
3.Is it possible to take a snapshot for all tables in one command?
4.Does HBase support to take automatic snapshots based on configured
timeout?
Please also let us know if there is any better way to take HBase tables
backup.
Thanks in advance.
-OC
Re: HBase snapshots
Posted by Ted Yu <yu...@gmail.com>.
Thanks for the detailed answers, Matteo.
w.r.t. incremental snapshots, there is HBASE-7912.
When Multi-WAL feature, HBASE-5699, is implemented, it would be more
efficient to support capturing incremental changes as proposed by
HBASE-7912.
Cheers
On Tue, Nov 19, 2013 at 9:13 AM, Matteo Bertozzi <th...@gmail.com>wrote:
> > 1.Is there a way to take incremental snapshots on HBase tables?
> >
>
> Due to the nature of hbase, when hfiles gets compacted your table will not
> share files with the snapshot causing the next snapshot to don't share
> hfiles (or share a few) with the previous snapshot. The other approach will
> be using logs to have an incremental snapshots but logs contains different
> tables, so you may end up with more space used, and for sure more time to
> restore since you must replay the logs.
>
>
> > 2.Can we export snapshots to outside of HDFS(e.g backup server)?
> >
>
> the ExportSnapshot tool takes a destination path as argument, so as long as
> the destination filesystem is supported by the FileSystem class you can.
> (e.g. you can export with -copy-to=file:///mnt/backup-mount-point/hbase)
>
>
> > 3.Is it possible to take a snapshot for all tables in one command?
> >
>
> No, but you can do a simple script that does a for each table in
> admin.listTable(), admin.snapshot(table, snapshotName)
>
>
> > 4.Does HBase support to take automatic snapshots based on configured
> > timeout?
> >
>
> No, hbase doesn't do job scheduling.
> you may use a cron job to execute a shell command, "echo snapshot
> 'tableName', 'snapshotName' | hbase shell"
>
>
> >
> > Please also let us know if there is any better way to take HBase tables
> > backup.
> >
>
> There are different solutions that you can use: Replication, CopyTable,
> HDFS files copy + CopyTable and maybe others it depends on what you're
> looking for.
> There is an article here that explains how to use CopyTable as backup
> tool:
>
> http://blog.cloudera.com/blog/2012/06/online-hbase-backups-with-copytable-2/
>
Re: HBase snapshots
Posted by oc tsdb <oc...@gmail.com>.
Hi Matteo,
Thanks a lot for your quick and detailed explanation.
-OC
On Tue, Nov 19, 2013 at 10:43 PM, Matteo Bertozzi
<th...@gmail.com>wrote:
> > 1.Is there a way to take incremental snapshots on HBase tables?
> >
>
> Due to the nature of hbase, when hfiles gets compacted your table will not
> share files with the snapshot causing the next snapshot to don't share
> hfiles (or share a few) with the previous snapshot. The other approach will
> be using logs to have an incremental snapshots but logs contains different
> tables, so you may end up with more space used, and for sure more time to
> restore since you must replay the logs.
>
>
> > 2.Can we export snapshots to outside of HDFS(e.g backup server)?
> >
>
> the ExportSnapshot tool takes a destination path as argument, so as long as
> the destination filesystem is supported by the FileSystem class you can.
> (e.g. you can export with -copy-to=file:///mnt/backup-mount-point/hbase)
>
>
> > 3.Is it possible to take a snapshot for all tables in one command?
> >
>
> No, but you can do a simple script that does a for each table in
> admin.listTable(), admin.snapshot(table, snapshotName)
>
>
> > 4.Does HBase support to take automatic snapshots based on configured
> > timeout?
> >
>
> No, hbase doesn't do job scheduling.
> you may use a cron job to execute a shell command, "echo snapshot
> 'tableName', 'snapshotName' | hbase shell"
>
>
> >
> > Please also let us know if there is any better way to take HBase tables
> > backup.
> >
>
> There are different solutions that you can use: Replication, CopyTable,
> HDFS files copy + CopyTable and maybe others it depends on what you're
> looking for.
> There is an article here that explains how to use CopyTable as backup
> tool:
>
> http://blog.cloudera.com/blog/2012/06/online-hbase-backups-with-copytable-2/
>
Re: HBase snapshots
Posted by Matteo Bertozzi <th...@gmail.com>.
> 1.Is there a way to take incremental snapshots on HBase tables?
>
Due to the nature of hbase, when hfiles gets compacted your table will not
share files with the snapshot causing the next snapshot to don't share
hfiles (or share a few) with the previous snapshot. The other approach will
be using logs to have an incremental snapshots but logs contains different
tables, so you may end up with more space used, and for sure more time to
restore since you must replay the logs.
> 2.Can we export snapshots to outside of HDFS(e.g backup server)?
>
the ExportSnapshot tool takes a destination path as argument, so as long as
the destination filesystem is supported by the FileSystem class you can.
(e.g. you can export with -copy-to=file:///mnt/backup-mount-point/hbase)
> 3.Is it possible to take a snapshot for all tables in one command?
>
No, but you can do a simple script that does a for each table in
admin.listTable(), admin.snapshot(table, snapshotName)
> 4.Does HBase support to take automatic snapshots based on configured
> timeout?
>
No, hbase doesn't do job scheduling.
you may use a cron job to execute a shell command, "echo snapshot
'tableName', 'snapshotName' | hbase shell"
>
> Please also let us know if there is any better way to take HBase tables
> backup.
>
There are different solutions that you can use: Replication, CopyTable,
HDFS files copy + CopyTable and maybe others it depends on what you're
looking for.
There is an article here that explains how to use CopyTable as backup
tool:
http://blog.cloudera.com/blog/2012/06/online-hbase-backups-with-copytable-2/