You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Siddharth Karandikar <si...@gmail.com> on 2013/08/01 15:35:06 UTC
Import HBase snapshots possible?
Hi there,
I am testing out newly added snapshot capability, ExportSnapshot in particular.
Its working fine for me. I am able to run ExportSnapshot properly.
But the biggest (noob) issue is, once exported, is there any way to
import those snapshots back in hbase? I don't see any ImportSnapshot
util there.
Thanks,
Siddharth
Re: Import HBase snapshots possible?
Posted by Siddharth Karandikar <si...@gmail.com>.
Btw, I am running on hbase-0.95.1-hadoop1.
On Thu, Aug 1, 2013 at 7:05 PM, Siddharth Karandikar
<si...@gmail.com> wrote:
> Hi there,
>
> I am testing out newly added snapshot capability, ExportSnapshot in particular.
> Its working fine for me. I am able to run ExportSnapshot properly.
>
> But the biggest (noob) issue is, once exported, is there any way to
> import those snapshots back in hbase? I don't see any ImportSnapshot
> util there.
>
>
> Thanks,
> Siddharth
Re: Import HBase snapshots possible?
Posted by Jignesh Patel <ji...@gmail.com>.
We have two requirements
1. Change the name of table1
2. Modify primary key of the table2.
Table and table 2 are not intern linked(those are different db).
Can we use above mention export and import functionality in both cases.
On Fri, Aug 2, 2013 at 3:34 AM, Siddharth Karandikar <
siddharth.karandikar@gmail.com> wrote:
> Hi Matteo,
>
> Thanks a lot for all your help and detailed explanation. I could
> finally get it running.
>
> I am now running it like -
> ./bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dfs.default.name=hdfs://10.209.17.88:9000/
> -Dhbase.rootdir=hdfs://10.209.17.88:9000/hbase-ss/
> -snapshot s1
> -copy-to /hbase/
>
> My HBase is running with hbase.rootdir=hdfs://10.209.17.88:9000/hbase.
>
> See how I am specifying fs.default.name now. Without that it used to
> fail for source location with - "IllegalArgumentException: Wrong FS:
> hdfs://10.209.17.88:9000/hbase-ss/s2/.hbase-snapshot/s2/.snapshotinfo
> expected: file:///"
> I am still understanding how Configuration, FS type and checkPath are
> working underneath, but for now things are working for me.
>
> Thanks again.
>
> Siddharth
>
>
> On Thu, Aug 1, 2013 at 8:14 PM, Matteo Bertozzi <th...@gmail.com>
> wrote:
> > You can't just copy the .snapshot folder... so now you've the RSs that is
> > failing since the files for the cloned table are not available..
> >
> > when you specify the hbase.rootdir you've just to specify the
> hbase.rootdir
> > the one in /etc/hbase-site.xml which doesn't contain the name of the
> > snapshot/table that you want to export (e.g.
> > hdfs://10.209.17.88:9000/hbasenot hdfs://
> > 10.209.17.88:9000/hbase/s2)
> >
> >
> >
> > Matteo
> >
> >
> >
> > On Thu, Aug 1, 2013 at 3:25 PM, Siddharth Karandikar <
> > siddharth.karandikar@gmail.com> wrote:
> >
> >> Its failing with '//' as well as '///'. Error suggests that it needs
> local
> >> fs.
> >>
> >> 3 ///
> >> ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
> >> org.apache.hadoop.hbase.snapshot.ExportSnapshot
> >> -Dhbase.rootdir=hdfs:///10.209.17.88:9000/hbase/s2 -snapshot s2
> >> -copy-to /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
> >> Exception in thread "main" java.io.IOException: Incomplete HDFS URI,
> >> no host: hdfs:///10.209.17.88:9000/hbase/s2
> >> at
> >>
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:85)
> >> at
> >> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
> >> at
> org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
> >> at
> org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
> >> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
> >> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
> >> at
> >> org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:860)
> >> at
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:594)
> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >> at
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
> >> at
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
> >>
> >> 2 //
> >>
> >> ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
> >> org.apache.hadoop.hbase.snapshot.ExportSnapshot
> >> -Dhbase.rootdir=hdfs://10.209.17.88:9000/hbase/s2 -snapshot s2
> >> -copy-to /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
> >> Exception in thread "main" java.lang.IllegalArgumentException: Wrong
> >> FS: hdfs://10.209.17.88:9000/hbase/s2/.hbase-snapshot/s2/.snapshotinfo,
> >> expected: file:///
> >> at
> org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
> >> at
> >>
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
> >> at
> >>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
> >> at
> >>
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> >> at
> >>
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
> >> at
> >>
> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
> >> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
> >> at
> >>
> org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:296)
> >> at
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:371)
> >> at
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:618)
> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >> at
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
> >> at
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
> >>
> >>
> >>
> >>
> >> Btw, I tried one more thing. From my HDFS location, I just did a copy
> like
> >> -
> >> ssk01:~/siddharth/tools/hadoop-1.1.2 # ./bin/hadoop fs -copyToLocal
> >> hdfs://10.209.17.88:9000/hbase/s1/.hbase-snapshot/s1
> >> /root/siddharth/tools/hbase-0.95.1-hadoop1/data/.hbase-snapshot/
> >>
> >> After doing this, I am able to see s1 in 'list_snapshots'. But it is
> >> failing at 'clone_snapshot'.
> >>
> >> hbase(main):014:0> clone_snapshot 's1', 'ts1'
> >>
> >> ERROR: java.io.IOException: Table 'ts1' not yet enabled, after 199617ms.
> >>
> >> Here is some help for this command:
> >> Create a new table by cloning the snapshot content.
> >> There're no copies of data involved.
> >> And writing on the newly created table will not influence the snapshot
> >> data.
> >>
> >> Examples:
> >> hbase> clone_snapshot 'snapshotName', 'tableName'
> >>
> >>
> >>
> >> On Thu, Aug 1, 2013 at 7:44 PM, Matteo Bertozzi <
> theo.bertozzi@gmail.com>
> >> wrote:
> >> > you have to use 3 slashes otherwise is interpreted as local
> file-system
> >> path
> >> > -Dhbase.rootdir=hdfs:///10.209.17.88:9000/hbase
> >> >
> >> > Matteo
> >> >
> >> >
> >> >
> >> > On Thu, Aug 1, 2013 at 3:09 PM, Siddharth Karandikar <
> >> > siddharth.karandikar@gmail.com> wrote:
> >> >
> >> >> Tried what you suggested. Here is what I get -
> >> >>
> >> >> ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
> >> >> org.apache.hadoop.hbase.snapshot.ExportSnapshot
> >> >> -Dhbase.rootdir=hdfs://10.209.17.88:9000/hbase -snapshot s1 -copy-to
> >> >> /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
> >> >> Exception in thread "main" java.lang.IllegalArgumentException: Wrong
> >> >> FS: hdfs://10.209.17.88:9000/hbase/.hbase-snapshot/s1/.snapshotinfo,
> >> >> expected: file:///
> >> >> at
> >> org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
> >> >> at
> >> >>
> >>
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
> >> >> at
> >> >>
> >>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
> >> >> at
> >> >>
> >>
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> >> >> at
> >> >>
> >>
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
> >> >> at
> >> >>
> >>
> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
> >> >> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
> >> >> at
> >> >>
> >>
> org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:296)
> >> >> at
> >> >>
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:371)
> >> >> at
> >> >>
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:618)
> >> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >> >> at
> >> >>
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
> >> >> at
> >> >>
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
> >> >>
> >> >>
> >> >> Am I missing something?
> >> >>
> >> >>
> >> >> Thanks,
> >> >> Siddharth
> >> >>
> >> >>
> >> >> On Thu, Aug 1, 2013 at 7:31 PM, Matteo Bertozzi <
> >> theo.bertozzi@gmail.com>
> >> >> wrote:
> >> >> > Ok, so to export a snapshot from your HBase cluster, you can do
> >> >> > $ bin/hbase class
> org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
> >> >> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/my-backup-dir
> >> >> >
> >> >> > Now on cluster2, hdfs:///srv2:8082 you've your my-backup-dir that
> >> >> contains
> >> >> > the exported snapshot (note that the snapshot is under hidden dirs
> >> >> > .snapshots, and .archive)
> >> >> >
> >> >> > Now if you want to restore the snapshot, you have to export it
> back
> >> an
> >> >> > HBase cluster.
> >> >> > So on cluster2, you can do:
> >> >> > $ bin/hbase class
> >> org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot -D
> >> >> > hbase.rootdir=hdfs:///srv2:8082/my-backup-dir -snapshot MySnapshot
> >> >> -copy-to
> >> >> > hdfs:///hbaseSrv:8082/hbase
> >> >> >
> >> >> >
> >> >> > so, to recap
> >> >> > - You take a snapshot
> >> >> > - You Export the snapshot from HBase Cluster-1 -> to a simple HDFS
> >> dir
> >> >> in
> >> >> > Cluster-2
> >> >> > - Then you want to restore
> >> >> > - You Export the snapshot from HDFS dir in Cluster-2 to HBase
> Cluster
> >> >> (it
> >> >> > can be a different one from the original)
> >> >> > - From the hbase shell you can just: clone_snapshot
> 'snapshotName',
> >> >> > 'newTableName' if the table does not exists or use restore_snapshot
> >> >> > 'snapshotName', if there's a table with the same name
> >> >> >
> >> >> >
> >> >> > Matteo
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Thu, Aug 1, 2013 at 2:54 PM, Siddharth Karandikar <
> >> >> > siddharth.karandikar@gmail.com> wrote:
> >> >> >
> >> >> >> Yeah, thats right. But the issue is, hdfs that I am exporting to
> is
> >> >> >> not under HBase.
> >> >> >> Can you please provide some example command to do this...
> >> >> >>
> >> >> >>
> >> >> >> Thanks,
> >> >> >> Siddharth
> >> >> >>
> >> >> >> On Thu, Aug 1, 2013 at 7:17 PM, Matteo Bertozzi <
> >> >> theo.bertozzi@gmail.com>
> >> >> >> wrote:
> >> >> >> > Yes, the export an HDFS path.
> >> >> >> > $ bin/hbase class
> >> org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
> >> >> >> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/hbase
> >> >> >> >
> >> >> >> > so you can export to some /my-backup-dir on your HDFS
> >> >> >> > and then you've to export back to an hbase cluster, when you
> want
> >> to
> >> >> >> > restore it
> >> >> >> >
> >> >> >> > Matteo
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > On Thu, Aug 1, 2013 at 2:45 PM, Siddharth Karandikar <
> >> >> >> > siddharth.karandikar@gmail.com> wrote:
> >> >> >> >
> >> >> >> >> Can't I export it to plain HDFS? I think that would be very
> >> useful.
> >> >> >> >>
> >> >> >> >> On Thu, Aug 1, 2013 at 7:08 PM, Matteo Bertozzi <
> >> >> >> theo.bertozzi@gmail.com>
> >> >> >> >> wrote:
> >> >> >> >> > The ExportSnapshot will export the snapshot data+metadata, in
> >> >> theory,
> >> >> >> to
> >> >> >> >> > another hbase cluster.
> >> >> >> >> > so on the second cluster you'll now be able to do
> >> "list_snapshots"
> >> >> >> from
> >> >> >> >> > shell and see the exported snapshot.
> >> >> >> >> > now you can simply do clone_snapshot "snapshot_name",
> >> >> "new_table_name"
> >> >> >> >> and
> >> >> >> >> > you're restoring a snapshot on the second cluster
> >> >> >> >> >
> >> >> >> >> > assuming that you have removed the snapshot from cluster1 and
> >> you
> >> >> >> want to
> >> >> >> >> > export back your snapshot..
> >> >> >> >> > you just use ExportSnapshot again to move the snapshot from
> >> >> cluster2
> >> >> >> to
> >> >> >> >> > cluster1
> >> >> >> >> > and same as before you do a clone_snapshot to restore it
> >> >> >> >> >
> >> >> >> >> > Matteo
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> > On Thu, Aug 1, 2013 at 2:35 PM, Siddharth Karandikar <
> >> >> >> >> > siddharth.karandikar@gmail.com> wrote:
> >> >> >> >> >
> >> >> >> >> >> Hi there,
> >> >> >> >> >>
> >> >> >> >> >> I am testing out newly added snapshot capability,
> >> ExportSnapshot
> >> >> in
> >> >> >> >> >> particular.
> >> >> >> >> >> Its working fine for me. I am able to run ExportSnapshot
> >> properly.
> >> >> >> >> >>
> >> >> >> >> >> But the biggest (noob) issue is, once exported, is there any
> >> way
> >> >> to
> >> >> >> >> >> import those snapshots back in hbase? I don't see any
> >> >> ImportSnapshot
> >> >> >> >> >> util there.
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >> Thanks,
> >> >> >> >> >> Siddharth
> >> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
>
Re: Import HBase snapshots possible?
Posted by Siddharth Karandikar <si...@gmail.com>.
Hi Matteo,
Thanks a lot for all your help and detailed explanation. I could
finally get it running.
I am now running it like -
./bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot
-Dfs.default.name=hdfs://10.209.17.88:9000/
-Dhbase.rootdir=hdfs://10.209.17.88:9000/hbase-ss/
-snapshot s1
-copy-to /hbase/
My HBase is running with hbase.rootdir=hdfs://10.209.17.88:9000/hbase.
See how I am specifying fs.default.name now. Without that it used to
fail for source location with - "IllegalArgumentException: Wrong FS:
hdfs://10.209.17.88:9000/hbase-ss/s2/.hbase-snapshot/s2/.snapshotinfo
expected: file:///"
I am still understanding how Configuration, FS type and checkPath are
working underneath, but for now things are working for me.
Thanks again.
Siddharth
On Thu, Aug 1, 2013 at 8:14 PM, Matteo Bertozzi <th...@gmail.com> wrote:
> You can't just copy the .snapshot folder... so now you've the RSs that is
> failing since the files for the cloned table are not available..
>
> when you specify the hbase.rootdir you've just to specify the hbase.rootdir
> the one in /etc/hbase-site.xml which doesn't contain the name of the
> snapshot/table that you want to export (e.g.
> hdfs://10.209.17.88:9000/hbasenot hdfs://
> 10.209.17.88:9000/hbase/s2)
>
>
>
> Matteo
>
>
>
> On Thu, Aug 1, 2013 at 3:25 PM, Siddharth Karandikar <
> siddharth.karandikar@gmail.com> wrote:
>
>> Its failing with '//' as well as '///'. Error suggests that it needs local
>> fs.
>>
>> 3 ///
>> ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot
>> -Dhbase.rootdir=hdfs:///10.209.17.88:9000/hbase/s2 -snapshot s2
>> -copy-to /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
>> Exception in thread "main" java.io.IOException: Incomplete HDFS URI,
>> no host: hdfs:///10.209.17.88:9000/hbase/s2
>> at
>> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:85)
>> at
>> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
>> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
>> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
>> at
>> org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:860)
>> at
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:594)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
>> at
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
>>
>> 2 //
>>
>> ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot
>> -Dhbase.rootdir=hdfs://10.209.17.88:9000/hbase/s2 -snapshot s2
>> -copy-to /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
>> Exception in thread "main" java.lang.IllegalArgumentException: Wrong
>> FS: hdfs://10.209.17.88:9000/hbase/s2/.hbase-snapshot/s2/.snapshotinfo,
>> expected: file:///
>> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
>> at
>> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
>> at
>> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
>> at
>> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>> at
>> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
>> at
>> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
>> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
>> at
>> org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:296)
>> at
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:371)
>> at
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:618)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
>> at
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
>>
>>
>>
>>
>> Btw, I tried one more thing. From my HDFS location, I just did a copy like
>> -
>> ssk01:~/siddharth/tools/hadoop-1.1.2 # ./bin/hadoop fs -copyToLocal
>> hdfs://10.209.17.88:9000/hbase/s1/.hbase-snapshot/s1
>> /root/siddharth/tools/hbase-0.95.1-hadoop1/data/.hbase-snapshot/
>>
>> After doing this, I am able to see s1 in 'list_snapshots'. But it is
>> failing at 'clone_snapshot'.
>>
>> hbase(main):014:0> clone_snapshot 's1', 'ts1'
>>
>> ERROR: java.io.IOException: Table 'ts1' not yet enabled, after 199617ms.
>>
>> Here is some help for this command:
>> Create a new table by cloning the snapshot content.
>> There're no copies of data involved.
>> And writing on the newly created table will not influence the snapshot
>> data.
>>
>> Examples:
>> hbase> clone_snapshot 'snapshotName', 'tableName'
>>
>>
>>
>> On Thu, Aug 1, 2013 at 7:44 PM, Matteo Bertozzi <th...@gmail.com>
>> wrote:
>> > you have to use 3 slashes otherwise is interpreted as local file-system
>> path
>> > -Dhbase.rootdir=hdfs:///10.209.17.88:9000/hbase
>> >
>> > Matteo
>> >
>> >
>> >
>> > On Thu, Aug 1, 2013 at 3:09 PM, Siddharth Karandikar <
>> > siddharth.karandikar@gmail.com> wrote:
>> >
>> >> Tried what you suggested. Here is what I get -
>> >>
>> >> ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
>> >> org.apache.hadoop.hbase.snapshot.ExportSnapshot
>> >> -Dhbase.rootdir=hdfs://10.209.17.88:9000/hbase -snapshot s1 -copy-to
>> >> /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
>> >> Exception in thread "main" java.lang.IllegalArgumentException: Wrong
>> >> FS: hdfs://10.209.17.88:9000/hbase/.hbase-snapshot/s1/.snapshotinfo,
>> >> expected: file:///
>> >> at
>> org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
>> >> at
>> >>
>> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
>> >> at
>> >>
>> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
>> >> at
>> >>
>> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>> >> at
>> >>
>> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
>> >> at
>> >>
>> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
>> >> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
>> >> at
>> >>
>> org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:296)
>> >> at
>> >>
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:371)
>> >> at
>> >>
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:618)
>> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> >> at
>> >>
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
>> >> at
>> >>
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
>> >>
>> >>
>> >> Am I missing something?
>> >>
>> >>
>> >> Thanks,
>> >> Siddharth
>> >>
>> >>
>> >> On Thu, Aug 1, 2013 at 7:31 PM, Matteo Bertozzi <
>> theo.bertozzi@gmail.com>
>> >> wrote:
>> >> > Ok, so to export a snapshot from your HBase cluster, you can do
>> >> > $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
>> >> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/my-backup-dir
>> >> >
>> >> > Now on cluster2, hdfs:///srv2:8082 you've your my-backup-dir that
>> >> contains
>> >> > the exported snapshot (note that the snapshot is under hidden dirs
>> >> > .snapshots, and .archive)
>> >> >
>> >> > Now if you want to restore the snapshot, you have to export it back
>> an
>> >> > HBase cluster.
>> >> > So on cluster2, you can do:
>> >> > $ bin/hbase class
>> org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot -D
>> >> > hbase.rootdir=hdfs:///srv2:8082/my-backup-dir -snapshot MySnapshot
>> >> -copy-to
>> >> > hdfs:///hbaseSrv:8082/hbase
>> >> >
>> >> >
>> >> > so, to recap
>> >> > - You take a snapshot
>> >> > - You Export the snapshot from HBase Cluster-1 -> to a simple HDFS
>> dir
>> >> in
>> >> > Cluster-2
>> >> > - Then you want to restore
>> >> > - You Export the snapshot from HDFS dir in Cluster-2 to HBase Cluster
>> >> (it
>> >> > can be a different one from the original)
>> >> > - From the hbase shell you can just: clone_snapshot 'snapshotName',
>> >> > 'newTableName' if the table does not exists or use restore_snapshot
>> >> > 'snapshotName', if there's a table with the same name
>> >> >
>> >> >
>> >> > Matteo
>> >> >
>> >> >
>> >> >
>> >> > On Thu, Aug 1, 2013 at 2:54 PM, Siddharth Karandikar <
>> >> > siddharth.karandikar@gmail.com> wrote:
>> >> >
>> >> >> Yeah, thats right. But the issue is, hdfs that I am exporting to is
>> >> >> not under HBase.
>> >> >> Can you please provide some example command to do this...
>> >> >>
>> >> >>
>> >> >> Thanks,
>> >> >> Siddharth
>> >> >>
>> >> >> On Thu, Aug 1, 2013 at 7:17 PM, Matteo Bertozzi <
>> >> theo.bertozzi@gmail.com>
>> >> >> wrote:
>> >> >> > Yes, the export an HDFS path.
>> >> >> > $ bin/hbase class
>> org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
>> >> >> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/hbase
>> >> >> >
>> >> >> > so you can export to some /my-backup-dir on your HDFS
>> >> >> > and then you've to export back to an hbase cluster, when you want
>> to
>> >> >> > restore it
>> >> >> >
>> >> >> > Matteo
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > On Thu, Aug 1, 2013 at 2:45 PM, Siddharth Karandikar <
>> >> >> > siddharth.karandikar@gmail.com> wrote:
>> >> >> >
>> >> >> >> Can't I export it to plain HDFS? I think that would be very
>> useful.
>> >> >> >>
>> >> >> >> On Thu, Aug 1, 2013 at 7:08 PM, Matteo Bertozzi <
>> >> >> theo.bertozzi@gmail.com>
>> >> >> >> wrote:
>> >> >> >> > The ExportSnapshot will export the snapshot data+metadata, in
>> >> theory,
>> >> >> to
>> >> >> >> > another hbase cluster.
>> >> >> >> > so on the second cluster you'll now be able to do
>> "list_snapshots"
>> >> >> from
>> >> >> >> > shell and see the exported snapshot.
>> >> >> >> > now you can simply do clone_snapshot "snapshot_name",
>> >> "new_table_name"
>> >> >> >> and
>> >> >> >> > you're restoring a snapshot on the second cluster
>> >> >> >> >
>> >> >> >> > assuming that you have removed the snapshot from cluster1 and
>> you
>> >> >> want to
>> >> >> >> > export back your snapshot..
>> >> >> >> > you just use ExportSnapshot again to move the snapshot from
>> >> cluster2
>> >> >> to
>> >> >> >> > cluster1
>> >> >> >> > and same as before you do a clone_snapshot to restore it
>> >> >> >> >
>> >> >> >> > Matteo
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > On Thu, Aug 1, 2013 at 2:35 PM, Siddharth Karandikar <
>> >> >> >> > siddharth.karandikar@gmail.com> wrote:
>> >> >> >> >
>> >> >> >> >> Hi there,
>> >> >> >> >>
>> >> >> >> >> I am testing out newly added snapshot capability,
>> ExportSnapshot
>> >> in
>> >> >> >> >> particular.
>> >> >> >> >> Its working fine for me. I am able to run ExportSnapshot
>> properly.
>> >> >> >> >>
>> >> >> >> >> But the biggest (noob) issue is, once exported, is there any
>> way
>> >> to
>> >> >> >> >> import those snapshots back in hbase? I don't see any
>> >> ImportSnapshot
>> >> >> >> >> util there.
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> Thanks,
>> >> >> >> >> Siddharth
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>>
Re: Import HBase snapshots possible?
Posted by Matteo Bertozzi <th...@gmail.com>.
You can't just copy the .snapshot folder... so now you've the RSs that is
failing since the files for the cloned table are not available..
when you specify the hbase.rootdir you've just to specify the hbase.rootdir
the one in /etc/hbase-site.xml which doesn't contain the name of the
snapshot/table that you want to export (e.g.
hdfs://10.209.17.88:9000/hbasenot hdfs://
10.209.17.88:9000/hbase/s2)
Matteo
On Thu, Aug 1, 2013 at 3:25 PM, Siddharth Karandikar <
siddharth.karandikar@gmail.com> wrote:
> Its failing with '//' as well as '///'. Error suggests that it needs local
> fs.
>
> 3 ///
> ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
> org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dhbase.rootdir=hdfs:///10.209.17.88:9000/hbase/s2 -snapshot s2
> -copy-to /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
> Exception in thread "main" java.io.IOException: Incomplete HDFS URI,
> no host: hdfs:///10.209.17.88:9000/hbase/s2
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:85)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
> at
> org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:860)
> at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:594)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
> at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
>
> 2 //
>
> ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
> org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dhbase.rootdir=hdfs://10.209.17.88:9000/hbase/s2 -snapshot s2
> -copy-to /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
> Exception in thread "main" java.lang.IllegalArgumentException: Wrong
> FS: hdfs://10.209.17.88:9000/hbase/s2/.hbase-snapshot/s2/.snapshotinfo,
> expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> at
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:296)
> at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:371)
> at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:618)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
> at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
>
>
>
>
> Btw, I tried one more thing. From my HDFS location, I just did a copy like
> -
> ssk01:~/siddharth/tools/hadoop-1.1.2 # ./bin/hadoop fs -copyToLocal
> hdfs://10.209.17.88:9000/hbase/s1/.hbase-snapshot/s1
> /root/siddharth/tools/hbase-0.95.1-hadoop1/data/.hbase-snapshot/
>
> After doing this, I am able to see s1 in 'list_snapshots'. But it is
> failing at 'clone_snapshot'.
>
> hbase(main):014:0> clone_snapshot 's1', 'ts1'
>
> ERROR: java.io.IOException: Table 'ts1' not yet enabled, after 199617ms.
>
> Here is some help for this command:
> Create a new table by cloning the snapshot content.
> There're no copies of data involved.
> And writing on the newly created table will not influence the snapshot
> data.
>
> Examples:
> hbase> clone_snapshot 'snapshotName', 'tableName'
>
>
>
> On Thu, Aug 1, 2013 at 7:44 PM, Matteo Bertozzi <th...@gmail.com>
> wrote:
> > you have to use 3 slashes otherwise is interpreted as local file-system
> path
> > -Dhbase.rootdir=hdfs:///10.209.17.88:9000/hbase
> >
> > Matteo
> >
> >
> >
> > On Thu, Aug 1, 2013 at 3:09 PM, Siddharth Karandikar <
> > siddharth.karandikar@gmail.com> wrote:
> >
> >> Tried what you suggested. Here is what I get -
> >>
> >> ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
> >> org.apache.hadoop.hbase.snapshot.ExportSnapshot
> >> -Dhbase.rootdir=hdfs://10.209.17.88:9000/hbase -snapshot s1 -copy-to
> >> /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
> >> Exception in thread "main" java.lang.IllegalArgumentException: Wrong
> >> FS: hdfs://10.209.17.88:9000/hbase/.hbase-snapshot/s1/.snapshotinfo,
> >> expected: file:///
> >> at
> org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
> >> at
> >>
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
> >> at
> >>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
> >> at
> >>
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> >> at
> >>
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
> >> at
> >>
> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
> >> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
> >> at
> >>
> org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:296)
> >> at
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:371)
> >> at
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:618)
> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >> at
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
> >> at
> >>
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
> >>
> >>
> >> Am I missing something?
> >>
> >>
> >> Thanks,
> >> Siddharth
> >>
> >>
> >> On Thu, Aug 1, 2013 at 7:31 PM, Matteo Bertozzi <
> theo.bertozzi@gmail.com>
> >> wrote:
> >> > Ok, so to export a snapshot from your HBase cluster, you can do
> >> > $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
> >> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/my-backup-dir
> >> >
> >> > Now on cluster2, hdfs:///srv2:8082 you've your my-backup-dir that
> >> contains
> >> > the exported snapshot (note that the snapshot is under hidden dirs
> >> > .snapshots, and .archive)
> >> >
> >> > Now if you want to restore the snapshot, you have to export it back
> an
> >> > HBase cluster.
> >> > So on cluster2, you can do:
> >> > $ bin/hbase class
> org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot -D
> >> > hbase.rootdir=hdfs:///srv2:8082/my-backup-dir -snapshot MySnapshot
> >> -copy-to
> >> > hdfs:///hbaseSrv:8082/hbase
> >> >
> >> >
> >> > so, to recap
> >> > - You take a snapshot
> >> > - You Export the snapshot from HBase Cluster-1 -> to a simple HDFS
> dir
> >> in
> >> > Cluster-2
> >> > - Then you want to restore
> >> > - You Export the snapshot from HDFS dir in Cluster-2 to HBase Cluster
> >> (it
> >> > can be a different one from the original)
> >> > - From the hbase shell you can just: clone_snapshot 'snapshotName',
> >> > 'newTableName' if the table does not exists or use restore_snapshot
> >> > 'snapshotName', if there's a table with the same name
> >> >
> >> >
> >> > Matteo
> >> >
> >> >
> >> >
> >> > On Thu, Aug 1, 2013 at 2:54 PM, Siddharth Karandikar <
> >> > siddharth.karandikar@gmail.com> wrote:
> >> >
> >> >> Yeah, thats right. But the issue is, hdfs that I am exporting to is
> >> >> not under HBase.
> >> >> Can you please provide some example command to do this...
> >> >>
> >> >>
> >> >> Thanks,
> >> >> Siddharth
> >> >>
> >> >> On Thu, Aug 1, 2013 at 7:17 PM, Matteo Bertozzi <
> >> theo.bertozzi@gmail.com>
> >> >> wrote:
> >> >> > Yes, the export an HDFS path.
> >> >> > $ bin/hbase class
> org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
> >> >> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/hbase
> >> >> >
> >> >> > so you can export to some /my-backup-dir on your HDFS
> >> >> > and then you've to export back to an hbase cluster, when you want
> to
> >> >> > restore it
> >> >> >
> >> >> > Matteo
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Thu, Aug 1, 2013 at 2:45 PM, Siddharth Karandikar <
> >> >> > siddharth.karandikar@gmail.com> wrote:
> >> >> >
> >> >> >> Can't I export it to plain HDFS? I think that would be very
> useful.
> >> >> >>
> >> >> >> On Thu, Aug 1, 2013 at 7:08 PM, Matteo Bertozzi <
> >> >> theo.bertozzi@gmail.com>
> >> >> >> wrote:
> >> >> >> > The ExportSnapshot will export the snapshot data+metadata, in
> >> theory,
> >> >> to
> >> >> >> > another hbase cluster.
> >> >> >> > so on the second cluster you'll now be able to do
> "list_snapshots"
> >> >> from
> >> >> >> > shell and see the exported snapshot.
> >> >> >> > now you can simply do clone_snapshot "snapshot_name",
> >> "new_table_name"
> >> >> >> and
> >> >> >> > you're restoring a snapshot on the second cluster
> >> >> >> >
> >> >> >> > assuming that you have removed the snapshot from cluster1 and
> you
> >> >> want to
> >> >> >> > export back your snapshot..
> >> >> >> > you just use ExportSnapshot again to move the snapshot from
> >> cluster2
> >> >> to
> >> >> >> > cluster1
> >> >> >> > and same as before you do a clone_snapshot to restore it
> >> >> >> >
> >> >> >> > Matteo
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > On Thu, Aug 1, 2013 at 2:35 PM, Siddharth Karandikar <
> >> >> >> > siddharth.karandikar@gmail.com> wrote:
> >> >> >> >
> >> >> >> >> Hi there,
> >> >> >> >>
> >> >> >> >> I am testing out newly added snapshot capability,
> ExportSnapshot
> >> in
> >> >> >> >> particular.
> >> >> >> >> Its working fine for me. I am able to run ExportSnapshot
> properly.
> >> >> >> >>
> >> >> >> >> But the biggest (noob) issue is, once exported, is there any
> way
> >> to
> >> >> >> >> import those snapshots back in hbase? I don't see any
> >> ImportSnapshot
> >> >> >> >> util there.
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> Thanks,
> >> >> >> >> Siddharth
> >> >> >> >>
> >> >> >>
> >> >>
> >>
>
Re: Import HBase snapshots possible?
Posted by Siddharth Karandikar <si...@gmail.com>.
Its failing with '//' as well as '///'. Error suggests that it needs local fs.
3 ///
ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
org.apache.hadoop.hbase.snapshot.ExportSnapshot
-Dhbase.rootdir=hdfs:///10.209.17.88:9000/hbase/s2 -snapshot s2
-copy-to /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
Exception in thread "main" java.io.IOException: Incomplete HDFS URI,
no host: hdfs:///10.209.17.88:9000/hbase/s2
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:85)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
at org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:860)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:594)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
2 //
ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
org.apache.hadoop.hbase.snapshot.ExportSnapshot
-Dhbase.rootdir=hdfs://10.209.17.88:9000/hbase/s2 -snapshot s2
-copy-to /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
Exception in thread "main" java.lang.IllegalArgumentException: Wrong
FS: hdfs://10.209.17.88:9000/hbase/s2/.hbase-snapshot/s2/.snapshotinfo,
expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
at org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:296)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:371)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:618)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
Btw, I tried one more thing. From my HDFS location, I just did a copy like -
ssk01:~/siddharth/tools/hadoop-1.1.2 # ./bin/hadoop fs -copyToLocal
hdfs://10.209.17.88:9000/hbase/s1/.hbase-snapshot/s1
/root/siddharth/tools/hbase-0.95.1-hadoop1/data/.hbase-snapshot/
After doing this, I am able to see s1 in 'list_snapshots'. But it is
failing at 'clone_snapshot'.
hbase(main):014:0> clone_snapshot 's1', 'ts1'
ERROR: java.io.IOException: Table 'ts1' not yet enabled, after 199617ms.
Here is some help for this command:
Create a new table by cloning the snapshot content.
There're no copies of data involved.
And writing on the newly created table will not influence the snapshot data.
Examples:
hbase> clone_snapshot 'snapshotName', 'tableName'
On Thu, Aug 1, 2013 at 7:44 PM, Matteo Bertozzi <th...@gmail.com> wrote:
> you have to use 3 slashes otherwise is interpreted as local file-system path
> -Dhbase.rootdir=hdfs:///10.209.17.88:9000/hbase
>
> Matteo
>
>
>
> On Thu, Aug 1, 2013 at 3:09 PM, Siddharth Karandikar <
> siddharth.karandikar@gmail.com> wrote:
>
>> Tried what you suggested. Here is what I get -
>>
>> ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot
>> -Dhbase.rootdir=hdfs://10.209.17.88:9000/hbase -snapshot s1 -copy-to
>> /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
>> Exception in thread "main" java.lang.IllegalArgumentException: Wrong
>> FS: hdfs://10.209.17.88:9000/hbase/.hbase-snapshot/s1/.snapshotinfo,
>> expected: file:///
>> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
>> at
>> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
>> at
>> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
>> at
>> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>> at
>> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
>> at
>> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
>> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
>> at
>> org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:296)
>> at
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:371)
>> at
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:618)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
>> at
>> org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
>>
>>
>> Am I missing something?
>>
>>
>> Thanks,
>> Siddharth
>>
>>
>> On Thu, Aug 1, 2013 at 7:31 PM, Matteo Bertozzi <th...@gmail.com>
>> wrote:
>> > Ok, so to export a snapshot from your HBase cluster, you can do
>> > $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
>> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/my-backup-dir
>> >
>> > Now on cluster2, hdfs:///srv2:8082 you've your my-backup-dir that
>> contains
>> > the exported snapshot (note that the snapshot is under hidden dirs
>> > .snapshots, and .archive)
>> >
>> > Now if you want to restore the snapshot, you have to export it back an
>> > HBase cluster.
>> > So on cluster2, you can do:
>> > $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot -D
>> > hbase.rootdir=hdfs:///srv2:8082/my-backup-dir -snapshot MySnapshot
>> -copy-to
>> > hdfs:///hbaseSrv:8082/hbase
>> >
>> >
>> > so, to recap
>> > - You take a snapshot
>> > - You Export the snapshot from HBase Cluster-1 -> to a simple HDFS dir
>> in
>> > Cluster-2
>> > - Then you want to restore
>> > - You Export the snapshot from HDFS dir in Cluster-2 to HBase Cluster
>> (it
>> > can be a different one from the original)
>> > - From the hbase shell you can just: clone_snapshot 'snapshotName',
>> > 'newTableName' if the table does not exists or use restore_snapshot
>> > 'snapshotName', if there's a table with the same name
>> >
>> >
>> > Matteo
>> >
>> >
>> >
>> > On Thu, Aug 1, 2013 at 2:54 PM, Siddharth Karandikar <
>> > siddharth.karandikar@gmail.com> wrote:
>> >
>> >> Yeah, thats right. But the issue is, hdfs that I am exporting to is
>> >> not under HBase.
>> >> Can you please provide some example command to do this...
>> >>
>> >>
>> >> Thanks,
>> >> Siddharth
>> >>
>> >> On Thu, Aug 1, 2013 at 7:17 PM, Matteo Bertozzi <
>> theo.bertozzi@gmail.com>
>> >> wrote:
>> >> > Yes, the export an HDFS path.
>> >> > $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
>> >> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/hbase
>> >> >
>> >> > so you can export to some /my-backup-dir on your HDFS
>> >> > and then you've to export back to an hbase cluster, when you want to
>> >> > restore it
>> >> >
>> >> > Matteo
>> >> >
>> >> >
>> >> >
>> >> > On Thu, Aug 1, 2013 at 2:45 PM, Siddharth Karandikar <
>> >> > siddharth.karandikar@gmail.com> wrote:
>> >> >
>> >> >> Can't I export it to plain HDFS? I think that would be very useful.
>> >> >>
>> >> >> On Thu, Aug 1, 2013 at 7:08 PM, Matteo Bertozzi <
>> >> theo.bertozzi@gmail.com>
>> >> >> wrote:
>> >> >> > The ExportSnapshot will export the snapshot data+metadata, in
>> theory,
>> >> to
>> >> >> > another hbase cluster.
>> >> >> > so on the second cluster you'll now be able to do "list_snapshots"
>> >> from
>> >> >> > shell and see the exported snapshot.
>> >> >> > now you can simply do clone_snapshot "snapshot_name",
>> "new_table_name"
>> >> >> and
>> >> >> > you're restoring a snapshot on the second cluster
>> >> >> >
>> >> >> > assuming that you have removed the snapshot from cluster1 and you
>> >> want to
>> >> >> > export back your snapshot..
>> >> >> > you just use ExportSnapshot again to move the snapshot from
>> cluster2
>> >> to
>> >> >> > cluster1
>> >> >> > and same as before you do a clone_snapshot to restore it
>> >> >> >
>> >> >> > Matteo
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > On Thu, Aug 1, 2013 at 2:35 PM, Siddharth Karandikar <
>> >> >> > siddharth.karandikar@gmail.com> wrote:
>> >> >> >
>> >> >> >> Hi there,
>> >> >> >>
>> >> >> >> I am testing out newly added snapshot capability, ExportSnapshot
>> in
>> >> >> >> particular.
>> >> >> >> Its working fine for me. I am able to run ExportSnapshot properly.
>> >> >> >>
>> >> >> >> But the biggest (noob) issue is, once exported, is there any way
>> to
>> >> >> >> import those snapshots back in hbase? I don't see any
>> ImportSnapshot
>> >> >> >> util there.
>> >> >> >>
>> >> >> >>
>> >> >> >> Thanks,
>> >> >> >> Siddharth
>> >> >> >>
>> >> >>
>> >>
>>
Re: Import HBase snapshots possible?
Posted by Matteo Bertozzi <th...@gmail.com>.
you have to use 3 slashes otherwise is interpreted as local file-system path
-Dhbase.rootdir=hdfs:///10.209.17.88:9000/hbase
Matteo
On Thu, Aug 1, 2013 at 3:09 PM, Siddharth Karandikar <
siddharth.karandikar@gmail.com> wrote:
> Tried what you suggested. Here is what I get -
>
> ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
> org.apache.hadoop.hbase.snapshot.ExportSnapshot
> -Dhbase.rootdir=hdfs://10.209.17.88:9000/hbase -snapshot s1 -copy-to
> /root/siddharth/tools/hbase-0.95.1-hadoop1/data/
> Exception in thread "main" java.lang.IllegalArgumentException: Wrong
> FS: hdfs://10.209.17.88:9000/hbase/.hbase-snapshot/s1/.snapshotinfo,
> expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> at
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:296)
> at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:371)
> at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:618)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
> at
> org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
>
>
> Am I missing something?
>
>
> Thanks,
> Siddharth
>
>
> On Thu, Aug 1, 2013 at 7:31 PM, Matteo Bertozzi <th...@gmail.com>
> wrote:
> > Ok, so to export a snapshot from your HBase cluster, you can do
> > $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/my-backup-dir
> >
> > Now on cluster2, hdfs:///srv2:8082 you've your my-backup-dir that
> contains
> > the exported snapshot (note that the snapshot is under hidden dirs
> > .snapshots, and .archive)
> >
> > Now if you want to restore the snapshot, you have to export it back an
> > HBase cluster.
> > So on cluster2, you can do:
> > $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot -D
> > hbase.rootdir=hdfs:///srv2:8082/my-backup-dir -snapshot MySnapshot
> -copy-to
> > hdfs:///hbaseSrv:8082/hbase
> >
> >
> > so, to recap
> > - You take a snapshot
> > - You Export the snapshot from HBase Cluster-1 -> to a simple HDFS dir
> in
> > Cluster-2
> > - Then you want to restore
> > - You Export the snapshot from HDFS dir in Cluster-2 to HBase Cluster
> (it
> > can be a different one from the original)
> > - From the hbase shell you can just: clone_snapshot 'snapshotName',
> > 'newTableName' if the table does not exists or use restore_snapshot
> > 'snapshotName', if there's a table with the same name
> >
> >
> > Matteo
> >
> >
> >
> > On Thu, Aug 1, 2013 at 2:54 PM, Siddharth Karandikar <
> > siddharth.karandikar@gmail.com> wrote:
> >
> >> Yeah, thats right. But the issue is, hdfs that I am exporting to is
> >> not under HBase.
> >> Can you please provide some example command to do this...
> >>
> >>
> >> Thanks,
> >> Siddharth
> >>
> >> On Thu, Aug 1, 2013 at 7:17 PM, Matteo Bertozzi <
> theo.bertozzi@gmail.com>
> >> wrote:
> >> > Yes, the export an HDFS path.
> >> > $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
> >> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/hbase
> >> >
> >> > so you can export to some /my-backup-dir on your HDFS
> >> > and then you've to export back to an hbase cluster, when you want to
> >> > restore it
> >> >
> >> > Matteo
> >> >
> >> >
> >> >
> >> > On Thu, Aug 1, 2013 at 2:45 PM, Siddharth Karandikar <
> >> > siddharth.karandikar@gmail.com> wrote:
> >> >
> >> >> Can't I export it to plain HDFS? I think that would be very useful.
> >> >>
> >> >> On Thu, Aug 1, 2013 at 7:08 PM, Matteo Bertozzi <
> >> theo.bertozzi@gmail.com>
> >> >> wrote:
> >> >> > The ExportSnapshot will export the snapshot data+metadata, in
> theory,
> >> to
> >> >> > another hbase cluster.
> >> >> > so on the second cluster you'll now be able to do "list_snapshots"
> >> from
> >> >> > shell and see the exported snapshot.
> >> >> > now you can simply do clone_snapshot "snapshot_name",
> "new_table_name"
> >> >> and
> >> >> > you're restoring a snapshot on the second cluster
> >> >> >
> >> >> > assuming that you have removed the snapshot from cluster1 and you
> >> want to
> >> >> > export back your snapshot..
> >> >> > you just use ExportSnapshot again to move the snapshot from
> cluster2
> >> to
> >> >> > cluster1
> >> >> > and same as before you do a clone_snapshot to restore it
> >> >> >
> >> >> > Matteo
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Thu, Aug 1, 2013 at 2:35 PM, Siddharth Karandikar <
> >> >> > siddharth.karandikar@gmail.com> wrote:
> >> >> >
> >> >> >> Hi there,
> >> >> >>
> >> >> >> I am testing out newly added snapshot capability, ExportSnapshot
> in
> >> >> >> particular.
> >> >> >> Its working fine for me. I am able to run ExportSnapshot properly.
> >> >> >>
> >> >> >> But the biggest (noob) issue is, once exported, is there any way
> to
> >> >> >> import those snapshots back in hbase? I don't see any
> ImportSnapshot
> >> >> >> util there.
> >> >> >>
> >> >> >>
> >> >> >> Thanks,
> >> >> >> Siddharth
> >> >> >>
> >> >>
> >>
>
Re: Import HBase snapshots possible?
Posted by Siddharth Karandikar <si...@gmail.com>.
Tried what you suggested. Here is what I get -
ssk01:~/siddharth/tools/hbase-0.95.1-hadoop1 # ./bin/hbase
org.apache.hadoop.hbase.snapshot.ExportSnapshot
-Dhbase.rootdir=hdfs://10.209.17.88:9000/hbase -snapshot s1 -copy-to
/root/siddharth/tools/hbase-0.95.1-hadoop1/data/
Exception in thread "main" java.lang.IllegalArgumentException: Wrong
FS: hdfs://10.209.17.88:9000/hbase/.hbase-snapshot/s1/.snapshotinfo,
expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
at org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:296)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:371)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:618)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:690)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:694)
Am I missing something?
Thanks,
Siddharth
On Thu, Aug 1, 2013 at 7:31 PM, Matteo Bertozzi <th...@gmail.com> wrote:
> Ok, so to export a snapshot from your HBase cluster, you can do
> $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
> -snapshot MySnapshot -copy-to hdfs:///srv2:8082/my-backup-dir
>
> Now on cluster2, hdfs:///srv2:8082 you've your my-backup-dir that contains
> the exported snapshot (note that the snapshot is under hidden dirs
> .snapshots, and .archive)
>
> Now if you want to restore the snapshot, you have to export it back an
> HBase cluster.
> So on cluster2, you can do:
> $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot -D
> hbase.rootdir=hdfs:///srv2:8082/my-backup-dir -snapshot MySnapshot -copy-to
> hdfs:///hbaseSrv:8082/hbase
>
>
> so, to recap
> - You take a snapshot
> - You Export the snapshot from HBase Cluster-1 -> to a simple HDFS dir in
> Cluster-2
> - Then you want to restore
> - You Export the snapshot from HDFS dir in Cluster-2 to HBase Cluster (it
> can be a different one from the original)
> - From the hbase shell you can just: clone_snapshot 'snapshotName',
> 'newTableName' if the table does not exists or use restore_snapshot
> 'snapshotName', if there's a table with the same name
>
>
> Matteo
>
>
>
> On Thu, Aug 1, 2013 at 2:54 PM, Siddharth Karandikar <
> siddharth.karandikar@gmail.com> wrote:
>
>> Yeah, thats right. But the issue is, hdfs that I am exporting to is
>> not under HBase.
>> Can you please provide some example command to do this...
>>
>>
>> Thanks,
>> Siddharth
>>
>> On Thu, Aug 1, 2013 at 7:17 PM, Matteo Bertozzi <th...@gmail.com>
>> wrote:
>> > Yes, the export an HDFS path.
>> > $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
>> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/hbase
>> >
>> > so you can export to some /my-backup-dir on your HDFS
>> > and then you've to export back to an hbase cluster, when you want to
>> > restore it
>> >
>> > Matteo
>> >
>> >
>> >
>> > On Thu, Aug 1, 2013 at 2:45 PM, Siddharth Karandikar <
>> > siddharth.karandikar@gmail.com> wrote:
>> >
>> >> Can't I export it to plain HDFS? I think that would be very useful.
>> >>
>> >> On Thu, Aug 1, 2013 at 7:08 PM, Matteo Bertozzi <
>> theo.bertozzi@gmail.com>
>> >> wrote:
>> >> > The ExportSnapshot will export the snapshot data+metadata, in theory,
>> to
>> >> > another hbase cluster.
>> >> > so on the second cluster you'll now be able to do "list_snapshots"
>> from
>> >> > shell and see the exported snapshot.
>> >> > now you can simply do clone_snapshot "snapshot_name", "new_table_name"
>> >> and
>> >> > you're restoring a snapshot on the second cluster
>> >> >
>> >> > assuming that you have removed the snapshot from cluster1 and you
>> want to
>> >> > export back your snapshot..
>> >> > you just use ExportSnapshot again to move the snapshot from cluster2
>> to
>> >> > cluster1
>> >> > and same as before you do a clone_snapshot to restore it
>> >> >
>> >> > Matteo
>> >> >
>> >> >
>> >> >
>> >> > On Thu, Aug 1, 2013 at 2:35 PM, Siddharth Karandikar <
>> >> > siddharth.karandikar@gmail.com> wrote:
>> >> >
>> >> >> Hi there,
>> >> >>
>> >> >> I am testing out newly added snapshot capability, ExportSnapshot in
>> >> >> particular.
>> >> >> Its working fine for me. I am able to run ExportSnapshot properly.
>> >> >>
>> >> >> But the biggest (noob) issue is, once exported, is there any way to
>> >> >> import those snapshots back in hbase? I don't see any ImportSnapshot
>> >> >> util there.
>> >> >>
>> >> >>
>> >> >> Thanks,
>> >> >> Siddharth
>> >> >>
>> >>
>>
Re: Import HBase snapshots possible?
Posted by Matteo Bertozzi <th...@gmail.com>.
Ok, so to export a snapshot from your HBase cluster, you can do
$ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
-snapshot MySnapshot -copy-to hdfs:///srv2:8082/my-backup-dir
Now on cluster2, hdfs:///srv2:8082 you've your my-backup-dir that contains
the exported snapshot (note that the snapshot is under hidden dirs
.snapshots, and .archive)
Now if you want to restore the snapshot, you have to export it back an
HBase cluster.
So on cluster2, you can do:
$ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot -D
hbase.rootdir=hdfs:///srv2:8082/my-backup-dir -snapshot MySnapshot -copy-to
hdfs:///hbaseSrv:8082/hbase
so, to recap
- You take a snapshot
- You Export the snapshot from HBase Cluster-1 -> to a simple HDFS dir in
Cluster-2
- Then you want to restore
- You Export the snapshot from HDFS dir in Cluster-2 to HBase Cluster (it
can be a different one from the original)
- From the hbase shell you can just: clone_snapshot 'snapshotName',
'newTableName' if the table does not exists or use restore_snapshot
'snapshotName', if there's a table with the same name
Matteo
On Thu, Aug 1, 2013 at 2:54 PM, Siddharth Karandikar <
siddharth.karandikar@gmail.com> wrote:
> Yeah, thats right. But the issue is, hdfs that I am exporting to is
> not under HBase.
> Can you please provide some example command to do this...
>
>
> Thanks,
> Siddharth
>
> On Thu, Aug 1, 2013 at 7:17 PM, Matteo Bertozzi <th...@gmail.com>
> wrote:
> > Yes, the export an HDFS path.
> > $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
> > -snapshot MySnapshot -copy-to hdfs:///srv2:8082/hbase
> >
> > so you can export to some /my-backup-dir on your HDFS
> > and then you've to export back to an hbase cluster, when you want to
> > restore it
> >
> > Matteo
> >
> >
> >
> > On Thu, Aug 1, 2013 at 2:45 PM, Siddharth Karandikar <
> > siddharth.karandikar@gmail.com> wrote:
> >
> >> Can't I export it to plain HDFS? I think that would be very useful.
> >>
> >> On Thu, Aug 1, 2013 at 7:08 PM, Matteo Bertozzi <
> theo.bertozzi@gmail.com>
> >> wrote:
> >> > The ExportSnapshot will export the snapshot data+metadata, in theory,
> to
> >> > another hbase cluster.
> >> > so on the second cluster you'll now be able to do "list_snapshots"
> from
> >> > shell and see the exported snapshot.
> >> > now you can simply do clone_snapshot "snapshot_name", "new_table_name"
> >> and
> >> > you're restoring a snapshot on the second cluster
> >> >
> >> > assuming that you have removed the snapshot from cluster1 and you
> want to
> >> > export back your snapshot..
> >> > you just use ExportSnapshot again to move the snapshot from cluster2
> to
> >> > cluster1
> >> > and same as before you do a clone_snapshot to restore it
> >> >
> >> > Matteo
> >> >
> >> >
> >> >
> >> > On Thu, Aug 1, 2013 at 2:35 PM, Siddharth Karandikar <
> >> > siddharth.karandikar@gmail.com> wrote:
> >> >
> >> >> Hi there,
> >> >>
> >> >> I am testing out newly added snapshot capability, ExportSnapshot in
> >> >> particular.
> >> >> Its working fine for me. I am able to run ExportSnapshot properly.
> >> >>
> >> >> But the biggest (noob) issue is, once exported, is there any way to
> >> >> import those snapshots back in hbase? I don't see any ImportSnapshot
> >> >> util there.
> >> >>
> >> >>
> >> >> Thanks,
> >> >> Siddharth
> >> >>
> >>
>
Re: Import HBase snapshots possible?
Posted by Siddharth Karandikar <si...@gmail.com>.
Yeah, thats right. But the issue is, hdfs that I am exporting to is
not under HBase.
Can you please provide some example command to do this...
Thanks,
Siddharth
On Thu, Aug 1, 2013 at 7:17 PM, Matteo Bertozzi <th...@gmail.com> wrote:
> Yes, the export an HDFS path.
> $ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
> -snapshot MySnapshot -copy-to hdfs:///srv2:8082/hbase
>
> so you can export to some /my-backup-dir on your HDFS
> and then you've to export back to an hbase cluster, when you want to
> restore it
>
> Matteo
>
>
>
> On Thu, Aug 1, 2013 at 2:45 PM, Siddharth Karandikar <
> siddharth.karandikar@gmail.com> wrote:
>
>> Can't I export it to plain HDFS? I think that would be very useful.
>>
>> On Thu, Aug 1, 2013 at 7:08 PM, Matteo Bertozzi <th...@gmail.com>
>> wrote:
>> > The ExportSnapshot will export the snapshot data+metadata, in theory, to
>> > another hbase cluster.
>> > so on the second cluster you'll now be able to do "list_snapshots" from
>> > shell and see the exported snapshot.
>> > now you can simply do clone_snapshot "snapshot_name", "new_table_name"
>> and
>> > you're restoring a snapshot on the second cluster
>> >
>> > assuming that you have removed the snapshot from cluster1 and you want to
>> > export back your snapshot..
>> > you just use ExportSnapshot again to move the snapshot from cluster2 to
>> > cluster1
>> > and same as before you do a clone_snapshot to restore it
>> >
>> > Matteo
>> >
>> >
>> >
>> > On Thu, Aug 1, 2013 at 2:35 PM, Siddharth Karandikar <
>> > siddharth.karandikar@gmail.com> wrote:
>> >
>> >> Hi there,
>> >>
>> >> I am testing out newly added snapshot capability, ExportSnapshot in
>> >> particular.
>> >> Its working fine for me. I am able to run ExportSnapshot properly.
>> >>
>> >> But the biggest (noob) issue is, once exported, is there any way to
>> >> import those snapshots back in hbase? I don't see any ImportSnapshot
>> >> util there.
>> >>
>> >>
>> >> Thanks,
>> >> Siddharth
>> >>
>>
Re: Import HBase snapshots possible?
Posted by Matteo Bertozzi <th...@gmail.com>.
Yes, the export an HDFS path.
$ bin/hbase class org.apache.hadoop.hbase.snapshot.tool.ExportSnapshot
-snapshot MySnapshot -copy-to hdfs:///srv2:8082/hbase
so you can export to some /my-backup-dir on your HDFS
and then you've to export back to an hbase cluster, when you want to
restore it
Matteo
On Thu, Aug 1, 2013 at 2:45 PM, Siddharth Karandikar <
siddharth.karandikar@gmail.com> wrote:
> Can't I export it to plain HDFS? I think that would be very useful.
>
> On Thu, Aug 1, 2013 at 7:08 PM, Matteo Bertozzi <th...@gmail.com>
> wrote:
> > The ExportSnapshot will export the snapshot data+metadata, in theory, to
> > another hbase cluster.
> > so on the second cluster you'll now be able to do "list_snapshots" from
> > shell and see the exported snapshot.
> > now you can simply do clone_snapshot "snapshot_name", "new_table_name"
> and
> > you're restoring a snapshot on the second cluster
> >
> > assuming that you have removed the snapshot from cluster1 and you want to
> > export back your snapshot..
> > you just use ExportSnapshot again to move the snapshot from cluster2 to
> > cluster1
> > and same as before you do a clone_snapshot to restore it
> >
> > Matteo
> >
> >
> >
> > On Thu, Aug 1, 2013 at 2:35 PM, Siddharth Karandikar <
> > siddharth.karandikar@gmail.com> wrote:
> >
> >> Hi there,
> >>
> >> I am testing out newly added snapshot capability, ExportSnapshot in
> >> particular.
> >> Its working fine for me. I am able to run ExportSnapshot properly.
> >>
> >> But the biggest (noob) issue is, once exported, is there any way to
> >> import those snapshots back in hbase? I don't see any ImportSnapshot
> >> util there.
> >>
> >>
> >> Thanks,
> >> Siddharth
> >>
>
Re: Import HBase snapshots possible?
Posted by Siddharth Karandikar <si...@gmail.com>.
Can't I export it to plain HDFS? I think that would be very useful.
On Thu, Aug 1, 2013 at 7:08 PM, Matteo Bertozzi <th...@gmail.com> wrote:
> The ExportSnapshot will export the snapshot data+metadata, in theory, to
> another hbase cluster.
> so on the second cluster you'll now be able to do "list_snapshots" from
> shell and see the exported snapshot.
> now you can simply do clone_snapshot "snapshot_name", "new_table_name" and
> you're restoring a snapshot on the second cluster
>
> assuming that you have removed the snapshot from cluster1 and you want to
> export back your snapshot..
> you just use ExportSnapshot again to move the snapshot from cluster2 to
> cluster1
> and same as before you do a clone_snapshot to restore it
>
> Matteo
>
>
>
> On Thu, Aug 1, 2013 at 2:35 PM, Siddharth Karandikar <
> siddharth.karandikar@gmail.com> wrote:
>
>> Hi there,
>>
>> I am testing out newly added snapshot capability, ExportSnapshot in
>> particular.
>> Its working fine for me. I am able to run ExportSnapshot properly.
>>
>> But the biggest (noob) issue is, once exported, is there any way to
>> import those snapshots back in hbase? I don't see any ImportSnapshot
>> util there.
>>
>>
>> Thanks,
>> Siddharth
>>
Re: Import HBase snapshots possible?
Posted by Matteo Bertozzi <th...@gmail.com>.
The ExportSnapshot will export the snapshot data+metadata, in theory, to
another hbase cluster.
so on the second cluster you'll now be able to do "list_snapshots" from
shell and see the exported snapshot.
now you can simply do clone_snapshot "snapshot_name", "new_table_name" and
you're restoring a snapshot on the second cluster
assuming that you have removed the snapshot from cluster1 and you want to
export back your snapshot..
you just use ExportSnapshot again to move the snapshot from cluster2 to
cluster1
and same as before you do a clone_snapshot to restore it
Matteo
On Thu, Aug 1, 2013 at 2:35 PM, Siddharth Karandikar <
siddharth.karandikar@gmail.com> wrote:
> Hi there,
>
> I am testing out newly added snapshot capability, ExportSnapshot in
> particular.
> Its working fine for me. I am able to run ExportSnapshot properly.
>
> But the biggest (noob) issue is, once exported, is there any way to
> import those snapshots back in hbase? I don't see any ImportSnapshot
> util there.
>
>
> Thanks,
> Siddharth
>