You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Rita <rm...@gmail.com> on 2012/11/17 15:31:39 UTC
migrating hbase table from one cluster to another
Hello,
My old cluster is running 0.90.4 and the new cluster is running 0.92.1
I managed to copy the data from old cluster to new cluster using
hadoop distcp -i hftp://oldcluster:50070/hbase/foo.taq /tmp
I even copied it to the newcluster :/hbase
I am not sure what I need to do next to have hbase recognize the data
--
--- Get your facts first, then you can distort them as you please.--
Re: migrating hbase table from one cluster to another
Posted by Ted Yu <yu...@gmail.com>.
Actually the procedure is not complicated.
Our group recently performed an upgrade of 0.90.x cluster to 0.92 where we
need to merge adjacent regions from the original table into fewer regions
in the new table. This was due to the overly small region size in 0.90.x
cluster.
That required scripting and moving HFiles around.
In terms of compatibility across major releases, we have been consistent:
(one-way) auto-upgrade is provided from one major release to the next major
release.
Cheers
On Sat, Nov 17, 2012 at 10:10 AM, Rita <rm...@gmail.com> wrote:
> Wow. Sounds complicated.
>
> Let me try copying the entire hbase directory over. And then restart the
> cluster.
>
> For future release of Hbase the upgrades and table compatibility should be
> better supported correct?
>
>
> On Sat, Nov 17, 2012 at 12:18 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Assuming the tables from 0.90.4 cluster are absent in 0.92.1 cluster, you
> > can (I haven't tried this):
> > 1. sideline the data you just copied from 0.90.4 cluster
> > 2. create table(s) in 0.92.1 cluster with same schema as those in 0.90.4
> > cluster - you need to presplit new table with the exact same region
> > boundaries as the old table
> > 3. move back the sidelined data into corresponding regions
> >
> > Thanks
> >
> > On Sat, Nov 17, 2012 at 8:47 AM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > So the .META. table in your 0.92.1 cluster doesn't have rows for the
> > > copied tables.
> > >
> > > Looks like you need:
> > > https://issues.apache.org/jira/browse/HBASE-3271
> > > Allow .META. table to be exported
> > >
> > > But it is in 0.94.2 and above.
> > >
> > > Cheers
> > >
> > >
> > > On Sat, Nov 17, 2012 at 8:41 AM, Rita <rm...@gmail.com> wrote:
> > >
> > >> Do I need to restart the cluster? My 0.92.1 already has several hbase
> > >> tables.
> > >>
> > >> On Sat, Nov 17, 2012 at 9:54 AM, <yu...@gmail.com> wrote:
> > >>
> > >> > When you start the new cluster with 0.92.1, hbase would migrate your
> > >> data
> > >> > to 0.92 format.
> > >> >
> > >> > Thanks
> > >> >
> > >> >
> > >> >
> > >> > On Nov 17, 2012, at 6:31 AM, Rita <rm...@gmail.com> wrote:
> > >> >
> > >> > > Hello,
> > >> > >
> > >> > > My old cluster is running 0.90.4 and the new cluster is running
> > 0.92.1
> > >> > >
> > >> > > I managed to copy the data from old cluster to new cluster using
> > >> > >
> > >> > > hadoop distcp -i hftp://oldcluster:50070/hbase/foo.taq /tmp
> > >> > >
> > >> > > I even copied it to the newcluster :/hbase
> > >> > >
> > >> > > I am not sure what I need to do next to have hbase recognize the
> > data
> > >> > >
> > >> > > --
> > >> > > --- Get your facts first, then you can distort them as you
> please.--
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> --- Get your facts first, then you can distort them as you please.--
> > >>
> > >
> > >
> >
>
>
>
> --
> --- Get your facts first, then you can distort them as you please.--
>
Re: migrating hbase table from one cluster to another
Posted by Rita <rm...@gmail.com>.
Wow. Sounds complicated.
Let me try copying the entire hbase directory over. And then restart the
cluster.
For future release of Hbase the upgrades and table compatibility should be
better supported correct?
On Sat, Nov 17, 2012 at 12:18 PM, Ted Yu <yu...@gmail.com> wrote:
> Assuming the tables from 0.90.4 cluster are absent in 0.92.1 cluster, you
> can (I haven't tried this):
> 1. sideline the data you just copied from 0.90.4 cluster
> 2. create table(s) in 0.92.1 cluster with same schema as those in 0.90.4
> cluster - you need to presplit new table with the exact same region
> boundaries as the old table
> 3. move back the sidelined data into corresponding regions
>
> Thanks
>
> On Sat, Nov 17, 2012 at 8:47 AM, Ted Yu <yu...@gmail.com> wrote:
>
> > So the .META. table in your 0.92.1 cluster doesn't have rows for the
> > copied tables.
> >
> > Looks like you need:
> > https://issues.apache.org/jira/browse/HBASE-3271
> > Allow .META. table to be exported
> >
> > But it is in 0.94.2 and above.
> >
> > Cheers
> >
> >
> > On Sat, Nov 17, 2012 at 8:41 AM, Rita <rm...@gmail.com> wrote:
> >
> >> Do I need to restart the cluster? My 0.92.1 already has several hbase
> >> tables.
> >>
> >> On Sat, Nov 17, 2012 at 9:54 AM, <yu...@gmail.com> wrote:
> >>
> >> > When you start the new cluster with 0.92.1, hbase would migrate your
> >> data
> >> > to 0.92 format.
> >> >
> >> > Thanks
> >> >
> >> >
> >> >
> >> > On Nov 17, 2012, at 6:31 AM, Rita <rm...@gmail.com> wrote:
> >> >
> >> > > Hello,
> >> > >
> >> > > My old cluster is running 0.90.4 and the new cluster is running
> 0.92.1
> >> > >
> >> > > I managed to copy the data from old cluster to new cluster using
> >> > >
> >> > > hadoop distcp -i hftp://oldcluster:50070/hbase/foo.taq /tmp
> >> > >
> >> > > I even copied it to the newcluster :/hbase
> >> > >
> >> > > I am not sure what I need to do next to have hbase recognize the
> data
> >> > >
> >> > > --
> >> > > --- Get your facts first, then you can distort them as you please.--
> >> >
> >>
> >>
> >>
> >> --
> >> --- Get your facts first, then you can distort them as you please.--
> >>
> >
> >
>
--
--- Get your facts first, then you can distort them as you please.--
Re: migrating hbase table from one cluster to another
Posted by Ted Yu <yu...@gmail.com>.
Assuming the tables from 0.90.4 cluster are absent in 0.92.1 cluster, you
can (I haven't tried this):
1. sideline the data you just copied from 0.90.4 cluster
2. create table(s) in 0.92.1 cluster with same schema as those in 0.90.4
cluster - you need to presplit new table with the exact same region
boundaries as the old table
3. move back the sidelined data into corresponding regions
Thanks
On Sat, Nov 17, 2012 at 8:47 AM, Ted Yu <yu...@gmail.com> wrote:
> So the .META. table in your 0.92.1 cluster doesn't have rows for the
> copied tables.
>
> Looks like you need:
> https://issues.apache.org/jira/browse/HBASE-3271
> Allow .META. table to be exported
>
> But it is in 0.94.2 and above.
>
> Cheers
>
>
> On Sat, Nov 17, 2012 at 8:41 AM, Rita <rm...@gmail.com> wrote:
>
>> Do I need to restart the cluster? My 0.92.1 already has several hbase
>> tables.
>>
>> On Sat, Nov 17, 2012 at 9:54 AM, <yu...@gmail.com> wrote:
>>
>> > When you start the new cluster with 0.92.1, hbase would migrate your
>> data
>> > to 0.92 format.
>> >
>> > Thanks
>> >
>> >
>> >
>> > On Nov 17, 2012, at 6:31 AM, Rita <rm...@gmail.com> wrote:
>> >
>> > > Hello,
>> > >
>> > > My old cluster is running 0.90.4 and the new cluster is running 0.92.1
>> > >
>> > > I managed to copy the data from old cluster to new cluster using
>> > >
>> > > hadoop distcp -i hftp://oldcluster:50070/hbase/foo.taq /tmp
>> > >
>> > > I even copied it to the newcluster :/hbase
>> > >
>> > > I am not sure what I need to do next to have hbase recognize the data
>> > >
>> > > --
>> > > --- Get your facts first, then you can distort them as you please.--
>> >
>>
>>
>>
>> --
>> --- Get your facts first, then you can distort them as you please.--
>>
>
>
Re: migrating hbase table from one cluster to another
Posted by Ted Yu <yu...@gmail.com>.
So the .META. table in your 0.92.1 cluster doesn't have rows for the copied
tables.
Looks like you need:
https://issues.apache.org/jira/browse/HBASE-3271
Allow .META. table to be exported
But it is in 0.94.2 and above.
Cheers
On Sat, Nov 17, 2012 at 8:41 AM, Rita <rm...@gmail.com> wrote:
> Do I need to restart the cluster? My 0.92.1 already has several hbase
> tables.
>
> On Sat, Nov 17, 2012 at 9:54 AM, <yu...@gmail.com> wrote:
>
> > When you start the new cluster with 0.92.1, hbase would migrate your data
> > to 0.92 format.
> >
> > Thanks
> >
> >
> >
> > On Nov 17, 2012, at 6:31 AM, Rita <rm...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > My old cluster is running 0.90.4 and the new cluster is running 0.92.1
> > >
> > > I managed to copy the data from old cluster to new cluster using
> > >
> > > hadoop distcp -i hftp://oldcluster:50070/hbase/foo.taq /tmp
> > >
> > > I even copied it to the newcluster :/hbase
> > >
> > > I am not sure what I need to do next to have hbase recognize the data
> > >
> > > --
> > > --- Get your facts first, then you can distort them as you please.--
> >
>
>
>
> --
> --- Get your facts first, then you can distort them as you please.--
>
Re: migrating hbase table from one cluster to another
Posted by Rita <rm...@gmail.com>.
Do I need to restart the cluster? My 0.92.1 already has several hbase
tables.
On Sat, Nov 17, 2012 at 9:54 AM, <yu...@gmail.com> wrote:
> When you start the new cluster with 0.92.1, hbase would migrate your data
> to 0.92 format.
>
> Thanks
>
>
>
> On Nov 17, 2012, at 6:31 AM, Rita <rm...@gmail.com> wrote:
>
> > Hello,
> >
> > My old cluster is running 0.90.4 and the new cluster is running 0.92.1
> >
> > I managed to copy the data from old cluster to new cluster using
> >
> > hadoop distcp -i hftp://oldcluster:50070/hbase/foo.taq /tmp
> >
> > I even copied it to the newcluster :/hbase
> >
> > I am not sure what I need to do next to have hbase recognize the data
> >
> > --
> > --- Get your facts first, then you can distort them as you please.--
>
--
--- Get your facts first, then you can distort them as you please.--
Re: migrating hbase table from one cluster to another
Posted by yu...@gmail.com.
When you start the new cluster with 0.92.1, hbase would migrate your data to 0.92 format.
Thanks
On Nov 17, 2012, at 6:31 AM, Rita <rm...@gmail.com> wrote:
> Hello,
>
> My old cluster is running 0.90.4 and the new cluster is running 0.92.1
>
> I managed to copy the data from old cluster to new cluster using
>
> hadoop distcp -i hftp://oldcluster:50070/hbase/foo.taq /tmp
>
> I even copied it to the newcluster :/hbase
>
> I am not sure what I need to do next to have hbase recognize the data
>
> --
> --- Get your facts first, then you can distort them as you please.--