You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Venkatesh <vr...@aol.com> on 2011/03/29 23:27:25 UTC
Export/Import and # of regions
Hi,
If I export existing table using Export MR job, truncate the table, increase region size, & do a Import
will it make use of the new region size?
thanks
V
Re: Export/Import and # of regions
Posted by Jean-Daniel Cryans <jd...@apache.org>.
> Thanks J-D..Using 0.20.6..I don't see that method with pre-split in 0.20.6 API spec
It's new from 0.89, please consider upgrading.
>
> 1) Will the data still be accessible if I Import the data to a new table? (purely for backup reasons)
> I tried on small data set..I could..
> Before I do export/Import on large table, want to make sure..
It's indeed a way to backup a table
http://blog.sematext.com/2011/03/11/hbase-backup-options/
> 2) Data exported using 0.20.6, can it be imported using 0.90.1? (i could use pre-split in this case)
Yes.
J-D
Re: Export/Import and # of regions
Posted by Venkatesh <vr...@aol.com>.
Thanks J-D..Using 0.20.6..I don't see that method with pre-split in 0.20.6 API spec
1) Will the data still be accessible if I Import the data to a new table? (purely for backup reasons)
I tried on small data set..I could..
Before I do export/Import on large table, want to make sure..
2) Data exported using 0.20.6, can it be imported using 0.90.1? (i could use pre-split in this case)
-----Original Message-----
From: Jean-Daniel Cryans <jd...@apache.org>
To: user@hbase.apache.org
Sent: Tue, Mar 29, 2011 5:38 pm
Subject: Re: Export/Import and # of regions
Pre-splitting was discussed a few times on the mailing list today, and
a few times in the past weeks, for example:
http://search-hadoop.com/m/XB9Vr1gQc66
Import works on a pre-existing table so it won't recreate it. Also it
doesn't know how your key space is constructed, so it cannot guess the
start/stop row keys for you.
J-D
On Tue, Mar 29, 2011 at 2:33 PM, Venkatesh <vr...@aol.com> wrote:
> Thanks J-D
>
> We have way too much data & it won't fit in 1 region.Is Import smart enough
create
> reqd # of regions?
>
> Cld u pl. elaborate on pre-split table creation? steps?
>
> Reason I'm doing this exercise is reduce # of regions in our cluster (in the
absence of additional hardware
>> 25K regions on 20 node)
>
>
>
>
>
>
>
>
>
>
> -----Original Message-----
> From: Jean-Daniel Cryans <jd...@apache.org>
> To: user@hbase.apache.org
> Sent: Tue, Mar 29, 2011 5:29 pm
> Subject: Re: Export/Import and # of regions
>
>
> Yes but you'll start with a single region, instead of truncating you
>
> probably want instead to create a pre-split table.
>
>
>
> J-D
>
>
>
> On Tue, Mar 29, 2011 at 2:27 PM, Venkatesh <vr...@aol.com> wrote:
>
>>
>
>>
>
>>
>
>> Hi,
>
>> If I export existing table using Export MR job, truncate the table, increase
>
> region size, & do a Import
>
>> will it make use of the new region size?
>
>>
>
>> thanks
>
>> V
>
>>
>
>>
>
>>
>
>
>
>
Re: Export/Import and # of regions
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Pre-splitting was discussed a few times on the mailing list today, and
a few times in the past weeks, for example:
http://search-hadoop.com/m/XB9Vr1gQc66
Import works on a pre-existing table so it won't recreate it. Also it
doesn't know how your key space is constructed, so it cannot guess the
start/stop row keys for you.
J-D
On Tue, Mar 29, 2011 at 2:33 PM, Venkatesh <vr...@aol.com> wrote:
> Thanks J-D
>
> We have way too much data & it won't fit in 1 region.Is Import smart enough create
> reqd # of regions?
>
> Cld u pl. elaborate on pre-split table creation? steps?
>
> Reason I'm doing this exercise is reduce # of regions in our cluster (in the absence of additional hardware
>> 25K regions on 20 node)
>
>
>
>
>
>
>
>
>
>
> -----Original Message-----
> From: Jean-Daniel Cryans <jd...@apache.org>
> To: user@hbase.apache.org
> Sent: Tue, Mar 29, 2011 5:29 pm
> Subject: Re: Export/Import and # of regions
>
>
> Yes but you'll start with a single region, instead of truncating you
>
> probably want instead to create a pre-split table.
>
>
>
> J-D
>
>
>
> On Tue, Mar 29, 2011 at 2:27 PM, Venkatesh <vr...@aol.com> wrote:
>
>>
>
>>
>
>>
>
>> Hi,
>
>> If I export existing table using Export MR job, truncate the table, increase
>
> region size, & do a Import
>
>> will it make use of the new region size?
>
>>
>
>> thanks
>
>> V
>
>>
>
>>
>
>>
>
>
>
>
Re: Export/Import and # of regions
Posted by Venkatesh <vr...@aol.com>.
Thanks J-D
We have way too much data & it won't fit in 1 region.Is Import smart enough create
reqd # of regions?
Cld u pl. elaborate on pre-split table creation? steps?
Reason I'm doing this exercise is reduce # of regions in our cluster (in the absence of additional hardware
> 25K regions on 20 node)
-----Original Message-----
From: Jean-Daniel Cryans <jd...@apache.org>
To: user@hbase.apache.org
Sent: Tue, Mar 29, 2011 5:29 pm
Subject: Re: Export/Import and # of regions
Yes but you'll start with a single region, instead of truncating you
probably want instead to create a pre-split table.
J-D
On Tue, Mar 29, 2011 at 2:27 PM, Venkatesh <vr...@aol.com> wrote:
>
>
>
> Hi,
> If I export existing table using Export MR job, truncate the table, increase
region size, & do a Import
> will it make use of the new region size?
>
> thanks
> V
>
>
>
Re: Export/Import and # of regions
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Yes but you'll start with a single region, instead of truncating you
probably want instead to create a pre-split table.
J-D
On Tue, Mar 29, 2011 at 2:27 PM, Venkatesh <vr...@aol.com> wrote:
>
>
>
> Hi,
> If I export existing table using Export MR job, truncate the table, increase region size, & do a Import
> will it make use of the new region size?
>
> thanks
> V
>
>
>