You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Jinsong Hu <ji...@hotmail.com> on 2010/09/01 20:28:12 UTC
truncate large table suggestion
Hi, Team:
I have noticed that the truncate/drop table with large amount of data
fails and actually corrupt the hbase. In the worse case, we can't even
create the table with the same name any more and I was forced to dump the
whole hbase records and recreate all tables again.
I noticed there is a rename command and I tried that one, it turns out it
actually works. One problem it causes is that the hbase master run out of
memory after I run the command. but after restart it, I am able to recreate
the table since the old table is renamed.
I then disable the renamed table and drop it, but it is not really
successful. the table disappears from the list command, but if I recreate a
table with the new name, the shell says it exist already. the regionserver
shows that they still have regions with the renamed table.
In the end, I have to write a utility to remove data from the renamed
table, I will try to see if this trick works although it is expensive.
Can the disable/truncate or drop for large table be fixed ?
Jimmy.
Re: truncate large table suggestion
Posted by Jinsong Hu <ji...@hotmail.com>.
unfortunately. I tried flush the table and disable, and then drop, and it
doesn't work.
I even wrote a utility to remove all records from the large table and then
do so,
and it doesn't work either. strangely. I looked at the web UI, and still see
many regions
even the number of rows in the table is 0.
the hbase that I use is CDH3 's latest version, which is hbase 0.89 version
released
in June 2010.
Jinsong
--------------------------------------------------
From: "Jean-Daniel Cryans" <jd...@apache.org>
Sent: Wednesday, September 01, 2010 11:36 AM
To: <us...@hbase.apache.org>
Subject: Re: truncate large table suggestion
> One trick is to pre- force flush the table. Also try out the new 0.89,
> it has 2 fixes regarding a race condition between the BaseScanner and
> the closing of regions. The release candidate is here
> http://people.apache.org/~jdcryans/hbase-0.89.20100830-candidate-1
>
> J-D
>
> On Wed, Sep 1, 2010 at 11:28 AM, Jinsong Hu <ji...@hotmail.com>
> wrote:
>> Hi, Team:
>> I have noticed that the truncate/drop table with large amount of data
>> fails
>> and actually corrupt the hbase. In the worse case, we can't even
>> create the table with the same name any more and I was forced to dump the
>> whole hbase records and recreate all tables again.
>> I noticed there is a rename command and I tried that one, it turns out
>> it
>> actually works. One problem it causes is that the hbase master run out of
>> memory after I run the command. but after restart it, I am able to
>> recreate
>> the table since the old table is renamed.
>> I then disable the renamed table and drop it, but it is not really
>> successful. the table disappears from the list command, but if I recreate
>> a
>> table with the new name, the shell says it exist already. the
>> regionserver
>> shows that they still have regions with the renamed table.
>> In the end, I have to write a utility to remove data from the renamed
>> table, I will try to see if this trick works although it is expensive.
>>
>>
>> Can the disable/truncate or drop for large table be fixed ?
>>
>> Jimmy.
>>
>
Re: truncate large table suggestion
Posted by Jean-Daniel Cryans <jd...@apache.org>.
That version doesn't have the fixes I referred to, and disabling large
tables will likely hit the race condition.
J-D
On Wed, Sep 1, 2010 at 2:47 PM, Jinsong Hu <ji...@hotmail.com> wrote:
> unfortunately. I tried flush the table and disable, and then drop, and it
> doesn't work.
> I even wrote a utility to remove all records from the large table and then
> do so,
> and it doesn't work either. strangely. I looked at the web UI, and still see
> many regions
> even the number of rows in the table is 0.
>
> the hbase that I use is CDH3 's latest version, which is hbase 0.89 version
> released
> in June 2010.
>
> Jinsong
>
>
> --------------------------------------------------
> From: "Jean-Daniel Cryans" <jd...@apache.org>
> Sent: Wednesday, September 01, 2010 11:36 AM
> To: <us...@hbase.apache.org>
> Subject: Re: truncate large table suggestion
>
>> One trick is to pre- force flush the table. Also try out the new 0.89,
>> it has 2 fixes regarding a race condition between the BaseScanner and
>> the closing of regions. The release candidate is here
>> http://people.apache.org/~jdcryans/hbase-0.89.20100830-candidate-1
>>
>> J-D
>>
>> On Wed, Sep 1, 2010 at 11:28 AM, Jinsong Hu <ji...@hotmail.com>
>> wrote:
>>>
>>> Hi, Team:
>>> I have noticed that the truncate/drop table with large amount of data
>>> fails
>>> and actually corrupt the hbase. In the worse case, we can't even
>>> create the table with the same name any more and I was forced to dump the
>>> whole hbase records and recreate all tables again.
>>> I noticed there is a rename command and I tried that one, it turns out
>>> it
>>> actually works. One problem it causes is that the hbase master run out of
>>> memory after I run the command. but after restart it, I am able to
>>> recreate
>>> the table since the old table is renamed.
>>> I then disable the renamed table and drop it, but it is not really
>>> successful. the table disappears from the list command, but if I recreate
>>> a
>>> table with the new name, the shell says it exist already. the
>>> regionserver
>>> shows that they still have regions with the renamed table.
>>> In the end, I have to write a utility to remove data from the renamed
>>> table, I will try to see if this trick works although it is expensive.
>>>
>>>
>>> Can the disable/truncate or drop for large table be fixed ?
>>>
>>> Jimmy.
>>>
>>
>
Re: truncate large table suggestion
Posted by Jean-Daniel Cryans <jd...@apache.org>.
One trick is to pre- force flush the table. Also try out the new 0.89,
it has 2 fixes regarding a race condition between the BaseScanner and
the closing of regions. The release candidate is here
http://people.apache.org/~jdcryans/hbase-0.89.20100830-candidate-1
J-D
On Wed, Sep 1, 2010 at 11:28 AM, Jinsong Hu <ji...@hotmail.com> wrote:
> Hi, Team:
> I have noticed that the truncate/drop table with large amount of data fails
> and actually corrupt the hbase. In the worse case, we can't even
> create the table with the same name any more and I was forced to dump the
> whole hbase records and recreate all tables again.
> I noticed there is a rename command and I tried that one, it turns out it
> actually works. One problem it causes is that the hbase master run out of
> memory after I run the command. but after restart it, I am able to recreate
> the table since the old table is renamed.
> I then disable the renamed table and drop it, but it is not really
> successful. the table disappears from the list command, but if I recreate a
> table with the new name, the shell says it exist already. the regionserver
> shows that they still have regions with the renamed table.
> In the end, I have to write a utility to remove data from the renamed
> table, I will try to see if this trick works although it is expensive.
>
>
> Can the disable/truncate or drop for large table be fixed ?
>
> Jimmy.
>