You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by liushaohui <li...@xiaomi.com> on 2013/08/05 13:28:35 UTC
how to clean all data in hbase table without truncate
hi all:
In our shared secure hbase cluster, users often wants to clean all
tested data in their tables. Currently, hbase supports the
truncate method , but its impl is based on disable table-> delete table
-> create table.
There are two drawbacks in this impl.
1, Region split info. Truncate creates a new table with just a region.
It loses the region split info, which is not suitable for pre-split tables.
2, Table ACLs. In the deletion of table, the tables' acls will be
deleted too. So every truncate need to be operated by hbase admin.
Is there any other effective method to clean all data of a table?
What about implement a real truncate method in hmaster which keeps
region split info and table acls?
Thanks
- liushaohui
Re: how to clean all data in hbase table without truncate
Posted by liushaohui <li...@xiaomi.com>.
Ok. Thanks, Matteo
- liushaohui
On 08/06/2013 09:08 PM, Matteo Bertozzi wrote:
> let me post my draft patch, and feel free to nuke it. but there's a lot of
> boring stuff code that you can use (all the observer interfaces to add & co)
>
> Matteo
>
>
>
> On Tue, Aug 6, 2013 at 10:56 AM, liushaohui <li...@xiaomi.com> wrote:
>
>> Thanks, Jean-Marc.
>>
>> There is truncate_preserve in HBASE-5525. But it drops the acls info.
>>
>> I will try to impl truncate_preserver_acl by add a new api in hmaster.
>>
>> HBASE-8332 discussed this problem, too. But no implement.
>>
>>
>>
>> On 08/05/2013 08:03 PM, Jean-Marc Spaggiari wrote:
>>
>>> truncate_preserve (from shell) will still do the disable, but it will not
>>> drop the regions split. Can you try and see if it keep he ACLs too? If
>>> not,
>>> might be a good idea to do something like "truncate_preserve_acl" too...
>>>
>>> JM
>>>
>>> 2013/8/5 liushaohui <li...@xiaomi.com>
>>>
>>> hi all:
>>>> In our shared secure hbase cluster, users often wants to clean all
>>>> tested data in their tables. Currently, hbase supports the
>>>>
>>>> truncate method , but its impl is based on disable table-> delete table
>>>> ->
>>>> create table.
>>>>
>>>> There are two drawbacks in this impl.
>>>>
>>>> 1, Region split info. Truncate creates a new table with just a region. It
>>>> loses the region split info, which is not suitable for pre-split tables.
>>>>
>>>> 2, Table ACLs. In the deletion of table, the tables' acls will be
>>>> deleted
>>>> too. So every truncate need to be operated by hbase admin.
>>>>
>>>>
>>>> Is there any other effective method to clean all data of a table?
>>>>
>>>> What about implement a real truncate method in hmaster which keeps region
>>>> split info and table acls?
>>>>
>>>> Thanks
>>>>
>>>>
>>>> - liushaohui
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
Re: how to clean all data in hbase table without truncate
Posted by Matteo Bertozzi <th...@gmail.com>.
let me post my draft patch, and feel free to nuke it. but there's a lot of
boring stuff code that you can use (all the observer interfaces to add & co)
Matteo
On Tue, Aug 6, 2013 at 10:56 AM, liushaohui <li...@xiaomi.com> wrote:
> Thanks, Jean-Marc.
>
> There is truncate_preserve in HBASE-5525. But it drops the acls info.
>
> I will try to impl truncate_preserver_acl by add a new api in hmaster.
>
> HBASE-8332 discussed this problem, too. But no implement.
>
>
>
> On 08/05/2013 08:03 PM, Jean-Marc Spaggiari wrote:
>
>> truncate_preserve (from shell) will still do the disable, but it will not
>> drop the regions split. Can you try and see if it keep he ACLs too? If
>> not,
>> might be a good idea to do something like "truncate_preserve_acl" too...
>>
>> JM
>>
>> 2013/8/5 liushaohui <li...@xiaomi.com>
>>
>> hi all:
>>>
>>> In our shared secure hbase cluster, users often wants to clean all
>>> tested data in their tables. Currently, hbase supports the
>>>
>>> truncate method , but its impl is based on disable table-> delete table
>>> ->
>>> create table.
>>>
>>> There are two drawbacks in this impl.
>>>
>>> 1, Region split info. Truncate creates a new table with just a region. It
>>> loses the region split info, which is not suitable for pre-split tables.
>>>
>>> 2, Table ACLs. In the deletion of table, the tables' acls will be
>>> deleted
>>> too. So every truncate need to be operated by hbase admin.
>>>
>>>
>>> Is there any other effective method to clean all data of a table?
>>>
>>> What about implement a real truncate method in hmaster which keeps region
>>> split info and table acls?
>>>
>>> Thanks
>>>
>>>
>>> - liushaohui
>>>
>>>
>>>
>>>
>>>
>>>
>
Re: how to clean all data in hbase table without truncate
Posted by liushaohui <li...@xiaomi.com>.
Thanks, Jean-Marc.
There is truncate_preserve in HBASE-5525. But it drops the acls info.
I will try to impl truncate_preserver_acl by add a new api in hmaster.
HBASE-8332 discussed this problem, too. But no implement.
On 08/05/2013 08:03 PM, Jean-Marc Spaggiari wrote:
> truncate_preserve (from shell) will still do the disable, but it will not
> drop the regions split. Can you try and see if it keep he ACLs too? If not,
> might be a good idea to do something like "truncate_preserve_acl" too...
>
> JM
>
> 2013/8/5 liushaohui <li...@xiaomi.com>
>
>> hi all:
>>
>> In our shared secure hbase cluster, users often wants to clean all
>> tested data in their tables. Currently, hbase supports the
>>
>> truncate method , but its impl is based on disable table-> delete table ->
>> create table.
>>
>> There are two drawbacks in this impl.
>>
>> 1, Region split info. Truncate creates a new table with just a region. It
>> loses the region split info, which is not suitable for pre-split tables.
>>
>> 2, Table ACLs. In the deletion of table, the tables' acls will be deleted
>> too. So every truncate need to be operated by hbase admin.
>>
>>
>> Is there any other effective method to clean all data of a table?
>>
>> What about implement a real truncate method in hmaster which keeps region
>> split info and table acls?
>>
>> Thanks
>>
>>
>> - liushaohui
>>
>>
>>
>>
>>
Re: how to clean all data in hbase table without truncate
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
truncate_preserve (from shell) will still do the disable, but it will not
drop the regions split. Can you try and see if it keep he ACLs too? If not,
might be a good idea to do something like "truncate_preserve_acl" too...
JM
2013/8/5 liushaohui <li...@xiaomi.com>
> hi all:
>
> In our shared secure hbase cluster, users often wants to clean all
> tested data in their tables. Currently, hbase supports the
>
> truncate method , but its impl is based on disable table-> delete table ->
> create table.
>
> There are two drawbacks in this impl.
>
> 1, Region split info. Truncate creates a new table with just a region. It
> loses the region split info, which is not suitable for pre-split tables.
>
> 2, Table ACLs. In the deletion of table, the tables' acls will be deleted
> too. So every truncate need to be operated by hbase admin.
>
>
> Is there any other effective method to clean all data of a table?
>
> What about implement a real truncate method in hmaster which keeps region
> split info and table acls?
>
> Thanks
>
>
> - liushaohui
>
>
>
>
>