[HBASE] [Freeing Up Region and/or merging region]

[Rohan Rai <ro...@inmobi.com>]

On suggestion by JDC

Creating a separate thread

On a smaller cluster pre allocating a certain memory to region leads to
continuous increase in heap space usage
with increasing usage.

Hence clean up of data is performed by me.

But region having no data does not get de allocated automatically and it
still occupies the preallocated memory.

I was a looking for a clean mechanism to achieve that.


Either merging two continuous empty region or deleting a region cleanly.

Unfortunately my efforts to mess directly with metadata and cleaning up
of region dirs on cluster
has resulted in my tables getting corrupt.

Basically any way to cleanly get back those precious memory.

Regards
Rohan




________________________________

The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.

Re: [HBASE] [Freeing Up Region and/or merging region]

[Jean-Daniel Cryans <jd...@apache.org>]

(thanks for the new thread!)

So a region itself doesn't use a lot of memory (plus if it's empty),
it's really just a few objects at most.

On the problem of merging regions, I've been spending this whole week
working on a tool to exactly that in an online fashion (there's
already a tool to do it offline). I have something that works right
now (we were able to shed off 5000 regions already) but it's
unreliable facing failures. Add yourself as a Watcher on the following
jira to be notified when I add the script:
https://issues.apache.org/jira/browse/HBASE-1621

J-D

On Fri, Aug 26, 2011 at 10:27 AM, Rohan Rai <ro...@inmobi.com> wrote:
> On suggestion by JDC
>
> Creating a separate thread
>
> On a smaller cluster pre allocating a certain memory to region leads to
> continuous increase in heap space usage
> with increasing usage.
>
> Hence clean up of data is performed by me.
>
> But region having no data does not get de allocated automatically and it
> still occupies the preallocated memory.
>
> I was a looking for a clean mechanism to achieve that.
>
>
> Either merging two continuous empty region or deleting a region cleanly.
>
> Unfortunately my efforts to mess directly with metadata and cleaning up
> of region dirs on cluster
> has resulted in my tables getting corrupt.
>
> Basically any way to cleanly get back those precious memory.
>
> Regards
> Rohan
>
>
>
>
> ________________________________
>
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify us
> immediately by responding to this email and then delete it from your system.
> The firm is neither liable for the proper and complete transmission of the
> information contained in this communication nor for any delay in its
> receipt.
>