HBASE-7846 : is it safe to use on 0.94.4 ?

[Viral Bajaria <vi...@gmail.com>]

Hi,

Just wanted to check if it's safe to use the JIRA mentioned in the subject
i.e. https://issues.apache.org/jira/browse/HBASE-7846

Thanks,
Viral

Re: HBASE-7846 : is it safe to use on 0.94.4 ?

[Viral Bajaria <vi...@gmail.com>]

Found this while going through the online merge jira...
https://issues.apache.org/jira/browse/HBASE-8217

The comments were interesting and I as an user would agree to the fact that
supplying a patch is good and it's on me to decide whether I should use it
or not. The core committee obviously is pushing for 0.96 but given that we
have clusters running older versions, I feel this tool would have helped my
current case immensely. I am all for disabling a region and then doing a
merge. To shutdown the whole cluster and put things in maintenance mode is
something that I would love to avoid. On a side note, it will help me avoid
staying up all night :-)

I will give this patch a try and provide my feedback. Maybe it might help
some other folks out there.

Thanks,
Viral

On Wed, Jul 3, 2013 at 4:21 AM, Ted Yu <yu...@gmail.com> wrote:

> Would online merge help (https://issues.apache.org/jira/browse/HBASE-7403)
> ?
>
> The feature is not in 0.94 though.
>
> Cheers
>

Re: HBASE-7846 : is it safe to use on 0.94.4 ?

[Ted Yu <yu...@gmail.com>]

Would online merge help (https://issues.apache.org/jira/browse/HBASE-7403) ?

The feature is not in 0.94 though.

Cheers

On Wed, Jul 3, 2013 at 3:58 AM, Viral Bajaria <vi...@gmail.com>wrote:

> I ended up writing a tool which helps merge the table regions into a target
> # of regions. For example if you want to go from N --> N/8, then the tool
> figures out the grouping and merges them in one pass. I will put it up in a
> github repo soon and share it here.
>
> The sad part of this approach is the downtime required. It's taking over 2
> hours on my test cluster which is less than 30% of the production table
> size. In absolute value, the table has over 100 regions and I am merging it
> down to 20 or so and it has 20GB of compressed (lzo) data.
>
> Is there a better way to achieve this ? If not, should I open a JIRA to
> explore the chance of running the Merge util on a disabled table rather
> than having to shutdown the entire cluster ? It will also be great to
> ignore compaction when merging the table and then do it as a later step
> since that can happen online. Just throwing some ideas here.
>
> Thanks,
> Viral
>
> On Tue, Jul 2, 2013 at 11:22 AM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
> > Hi Viral,
> >
> > It was working fine when I did it. I'm not sure you can still apply it
> > to a recent HBase version because some code change. But I can take a
> > look to see if I can rebase it...
> >
> > JM
> >
>

Re: HBASE-7846 : is it safe to use on 0.94.4 ?

[Viral Bajaria <vi...@gmail.com>]

I ended up writing a tool which helps merge the table regions into a target
# of regions. For example if you want to go from N --> N/8, then the tool
figures out the grouping and merges them in one pass. I will put it up in a
github repo soon and share it here.

The sad part of this approach is the downtime required. It's taking over 2
hours on my test cluster which is less than 30% of the production table
size. In absolute value, the table has over 100 regions and I am merging it
down to 20 or so and it has 20GB of compressed (lzo) data.

Is there a better way to achieve this ? If not, should I open a JIRA to
explore the chance of running the Merge util on a disabled table rather
than having to shutdown the entire cluster ? It will also be great to
ignore compaction when merging the table and then do it as a later step
since that can happen online. Just throwing some ideas here.

Thanks,
Viral

On Tue, Jul 2, 2013 at 11:22 AM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> Hi Viral,
>
> It was working fine when I did it. I'm not sure you can still apply it
> to a recent HBase version because some code change. But I can take a
> look to see if I can rebase it...
>
> JM
>

Re: HBASE-7846 : is it safe to use on 0.94.4 ?

[Jean-Marc Spaggiari <je...@spaggiari.org>]

Hi Viral,

It was working fine when I did it. I'm not sure you can still apply it
to a recent HBase version because some code change. But I can take a
look to see if I can rebase it...

JM

2013/7/1 Viral Bajaria <vi...@gmail.com>:
> Hi,
>
> Just wanted to check if it's safe to use the JIRA mentioned in the subject
> i.e. https://issues.apache.org/jira/browse/HBASE-7846
>
> Thanks,
> Viral