You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by rahul malviya <ma...@gmail.com> on 2015/05/10 00:05:45 UTC
How to Restore the block locality of a RegionServer ?
Hi,
My HBase cluster went through a rough patch recently where lot of region
server started dying because of sudden increase in amount of data being
funneled to the HBase cluster and we have to place a auto start script for
regionservers.
After this all my data locality is lost which does not seems to recover
even after compaction. This has degraded the performance by a factor of 4.
So I want to know is their a way to restore the data locality of my HBase
cluster.
I am using hbase-0.98.6-cdh5.2.0.
Thanks,
Rahul
Re: How to Restore the block locality of a RegionServer ?
Posted by rahul malviya <ma...@gmail.com>.
Data locality if completely restored after major compaction.
Thanks everyone.
Rahul
On Sat, May 9, 2015 at 9:36 PM, rahul malviya <ma...@gmail.com>
wrote:
> Thanks for suggestion. I will try this and update the thread if the data
> locality is restored.
>
> Thanks,
> Rahul
>
> On Sat, May 9, 2015 at 6:36 PM, Dave Latham <la...@davelink.net> wrote:
>
>> Major compactions will fix locality, so long as there is space on the
>> local data nodes and they actually happen. Also, if there is already
>> only a single HFile in a store, major compaction may be skipped.
>> Newer versions of hbase have a parameter
>> hbase.hstore.min.locality.to.skip.major.compact that you can set to
>> perform major compaction if locality is below some threshold, even if
>> there is only one store file. (See HBASE-11195).
>>
>> Dave
>>
>> On Sat, May 9, 2015 at 5:31 PM, Bryan Beaudreault
>> <bb...@hubspot.com> wrote:
>> > Major compactions will restore locality to the cluster.
>> >
>> > On Sat, May 9, 2015 at 3:36 PM, Michael Segel <
>> michael_segel@hotmail.com>
>> > wrote:
>> >
>> >> First, understand why you had to create an ‘auto restart’ script.
>> >>
>> >> Taking down HBase completely (probably including zookeeper) and do a
>> full
>> >> restart would probably fix the issue of data locality.
>> >>
>> >>
>> >> > On May 9, 2015, at 5:05 PM, rahul malviya <
>> malviyarahul2001@gmail.com>
>> >> wrote:
>> >> >
>> >> > Hi,
>> >> >
>> >> > My HBase cluster went through a rough patch recently where lot of
>> region
>> >> > server started dying because of sudden increase in amount of data
>> being
>> >> > funneled to the HBase cluster and we have to place a auto start
>> script
>> >> for
>> >> > regionservers.
>> >> >
>> >> > After this all my data locality is lost which does not seems to
>> recover
>> >> > even after compaction. This has degraded the performance by a factor
>> of
>> >> 4.
>> >> > So I want to know is their a way to restore the data locality of my
>> HBase
>> >> > cluster.
>> >> >
>> >> > I am using hbase-0.98.6-cdh5.2.0.
>> >> >
>> >> > Thanks,
>> >> > Rahul
>> >>
>> >> The opinions expressed here are mine, while they may reflect a
>> cognitive
>> >> thought, that is purely accidental.
>> >> Use at your own risk.
>> >> Michael Segel
>> >> michael_segel (AT) hotmail.com
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>>
>
>
Re: How to Restore the block locality of a RegionServer ?
Posted by rahul malviya <ma...@gmail.com>.
Thanks for suggestion. I will try this and update the thread if the data
locality is restored.
Thanks,
Rahul
On Sat, May 9, 2015 at 6:36 PM, Dave Latham <la...@davelink.net> wrote:
> Major compactions will fix locality, so long as there is space on the
> local data nodes and they actually happen. Also, if there is already
> only a single HFile in a store, major compaction may be skipped.
> Newer versions of hbase have a parameter
> hbase.hstore.min.locality.to.skip.major.compact that you can set to
> perform major compaction if locality is below some threshold, even if
> there is only one store file. (See HBASE-11195).
>
> Dave
>
> On Sat, May 9, 2015 at 5:31 PM, Bryan Beaudreault
> <bb...@hubspot.com> wrote:
> > Major compactions will restore locality to the cluster.
> >
> > On Sat, May 9, 2015 at 3:36 PM, Michael Segel <michael_segel@hotmail.com
> >
> > wrote:
> >
> >> First, understand why you had to create an ‘auto restart’ script.
> >>
> >> Taking down HBase completely (probably including zookeeper) and do a
> full
> >> restart would probably fix the issue of data locality.
> >>
> >>
> >> > On May 9, 2015, at 5:05 PM, rahul malviya <malviyarahul2001@gmail.com
> >
> >> wrote:
> >> >
> >> > Hi,
> >> >
> >> > My HBase cluster went through a rough patch recently where lot of
> region
> >> > server started dying because of sudden increase in amount of data
> being
> >> > funneled to the HBase cluster and we have to place a auto start script
> >> for
> >> > regionservers.
> >> >
> >> > After this all my data locality is lost which does not seems to
> recover
> >> > even after compaction. This has degraded the performance by a factor
> of
> >> 4.
> >> > So I want to know is their a way to restore the data locality of my
> HBase
> >> > cluster.
> >> >
> >> > I am using hbase-0.98.6-cdh5.2.0.
> >> >
> >> > Thanks,
> >> > Rahul
> >>
> >> The opinions expressed here are mine, while they may reflect a cognitive
> >> thought, that is purely accidental.
> >> Use at your own risk.
> >> Michael Segel
> >> michael_segel (AT) hotmail.com
> >>
> >>
> >>
> >>
> >>
> >>
>
Re: How to Restore the block locality of a RegionServer ?
Posted by Dave Latham <la...@davelink.net>.
Major compactions will fix locality, so long as there is space on the
local data nodes and they actually happen. Also, if there is already
only a single HFile in a store, major compaction may be skipped.
Newer versions of hbase have a parameter
hbase.hstore.min.locality.to.skip.major.compact that you can set to
perform major compaction if locality is below some threshold, even if
there is only one store file. (See HBASE-11195).
Dave
On Sat, May 9, 2015 at 5:31 PM, Bryan Beaudreault
<bb...@hubspot.com> wrote:
> Major compactions will restore locality to the cluster.
>
> On Sat, May 9, 2015 at 3:36 PM, Michael Segel <mi...@hotmail.com>
> wrote:
>
>> First, understand why you had to create an ‘auto restart’ script.
>>
>> Taking down HBase completely (probably including zookeeper) and do a full
>> restart would probably fix the issue of data locality.
>>
>>
>> > On May 9, 2015, at 5:05 PM, rahul malviya <ma...@gmail.com>
>> wrote:
>> >
>> > Hi,
>> >
>> > My HBase cluster went through a rough patch recently where lot of region
>> > server started dying because of sudden increase in amount of data being
>> > funneled to the HBase cluster and we have to place a auto start script
>> for
>> > regionservers.
>> >
>> > After this all my data locality is lost which does not seems to recover
>> > even after compaction. This has degraded the performance by a factor of
>> 4.
>> > So I want to know is their a way to restore the data locality of my HBase
>> > cluster.
>> >
>> > I am using hbase-0.98.6-cdh5.2.0.
>> >
>> > Thanks,
>> > Rahul
>>
>> The opinions expressed here are mine, while they may reflect a cognitive
>> thought, that is purely accidental.
>> Use at your own risk.
>> Michael Segel
>> michael_segel (AT) hotmail.com
>>
>>
>>
>>
>>
>>
Re: How to Restore the block locality of a RegionServer ?
Posted by Bryan Beaudreault <bb...@hubspot.com>.
Major compactions will restore locality to the cluster.
On Sat, May 9, 2015 at 3:36 PM, Michael Segel <mi...@hotmail.com>
wrote:
> First, understand why you had to create an ‘auto restart’ script.
>
> Taking down HBase completely (probably including zookeeper) and do a full
> restart would probably fix the issue of data locality.
>
>
> > On May 9, 2015, at 5:05 PM, rahul malviya <ma...@gmail.com>
> wrote:
> >
> > Hi,
> >
> > My HBase cluster went through a rough patch recently where lot of region
> > server started dying because of sudden increase in amount of data being
> > funneled to the HBase cluster and we have to place a auto start script
> for
> > regionservers.
> >
> > After this all my data locality is lost which does not seems to recover
> > even after compaction. This has degraded the performance by a factor of
> 4.
> > So I want to know is their a way to restore the data locality of my HBase
> > cluster.
> >
> > I am using hbase-0.98.6-cdh5.2.0.
> >
> > Thanks,
> > Rahul
>
> The opinions expressed here are mine, while they may reflect a cognitive
> thought, that is purely accidental.
> Use at your own risk.
> Michael Segel
> michael_segel (AT) hotmail.com
>
>
>
>
>
>
Re: How to Restore the block locality of a RegionServer ?
Posted by Michael Segel <mi...@hotmail.com>.
First, understand why you had to create an ‘auto restart’ script.
Taking down HBase completely (probably including zookeeper) and do a full restart would probably fix the issue of data locality.
> On May 9, 2015, at 5:05 PM, rahul malviya <ma...@gmail.com> wrote:
>
> Hi,
>
> My HBase cluster went through a rough patch recently where lot of region
> server started dying because of sudden increase in amount of data being
> funneled to the HBase cluster and we have to place a auto start script for
> regionservers.
>
> After this all my data locality is lost which does not seems to recover
> even after compaction. This has degraded the performance by a factor of 4.
> So I want to know is their a way to restore the data locality of my HBase
> cluster.
>
> I am using hbase-0.98.6-cdh5.2.0.
>
> Thanks,
> Rahul
The opinions expressed here are mine, while they may reflect a cognitive thought, that is purely accidental.
Use at your own risk.
Michael Segel
michael_segel (AT) hotmail.com