You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Ted Yu <yu...@gmail.com> on 2010/03/06 00:21:19 UTC
Re: WrongRegionException
I want to hear other people's experience handling WrongRegionException,
especially in production.
Thanks
On Wed, Feb 17, 2010 at 1:54 PM, Zhenyu Zhong <zh...@gmail.com>wrote:
> Stack,
>
> As you described, I found a hole after the region with problem.
> After the problem region(3d9d1175a7f8bf861bf75638bb1eb231,
> 3ec1a5a1d8753d3b073a831969a70571),
> the following region has start and end key
> (402defb47815ce08e4d73d5be7f33d9f, 40dadd57c93ba2a75f26438ef2a7b244)
>
> Is there a way to get the missing region back? or is there a way to split
> the problem region so that it can find the appropriate endkey?
>
> Thanks a lot.
> zhenyu
>
>
> On Wed, Feb 17, 2010 at 2:05 PM, Stack <st...@duboce.net> wrote:
>
> > On Wed, Feb 17, 2010 at 8:33 AM, Zhenyu Zhong <zh...@gmail.com>
> > wrote:
> > >
> > > Is it possible that a region gets split but the status of the parent
> > region
> > > doesn't get update, either does the child region?
> > >
> >
> > WRE is usually indicative of something gone amiss, perhaps something
> > like you describe above. Can you hunt in your master log for regions
> > that had 3d9d1175a7f8bf861bf75638bb1eb231 for a startkey? Its looking
> > like a split happened but only the lower-half made it into the .META.
> > See if you can see where the parent had been hosted. See if you can
> > find the split event. Is there a hole in your .META. just after this
> > region; i.e. the end key does not match the start key of next region?
> >
> > St.Ack
> >
>
Re: WrongRegionException
Posted by Ted Yu <yu...@gmail.com>.
I noticed there are more than 1 regions whose start key is empty. And for
some End Key, such as
com.hoovers.www\x2Fcompany\x2FCenter_Tech_Comm_Seguridad_Efectos_Y_Eq\x2Fyjrjfjrtk-1.html,
I don't see a Start Key with the same value.
Here is some of the regions reported by :60030/regionserver.jsp (I tried to
format better):
rules,com.hoovers.www\x2Fcompanyindex\x2FMaryland\x2FUpper_Marlboro\x2FFurniture_Mfg-1.html,1267146549075
474580346
com.hoovers.www\x2Fcompanyindex\x2FMaryland\x2FUpper_Marlboro\x2FFurniture_Mfg-1.html
com.hoovers.www\x2Fcompanyindex\x2FNevada\x2FMarketing_and_Advertising_Agencies-6.html
stores=2, storefiles=1, storefileSizeMB=131,
memstoreSizeMB=0, storefileIndexSizeMB=0
rules,com.hoovers.www\x2Fcompanyindex\x2FOhio\x2FSandusky\x2FMining_Support_Activities-1.html,1267146507504
1308878326
com.hoovers.www\x2Fcompanyindex\x2FOhio\x2FSandusky\x2FMining_Support_Activities-1.html
com.hoovers.www\x2Fcompanyindex\x2FTexas\x2FLewisville\x2FProperty_Escrow_and_Title-1.html
stores=2, storefiles=1, storefileSizeMB=153,
memstoreSizeMB=0, storefileIndexSizeMB=0
ruletable,,1267831180107
5981787 [No Start Key]
com.hoovers.www\x2Fcompany\x2FCenter_Tech_Comm_Seguridad_Efectos_Y_Eq\x2Fyjrjfjrtk-1.html
stores=2, storefiles=2, storefileSizeMB=85,
memstoreSizeMB=0, storefileIndexSizeMB=0
ruletable,com.hoovers.www\x2Fcompanyindex\x2FAlaska\x2FKarluk-1.html,1267860217595
1879191502
com.hoovers.www\x2Fcompanyindex\x2FAlaska\x2FKarluk-1.html
com.hoovers.www\x2Fcompanyindex\x2FGeorgia\x2FWarner_Robins\x2FComputing_and_Information_Technology-1.html
stores=2, storefiles=2, storefileSizeMB=109,
memstoreSizeMB=0, storefileIndexSizeMB=0
ruletable,com.hoovers.www\x2Fcompanyindex\x2FGeorgia\x2FWarner_Robins\x2FComputing_and_Information_Technology-1.html,1267860283574
1208738952
com.hoovers.www\x2Fcompanyindex\x2FGeorgia\x2FWarner_Robins\x2FComputing_and_Information_Technology-1.html
com.hoovers.www\x2Fcompanyindex\x2FMichigan\x2FBattle_Creek\x2FMarketing_and_Advertising_Services-1.html
stores=2, storefiles=2, storefileSizeMB=113, memstoreSizeMB=0,
storefileIndexSizeMB=0
ruletable,com.hoovers.www\x2Fcompanyindex\x2FMichigan\x2FBattle_Creek\x2FMarketing_and_Advertising_Services-1.html,1267860376861
1134018253
com.hoovers.www\x2Fcompanyindex\x2FMichigan\x2FBattle_Creek\x2FMarketing_and_Advertising_Services-1.html
com.hoovers.www\x2Fcompanyindex\x2FNew_Mexico\x2FVado\x2FBroadcasting_Industry-1.html
stores=2, storefiles=2, storefileSizeMB=98, memstoreSizeMB=0,
storefileIndexSizeMB=0
Besides dropping ruletable, what choice(s) do I have ?
On Fri, Mar 5, 2010 at 3:21 PM, Ted Yu <yu...@gmail.com> wrote:
> I want to hear other people's experience handling WrongRegionException,
> especially in production.
> Thanks
>
>
> On Wed, Feb 17, 2010 at 1:54 PM, Zhenyu Zhong <zh...@gmail.com>wrote:
>
>> Stack,
>>
>> As you described, I found a hole after the region with problem.
>> After the problem region(3d9d1175a7f8bf861bf75638bb1eb231,
>> 3ec1a5a1d8753d3b073a831969a70571),
>> the following region has start and end key
>> (402defb47815ce08e4d73d5be7f33d9f, 40dadd57c93ba2a75f26438ef2a7b244)
>>
>> Is there a way to get the missing region back? or is there a way to split
>> the problem region so that it can find the appropriate endkey?
>>
>> Thanks a lot.
>> zhenyu
>>
>>
>> On Wed, Feb 17, 2010 at 2:05 PM, Stack <st...@duboce.net> wrote:
>>
>> > On Wed, Feb 17, 2010 at 8:33 AM, Zhenyu Zhong <zh...@gmail.com>
>> > wrote:
>> > >
>> > > Is it possible that a region gets split but the status of the parent
>> > region
>> > > doesn't get update, either does the child region?
>> > >
>> >
>> > WRE is usually indicative of something gone amiss, perhaps something
>> > like you describe above. Can you hunt in your master log for regions
>> > that had 3d9d1175a7f8bf861bf75638bb1eb231 for a startkey? Its looking
>> > like a split happened but only the lower-half made it into the .META.
>> > See if you can see where the parent had been hosted. See if you can
>> > find the split event. Is there a hole in your .META. just after this
>> > region; i.e. the end key does not match the start key of next region?
>> >
>> > St.Ack
>> >
>>
>
>