You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "henry.kim" <he...@gmail.com> on 2012/08/09 12:36:12 UTC
how to delete a zombie table permanently?
hi, hbase users.
I got a problem when I am testing coprocessors which is released at base 0.92.1.
here is the hbase shell outputs
----------------
hbase(main):001:0> truncate 'blog'
Truncating 'blog' table (it may take a while):
ERROR: Unknown table blog!
Here is some help for this command:
Disables, drops and recreates the specified table.
hbase(main):002:0> list
TABLE
blog
counter
sidx
3 row(s) in 15.3610 seconds
----------------
yes, table 'blog' is a zombie I think.
there was some big stress from coprocessors which is using RegionServerObserver interfaces.
how could I fix this situation?
Re: how to delete a zombie table permanently?
Posted by anil gupta <an...@gmail.com>.
Hi Henry,
I would suggest you to try out 'hbck' utility of Hbase for dealing with
corrupt metadata in HBase. It's pretty effective and clean. If you use that
then you wont have to deal with manual cleanup most of the time.
Run: 'hbase hbck -help' command for more details.
Here is a useful link of HBase wiki:
http://hbase.apache.org/book/hbck.in.depth.html
HTH,
Anil Gupta
On Thu, Aug 9, 2012 at 5:00 AM, Jean-Marc Spaggiari <jean-marc@spaggiari.org
> wrote:
> Hi Henry,
>
> I faced the same issue not so long time ago...
>
> Can you take a look at what you have in zookeeper under /hbase/table ?
>
> If you table is there, that's why you see it on the list. Simply
> remove it from zookeeper.
>
> You can also take a look there:
> https://issues.apache.org/jira/browse/HBASE-6294
>
> JM
>
> 2012/8/9, henry.kim <he...@gmail.com>:
> > hi, hbase users.
> >
> > I got a problem when I am testing coprocessors which is released at base
> > 0.92.1.
> >
> > here is the hbase shell outputs
> >
> > ----------------
> > hbase(main):001:0> truncate 'blog'
> > Truncating 'blog' table (it may take a while):
> >
> > ERROR: Unknown table blog!
> >
> > Here is some help for this command:
> > Disables, drops and recreates the specified table.
> >
> >
> > hbase(main):002:0> list
> > TABLE
> >
> >
> > blog
> >
> >
> > counter
> >
> >
> > sidx
> >
> >
> > 3 row(s) in 15.3610 seconds
> > ----------------
> >
> > yes, table 'blog' is a zombie I think.
> >
> > there was some big stress from coprocessors which is using
> > RegionServerObserver interfaces.
> >
> > how could I fix this situation?
>
--
Thanks & Regards,
Anil Gupta
Re: how to delete a zombie table permanently?
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
You have deleted all the entries...
Only blog under table should have been deleted.
As proposed, have you tried to restart everything, including zookeeper?
Le 10 août 2012 08:52, "GMAIL(henry)" <he...@gmail.com> a écrit :
> yes, it's a same situation for me.
>
> ----
> iPhone에서 보냄
>
> 2012. 8. 10. 20:22 Miguel Costa <mi...@telecom.pt> 작성:
>
> > after the deletion did you restart HBase Master and region servers?
> >
> > I Had the same problem a while ago and solve it by deleted from zk and
> > then restart.
> >
> > Miguel Costa
> >
> > On 08/10/2012 09:33 AM, Henry JunYoung KIM wrote:
> >> Hi, JM.
> >>
> >> I followed procedures you mentioned.
> >>
> >> here is it.
> >>
> >> from zookeeper, I deleted it
> >>
> >> ---------------
> >> $] rmr /hbase/table/*
> >> $] ls /hbase/table
> >> Node does not exist: /hbase/table
> >> ---------------
> >>
> >> and then, I checked my hbase cluster.
> >>
> >> ---------------
> >> $> hbase hbck
> >> ,,,,
> >> Number of empty REGIONINFO_QUALIFIER rows in .META.: 0
> >> Exception in thread "main" java.lang.NullPointerException
> >> at
> org.apache.hadoop.hbase.zookeeper.ZKTable.getDisabledOrDisablingTables(ZKTable.java:397)
> >> at
> org.apache.hadoop.hbase.util.HBaseFsck$1.connect(HBaseFsck.java:996)
> >> at
> org.apache.hadoop.hbase.util.HBaseFsck$1.connect(HBaseFsck.java:991)
> >> at
> org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:360)
> >> at
> org.apache.hadoop.hbase.util.HBaseFsck.loadDisabledTables(HBaseFsck.java:991)
> >> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:360)
> >> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:383)
> >> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3121)
> >> ---------------
> >>
> >> OMG.
> >>
> >> ok, let's refresh master's cache which is from zk.
> >>
> >> ---------------
> >> $> hadoop-hbase-master restart
> >> $> hbase hbck
> >> …
> >> ERROR: Found inconsistency in table blog
> >> Summary:
> >> -ROOT- is okay.
> >> Number of regions: 1
> >> Deployed on: search-ddm-test10,60020,1344586183254
> >> .META. is okay.
> >> Number of regions: 1
> >> Deployed on: search-ddm-test6,60020,1344586183295
> >> Table blog is inconsistent.
> >> Number of regions: 3
> >> Deployed on: search-ddm-test5,60020,1344586183269
> search-ddm-test8,60020,1344586183257 search-ddm-test9,60020,1344586183260
> >> 8 inconsistencies detected.
> >> Status: INCONSISTENT
> >> ---------------
> >>
> >> now, it shows up the different situation for me. (thanks)
> >>
> >> from hbase shell, I could check the existence of the tables I want to
> delete.
> >>
> >> ---------------
> >> hbase(main):002:0> list
> >> TABLE
> >> blog
> >> sidx
> >> 2 row(s) in 0.0180 seconds
> >> ---------------
> >> try to fix it again.
> >>
> >> ---------------
> >> $> hbase hbck -fix
> >> …
> >> Number of empty REGIONINFO_QUALIFIER rows in .META.: 0
> >> 12/08/10 17:29:54 DEBUG util.HBaseFsck: There are 2 region info entries
> >> Summary:
> >> -ROOT- is okay.
> >> Number of regions: 1
> >> Deployed on: search-ddm-test10,60020,1344586183254
> >> .META. is okay.
> >> Number of regions: 1
> >> Deployed on: search-ddm-test6,60020,1344586183295
> >> 0 inconsistencies detected.
> >> Status: OK
> >> ---------------
> >>
> >> hbase shell says also
> >>
> >> ---------------
> >> hbase(main):009:0> list
> >> TABLE
> >> 0 row(s) in 0.0160 seconds
> >> ---------------
> >>
> >>
> >> not easy procedures to fix it for me.
> >> anyway, finally, in the conclusion, I fixed it.
> >>
> >>
> >> 2012. 8. 9., 오후 9:00, Jean-Marc Spaggiari <je...@spaggiari.org> 작성:
> >>
> >>> Hi Henry,
> >>>
> >>> I faced the same issue not so long time ago...
> >>>
> >>> Can you take a look at what you have in zookeeper under /hbase/table ?
> >>>
> >>> If you table is there, that's why you see it on the list. Simply
> >>> remove it from zookeeper.
> >>>
> >>> You can also take a look there:
> https://issues.apache.org/jira/browse/HBASE-6294
> >>>
> >>> JM
> >>>
> >>> 2012/8/9, henry.kim <he...@gmail.com>:
> >>>> hi, hbase users.
> >>>>
> >>>> I got a problem when I am testing coprocessors which is released at
> base
> >>>> 0.92.1.
> >>>>
> >>>> here is the hbase shell outputs
> >>>>
> >>>> ----------------
> >>>> hbase(main):001:0> truncate 'blog'
> >>>> Truncating 'blog' table (it may take a while):
> >>>>
> >>>> ERROR: Unknown table blog!
> >>>>
> >>>> Here is some help for this command:
> >>>> Disables, drops and recreates the specified table.
> >>>>
> >>>>
> >>>> hbase(main):002:0> list
> >>>> TABLE
> >>>>
> >>>>
> >>>> blog
> >>>>
> >>>>
> >>>> counter
> >>>>
> >>>>
> >>>> sidx
> >>>>
> >>>>
> >>>> 3 row(s) in 15.3610 seconds
> >>>> ----------------
> >>>>
> >>>> yes, table 'blog' is a zombie I think.
> >>>>
> >>>> there was some big stress from coprocessors which is using
> >>>> RegionServerObserver interfaces.
> >>>>
> >>>> how could I fix this situation?
> >
> >
>
Re: how to delete a zombie table permanently?
Posted by "GMAIL(henry)" <he...@gmail.com>.
yes, it's a same situation for me.
----
iPhone에서 보냄
2012. 8. 10. 20:22 Miguel Costa <mi...@telecom.pt> 작성:
> after the deletion did you restart HBase Master and region servers?
>
> I Had the same problem a while ago and solve it by deleted from zk and
> then restart.
>
> Miguel Costa
>
> On 08/10/2012 09:33 AM, Henry JunYoung KIM wrote:
>> Hi, JM.
>>
>> I followed procedures you mentioned.
>>
>> here is it.
>>
>> from zookeeper, I deleted it
>>
>> ---------------
>> $] rmr /hbase/table/*
>> $] ls /hbase/table
>> Node does not exist: /hbase/table
>> ---------------
>>
>> and then, I checked my hbase cluster.
>>
>> ---------------
>> $> hbase hbck
>> ,,,,
>> Number of empty REGIONINFO_QUALIFIER rows in .META.: 0
>> Exception in thread "main" java.lang.NullPointerException
>> at org.apache.hadoop.hbase.zookeeper.ZKTable.getDisabledOrDisablingTables(ZKTable.java:397)
>> at org.apache.hadoop.hbase.util.HBaseFsck$1.connect(HBaseFsck.java:996)
>> at org.apache.hadoop.hbase.util.HBaseFsck$1.connect(HBaseFsck.java:991)
>> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:360)
>> at org.apache.hadoop.hbase.util.HBaseFsck.loadDisabledTables(HBaseFsck.java:991)
>> at org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:360)
>> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:383)
>> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3121)
>> ---------------
>>
>> OMG.
>>
>> ok, let's refresh master's cache which is from zk.
>>
>> ---------------
>> $> hadoop-hbase-master restart
>> $> hbase hbck
>> …
>> ERROR: Found inconsistency in table blog
>> Summary:
>> -ROOT- is okay.
>> Number of regions: 1
>> Deployed on: search-ddm-test10,60020,1344586183254
>> .META. is okay.
>> Number of regions: 1
>> Deployed on: search-ddm-test6,60020,1344586183295
>> Table blog is inconsistent.
>> Number of regions: 3
>> Deployed on: search-ddm-test5,60020,1344586183269 search-ddm-test8,60020,1344586183257 search-ddm-test9,60020,1344586183260
>> 8 inconsistencies detected.
>> Status: INCONSISTENT
>> ---------------
>>
>> now, it shows up the different situation for me. (thanks)
>>
>> from hbase shell, I could check the existence of the tables I want to delete.
>>
>> ---------------
>> hbase(main):002:0> list
>> TABLE
>> blog
>> sidx
>> 2 row(s) in 0.0180 seconds
>> ---------------
>> try to fix it again.
>>
>> ---------------
>> $> hbase hbck -fix
>> …
>> Number of empty REGIONINFO_QUALIFIER rows in .META.: 0
>> 12/08/10 17:29:54 DEBUG util.HBaseFsck: There are 2 region info entries
>> Summary:
>> -ROOT- is okay.
>> Number of regions: 1
>> Deployed on: search-ddm-test10,60020,1344586183254
>> .META. is okay.
>> Number of regions: 1
>> Deployed on: search-ddm-test6,60020,1344586183295
>> 0 inconsistencies detected.
>> Status: OK
>> ---------------
>>
>> hbase shell says also
>>
>> ---------------
>> hbase(main):009:0> list
>> TABLE
>> 0 row(s) in 0.0160 seconds
>> ---------------
>>
>>
>> not easy procedures to fix it for me.
>> anyway, finally, in the conclusion, I fixed it.
>>
>>
>> 2012. 8. 9., 오후 9:00, Jean-Marc Spaggiari <je...@spaggiari.org> 작성:
>>
>>> Hi Henry,
>>>
>>> I faced the same issue not so long time ago...
>>>
>>> Can you take a look at what you have in zookeeper under /hbase/table ?
>>>
>>> If you table is there, that's why you see it on the list. Simply
>>> remove it from zookeeper.
>>>
>>> You can also take a look there: https://issues.apache.org/jira/browse/HBASE-6294
>>>
>>> JM
>>>
>>> 2012/8/9, henry.kim <he...@gmail.com>:
>>>> hi, hbase users.
>>>>
>>>> I got a problem when I am testing coprocessors which is released at base
>>>> 0.92.1.
>>>>
>>>> here is the hbase shell outputs
>>>>
>>>> ----------------
>>>> hbase(main):001:0> truncate 'blog'
>>>> Truncating 'blog' table (it may take a while):
>>>>
>>>> ERROR: Unknown table blog!
>>>>
>>>> Here is some help for this command:
>>>> Disables, drops and recreates the specified table.
>>>>
>>>>
>>>> hbase(main):002:0> list
>>>> TABLE
>>>>
>>>>
>>>> blog
>>>>
>>>>
>>>> counter
>>>>
>>>>
>>>> sidx
>>>>
>>>>
>>>> 3 row(s) in 15.3610 seconds
>>>> ----------------
>>>>
>>>> yes, table 'blog' is a zombie I think.
>>>>
>>>> there was some big stress from coprocessors which is using
>>>> RegionServerObserver interfaces.
>>>>
>>>> how could I fix this situation?
>
>
Re: how to delete a zombie table permanently?
Posted by Miguel Costa <mi...@telecom.pt>.
after the deletion did you restart HBase Master and region servers?
I Had the same problem a while ago and solve it by deleted from zk and
then restart.
Miguel Costa
On 08/10/2012 09:33 AM, Henry JunYoung KIM wrote:
> Hi, JM.
>
> I followed procedures you mentioned.
>
> here is it.
>
> from zookeeper, I deleted it
>
> ---------------
> $] rmr /hbase/table/*
> $] ls /hbase/table
> Node does not exist: /hbase/table
> ---------------
>
> and then, I checked my hbase cluster.
>
> ---------------
> $> hbase hbck
> ,,,,
> Number of empty REGIONINFO_QUALIFIER rows in .META.: 0
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.hadoop.hbase.zookeeper.ZKTable.getDisabledOrDisablingTables(ZKTable.java:397)
> at org.apache.hadoop.hbase.util.HBaseFsck$1.connect(HBaseFsck.java:996)
> at org.apache.hadoop.hbase.util.HBaseFsck$1.connect(HBaseFsck.java:991)
> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:360)
> at org.apache.hadoop.hbase.util.HBaseFsck.loadDisabledTables(HBaseFsck.java:991)
> at org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:360)
> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:383)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3121)
> ---------------
>
> OMG.
>
> ok, let's refresh master's cache which is from zk.
>
> ---------------
> $> hadoop-hbase-master restart
> $> hbase hbck
> …
> ERROR: Found inconsistency in table blog
> Summary:
> -ROOT- is okay.
> Number of regions: 1
> Deployed on: search-ddm-test10,60020,1344586183254
> .META. is okay.
> Number of regions: 1
> Deployed on: search-ddm-test6,60020,1344586183295
> Table blog is inconsistent.
> Number of regions: 3
> Deployed on: search-ddm-test5,60020,1344586183269 search-ddm-test8,60020,1344586183257 search-ddm-test9,60020,1344586183260
> 8 inconsistencies detected.
> Status: INCONSISTENT
> ---------------
>
> now, it shows up the different situation for me. (thanks)
>
> from hbase shell, I could check the existence of the tables I want to delete.
>
> ---------------
> hbase(main):002:0> list
> TABLE
> blog
> sidx
> 2 row(s) in 0.0180 seconds
> ---------------
> try to fix it again.
>
> ---------------
> $> hbase hbck -fix
> …
> Number of empty REGIONINFO_QUALIFIER rows in .META.: 0
> 12/08/10 17:29:54 DEBUG util.HBaseFsck: There are 2 region info entries
> Summary:
> -ROOT- is okay.
> Number of regions: 1
> Deployed on: search-ddm-test10,60020,1344586183254
> .META. is okay.
> Number of regions: 1
> Deployed on: search-ddm-test6,60020,1344586183295
> 0 inconsistencies detected.
> Status: OK
> ---------------
>
> hbase shell says also
>
> ---------------
> hbase(main):009:0> list
> TABLE
> 0 row(s) in 0.0160 seconds
> ---------------
>
>
> not easy procedures to fix it for me.
> anyway, finally, in the conclusion, I fixed it.
>
>
> 2012. 8. 9., 오후 9:00, Jean-Marc Spaggiari <je...@spaggiari.org> 작성:
>
>> Hi Henry,
>>
>> I faced the same issue not so long time ago...
>>
>> Can you take a look at what you have in zookeeper under /hbase/table ?
>>
>> If you table is there, that's why you see it on the list. Simply
>> remove it from zookeeper.
>>
>> You can also take a look there: https://issues.apache.org/jira/browse/HBASE-6294
>>
>> JM
>>
>> 2012/8/9, henry.kim <he...@gmail.com>:
>>> hi, hbase users.
>>>
>>> I got a problem when I am testing coprocessors which is released at base
>>> 0.92.1.
>>>
>>> here is the hbase shell outputs
>>>
>>> ----------------
>>> hbase(main):001:0> truncate 'blog'
>>> Truncating 'blog' table (it may take a while):
>>>
>>> ERROR: Unknown table blog!
>>>
>>> Here is some help for this command:
>>> Disables, drops and recreates the specified table.
>>>
>>>
>>> hbase(main):002:0> list
>>> TABLE
>>>
>>>
>>> blog
>>>
>>>
>>> counter
>>>
>>>
>>> sidx
>>>
>>>
>>> 3 row(s) in 15.3610 seconds
>>> ----------------
>>>
>>> yes, table 'blog' is a zombie I think.
>>>
>>> there was some big stress from coprocessors which is using
>>> RegionServerObserver interfaces.
>>>
>>> how could I fix this situation?
Re: how to delete a zombie table permanently?
Posted by Henry JunYoung KIM <he...@gmail.com>.
Hi, JM.
I followed procedures you mentioned.
here is it.
from zookeeper, I deleted it
---------------
$] rmr /hbase/table/*
$] ls /hbase/table
Node does not exist: /hbase/table
---------------
and then, I checked my hbase cluster.
---------------
$> hbase hbck
,,,,
Number of empty REGIONINFO_QUALIFIER rows in .META.: 0
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.hbase.zookeeper.ZKTable.getDisabledOrDisablingTables(ZKTable.java:397)
at org.apache.hadoop.hbase.util.HBaseFsck$1.connect(HBaseFsck.java:996)
at org.apache.hadoop.hbase.util.HBaseFsck$1.connect(HBaseFsck.java:991)
at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:360)
at org.apache.hadoop.hbase.util.HBaseFsck.loadDisabledTables(HBaseFsck.java:991)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:360)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:383)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3121)
---------------
OMG.
ok, let's refresh master's cache which is from zk.
---------------
$> hadoop-hbase-master restart
$> hbase hbck
…
ERROR: Found inconsistency in table blog
Summary:
-ROOT- is okay.
Number of regions: 1
Deployed on: search-ddm-test10,60020,1344586183254
.META. is okay.
Number of regions: 1
Deployed on: search-ddm-test6,60020,1344586183295
Table blog is inconsistent.
Number of regions: 3
Deployed on: search-ddm-test5,60020,1344586183269 search-ddm-test8,60020,1344586183257 search-ddm-test9,60020,1344586183260
8 inconsistencies detected.
Status: INCONSISTENT
---------------
now, it shows up the different situation for me. (thanks)
from hbase shell, I could check the existence of the tables I want to delete.
---------------
hbase(main):002:0> list
TABLE
blog
sidx
2 row(s) in 0.0180 seconds
---------------
try to fix it again.
---------------
$> hbase hbck -fix
…
Number of empty REGIONINFO_QUALIFIER rows in .META.: 0
12/08/10 17:29:54 DEBUG util.HBaseFsck: There are 2 region info entries
Summary:
-ROOT- is okay.
Number of regions: 1
Deployed on: search-ddm-test10,60020,1344586183254
.META. is okay.
Number of regions: 1
Deployed on: search-ddm-test6,60020,1344586183295
0 inconsistencies detected.
Status: OK
---------------
hbase shell says also
---------------
hbase(main):009:0> list
TABLE
0 row(s) in 0.0160 seconds
---------------
not easy procedures to fix it for me.
anyway, finally, in the conclusion, I fixed it.
2012. 8. 9., 오후 9:00, Jean-Marc Spaggiari <je...@spaggiari.org> 작성:
> Hi Henry,
>
> I faced the same issue not so long time ago...
>
> Can you take a look at what you have in zookeeper under /hbase/table ?
>
> If you table is there, that's why you see it on the list. Simply
> remove it from zookeeper.
>
> You can also take a look there: https://issues.apache.org/jira/browse/HBASE-6294
>
> JM
>
> 2012/8/9, henry.kim <he...@gmail.com>:
>> hi, hbase users.
>>
>> I got a problem when I am testing coprocessors which is released at base
>> 0.92.1.
>>
>> here is the hbase shell outputs
>>
>> ----------------
>> hbase(main):001:0> truncate 'blog'
>> Truncating 'blog' table (it may take a while):
>>
>> ERROR: Unknown table blog!
>>
>> Here is some help for this command:
>> Disables, drops and recreates the specified table.
>>
>>
>> hbase(main):002:0> list
>> TABLE
>>
>>
>> blog
>>
>>
>> counter
>>
>>
>> sidx
>>
>>
>> 3 row(s) in 15.3610 seconds
>> ----------------
>>
>> yes, table 'blog' is a zombie I think.
>>
>> there was some big stress from coprocessors which is using
>> RegionServerObserver interfaces.
>>
>> how could I fix this situation?
Re: how to delete a zombie table permanently?
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Hi Henry,
I faced the same issue not so long time ago...
Can you take a look at what you have in zookeeper under /hbase/table ?
If you table is there, that's why you see it on the list. Simply
remove it from zookeeper.
You can also take a look there: https://issues.apache.org/jira/browse/HBASE-6294
JM
2012/8/9, henry.kim <he...@gmail.com>:
> hi, hbase users.
>
> I got a problem when I am testing coprocessors which is released at base
> 0.92.1.
>
> here is the hbase shell outputs
>
> ----------------
> hbase(main):001:0> truncate 'blog'
> Truncating 'blog' table (it may take a while):
>
> ERROR: Unknown table blog!
>
> Here is some help for this command:
> Disables, drops and recreates the specified table.
>
>
> hbase(main):002:0> list
> TABLE
>
>
> blog
>
>
> counter
>
>
> sidx
>
>
> 3 row(s) in 15.3610 seconds
> ----------------
>
> yes, table 'blog' is a zombie I think.
>
> there was some big stress from coprocessors which is using
> RegionServerObserver interfaces.
>
> how could I fix this situation?