You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by elsif <el...@gmail.com> on 2009/09/16 03:25:50 UTC
NoServerForRegionException
We have an issue with 0.20.0 where we did a shutdown and a start up and
lost access to the tables.
Is there a tool to repair issues like this? How do we go about fixing
this manually?
Creating a new table while this is occurring seems to be assigned a
region server ok and can be scanned.
The master log reports the following over and over:
2009-09-15 18:17:13,552 INFO
org.apache.hadoop.hbase.master.ServerManager: 28 region servers, 0 dead,
average load 0.10714285714285714
2009-09-15 18:17:13,606 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.metaScanner scanning meta region {server:
10.4.30.37:60020, regionname: .META.,,1, startKey: <>}
2009-09-15 18:17:13,607 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.rootScanner scanning meta region {server:
10.4.30.19:60020, regionname: -ROOT-,,0, startKey: <>}
2009-09-15 18:17:13,615 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.rootScanner scan of 1 row(s) of meta region {server:
10.4.30.19:60020, regionname: -ROOT-,,0, startKey: <>} complete
2009-09-15 18:17:13,633 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.metaScanner scan of 3 row(s) of meta region {server:
10.4.30.37:60020, regionname: .META.,,1, startKey: <>} complete
2009-09-15 18:17:13,634 INFO org.apache.hadoop.hbase.master.BaseScanner:
All 1 .META. region(s) scanned
2009-09-15 18:17:13,834 INFO
org.apache.hadoop.hbase.master.ServerManager: Processing
MSG_REPORT_CLOSE: dropzone,,1253047705364: java.lang.IllegalAccessError:
Has not been initialized
at
org.apache.hadoop.hbase.regionserver.StoreFile.getMaxSequenceId(StoreFile.java:216)
at
org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:388)
at org.apache.hadoop.hbase.regionserver.Store.<init>(Store.java:217)
at
org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:1505)
at
org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:308)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateRegion(HRegionServer.java:1601)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:1570)
at
org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1485)
at java.lang.Thread.run(Unknown Source)
from hfs-030031,60020,1253063295720; 1 of 1
2009-09-15 18:17:13,834 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: region set as
unassigned: dropzone,,1253047705364
Each region server reports:
2009-09-15 18:18:13,543 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
dropzone,,1253047705364
2009-09-15 18:18:13,544 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Worker:
MSG_REGION_OPEN: dropzone,,1253047705364
2009-09-15 18:18:13,560 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Error opening
dropzone,,1253047705364
java.lang.IllegalAccessError: Has not been initialized
at
org.apache.hadoop.hbase.regionserver.StoreFile.getMaxSequenceId(StoreFile.java:216)
at
org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:388)
at org.apache.hadoop.hbase.regionserver.Store.<init>(Store.java:217)
at
org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:1505)
at
org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:308)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateRegion(HRegionServer.java:1601)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:1570)
at
org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1485)
at java.lang.Thread.run(Unknown Source)
Re: NoServerForRegionException
Posted by stack <st...@duboce.net>.
Related, I committed elsif's compression validation tool yesterday. Its
part of the 0.20.1 and I added documentation to the compression how-to up on
the wiki.
St.Ack
On Tue, Oct 6, 2009 at 12:39 AM, Shotaro Kamio <ka...@gmail.com> wrote:
> Hi,
>
> I saw the same error when I migrate hbase from 0.20.0 RC1 to 0.20.0.
> In my case, it was because I copied the config on hbase 0.20.0RC1 with
> lzo compression to hbase 0.20.0. But I forgot to copy lzo libraries to
> lib/ directory. Then, I couldn't properly write data into hbase.
> After the library is installed, it works well.
>
> So, I suggest you to check your config if you use lzo compression.
>
>
> Cheers,
> Shotaro
>
>
>
> On Wed, Sep 16, 2009 at 10:25 AM, elsif <el...@gmail.com> wrote:
> > We have an issue with 0.20.0 where we did a shutdown and a start up and
> > lost access to the tables.
> >
> > Is there a tool to repair issues like this? How do we go about fixing
> > this manually?
> >
> > Creating a new table while this is occurring seems to be assigned a
> > region server ok and can be scanned.
> >
> > The master log reports the following over and over:
> >
> > 2009-09-15 18:17:13,552 INFO
> > org.apache.hadoop.hbase.master.ServerManager: 28 region servers, 0 dead,
> > average load 0.10714285714285714
> > 2009-09-15 18:17:13,606 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.metaScanner scanning meta region {server:
> > 10.4.30.37:60020, regionname: .META.,,1, startKey: <>}
> > 2009-09-15 18:17:13,607 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.rootScanner scanning meta region {server:
> > 10.4.30.19:60020, regionname: -ROOT-,,0, startKey: <>}
> > 2009-09-15 18:17:13,615 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.rootScanner scan of 1 row(s) of meta region {server:
> > 10.4.30.19:60020, regionname: -ROOT-,,0, startKey: <>} complete
> > 2009-09-15 18:17:13,633 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > RegionManager.metaScanner scan of 3 row(s) of meta region {server:
> > 10.4.30.37:60020, regionname: .META.,,1, startKey: <>} complete
> > 2009-09-15 18:17:13,634 INFO org.apache.hadoop.hbase.master.BaseScanner:
> > All 1 .META. region(s) scanned
> > 2009-09-15 18:17:13,834 INFO
> > org.apache.hadoop.hbase.master.ServerManager: Processing
> > MSG_REPORT_CLOSE: dropzone,,1253047705364: java.lang.IllegalAccessError:
> > Has not been initialized
> > at
> >
> org.apache.hadoop.hbase.regionserver.StoreFile.getMaxSequenceId(StoreFile.java:216)
> > at
> > org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:388)
> > at
> org.apache.hadoop.hbase.regionserver.Store.<init>(Store.java:217)
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:1505)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:308)
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateRegion(HRegionServer.java:1601)
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:1570)
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1485)
> > at java.lang.Thread.run(Unknown Source)
> > from hfs-030031,60020,1253063295720; 1 of 1
> > 2009-09-15 18:17:13,834 INFO
> > org.apache.hadoop.hbase.master.RegionServerOperation: region set as
> > unassigned: dropzone,,1253047705364
> >
> >
> > Each region server reports:
> >
> > 2009-09-15 18:18:13,543 INFO
> > org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
> > dropzone,,1253047705364
> > 2009-09-15 18:18:13,544 INFO
> > org.apache.hadoop.hbase.regionserver.HRegionServer: Worker:
> > MSG_REGION_OPEN: dropzone,,1253047705364
> > 2009-09-15 18:18:13,560 ERROR
> > org.apache.hadoop.hbase.regionserver.HRegionServer: Error opening
> > dropzone,,1253047705364
> > java.lang.IllegalAccessError: Has not been initialized
> > at
> >
> org.apache.hadoop.hbase.regionserver.StoreFile.getMaxSequenceId(StoreFile.java:216)
> > at
> > org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:388)
> > at
> org.apache.hadoop.hbase.regionserver.Store.<init>(Store.java:217)
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:1505)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:308)
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateRegion(HRegionServer.java:1601)
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:1570)
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1485)
> > at java.lang.Thread.run(Unknown Source)
> >
> >
>
>
>
> --
> Shotaro Kamio
>
Re: NoServerForRegionException
Posted by Shotaro Kamio <ka...@gmail.com>.
Hi,
I saw the same error when I migrate hbase from 0.20.0 RC1 to 0.20.0.
In my case, it was because I copied the config on hbase 0.20.0RC1 with
lzo compression to hbase 0.20.0. But I forgot to copy lzo libraries to
lib/ directory. Then, I couldn't properly write data into hbase.
After the library is installed, it works well.
So, I suggest you to check your config if you use lzo compression.
Cheers,
Shotaro
On Wed, Sep 16, 2009 at 10:25 AM, elsif <el...@gmail.com> wrote:
> We have an issue with 0.20.0 where we did a shutdown and a start up and
> lost access to the tables.
>
> Is there a tool to repair issues like this? How do we go about fixing
> this manually?
>
> Creating a new table while this is occurring seems to be assigned a
> region server ok and can be scanned.
>
> The master log reports the following over and over:
>
> 2009-09-15 18:17:13,552 INFO
> org.apache.hadoop.hbase.master.ServerManager: 28 region servers, 0 dead,
> average load 0.10714285714285714
> 2009-09-15 18:17:13,606 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.metaScanner scanning meta region {server:
> 10.4.30.37:60020, regionname: .META.,,1, startKey: <>}
> 2009-09-15 18:17:13,607 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scanning meta region {server:
> 10.4.30.19:60020, regionname: -ROOT-,,0, startKey: <>}
> 2009-09-15 18:17:13,615 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scan of 1 row(s) of meta region {server:
> 10.4.30.19:60020, regionname: -ROOT-,,0, startKey: <>} complete
> 2009-09-15 18:17:13,633 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.metaScanner scan of 3 row(s) of meta region {server:
> 10.4.30.37:60020, regionname: .META.,,1, startKey: <>} complete
> 2009-09-15 18:17:13,634 INFO org.apache.hadoop.hbase.master.BaseScanner:
> All 1 .META. region(s) scanned
> 2009-09-15 18:17:13,834 INFO
> org.apache.hadoop.hbase.master.ServerManager: Processing
> MSG_REPORT_CLOSE: dropzone,,1253047705364: java.lang.IllegalAccessError:
> Has not been initialized
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.getMaxSequenceId(StoreFile.java:216)
> at
> org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:388)
> at org.apache.hadoop.hbase.regionserver.Store.<init>(Store.java:217)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:1505)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:308)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateRegion(HRegionServer.java:1601)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:1570)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1485)
> at java.lang.Thread.run(Unknown Source)
> from hfs-030031,60020,1253063295720; 1 of 1
> 2009-09-15 18:17:13,834 INFO
> org.apache.hadoop.hbase.master.RegionServerOperation: region set as
> unassigned: dropzone,,1253047705364
>
>
> Each region server reports:
>
> 2009-09-15 18:18:13,543 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
> dropzone,,1253047705364
> 2009-09-15 18:18:13,544 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Worker:
> MSG_REGION_OPEN: dropzone,,1253047705364
> 2009-09-15 18:18:13,560 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer: Error opening
> dropzone,,1253047705364
> java.lang.IllegalAccessError: Has not been initialized
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.getMaxSequenceId(StoreFile.java:216)
> at
> org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:388)
> at org.apache.hadoop.hbase.regionserver.Store.<init>(Store.java:217)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:1505)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:308)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateRegion(HRegionServer.java:1601)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:1570)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1485)
> at java.lang.Thread.run(Unknown Source)
>
>
--
Shotaro Kamio