You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Lars George (JIRA)" <ji...@apache.org> on 2008/02/26 01:59:51 UTC
[jira] Issue Comment Edited: (HBASE-471) IllegalStateException
thrown in client after region was split and deleted
[ https://issues.apache.org/jira/browse/HBASE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12572321#action_12572321 ]
larsgeorge edited comment on HBASE-471 at 2/25/08 4:59 PM:
------------------------------------------------------------
Sometimes I get this error instead:
[2008-02-25 16:37:06,162] ERROR [TP-Processor1] archive.MultilingualArchive - An error occurred.
org.apache.hadoop.hbase.WrongRegionException: org.apache.hadoop.hbase.WrongRegionException: Requested row out of range for HRegion pdc-
docs,US20070223445_20070927,1203985191984, startKey='US20070223445_20070927', getEndKey()='US20070226683_20070927', row='US20070271984_
20071129'
at org.apache.hadoop.hbase.HRegion.checkRow(HRegion.java:1486)
at org.apache.hadoop.hbase.HRegion.obtainRowLock(HRegion.java:1531)
at org.apache.hadoop.hbase.HRegion.batchUpdate(HRegion.java:1226)
at org.apache.hadoop.hbase.HRegionServer.batchUpdate(HRegionServer.java:1433)
I assume they are related, just a different effect of the same problem, i.e. the client out of sync with the master?
Here a full example with regions trace on the master:
[2008-02-25 16:55:02,375] ERROR [TP-Processor3] archive.MultilingualArchive - An error occurred.
org.apache.hadoop.hbase.WrongRegionException: org.apache.hadoop.hbase.WrongRegionException: Requested row out of range for HRegion pdc-docs,US20070248463_20071025,1203986650827, startKey='US20070248463_20071025', getEndKey()='US20070251593_20071101', row='US7052594_20060530'
at org.apache.hadoop.hbase.HRegion.checkRow(HRegion.java:1486)
at org.apache.hadoop.hbase.HRegion.obtainRowLock(HRegion.java:1531)
at org.apache.hadoop.hbase.HRegion.batchUpdate(HRegion.java:1226)
at org.apache.hadoop.hbase.HRegionServer.batchUpdate(HRegionServer.java:1433)
2008-02-25 16:54:02,517 DEBUG org.apache.hadoop.hbase.HMaster: HMaster.metaScanner regioninfo: {regionname: pdc-docs,US20070248463_20071025,1203986650827, startKey: <US20070248463_20071025>, endKey: <US20070251593_20071101>, encodedName: 687614528, tableDesc: {name: pdc-docs, families: {contents:={name: contents, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}, language:={name: language, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}, mimetype:={name: mimetype, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}}}}, server: 192.168.105.21:60020, startCode: 1203949130468
2008-02-25 16:55:02,521 DEBUG org.apache.hadoop.hbase.HMaster: HMaster.metaScanner regioninfo: {regionname: pdc-docs,US20070248463_20071025,1203986650827, startKey: <US20070248463_20071025>, endKey: <US20070251593_20071101>, encodedName: 687614528, tableDesc: {name: pdc-docs, families: {contents:={name: contents, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}, language:={name: language, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}, mimetype:={name: mimetype, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}}}}, server: 192.168.105.21:60020, startCode: 1203949130468
2008-02-25 16:56:02,525 DEBUG org.apache.hadoop.hbase.HMaster: HMaster.metaScanner regioninfo: {regionname: pdc-docs,US20070248463_20071025,1203986650827, startKey: <US20070248463_20071025>, endKey: <US20070251593_20071101>, encodedName: 687614528, tableDesc: {name: pdc-docs, families: {contents:={name: contents, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}, language:={name: language, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}, mimetype:={name: mimetype, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}}}}, server: 192.168.105.21:60020, startCode: 1203949130468
2008-02-25 16:57:02,524 DEBUG org.apache.hadoop.hbase.HMaster: HMaster.metaScanner regioninfo: {regionname: pdc-docs,US20070248463_20071025,1203986650827, startKey: <US20070248463_20071025>, endKey: <US20070251593_20071101>, encodedName: 687614528, tableDesc: {name: pdc-docs, families: {contents:={name: contents, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}, language:={name: language, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}, mimetype:={name: mimetype, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}}}}, server: 192.168.105.21:60020, startCode: 1203949130468
2008-02-25 16:58:02,532 DEBUG org.apache.hadoop.hbase.HMaster: HMaster.metaScanner regioninfo: {regionname: pdc-docs,US20070248463_20071025,1203986650827, startKey: <US20070248463_20071025>, endKey: <US20070251593_20071101>, encodedName: 687614528, tableDesc: {name: pdc-docs, families: {contents:={name: contents, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}, language:={name: language, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}, mimetype:={name: mimetype, max versions: 3, compression: NONE, in memory: false, max length: 2147483647, bloom filter: none}}}}, server: 192.168.105.21:60020, startCode: 1203949130468
was (Author: larsgeorge):
Sometimes I get this error instead:
[2008-02-25 16:37:06,162] ERROR [TP-Processor1] archive.MultilingualArchive - An error occurred.
org.apache.hadoop.hbase.WrongRegionException: org.apache.hadoop.hbase.WrongRegionException: Requested row out of range for HRegion pdc-
docs,US20070223445_20070927,1203985191984, startKey='US20070223445_20070927', getEndKey()='US20070226683_20070927', row='US20070271984_
20071129'
at org.apache.hadoop.hbase.HRegion.checkRow(HRegion.java:1486)
at org.apache.hadoop.hbase.HRegion.obtainRowLock(HRegion.java:1531)
at org.apache.hadoop.hbase.HRegion.batchUpdate(HRegion.java:1226)
at org.apache.hadoop.hbase.HRegionServer.batchUpdate(HRegionServer.java:1433)
I assume they are related, just a different effect of the same problem, i.e. the client out of sync with the master?
> IllegalStateException thrown in client after region was split and deleted
> -------------------------------------------------------------------------
>
> Key: HBASE-471
> URL: https://issues.apache.org/jira/browse/HBASE-471
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.1.0
> Environment: Linux Debian, HBase 0.16.0
> Reporter: Lars George
>
> For some reason a client sometimes fails to locate a row with a IllegalStateException when the region was split and deleted.
> > [2008-02-25 16:12:39,171] ERROR [http-80-Processor20] archive.MultilingualArchive - getDocument: An error occurred.
> > java.lang.IllegalStateException: region offline: pdc-docs,US7039976_20060509,1203981958556
> > at org.apache.hadoop.hbase.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:432)
> > at org.apache.hadoop.hbase.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:343)
> > at org.apache.hadoop.hbase.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:306)
> > at org.apache.hadoop.hbase.HTable.getRegionLocation(HTable.java:102)
> > at org.apache.hadoop.hbase.HTable.get(HTable.java:280)
> Tracing the region on the master shows this:
> > 2008-02-25 16:09:38,761 DEBUG org.apache.hadoop.hbase.HMaster: Received MSG_REGION_SPLIT :
> +pdc-docs,US7039976_20060509,1203981958556 from 192.168.105.21:60020
> > 2008-02-25 16:09:38,761 INFO org.apache.hadoop.hbase.HMaster: region pdc-docs,US7039976_20060509,1203981958556 split. New regions
> +are: pdc-docs,US7039976_20060509,1203984578345, pdc-docs,US7046359_20060516,1203984578345
> > 2008-02-25 16:10:02,470 DEBUG org.apache.hadoop.hbase.HMaster: HMaster.metaScanner regioninfo: {regionname:
> +pdc-docs,US7039976_20060509,1203981958556, startKey: <US7039976_20060509>, endKey: <US7053021_20060530>, encodedName: 1260314009,
> +offline: true, split: true, tableDesc: {name: pdc-docs, families: {contents:={name: contents, max versions: 3, compression: NONE, in
> +memory: false, max length: 2147483647, bloom filter: none}, language:={name: language, max versions: 3, compression: NONE, in
> +memory: false, max length: 2147483647, bloom filter: none}, mimetype:={name: mimetype, max versions: 3, compression: NONE, in
> +memory: false, max length: 2147483647, bloom filter: none}}}}, server: 192.168.105.21:60020, startCode: 1203949130468
> > 2008-02-25 16:10:02,513 DEBUG org.apache.hadoop.hbase.HMaster: pdc-docs,US7039976_20060509,1203984578345 no longer has references
> +to pdc-docs,US7039976_20060509,1203981958556
> > 2008-02-25 16:10:02,516 DEBUG org.apache.hadoop.hbase.HMaster: pdc-docs,US7046359_20060516,1203984578345 no longer has references
> +to pdc-docs,US7039976_20060509,1203981958556
> > 2008-02-25 16:10:02,516 INFO org.apache.hadoop.hbase.HMaster: Deleting region pdc-docs,US7039976_20060509,1203981958556 because
> +daughter splits no longer hold references
> After discussion with st^ack it seems that the server is simply not retrying IllegalStateException but IOExceptions only. Also see HBASE-452 which should be addressed at the same time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.