You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Mubarak Seyed (Created) (JIRA)" <ji...@apache.org> on 2012/01/11 23:12:39 UTC

[jira] [Created] (HBASE-5181) Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)

Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)
--------------------------------------------------------------------------------------------------

                 Key: HBASE-5181
                 URL: https://issues.apache.org/jira/browse/HBASE-5181
             Project: HBase
          Issue Type: Bug
          Components: master
    Affects Versions: 0.90.5, 0.92.0
            Reporter: Mubarak Seyed
            Priority: Minor


When master fail-over happens, if we have number of RITs under /hbase/unassigned and if we have stale znode(s) (encoded region names) under /hbase/unassigned, we are getting

{code}
2011-12-30 10:27:35,623 INFO org.apache.hadoop.hbase.master.HMaster: Master startup proceeding: master failover 
2011-12-30 10:27:36,002 INFO org.apache.hadoop.hbase.master.AssignmentManager: Failed-over master needs to process 1717 regions in transition 
2011-12-30 10:27:36,004 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown. 
java.lang.ArrayIndexOutOfBoundsException: -256 
at org.apache.hadoop.hbase.executor.RegionTransitionData.readFields(RegionTransitionData.java:148) 
at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:105) 
at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:75) 
at org.apache.hadoop.hbase.executor.RegionTransitionData.fromBytes(RegionTransitionData.java:198) 
at org.apache.hadoop.hbase.zookeeper.ZKAssign.getData(ZKAssign.java:743) 
at org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:262) 
at org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:223) 
at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:401) 
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:283)
{code}

and there is no clue on how to clean-up the stale znode(s) from unassigned using zkCli.sh (del /hbase/unassigned/<bad region name>). It would be good if we include the bad region name in IOException from RegionTransitionData.readFields().

{code}

@Override
  public void readFields(DataInput in) throws IOException {
    // the event type byte
    eventType = EventType.values()[in.readShort()];
    // the timestamp
    stamp = in.readLong();
    // the encoded name of the region being transitioned
    regionName = Bytes.readByteArray(in);
    // remaining fields are optional so prefixed with boolean
    // the name of the regionserver sending the data
    if (in.readBoolean()) {
      byte [] versionedBytes = Bytes.readByteArray(in);
      this.origin = ServerName.parseVersionedServerName(versionedBytes);
    }
    if (in.readBoolean()) {
      this.payload = Bytes.readByteArray(in);
    }
  }
{code}

If the code execution has survived until regionName then we can include the regionName in IOException with error message to clean-up the stale znode(s) under /hbase/unassigned.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5181) Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)

Posted by "Mubarak Seyed (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13184514#comment-13184514 ] 

Mubarak Seyed commented on HBASE-5181:
--------------------------------------

Is there any suggestion on error message? How about

throw new IOException("There could be a stale region-in-transition in ZK." +
        		" The bad region is " + Bytes.toString(regionName) +
        		". Try deleting the region-in-transition using 'del /hbase/unassigned/"
        		+ Bytes.toString(regionName) + "' command over a ZK connection (in zkCli.sh)", ioe);
                
> Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5181
>                 URL: https://issues.apache.org/jira/browse/HBASE-5181
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.0, 0.90.5
>            Reporter: Mubarak Seyed
>            Assignee: Mubarak Seyed
>            Priority: Minor
>              Labels: noob
>
> When master fail-over happens, if we have number of RITs under /hbase/unassigned and if we have stale znode(s) (encoded region names) under /hbase/unassigned, we are getting
> {code}
> 2011-12-30 10:27:35,623 INFO org.apache.hadoop.hbase.master.HMaster: Master startup proceeding: master failover 
> 2011-12-30 10:27:36,002 INFO org.apache.hadoop.hbase.master.AssignmentManager: Failed-over master needs to process 1717 regions in transition 
> 2011-12-30 10:27:36,004 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown. 
> java.lang.ArrayIndexOutOfBoundsException: -256 
> at org.apache.hadoop.hbase.executor.RegionTransitionData.readFields(RegionTransitionData.java:148) 
> at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:105) 
> at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:75) 
> at org.apache.hadoop.hbase.executor.RegionTransitionData.fromBytes(RegionTransitionData.java:198) 
> at org.apache.hadoop.hbase.zookeeper.ZKAssign.getData(ZKAssign.java:743) 
> at org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:262) 
> at org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:223) 
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:401) 
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:283)
> {code}
> and there is no clue on how to clean-up the stale znode(s) from unassigned using zkCli.sh (del /hbase/unassigned/<bad region name>). It would be good if we include the bad region name in IOException from RegionTransitionData.readFields().
> {code}
> @Override
>   public void readFields(DataInput in) throws IOException {
>     // the event type byte
>     eventType = EventType.values()[in.readShort()];
>     // the timestamp
>     stamp = in.readLong();
>     // the encoded name of the region being transitioned
>     regionName = Bytes.readByteArray(in);
>     // remaining fields are optional so prefixed with boolean
>     // the name of the regionserver sending the data
>     if (in.readBoolean()) {
>       byte [] versionedBytes = Bytes.readByteArray(in);
>       this.origin = ServerName.parseVersionedServerName(versionedBytes);
>     }
>     if (in.readBoolean()) {
>       this.payload = Bytes.readByteArray(in);
>     }
>   }
> {code}
> If the code execution has survived until regionName then we can include the regionName in IOException with error message to clean-up the stale znode(s) under /hbase/unassigned.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5181) Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13184551#comment-13184551 ] 

Zhihong Yu commented on HBASE-5181:
-----------------------------------

The message is certainly detailed :-)
Please remember to replace '/hbase' with the value of zookeeper.znode.parent
                
> Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5181
>                 URL: https://issues.apache.org/jira/browse/HBASE-5181
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.0, 0.90.5
>            Reporter: Mubarak Seyed
>            Assignee: Mubarak Seyed
>            Priority: Minor
>              Labels: noob
>
> When master fail-over happens, if we have number of RITs under /hbase/unassigned and if we have stale znode(s) (encoded region names) under /hbase/unassigned, we are getting
> {code}
> 2011-12-30 10:27:35,623 INFO org.apache.hadoop.hbase.master.HMaster: Master startup proceeding: master failover 
> 2011-12-30 10:27:36,002 INFO org.apache.hadoop.hbase.master.AssignmentManager: Failed-over master needs to process 1717 regions in transition 
> 2011-12-30 10:27:36,004 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown. 
> java.lang.ArrayIndexOutOfBoundsException: -256 
> at org.apache.hadoop.hbase.executor.RegionTransitionData.readFields(RegionTransitionData.java:148) 
> at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:105) 
> at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:75) 
> at org.apache.hadoop.hbase.executor.RegionTransitionData.fromBytes(RegionTransitionData.java:198) 
> at org.apache.hadoop.hbase.zookeeper.ZKAssign.getData(ZKAssign.java:743) 
> at org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:262) 
> at org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:223) 
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:401) 
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:283)
> {code}
> and there is no clue on how to clean-up the stale znode(s) from unassigned using zkCli.sh (del /hbase/unassigned/<bad region name>). It would be good if we include the bad region name in IOException from RegionTransitionData.readFields().
> {code}
> @Override
>   public void readFields(DataInput in) throws IOException {
>     // the event type byte
>     eventType = EventType.values()[in.readShort()];
>     // the timestamp
>     stamp = in.readLong();
>     // the encoded name of the region being transitioned
>     regionName = Bytes.readByteArray(in);
>     // remaining fields are optional so prefixed with boolean
>     // the name of the regionserver sending the data
>     if (in.readBoolean()) {
>       byte [] versionedBytes = Bytes.readByteArray(in);
>       this.origin = ServerName.parseVersionedServerName(versionedBytes);
>     }
>     if (in.readBoolean()) {
>       this.payload = Bytes.readByteArray(in);
>     }
>   }
> {code}
> If the code execution has survived until regionName then we can include the regionName in IOException with error message to clean-up the stale znode(s) under /hbase/unassigned.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Assigned] (HBASE-5181) Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)

Posted by "Zhihong Yu (Assigned) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhihong Yu reassigned HBASE-5181:
---------------------------------

    Assignee: Mubarak Seyed
    
> Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5181
>                 URL: https://issues.apache.org/jira/browse/HBASE-5181
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.0, 0.90.5
>            Reporter: Mubarak Seyed
>            Assignee: Mubarak Seyed
>            Priority: Minor
>              Labels: noob
>
> When master fail-over happens, if we have number of RITs under /hbase/unassigned and if we have stale znode(s) (encoded region names) under /hbase/unassigned, we are getting
> {code}
> 2011-12-30 10:27:35,623 INFO org.apache.hadoop.hbase.master.HMaster: Master startup proceeding: master failover 
> 2011-12-30 10:27:36,002 INFO org.apache.hadoop.hbase.master.AssignmentManager: Failed-over master needs to process 1717 regions in transition 
> 2011-12-30 10:27:36,004 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown. 
> java.lang.ArrayIndexOutOfBoundsException: -256 
> at org.apache.hadoop.hbase.executor.RegionTransitionData.readFields(RegionTransitionData.java:148) 
> at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:105) 
> at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:75) 
> at org.apache.hadoop.hbase.executor.RegionTransitionData.fromBytes(RegionTransitionData.java:198) 
> at org.apache.hadoop.hbase.zookeeper.ZKAssign.getData(ZKAssign.java:743) 
> at org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:262) 
> at org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:223) 
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:401) 
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:283)
> {code}
> and there is no clue on how to clean-up the stale znode(s) from unassigned using zkCli.sh (del /hbase/unassigned/<bad region name>). It would be good if we include the bad region name in IOException from RegionTransitionData.readFields().
> {code}
> @Override
>   public void readFields(DataInput in) throws IOException {
>     // the event type byte
>     eventType = EventType.values()[in.readShort()];
>     // the timestamp
>     stamp = in.readLong();
>     // the encoded name of the region being transitioned
>     regionName = Bytes.readByteArray(in);
>     // remaining fields are optional so prefixed with boolean
>     // the name of the regionserver sending the data
>     if (in.readBoolean()) {
>       byte [] versionedBytes = Bytes.readByteArray(in);
>       this.origin = ServerName.parseVersionedServerName(versionedBytes);
>     }
>     if (in.readBoolean()) {
>       this.payload = Bytes.readByteArray(in);
>     }
>   }
> {code}
> If the code execution has survived until regionName then we can include the regionName in IOException with error message to clean-up the stale znode(s) under /hbase/unassigned.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5181) Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)

Posted by "Mubarak Seyed (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mubarak Seyed updated HBASE-5181:
---------------------------------

    Attachment: HBASE-5181-v1-trunk.patch

The attached file is a patch.
                
> Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5181
>                 URL: https://issues.apache.org/jira/browse/HBASE-5181
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.0, 0.90.5
>            Reporter: Mubarak Seyed
>            Assignee: Mubarak Seyed
>            Priority: Minor
>              Labels: noob
>         Attachments: HBASE-5181-v1-trunk.patch
>
>
> When master fail-over happens, if we have number of RITs under /hbase/unassigned and if we have stale znode(s) (encoded region names) under /hbase/unassigned, we are getting
> {code}
> 2011-12-30 10:27:35,623 INFO org.apache.hadoop.hbase.master.HMaster: Master startup proceeding: master failover 
> 2011-12-30 10:27:36,002 INFO org.apache.hadoop.hbase.master.AssignmentManager: Failed-over master needs to process 1717 regions in transition 
> 2011-12-30 10:27:36,004 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown. 
> java.lang.ArrayIndexOutOfBoundsException: -256 
> at org.apache.hadoop.hbase.executor.RegionTransitionData.readFields(RegionTransitionData.java:148) 
> at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:105) 
> at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:75) 
> at org.apache.hadoop.hbase.executor.RegionTransitionData.fromBytes(RegionTransitionData.java:198) 
> at org.apache.hadoop.hbase.zookeeper.ZKAssign.getData(ZKAssign.java:743) 
> at org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:262) 
> at org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:223) 
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:401) 
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:283)
> {code}
> and there is no clue on how to clean-up the stale znode(s) from unassigned using zkCli.sh (del /hbase/unassigned/<bad region name>). It would be good if we include the bad region name in IOException from RegionTransitionData.readFields().
> {code}
> @Override
>   public void readFields(DataInput in) throws IOException {
>     // the event type byte
>     eventType = EventType.values()[in.readShort()];
>     // the timestamp
>     stamp = in.readLong();
>     // the encoded name of the region being transitioned
>     regionName = Bytes.readByteArray(in);
>     // remaining fields are optional so prefixed with boolean
>     // the name of the regionserver sending the data
>     if (in.readBoolean()) {
>       byte [] versionedBytes = Bytes.readByteArray(in);
>       this.origin = ServerName.parseVersionedServerName(versionedBytes);
>     }
>     if (in.readBoolean()) {
>       this.payload = Bytes.readByteArray(in);
>     }
>   }
> {code}
> If the code execution has survived until regionName then we can include the regionName in IOException with error message to clean-up the stale znode(s) under /hbase/unassigned.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5181) Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)

Posted by "Mubarak Seyed (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13184478#comment-13184478 ] 

Mubarak Seyed commented on HBASE-5181:
--------------------------------------

Working on corporate approval to contribute this patch. Thanks.
                
> Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5181
>                 URL: https://issues.apache.org/jira/browse/HBASE-5181
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.0, 0.90.5
>            Reporter: Mubarak Seyed
>            Priority: Minor
>              Labels: noob
>
> When master fail-over happens, if we have number of RITs under /hbase/unassigned and if we have stale znode(s) (encoded region names) under /hbase/unassigned, we are getting
> {code}
> 2011-12-30 10:27:35,623 INFO org.apache.hadoop.hbase.master.HMaster: Master startup proceeding: master failover 
> 2011-12-30 10:27:36,002 INFO org.apache.hadoop.hbase.master.AssignmentManager: Failed-over master needs to process 1717 regions in transition 
> 2011-12-30 10:27:36,004 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown. 
> java.lang.ArrayIndexOutOfBoundsException: -256 
> at org.apache.hadoop.hbase.executor.RegionTransitionData.readFields(RegionTransitionData.java:148) 
> at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:105) 
> at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:75) 
> at org.apache.hadoop.hbase.executor.RegionTransitionData.fromBytes(RegionTransitionData.java:198) 
> at org.apache.hadoop.hbase.zookeeper.ZKAssign.getData(ZKAssign.java:743) 
> at org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:262) 
> at org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:223) 
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:401) 
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:283)
> {code}
> and there is no clue on how to clean-up the stale znode(s) from unassigned using zkCli.sh (del /hbase/unassigned/<bad region name>). It would be good if we include the bad region name in IOException from RegionTransitionData.readFields().
> {code}
> @Override
>   public void readFields(DataInput in) throws IOException {
>     // the event type byte
>     eventType = EventType.values()[in.readShort()];
>     // the timestamp
>     stamp = in.readLong();
>     // the encoded name of the region being transitioned
>     regionName = Bytes.readByteArray(in);
>     // remaining fields are optional so prefixed with boolean
>     // the name of the regionserver sending the data
>     if (in.readBoolean()) {
>       byte [] versionedBytes = Bytes.readByteArray(in);
>       this.origin = ServerName.parseVersionedServerName(versionedBytes);
>     }
>     if (in.readBoolean()) {
>       this.payload = Bytes.readByteArray(in);
>     }
>   }
> {code}
> If the code execution has survived until regionName then we can include the regionName in IOException with error message to clean-up the stale znode(s) under /hbase/unassigned.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5181) Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13184472#comment-13184472 ] 

Zhihong Yu commented on HBASE-5181:
-----------------------------------

Thanks for the suggestion, Mubarak.

Do you want to attach a patch ?
                
> Improve error message when Master fail-over happens and ZK unassigned node contains stale znode(s)
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5181
>                 URL: https://issues.apache.org/jira/browse/HBASE-5181
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.0, 0.90.5
>            Reporter: Mubarak Seyed
>            Priority: Minor
>              Labels: noob
>
> When master fail-over happens, if we have number of RITs under /hbase/unassigned and if we have stale znode(s) (encoded region names) under /hbase/unassigned, we are getting
> {code}
> 2011-12-30 10:27:35,623 INFO org.apache.hadoop.hbase.master.HMaster: Master startup proceeding: master failover 
> 2011-12-30 10:27:36,002 INFO org.apache.hadoop.hbase.master.AssignmentManager: Failed-over master needs to process 1717 regions in transition 
> 2011-12-30 10:27:36,004 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown. 
> java.lang.ArrayIndexOutOfBoundsException: -256 
> at org.apache.hadoop.hbase.executor.RegionTransitionData.readFields(RegionTransitionData.java:148) 
> at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:105) 
> at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:75) 
> at org.apache.hadoop.hbase.executor.RegionTransitionData.fromBytes(RegionTransitionData.java:198) 
> at org.apache.hadoop.hbase.zookeeper.ZKAssign.getData(ZKAssign.java:743) 
> at org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:262) 
> at org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:223) 
> at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:401) 
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:283)
> {code}
> and there is no clue on how to clean-up the stale znode(s) from unassigned using zkCli.sh (del /hbase/unassigned/<bad region name>). It would be good if we include the bad region name in IOException from RegionTransitionData.readFields().
> {code}
> @Override
>   public void readFields(DataInput in) throws IOException {
>     // the event type byte
>     eventType = EventType.values()[in.readShort()];
>     // the timestamp
>     stamp = in.readLong();
>     // the encoded name of the region being transitioned
>     regionName = Bytes.readByteArray(in);
>     // remaining fields are optional so prefixed with boolean
>     // the name of the regionserver sending the data
>     if (in.readBoolean()) {
>       byte [] versionedBytes = Bytes.readByteArray(in);
>       this.origin = ServerName.parseVersionedServerName(versionedBytes);
>     }
>     if (in.readBoolean()) {
>       this.payload = Bytes.readByteArray(in);
>     }
>   }
> {code}
> If the code execution has survived until regionName then we can include the regionName in IOException with error message to clean-up the stale znode(s) under /hbase/unassigned.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira