You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2009/05/13 22:20:45 UTC

[jira] Created: (HBASE-1421) Processing a regionserver message -- OPEN, CLOSE, SPLIT, etc. -- and if we're carrying more than one message in payload, if exception, all messages that follow are dropped on floor

Processing a regionserver message -- OPEN, CLOSE, SPLIT, etc. -- and if we're carrying more than one message in payload, if exception, all messages that follow are dropped on floor
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

                 Key: HBASE-1421
                 URL: https://issues.apache.org/jira/browse/HBASE-1421
             Project: Hadoop HBase
          Issue Type: Bug
            Reporter: stack
            Priority: Blocker
             Fix For: 0.20.0


Just saw this in pset cluster.  Marking blocker. 

We had an incidence of HBASE-1344 on our 0.19.x era hbase cluster.  The report from the regionserver was carrying at least two open messages.  The first provoked the exception, the second open message was never processed.  Regionserver thought it had successfully opened region.  Master didn't know anything about it.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HBASE-1421) Processing a regionserver message -- OPEN, CLOSE, SPLIT, etc. -- and if we're carrying more than one message in payload, if exception, all messages that follow are dropped on floor

Posted by "stack (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710438#action_12710438 ] 

stack commented on HBASE-1421:
------------------------------

MSG_REPORT_PROCESS_OPEN doesn't seem to do anything master-side any more (I thought it used to update timers on master-side)?  This means that if the open message is lost, then we don't try open again seemingly?   Seems broke.

Otherwise, I went through the processing of messages returned by the regionserver and tried remove all places where we threw unchecked exceptions in particular.  I also changed the process message signatures so they don't throw even IOExceptions.  Instead we just log warnings since most of the time these are non-fatal anyway and even if they are damaging, we probably want to keep going with a warning log rather than throw an exception that can possibly do even more damage.

> Processing a regionserver message -- OPEN, CLOSE, SPLIT, etc. -- and if we're carrying more than one message in payload, if exception, all messages that follow are dropped on floor
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-1421
>                 URL: https://issues.apache.org/jira/browse/HBASE-1421
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>            Priority: Blocker
>             Fix For: 0.20.0
>
>         Attachments: 1421.patch
>
>
> Just saw this in pset cluster.  Marking blocker. 
> We had an incidence of HBASE-1344 on our 0.19.x era hbase cluster.  The report from the regionserver was carrying at least two open messages.  The first provoked the exception, the second open message was never processed.  Regionserver thought it had successfully opened region.  Master didn't know anything about it.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Assigned: (HBASE-1421) Processing a regionserver message -- OPEN, CLOSE, SPLIT, etc. -- and if we're carrying more than one message in payload, if exception, all messages that follow are dropped on floor

Posted by "stack (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack reassigned HBASE-1421:
----------------------------

    Assignee: stack

> Processing a regionserver message -- OPEN, CLOSE, SPLIT, etc. -- and if we're carrying more than one message in payload, if exception, all messages that follow are dropped on floor
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-1421
>                 URL: https://issues.apache.org/jira/browse/HBASE-1421
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: stack
>            Priority: Blocker
>             Fix For: 0.20.0
>
>         Attachments: 1421.patch
>
>
> Just saw this in pset cluster.  Marking blocker. 
> We had an incidence of HBASE-1344 on our 0.19.x era hbase cluster.  The report from the regionserver was carrying at least two open messages.  The first provoked the exception, the second open message was never processed.  Regionserver thought it had successfully opened region.  Master didn't know anything about it.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Resolved: (HBASE-1421) Processing a regionserver message -- OPEN, CLOSE, SPLIT, etc. -- and if we're carrying more than one message in payload, if exception, all messages that follow are dropped on floor

Posted by "stack (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack resolved HBASE-1421.
--------------------------

    Resolution: Fixed

Committed branch and trunk.

> Processing a regionserver message -- OPEN, CLOSE, SPLIT, etc. -- and if we're carrying more than one message in payload, if exception, all messages that follow are dropped on floor
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-1421
>                 URL: https://issues.apache.org/jira/browse/HBASE-1421
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>            Priority: Blocker
>             Fix For: 0.20.0
>
>         Attachments: 1421.patch
>
>
> Just saw this in pset cluster.  Marking blocker. 
> We had an incidence of HBASE-1344 on our 0.19.x era hbase cluster.  The report from the regionserver was carrying at least two open messages.  The first provoked the exception, the second open message was never processed.  Regionserver thought it had successfully opened region.  Master didn't know anything about it.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HBASE-1421) Processing a regionserver message -- OPEN, CLOSE, SPLIT, etc. -- and if we're carrying more than one message in payload, if exception, all messages that follow are dropped on floor

Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jean-Daniel Cryans updated HBASE-1421:
--------------------------------------

    Fix Version/s: 0.19.3

> Processing a regionserver message -- OPEN, CLOSE, SPLIT, etc. -- and if we're carrying more than one message in payload, if exception, all messages that follow are dropped on floor
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-1421
>                 URL: https://issues.apache.org/jira/browse/HBASE-1421
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: stack
>            Priority: Blocker
>             Fix For: 0.20.0, 0.19.3
>
>         Attachments: 1421.patch
>
>
> Just saw this in pset cluster.  Marking blocker. 
> We had an incidence of HBASE-1344 on our 0.19.x era hbase cluster.  The report from the regionserver was carrying at least two open messages.  The first provoked the exception, the second open message was never processed.  Regionserver thought it had successfully opened region.  Master didn't know anything about it.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HBASE-1421) Processing a regionserver message -- OPEN, CLOSE, SPLIT, etc. -- and if we're carrying more than one message in payload, if exception, all messages that follow are dropped on floor

Posted by "stack (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack updated HBASE-1421:
-------------------------

    Attachment: 1421.patch

> Processing a regionserver message -- OPEN, CLOSE, SPLIT, etc. -- and if we're carrying more than one message in payload, if exception, all messages that follow are dropped on floor
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-1421
>                 URL: https://issues.apache.org/jira/browse/HBASE-1421
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>            Priority: Blocker
>             Fix For: 0.20.0
>
>         Attachments: 1421.patch
>
>
> Just saw this in pset cluster.  Marking blocker. 
> We had an incidence of HBASE-1344 on our 0.19.x era hbase cluster.  The report from the regionserver was carrying at least two open messages.  The first provoked the exception, the second open message was never processed.  Regionserver thought it had successfully opened region.  Master didn't know anything about it.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.