You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jesse Yates (JIRA)" <ji...@apache.org> on 2012/08/13 23:30:38 UTC
[jira] [Created] (HBASE-6571) Generic multi-thread/cross-process
error handling framework
Jesse Yates created HBASE-6571:
----------------------------------
Summary: Generic multi-thread/cross-process error handling framework
Key: HBASE-6571
URL: https://issues.apache.org/jira/browse/HBASE-6571
Project: HBase
Issue Type: Sub-task
Reporter: Jesse Yates
Assignee: Jesse Yates
The idea for a generic inter-process error-handling framework came from working on HBASE-6055 (snapshots). Distributed snapshots require tight time constraints in taking a snapshot to minimize offline time in face of errors. However, we often need to coordinate errors between processes and the current Abortable framework is not sufficiently flexible to handle the multitude of situations that can occur when coordinating between all region servers, the master and zookeeper. Using this framework error handling for snapshots was a simple matter, amounting to maybe 200 LOC.
This seems to be a generally useful framework and can be used to easily add inter-process error handling in HBase. The most obvious immediate usage is as part of HBASE-5487 when coordinating multiple sub-tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6571) Generic multi-thread/cross-process
error handling framework
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437414#comment-13437414 ]
stack commented on HBASE-6571:
------------------------------
Doc is missing an author and date and link to the issue. I stuck some review up on RB. Agree this is something else than Abortable (Abortable could add an error to this error bus)
> Generic multi-thread/cross-process error handling framework
> -----------------------------------------------------------
>
> Key: HBASE-6571
> URL: https://issues.apache.org/jira/browse/HBASE-6571
> Project: HBase
> Issue Type: Sub-task
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: Distributed Error Monitoring.docx, java_HBASE-6571-v0.patch
>
>
> The idea for a generic inter-process error-handling framework came from working on HBASE-6055 (snapshots). Distributed snapshots require tight time constraints in taking a snapshot to minimize offline time in face of errors. However, we often need to coordinate errors between processes and the current Abortable framework is not sufficiently flexible to handle the multitude of situations that can occur when coordinating between all region servers, the master and zookeeper. Using this framework error handling for snapshots was a simple matter, amounting to maybe 200 LOC.
> This seems to be a generally useful framework and can be used to easily add inter-process error handling in HBase. The most obvious immediate usage is as part of HBASE-5487 when coordinating multiple sub-tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6571) Generic multi-thread/cross-process
error handling framework
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434343#comment-13434343 ]
Jesse Yates commented on HBASE-6571:
------------------------------------
@Andrew
bq. We should not have two error notification frameworks (this and Abortable) coexisting for long.
I'd argue that we can have both, but we can wrap the abortable code (which is a nice abstraction) into the general framework.
> Generic multi-thread/cross-process error handling framework
> -----------------------------------------------------------
>
> Key: HBASE-6571
> URL: https://issues.apache.org/jira/browse/HBASE-6571
> Project: HBase
> Issue Type: Sub-task
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: Distributed Error Monitoring.docx, java_HBASE-6571-v0.patch
>
>
> The idea for a generic inter-process error-handling framework came from working on HBASE-6055 (snapshots). Distributed snapshots require tight time constraints in taking a snapshot to minimize offline time in face of errors. However, we often need to coordinate errors between processes and the current Abortable framework is not sufficiently flexible to handle the multitude of situations that can occur when coordinating between all region servers, the master and zookeeper. Using this framework error handling for snapshots was a simple matter, amounting to maybe 200 LOC.
> This seems to be a generally useful framework and can be used to easily add inter-process error handling in HBase. The most obvious immediate usage is as part of HBASE-5487 when coordinating multiple sub-tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6571) Generic multi-thread/cross-process
error handling framework
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates resolved HBASE-6571.
--------------------------------
Resolution: Fixed
Closing - committed to https://github.com/jyates/hbase/tree/snapshots
> Generic multi-thread/cross-process error handling framework
> -----------------------------------------------------------
>
> Key: HBASE-6571
> URL: https://issues.apache.org/jira/browse/HBASE-6571
> Project: HBase
> Issue Type: Sub-task
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Fix For: hbase-6055
>
> Attachments: Distributed Error Monitoring.docx, java_HBASE-6571-v0.patch
>
>
> The idea for a generic inter-process error-handling framework came from working on HBASE-6055 (snapshots). Distributed snapshots require tight time constraints in taking a snapshot to minimize offline time in face of errors. However, we often need to coordinate errors between processes and the current Abortable framework is not sufficiently flexible to handle the multitude of situations that can occur when coordinating between all region servers, the master and zookeeper. Using this framework error handling for snapshots was a simple matter, amounting to maybe 200 LOC.
> This seems to be a generally useful framework and can be used to easily add inter-process error handling in HBase. The most obvious immediate usage is as part of HBASE-5487 when coordinating multiple sub-tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6571) Generic multi-thread/cross-process
error handling framework
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456446#comment-13456446 ]
Jesse Yates commented on HBASE-6571:
------------------------------------
+1 for commit from Hseih on RB... waiting another day for any other comments, but otherwise will commit to branch (modulo nits).
> Generic multi-thread/cross-process error handling framework
> -----------------------------------------------------------
>
> Key: HBASE-6571
> URL: https://issues.apache.org/jira/browse/HBASE-6571
> Project: HBase
> Issue Type: Sub-task
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Fix For: hbase-6055
>
> Attachments: Distributed Error Monitoring.docx, java_HBASE-6571-v0.patch
>
>
> The idea for a generic inter-process error-handling framework came from working on HBASE-6055 (snapshots). Distributed snapshots require tight time constraints in taking a snapshot to minimize offline time in face of errors. However, we often need to coordinate errors between processes and the current Abortable framework is not sufficiently flexible to handle the multitude of situations that can occur when coordinating between all region servers, the master and zookeeper. Using this framework error handling for snapshots was a simple matter, amounting to maybe 200 LOC.
> This seems to be a generally useful framework and can be used to easily add inter-process error handling in HBase. The most obvious immediate usage is as part of HBASE-5487 when coordinating multiple sub-tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6571) Generic multi-thread/cross-process
error handling framework
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437321#comment-13437321 ]
Jonathan Hsieh commented on HBASE-6571:
---------------------------------------
I'm not convinced that Abortable is at the same level as this framework and that "one shall stand and one shall fall" -- Abortable is essentially for killing a parent process (RS/HM/HBCK/Client) while this is more of just a thread or possibly remote process.
> Generic multi-thread/cross-process error handling framework
> -----------------------------------------------------------
>
> Key: HBASE-6571
> URL: https://issues.apache.org/jira/browse/HBASE-6571
> Project: HBase
> Issue Type: Sub-task
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: Distributed Error Monitoring.docx, java_HBASE-6571-v0.patch
>
>
> The idea for a generic inter-process error-handling framework came from working on HBASE-6055 (snapshots). Distributed snapshots require tight time constraints in taking a snapshot to minimize offline time in face of errors. However, we often need to coordinate errors between processes and the current Abortable framework is not sufficiently flexible to handle the multitude of situations that can occur when coordinating between all region servers, the master and zookeeper. Using this framework error handling for snapshots was a simple matter, amounting to maybe 200 LOC.
> This seems to be a generally useful framework and can be used to easily add inter-process error handling in HBase. The most obvious immediate usage is as part of HBASE-5487 when coordinating multiple sub-tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6571) Generic multi-thread/cross-process
error handling framework
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434334#comment-13434334 ]
Andrew Purtell commented on HBASE-6571:
---------------------------------------
Really like the fault injection facility.
We should not have two error notification frameworks (this and Abortable) coexisting for long.
> Generic multi-thread/cross-process error handling framework
> -----------------------------------------------------------
>
> Key: HBASE-6571
> URL: https://issues.apache.org/jira/browse/HBASE-6571
> Project: HBase
> Issue Type: Sub-task
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: Distributed Error Monitoring.docx, java_HBASE-6571-v0.patch
>
>
> The idea for a generic inter-process error-handling framework came from working on HBASE-6055 (snapshots). Distributed snapshots require tight time constraints in taking a snapshot to minimize offline time in face of errors. However, we often need to coordinate errors between processes and the current Abortable framework is not sufficiently flexible to handle the multitude of situations that can occur when coordinating between all region servers, the master and zookeeper. Using this framework error handling for snapshots was a simple matter, amounting to maybe 200 LOC.
> This seems to be a generally useful framework and can be used to easily add inter-process error handling in HBase. The most obvious immediate usage is as part of HBASE-5487 when coordinating multiple sub-tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6571) Generic multi-thread/cross-process
error handling framework
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hsieh updated HBASE-6571:
----------------------------------
Fix Version/s: hbase-6055
> Generic multi-thread/cross-process error handling framework
> -----------------------------------------------------------
>
> Key: HBASE-6571
> URL: https://issues.apache.org/jira/browse/HBASE-6571
> Project: HBase
> Issue Type: Sub-task
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Fix For: hbase-6055
>
> Attachments: Distributed Error Monitoring.docx, java_HBASE-6571-v0.patch
>
>
> The idea for a generic inter-process error-handling framework came from working on HBASE-6055 (snapshots). Distributed snapshots require tight time constraints in taking a snapshot to minimize offline time in face of errors. However, we often need to coordinate errors between processes and the current Abortable framework is not sufficiently flexible to handle the multitude of situations that can occur when coordinating between all region servers, the master and zookeeper. Using this framework error handling for snapshots was a simple matter, amounting to maybe 200 LOC.
> This seems to be a generally useful framework and can be used to easily add inter-process error handling in HBase. The most obvious immediate usage is as part of HBASE-5487 when coordinating multiple sub-tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6571) Generic multi-thread/cross-process
error handling framework
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434347#comment-13434347 ]
Andrew Purtell commented on HBASE-6571:
---------------------------------------
bq. we can wrap the abortable code (which is a nice abstraction) into the general framework.
This gets us to the same place.
> Generic multi-thread/cross-process error handling framework
> -----------------------------------------------------------
>
> Key: HBASE-6571
> URL: https://issues.apache.org/jira/browse/HBASE-6571
> Project: HBase
> Issue Type: Sub-task
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: Distributed Error Monitoring.docx, java_HBASE-6571-v0.patch
>
>
> The idea for a generic inter-process error-handling framework came from working on HBASE-6055 (snapshots). Distributed snapshots require tight time constraints in taking a snapshot to minimize offline time in face of errors. However, we often need to coordinate errors between processes and the current Abortable framework is not sufficiently flexible to handle the multitude of situations that can occur when coordinating between all region servers, the master and zookeeper. Using this framework error handling for snapshots was a simple matter, amounting to maybe 200 LOC.
> This seems to be a generally useful framework and can be used to easily add inter-process error handling in HBase. The most obvious immediate usage is as part of HBASE-5487 when coordinating multiple sub-tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6571) Generic multi-thread/cross-process
error handling framework
Posted by "Jesse Yates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-6571:
-------------------------------
Attachment: Distributed Error Monitoring.docx
java_HBASE-6571-v0.patch
Attaching patch and design doc, posting to RB momentarily.
> Generic multi-thread/cross-process error handling framework
> -----------------------------------------------------------
>
> Key: HBASE-6571
> URL: https://issues.apache.org/jira/browse/HBASE-6571
> Project: HBase
> Issue Type: Sub-task
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: Distributed Error Monitoring.docx, java_HBASE-6571-v0.patch
>
>
> The idea for a generic inter-process error-handling framework came from working on HBASE-6055 (snapshots). Distributed snapshots require tight time constraints in taking a snapshot to minimize offline time in face of errors. However, we often need to coordinate errors between processes and the current Abortable framework is not sufficiently flexible to handle the multitude of situations that can occur when coordinating between all region servers, the master and zookeeper. Using this framework error handling for snapshots was a simple matter, amounting to maybe 200 LOC.
> This seems to be a generally useful framework and can be used to easily add inter-process error handling in HBase. The most obvious immediate usage is as part of HBASE-5487 when coordinating multiple sub-tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6571) Generic multi-thread/cross-process
error handling framework
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437105#comment-13437105 ]
Jonathan Hsieh commented on HBASE-6571:
---------------------------------------
review board here: https://reviews.apache.org/r/6589/
> Generic multi-thread/cross-process error handling framework
> -----------------------------------------------------------
>
> Key: HBASE-6571
> URL: https://issues.apache.org/jira/browse/HBASE-6571
> Project: HBase
> Issue Type: Sub-task
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: Distributed Error Monitoring.docx, java_HBASE-6571-v0.patch
>
>
> The idea for a generic inter-process error-handling framework came from working on HBASE-6055 (snapshots). Distributed snapshots require tight time constraints in taking a snapshot to minimize offline time in face of errors. However, we often need to coordinate errors between processes and the current Abortable framework is not sufficiently flexible to handle the multitude of situations that can occur when coordinating between all region servers, the master and zookeeper. Using this framework error handling for snapshots was a simple matter, amounting to maybe 200 LOC.
> This seems to be a generally useful framework and can be used to easily add inter-process error handling in HBase. The most obvious immediate usage is as part of HBASE-5487 when coordinating multiple sub-tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira