[jira] [Issue Comment Edited] (HBASE-1730) Near-instantaneous online schema and table state updates

["Ted Yu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089829#comment-13089829 ] 

Ted Yu edited comment on HBASE-1730 at 8/23/11 10:45 PM:
---------------------------------------------------------

Stack, Ted, Nileema, Karthik, and I discussed the HBASE-1730 & HBASE-4213 methodologies at the HBase Hackathon yesterday.  Main Points:

1. 1730 & 4213 are not the same feature.  1730 is implementing online schema changes, 4213 is adding persistence to online schema changes during master failover.  We should think of the two JIRAs as disjoint in this way.
2. 1730 meets it's looser requirements.  Karthik & other developers had a handful of design concerns about 4213.  We should make sure the design & failure analysis is solid before committing that feature.
3. In the 4213/1730 overlap, there is some useful non-persistent functionality and design that should be incorporated into 1730 before commit.
4. What is the testing/maturity of the 4213 patch?  We have done extensive dev/cluster testing on 1730 internally & communicated this to the other committers.  The persistence on 4213 requires even more edge case analysis testing than this feature to meet its requirements.  None of us currently have any visibility into the 4213 testing & support.  @Subbu: Could you please elaborate more in JIRA/IRC about the maturity of your patch?

We would like to commit 1730 first since it is mature and iterate/solidify 4213 a little bit before committing.

      was (Author: nspiegelberg):
    Stack, Ted, Nileema, Karthik, and I discussed the HBASE-1730 & HBASE-4123 methodologies at the HBase Hackathon yesterday.  Main Points:

1. 1730 & 4123 are not the same feature.  1730 is implementing online schema changes, 4123 is adding persistence to online schema changes during master failover.  We should think of the two JIRAs as disjoint in this way.
2. 1730 meets it's looser requirements.  Karthik & other developers had a handful of design concerns about 4123.  We should make sure the design & failure analysis is solid before committing that feature.
3. In the 4123/1730 overlap, there is some useful non-persistent functionality and design that should be incorporated into 1730 before commit.
4. What is the testing/maturity of the 4123 patch?  We have done extensive dev/cluster testing on 1730 internally & communicated this to the other committers.  The persistence on 4123 requires even more edge case analysis testing than this feature to meet its requirements.  None of us currently have any visibility into the 4123 testing & support.  @Subbu: Could you please elaborate more in JIRA/IRC about the maturity of your patch?

We would like to commit 1730 first since it is mature and iterate/solidify 4123 a little bit before committing.
  
> Near-instantaneous online schema and table state updates
> --------------------------------------------------------
>
>                 Key: HBASE-1730
>                 URL: https://issues.apache.org/jira/browse/HBASE-1730
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>            Assignee: stack
>            Priority: Critical
>             Fix For: 0.92.0
>
>         Attachments: 1730-v2.patch, 1730-v3.patch, 1730.patch, HBASE-1730.patch
>
>
> We should not need to take a table offline to update HCD or HTD. 
> One option for that is putting HTDs and HCDs up into ZK, with mirror on disk catalog tables to be used only for cold init scenarios, as discussed on IRC. In this scheme, regionservers hosting regions of a table would watch permanent nodes in ZK associated with that table for schema updates and take appropriate actions out of the watcher. In effect, schema updates become another item in the ToDo list.
> {{/hbase/tables/<table-name>/schema}}
> Must be associated with a write locking scheme also handled with ZK primitives to avoid situations where one concurrent update clobbers another. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira