[jira] [Commented] (HBASE-1730) Near-instantaneous online schema and table state updates

["Andrew Purtell (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009468#comment-13009468 ] 

Andrew Purtell commented on HBASE-1730:
---------------------------------------

Discussed today on IRC in context of 0.92:

<St^Ack> apurtell: so, altering table online part of 0.92? done by may 1st?
<St^Ack> apurtell: its pretty critical
<St^Ack> not only for cps
<apurtell> i would agree
<apurtell> just the attrs right?
<apurtell> we can overlay HTD and HCD attrs in ZK, update HRIs in meta write behind
<St^Ack> apurtell: yes
<apurtell> St^Ack: yes i think we need this, that would take care of the CP case (we can watch attr updates and trigger loads); having admin triggered online merge also highly desirable (after split threshold upward adjustment)

<St^Ack> we can't put it up in zk as I used to think; dj_ryan points out that only transient data should be up in zk 'cos then we can just copy hdfs content to repro cluster elsewhere (otherwise have to copy out of zk too)
<apurtell> i want to mirror them up in zk
<apurtell> and change in zk first
<apurtell> and trigger actions via watches on the attr znodes
<apurtell> one action is write back to META of changes
<apurtell> other action could be to load a CP
<apurtell> other action would be to updated cached HRI or HRI-equivalent so we can e.g. change split threshold dynamically
<apurtell> so what is in zk is indeed transient
<apurtell> however we use it as a common point for online changes and use watches to replicate the changes cross cluster
<apurtell> ... well technically to trigger changes cross cluster :-)
<St^Ack> apurtell: oh, there is hbase-1730
<St^Ack> apurtell: stick your desire above into 1730... bit about mirroring

> Near-instantaneous online schema and table state updates
> --------------------------------------------------------
>
>                 Key: HBASE-1730
>                 URL: https://issues.apache.org/jira/browse/HBASE-1730
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>            Assignee: stack
>            Priority: Critical
>             Fix For: 0.92.0
>
>         Attachments: 1730-v2.patch, 1730-v3.patch, 1730.patch
>
>
> We should not need to take a table offline to update HCD or HTD. 
> One option for that is putting HTDs and HCDs up into ZK, with mirror on disk catalog tables to be used only for cold init scenarios, as discussed on IRC. In this scheme, regionservers hosting regions of a table would watch permanent nodes in ZK associated with that table for schema updates and take appropriate actions out of the watcher. In effect, schema updates become another item in the ToDo list.
> {{/hbase/tables/<table-name>/schema}}
> Must be associated with a write locking scheme also handled with ZK primitives to avoid situations where one concurrent update clobbers another. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira