You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2014/04/06 19:06:14 UTC

[jira] [Commented] (HBASE-9864) Notifications bus for use by cluster members keeping up-to-date on changes

    [ https://issues.apache.org/jira/browse/HBASE-9864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13961461#comment-13961461 ] 

Andrew Purtell commented on HBASE-9864:
---------------------------------------

See HBASE-10919. It could be useful if the envisioned distributed cache there could be propagated among RegionServers in the background with best effort. Update conflicts could be resolved with a latest update wins policy and cache misses during propagation would be fine.

Consider a trivial internal distributed non-persistent store where we reserve a small proportion of heap for in-memory storage and propagate updates from RegionServer to RegionServer using an epidemic gossip protocol over UDP. We could use this for the AccessController's table/CF ACL cache if in addition we support pinning of specific items to prevent eviction and provide an alternative update propagation option using Procedures to guarantee the global consistency of the update.

Then we would have for distributing internal state a lightweight option for lazy propagation when only that is needed (idempotent updates, eventual consistency) and a heavyweight option for assuring change notification or state updates are delivered to all RegionServers. Neither need be tightly coupled to ZooKeeper. The gossip protocol would have no relationship to ZooKeeper. The Procedure based update should be abstracted from ZK after HBASE-10909.

> Notifications bus for use by cluster members keeping up-to-date on changes
> --------------------------------------------------------------------------
>
>                 Key: HBASE-9864
>                 URL: https://issues.apache.org/jira/browse/HBASE-9864
>             Project: HBase
>          Issue Type: Brainstorming
>            Reporter: stack
>            Priority: Blocker
>             Fix For: 1.0.0
>
>
> In namespaces and acls, zk callbacks are used so all participating servers are notified when there is a change in acls/namespaces list.
> The new visibility tags feature coming in copies the same model of using zk with listeners for the features' particular notifications.
> Three systems each w/ their own implementation of the notifications all using zk w/ their own feature-specific watchers.
> Should probably unify.
> Do we have to go via zk?  Seems like all want to be notified when an hbase table is updated.  Could we tell servers directly rather than go via zk?



--
This message was sent by Atlassian JIRA
(v6.2#6252)