[jira] [Comment Edited] (ACCUMULO-4050) Serialize table configs into a single ZK node

["Christopher Tubbs (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ACCUMULO-4050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15485914#comment-15485914 ] 

Christopher Tubbs edited comment on ACCUMULO-4050 at 9/13/16 1:39 AM:
----------------------------------------------------------------------

The following is what I've been discussing with [~milleruntime]:

By "container", I mean an object that is composed of the configuration (set of key value pairs) deserialized from ZK, as well as functions to perform the serialization/deserialization, and which can atomically swap out the reference to the complete configuration. Its basic API is "getSnapshot()" to get a copy of the current configuration reference it contains.

By "snapshot", I mean an isolated immutable view of the entire set of configuration (k-v pairs) at a single point in time (for that context: table, namespace, etc.). It doesn't have anything to do with ZooKeeper. Snapshot, as I use it, refers to any such reference.

By "snapshot hierarchy", I mean, a special snapshot with all the relevant snapshot contexts linked together on demand: table conf snapshot -> namespace conf snapshot -> etc.

I've been using "container", because it contains a reference to a "snapshot", which can be passed when requested. But it could just as easily be called a "factory" or "provider" of the configuration object. It only holds the configuration, but does not provide methods to access individual configuration elements, like the "snapshot" object would.

Thoughts on cluster-wide synchronizing is mostly unrelated to this issue... but I suppose we could incorporate the ZNode Version into the snapshot object or its containers somehow. We could then add a ZNode version prerequisite to our RPCs which would ensure that TServers update their cache from ZK, before processing the client's request. This might only be appropriate for some RPCs. We could also have a new RPC for just verifying that it has been propagated. Any of that would definitely be follow-on work.



was (Author: ctubbsii):
The following is what I've been discussing with [~milleruntime]:

By "container", I mean an object that is composed of the configuration (set of key value pairs) deserialized from ZK, as well as functions to perform the serialization/deserialization, and which can atomically swap out the reference to the complete configuration. It's basic API is "getSnapshot()" to get a copy of the current configuration reference it contains.

By "snapshot", I mean an isolated immutable view of the entire set of configuration (k-v pairs) at a single point in time. It doesn't have anything to do with ZooKeeper. Snapshot, as I use it, refers to any such reference.

I've been using "container", because it contains a reference to a "snapshot", which can be passed when requested. But it could just as easily be called a "factory" or "provider" of the configuration object. It only holds the configuration, but does not provide methods to access individual configuration elements, like the "snapshot" object would.

Thoughts on cluster-wide synchronizing is mostly unrelated to this issue... but I suppose we could incorporate the ZNode Version into the snapshot object or its containers somehow. We could then add a ZNode version prerequisite to our RPCs which would ensure that TServers update their cache from ZK, before processing the client's request. This might only be appropriate for some RPCs. We could also have a new RPC for just verifying that it has been propagated. Any of that would definitely be follow-on work.


> Serialize table configs into a single ZK node
> ---------------------------------------------
>
>                 Key: ACCUMULO-4050
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4050
>             Project: Accumulo
>          Issue Type: Sub-task
>            Reporter: Christopher Tubbs
>             Fix For: 2.0.0
>
>
> Use a single ZK node to store table configs, rather than split across multiple configs. This should make it easier to implement atomic changes to the configuration, and should simplify reading loading/saving configuration from/to ZooKeeper.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)