You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Subru Krishnan (JIRA)" <ji...@apache.org> on 2017/08/03 19:47:00 UTC
[jira] [Comment Edited] (YARN-6900) ZooKeeper based implementation
of the FederationStateStore
[ https://issues.apache.org/jira/browse/YARN-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111986#comment-16111986 ]
Subru Krishnan edited comment on YARN-6900 at 8/3/17 7:46 PM:
--------------------------------------------------------------
Thanks [~elgoiri] for working on this. I looked at the patch, please find my comments below:
* Looks like we have common ZooKeeper interaction functionalities like *createAndStartCurator, getZKAuths, getZKAcls, create, delete, exists, getNodePath*,... etc which we can share with {{ZKRMStateStore}}. I suggest moving these to a common {{YarnZKUtils}} in _yarn-server-common_ so that it be reused. This should also be useful for YARN-6840.
* ZK_RM_STATE_STORE_PARENT_PATH --> ZK_FEDERATION_STATE_STORE_PARENT_PATH as this is federation state across clusters.
* Can we materialize *appsZNode, membershipZNode, policiesZNode* as we use them a lot and they don't change post initialization?
* Nit: can we have a hierarchy layout as part of the class Javadocs like in {{ZKRMStateStore}} for clarity?
* We should make {{ZookeeperFederationStateStore}} the default {{FederationStateStore}} implementation.
* Update the documentation {{hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md}} based on the above change.
* Lastly, can you take a look at the Yetus checkstyle/whitespace warnings?
was (Author: subru):
Thanks [~elgoiri] for working on this. I looked at the patch, please find my comments below:
* Looks like we have common ZooKeeper interaction functionalities like *createAndStartCurator, getZKAuths, getZKAcls, create, delete, exists, getNodePath*,... etc which we can share with {{ZKRMStateStore}}. I suggest moving these to a common {{YarnZKUtils}} in _yarn-server-common_ so that it be reused. This should also be useful for YARN-6840.
* ZK_RM_STATE_STORE_PARENT_PATH --> ZK_FEDERATION_STATE_STORE_PARENT_PATH as this is federation state across clusters.
* Can we materialize *appsZNode, membershipZNode, policiesZNode* as we use them a lot and they don't change post initialization?
* Nit: can we have a hierarchy layout as part of the class Javadocs like in {{ZKRMStateStore}} for clarity?
* We should {{ZookeeperFederationStateStore}} the default {{FederationStateStore}} implementation.
* Update the documentation {{hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/Federation.md}} based on the above change.
* Lastly, can you take a look at the Yetus checkstyle/whitespace warnings?
> ZooKeeper based implementation of the FederationStateStore
> ----------------------------------------------------------
>
> Key: YARN-6900
> URL: https://issues.apache.org/jira/browse/YARN-6900
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation, nodemanager, resourcemanager
> Reporter: Subru Krishnan
> Assignee: Inigo Goiri
> Attachments: YARN-6900-002.patch, YARN-6900-YARN-2915-000.patch, YARN-6900-YARN-2915-001.patch
>
>
> YARN-5408 defines the unified {{FederationStateStore}} API. Currently we only support SQL based stores, this JIRA tracks adding a ZooKeeper based implementation for simplifying deployment as it's already popularly used for {{RMStateStore}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org