You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2021/11/24 20:17:50 UTC
[GitHub] [helix] xyuanlu opened a new pull request #1913: Implement RestSnapShot and substitute the kv maps in HelixDataAccessorWrapper to a RestSnapShot object
xyuanlu opened a new pull request #1913:
URL: https://github.com/apache/helix/pull/1913
### Issues
- [X] My PR addresses the following Helix issues and references them in the PR description:
#1896
### Description
- [X] Here are some details about my PR, including screenshots of any UI changes:
This change Implement RestSnapShot and substitute the kv maps in HelixDataAccessorWrapper to a RestSnapShot object.
The RestSnapshot object will be consumed by functions in clusterMaintenanceService.
### Tests
- [X] The following tests are written for this issue:
NA. The current TestHelixDataAccessorWrapper already covers the added functions.
- The following is the result of the "mvn test" command on the appropriate module:
(If CI test fails due to known issue, please specify the issue and test PR locally. Then copy & paste the result of "mvn test" to here.)
### Changes that Break Backward Compatibility (Optional)
- My PR contains changes that break backward compatibility or previous assumptions for certain methods or API. They include:
(Consider including all behavior changes for public methods or API. Also include these changes in merge description so that other developers are aware of these changes. This allows them to make relevant code changes in feature branches accounting for the new method/API behavior.)
### Documentation (Optional)
- In case of new functionality, my PR adds documentation in the following wiki page:
(Link the GitHub wiki you added)
### Commits
- My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Code Quality
- My diff has been formatted using helix-style.xml
(helix-style-intellij.xml if IntelliJ IDE is used)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] junkaixue commented on a change in pull request #1913: Implement RestSnapShot and substitute the kv maps in HelixDataAccessorWrapper to a RestSnapShot object
Posted by GitBox <gi...@apache.org>.
junkaixue commented on a change in pull request #1913:
URL: https://github.com/apache/helix/pull/1913#discussion_r761407543
##########
File path: helix-common/src/main/java/org/apache/helix/Snapshot.java
##########
@@ -0,0 +1,25 @@
+package org.apache.helix;
Review comment:
Let's make a folder as datamodel?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] junkaixue merged pull request #1913: Implement RestSnapShot and substitute the kv maps in HelixDataAccessorWrapper to a RestSnapShot object
Posted by GitBox <gi...@apache.org>.
junkaixue merged pull request #1913:
URL: https://github.com/apache/helix/pull/1913
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly commented on a change in pull request #1913: Implement RestSnapShot and substitute the kv maps in HelixDataAccessorWrapper to a RestSnapShot object
Posted by GitBox <gi...@apache.org>.
narendly commented on a change in pull request #1913:
URL: https://github.com/apache/helix/pull/1913#discussion_r760583720
##########
File path: helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/api/OperationAbstractClass.java
##########
@@ -19,14 +19,22 @@
* under the License.
*/
+import java.util.ArrayList;
+import java.util.Arrays;
import java.util.Collection;
+import java.util.List;
import java.util.Map;
+import org.apache.helix.PropertyType;
import org.apache.helix.rest.clusterMaintenanceService.MaintenanceManagementInstanceInfo;
import org.apache.helix.rest.common.datamodel.RestSnapShot;
interface OperationAbstractClass {
+
+ List<PropertyType> PROPERTY_TYPE_LIST = new ArrayList<>(Arrays
Review comment:
Where is this used? Can or should this just be a static array?
##########
File path: helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/api/OperationAbstractClass.java
##########
@@ -19,14 +19,22 @@
* under the License.
*/
+import java.util.ArrayList;
+import java.util.Arrays;
import java.util.Collection;
+import java.util.List;
import java.util.Map;
+import org.apache.helix.PropertyType;
import org.apache.helix.rest.clusterMaintenanceService.MaintenanceManagementInstanceInfo;
import org.apache.helix.rest.common.datamodel.RestSnapShot;
interface OperationAbstractClass {
+
+ List<PropertyType> PROPERTY_TYPE_LIST = new ArrayList<>(Arrays
Review comment:
Where is this used? Can or should this just be a static final array?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] xyuanlu commented on pull request #1913: Implement RestSnapShot and substitute the kv maps in HelixDataAccessorWrapper to a RestSnapShot object
Posted by GitBox <gi...@apache.org>.
xyuanlu commented on pull request #1913:
URL: https://github.com/apache/helix/pull/1913#issuecomment-985099732
Push after rebase and resolve conflict.
This PR is ready to be merged. Approved by @junkaixue // Thanks!
Commit message:
Implement RestSnapShot and substitute the kv maps in HelixDataAccessorWrapper.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] xyuanlu commented on a change in pull request #1913: Implement RestSnapShot and substitute the kv maps in HelixDataAccessorWrapper to a RestSnapShot object
Posted by GitBox <gi...@apache.org>.
xyuanlu commented on a change in pull request #1913:
URL: https://github.com/apache/helix/pull/1913#discussion_r760529385
##########
File path: helix-rest/src/main/java/org/apache/helix/rest/common/HelixDataAccessorWrapper.java
##########
@@ -91,6 +93,35 @@ public HelixDataAccessorWrapper(ZKHelixDataAccessor dataAccessor,
super(dataAccessor);
_restClient = customRestClient;
_namespace = namespace;
+ _restSnapShot = new RestSnapshotWrapper(_clusterName);
+ }
+
+ static class RestSnapshotWrapper extends RestSnapShot {
Review comment:
TFTR. We need to access the generic getter and setter in DataAccessorWrapper, but the getter and setter is private in the RestSnapShot class. So I extends RestSnapShot tp have access to these private functions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] xyuanlu commented on a change in pull request #1913: Implement RestSnapShot and substitute the kv maps in HelixDataAccessorWrapper to a RestSnapShot object
Posted by GitBox <gi...@apache.org>.
xyuanlu commented on a change in pull request #1913:
URL: https://github.com/apache/helix/pull/1913#discussion_r761534120
##########
File path: helix-common/src/main/java/org/apache/helix/Snapshot.java
##########
@@ -0,0 +1,25 @@
+package org.apache.helix;
Review comment:
TFTR. Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] junkaixue commented on a change in pull request #1913: Implement RestSnapShot and substitute the kv maps in HelixDataAccessorWrapper to a RestSnapShot object
Posted by GitBox <gi...@apache.org>.
junkaixue commented on a change in pull request #1913:
URL: https://github.com/apache/helix/pull/1913#discussion_r756540805
##########
File path: helix-rest/src/main/java/org/apache/helix/rest/common/HelixDataAccessorWrapper.java
##########
@@ -205,25 +210,62 @@ public HelixDataAccessorWrapper(ZKHelixDataAccessor dataAccessor,
}
}
+ public RestSnapShot getRestSnapShot() {
+ return _restSnapShot;
+ }
+
@Override
public <T extends HelixProperty> T getProperty(PropertyKey key) {
- if (_propertyCache.containsKey(key)) {
- return (T) _propertyCache.get(key);
+ T property = _restSnapShot.getProperty(key);
+ if (property == null) {
+ property = super.getProperty(key);
+ _restSnapShot.updateProperty(key, property);
}
- T property = super.getProperty(key);
- _propertyCache.put(key, property);
+
return property;
}
@Override
public List<String> getChildNames(PropertyKey key) {
- if (_batchNameCache.containsKey(key)) {
- return _batchNameCache.get(key);
+ List<String> names = _restSnapShot.getChildNames(key);
+ if (names == null) {
+ names = super.getChildNames(key);
+ _restSnapShot.updateChildNames(key, names);
}
+ return names;
+ }
- List<String> names = super.getChildNames(key);
- _batchNameCache.put(key, names);
+ public void fetchIdealStatesExternalViewStateModel() {
Review comment:
Let's make comments for the user would like to implement these functions.
##########
File path: helix-rest/src/main/java/org/apache/helix/rest/common/datamodel/RestSnapShot.java
##########
@@ -19,18 +18,69 @@
* under the License.
*/
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.helix.HelixProperty;
+import org.apache.helix.PropertyKey;
+import org.apache.helix.PropertyType;
+
/* This Snapshot can extend Snapshot from common/core module
* once there is more generic snapshot.
+ * An Snapshot object should contain all the Helix related info that an implementation of
+ * OperationAbstractClass would need.
*/
+
+// TODO: Future: Support hierarchical Snapshot type for other services besides cluster MaintenanceService.
+
public class RestSnapShot {
- /* An Snapshot object should contain all the Helix related info that an implementation of
- * OperationAbstractClass would need.
- */
+ private final Map<PropertyKey, HelixProperty> _propertyCache;
+ private final Map<PropertyKey, List<String>> _childNodesCache;
+ private Set<PropertyType> _propertyTypes;
+ private String _clusterName;
+
+ public RestSnapShot(String clusterName) {
+ _propertyCache = new HashMap<>();
+ _childNodesCache = new HashMap<>();
+ _propertyTypes = new HashSet<>();
+ _clusterName = clusterName;
+ }
+
+ public <T extends HelixProperty> T getProperty(PropertyKey key) {
Review comment:
Do we support this kind of return type in SNAPSHOT for RoutingTable? If yes, let's keep this convention. Otherwise, this is more specific for snapshot public API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] junkaixue commented on a change in pull request #1913: Implement RestSnapShot and substitute the kv maps in HelixDataAccessorWrapper to a RestSnapShot object
Posted by GitBox <gi...@apache.org>.
junkaixue commented on a change in pull request #1913:
URL: https://github.com/apache/helix/pull/1913#discussion_r760508732
##########
File path: helix-rest/src/main/java/org/apache/helix/rest/common/HelixDataAccessorWrapper.java
##########
@@ -91,6 +93,35 @@ public HelixDataAccessorWrapper(ZKHelixDataAccessor dataAccessor,
super(dataAccessor);
_restClient = customRestClient;
_namespace = namespace;
+ _restSnapShot = new RestSnapshotWrapper(_clusterName);
+ }
+
+ static class RestSnapshotWrapper extends RestSnapShot {
Review comment:
Why we need this wrapper extend it? If we would like to make this RestSnapShot interface. We can have an impl for it. But if we already have a generic kv snapshot. We dont need to extend the class, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] xyuanlu commented on a change in pull request #1913: Implement RestSnapShot and substitute the kv maps in HelixDataAccessorWrapper to a RestSnapShot object
Posted by GitBox <gi...@apache.org>.
xyuanlu commented on a change in pull request #1913:
URL: https://github.com/apache/helix/pull/1913#discussion_r760597968
##########
File path: helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/api/OperationAbstractClass.java
##########
@@ -19,14 +19,22 @@
* under the License.
*/
+import java.util.ArrayList;
+import java.util.Arrays;
import java.util.Collection;
+import java.util.List;
import java.util.Map;
+import org.apache.helix.PropertyType;
import org.apache.helix.rest.clusterMaintenanceService.MaintenanceManagementInstanceInfo;
import org.apache.helix.rest.common.datamodel.RestSnapShot;
interface OperationAbstractClass {
+
+ List<PropertyType> PROPERTY_TYPE_LIST = new ArrayList<>(Arrays
Review comment:
This value will be read in clusterMaintenanceService take/freeInstance implementation. Advising what are the type of property needed when populating the snapshot.
Different child class may change this value, so may not be a static final.
##########
File path: helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/api/OperationAbstractClass.java
##########
@@ -19,14 +19,22 @@
* under the License.
*/
+import java.util.ArrayList;
+import java.util.Arrays;
import java.util.Collection;
+import java.util.List;
import java.util.Map;
+import org.apache.helix.PropertyType;
import org.apache.helix.rest.clusterMaintenanceService.MaintenanceManagementInstanceInfo;
import org.apache.helix.rest.common.datamodel.RestSnapShot;
interface OperationAbstractClass {
+
+ List<PropertyType> PROPERTY_TYPE_LIST = new ArrayList<>(Arrays
Review comment:
This value will be read in clusterMaintenanceService take/freeInstance implementation. Advising what are the type of property needed when populating the snapshot.
Different child classes may change this value, so may not be a static final.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] xyuanlu commented on a change in pull request #1913: Implement RestSnapShot and substitute the kv maps in HelixDataAccessorWrapper to a RestSnapShot object
Posted by GitBox <gi...@apache.org>.
xyuanlu commented on a change in pull request #1913:
URL: https://github.com/apache/helix/pull/1913#discussion_r758612442
##########
File path: helix-rest/src/main/java/org/apache/helix/rest/common/datamodel/RestSnapShot.java
##########
@@ -19,18 +18,69 @@
* under the License.
*/
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.helix.HelixProperty;
+import org.apache.helix.PropertyKey;
+import org.apache.helix.PropertyType;
+
/* This Snapshot can extend Snapshot from common/core module
* once there is more generic snapshot.
+ * An Snapshot object should contain all the Helix related info that an implementation of
+ * OperationAbstractClass would need.
*/
+
+// TODO: Future: Support hierarchical Snapshot type for other services besides cluster MaintenanceService.
+
public class RestSnapShot {
- /* An Snapshot object should contain all the Helix related info that an implementation of
- * OperationAbstractClass would need.
- */
+ private final Map<PropertyKey, HelixProperty> _propertyCache;
+ private final Map<PropertyKey, List<String>> _childNodesCache;
+ private Set<PropertyType> _propertyTypes;
+ private String _clusterName;
+
+ public RestSnapShot(String clusterName) {
+ _propertyCache = new HashMap<>();
+ _childNodesCache = new HashMap<>();
+ _propertyTypes = new HashSet<>();
+ _clusterName = clusterName;
+ }
+
+ public <T extends HelixProperty> T getProperty(PropertyKey key) {
Review comment:
TFTR. Updated to more specific functions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org