You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by "xyuanlu (via GitHub)" <gi...@apache.org> on 2023/08/11 01:16:18 UTC
[GitHub] [helix] xyuanlu opened a new pull request, #2588: Add API for Instance evacuation
xyuanlu opened a new pull request, #2588:
URL: https://github.com/apache/helix/pull/2588
### Issues
- [ ] My PR addresses the following Helix issues and references them in the PR description:
(#200 - Link your issue number here: You can write "Fixes #XXX". Please use the proper keyword so that the issue gets closed automatically. See https://docs.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue
Any of the following keywords can be used: close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved)
### Description
- [ ] Here are some details about my PR, including screenshots of any UI changes:
(Write a concise description including what, why, how)
### Tests
- [ ] The following tests are written for this issue:
(List the names of added unit/integration tests)
- The following is the result of the "mvn test" command on the appropriate module:
(If CI test fails due to known issue, please specify the issue and test PR locally. Then copy & paste the result of "mvn test" to here.)
### Changes that Break Backward Compatibility (Optional)
- My PR contains changes that break backward compatibility or previous assumptions for certain methods or API. They include:
(Consider including all behavior changes for public methods or API. Also include these changes in merge description so that other developers are aware of these changes. This allows them to make relevant code changes in feature branches accounting for the new method/API behavior.)
### Documentation (Optional)
- In case of new functionality, my PR adds documentation in the following wiki page:
(Link the GitHub wiki you added)
### Commits
- My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Code Quality
- My diff has been formatted using helix-style.xml
(helix-style-intellij.xml if IntelliJ IDE is used)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] xyuanlu merged pull request #2588: Add API for Instance evacuation
Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu merged PR #2588:
URL: https://github.com/apache/helix/pull/2588
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] xyuanlu commented on a diff in pull request #2588: Add API for Instance evacuation
Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2588:
URL: https://github.com/apache/helix/pull/2588#discussion_r1294079803
##########
helix-common/src/main/java/org/apache/helix/constants/InstanceConstants.java:
##########
@@ -8,4 +8,12 @@ public enum InstanceDisabledType {
USER_OPERATION,
DEFAULT_INSTANCE_DISABLE_TYPE
}
+
+ public enum InstanceOperation {
+ EVACUATE,
+ SWAP_IN,
+ SWAP_OUT,
+ ENABLE,
+ DISABLE;
Review Comment:
TFTR. I suppose user can `enable` and disabled instance?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] MarkGaox commented on a diff in pull request #2588: Add API for Instance evacuation
Posted by "MarkGaox (via GitHub)" <gi...@apache.org>.
MarkGaox commented on code in PR #2588:
URL: https://github.com/apache/helix/pull/2588#discussion_r1293977624
##########
helix-core/src/test/java/org/apache/helix/mock/MockHelixAdmin.java:
##########
@@ -306,6 +305,33 @@ public void enableInstance(String clusterName, List<String> instances, boolean e
}
+ @Override
+ public void setInstanceOperation(String clusterName, String instanceName,
+ InstanceConstants.InstanceOperation instanceOperation) {
+
+ String path = PropertyPathBuilder.instanceConfig(clusterName, instanceName);
+
+ if (!_baseDataAccessor.exists(path, 0)) {
+ throw new HelixException(
+ "Cluster " + clusterName + ", instance: " + instanceName + ", instance config does not exist");
+ }
+
+ _baseDataAccessor.update(path, new DataUpdater<ZNRecord>() {
+ @Override
+ public ZNRecord update(ZNRecord currentData) {
+ if (currentData == null) {
+ throw new HelixException(
+ "Cluster: " + clusterName + ", instance: " + instanceName + ", participant config is null");
+ }
+
+ InstanceConfig config = new InstanceConfig(currentData);
+ config.setInstanceOperation(instanceOperation); // we set instance enabled in instance config
+ return config.getRecord();
+ }
+ }, AccessOption.PERSISTENT);
+
Review Comment:
nit: empty line.
##########
helix-rest/src/test/java/org/apache/helix/rest/server/TestPerInstanceAccessor.java:
##########
@@ -486,6 +486,32 @@ public void updateInstance() throws IOException {
Assert.assertEquals(new HashSet<>(instanceConfig.getDisabledPartitionsMap()
.get(CLUSTER_NAME + dbName.substring(0, dbName.length() - 1))),
new HashSet<>(Arrays.asList(CLUSTER_NAME + dbName + "0", CLUSTER_NAME + dbName + "3")));
+
+ // test set instance operation
+ new JerseyUriRequestBuilder("clusters/{}/instances/{}?command=setInstanceOperation&instanceOperation=EVACUATE")
+ .format(CLUSTER_NAME, INSTANCE_NAME).post(this, entity);
+ instanceConfig = _configAccessor.getInstanceConfig(CLUSTER_NAME, INSTANCE_NAME);
+ Assert.assertEquals(
+ instanceConfig.getInstanceOperation(), InstanceConstants.InstanceOperation.EVACUATE.toString());
+ // set operation to be DISABLE
+ new JerseyUriRequestBuilder("clusters/{}/instances/{}?command=setInstanceOperation&instanceOperation=DISABLE")
+ .format(CLUSTER_NAME, INSTANCE_NAME).post(this, entity);
+ instanceConfig = _configAccessor.getInstanceConfig(CLUSTER_NAME, INSTANCE_NAME);
+ Assert.assertEquals(
+ instanceConfig.getInstanceOperation(), InstanceConstants.InstanceOperation.DISABLE.toString());
+ Assert.assertTrue(!instanceConfig.getInstanceEnabled());
+ // set operation to EVACUATE, expect error
+ new JerseyUriRequestBuilder("clusters/{}/instances/{}?command=setInstanceOperation&instanceOperation=EVACUATE")
+ .expectedReturnStatusCode(Response.Status.BAD_REQUEST.getStatusCode())
+ .format(CLUSTER_NAME, INSTANCE_NAME).post(this, entity);
+ // set back to enable
+ new JerseyUriRequestBuilder("clusters/{}/instances/{}?command=setInstanceOperation&instanceOperation=ENABLE")
+ .format(CLUSTER_NAME, INSTANCE_NAME).post(this, entity);
+ instanceConfig = _configAccessor.getInstanceConfig(CLUSTER_NAME, INSTANCE_NAME);
+ Assert.assertEquals(
+ instanceConfig.getInstanceOperation(), InstanceConstants.InstanceOperation.ENABLE.toString());
+ Assert.assertTrue(instanceConfig.getInstanceEnabled());
+
Review Comment:
Suggest to add a case when the `instanceOperation` QueryParam is null. E.x, "clusters/{}/instances/{}?command=setInstanceOperation"
##########
helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixAdmin.java:
##########
@@ -374,6 +374,43 @@ public void enableInstance(String clusterName, List<String> instances, boolean e
//enableInstance(clusterName, instances, enabled, null, null);
}
+ @Override
+ // TODO: Name may change in future
+ public void setInstanceOperation(String clusterName, String instanceName,
+ InstanceConstants.InstanceOperation instanceOperation) {
+
+ BaseDataAccessor<ZNRecord> baseAccessor = new ZkBaseDataAccessor<>(_zkClient);
+ String path = PropertyPathBuilder.instanceConfig(clusterName, instanceName);
+
+ if (!baseAccessor.exists(path, 0)) {
+ throw new HelixException(
+ "Cluster " + clusterName + ", instance: " + instanceName + ", instance config does not exist");
+ }
+
+ boolean succeeded = baseAccessor.update(path, new DataUpdater<ZNRecord>() {
Review Comment:
nit: looks like there is an extra space at the front.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] xyuanlu commented on a diff in pull request #2588: Add API for Instance evacuation
Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2588:
URL: https://github.com/apache/helix/pull/2588#discussion_r1295143027
##########
helix-common/src/main/java/org/apache/helix/constants/InstanceConstants.java:
##########
@@ -8,4 +8,12 @@ public enum InstanceDisabledType {
USER_OPERATION,
DEFAULT_INSTANCE_DISABLE_TYPE
}
+
+ public enum InstanceOperation {
+ EVACUATE,
+ SWAP_IN,
+ SWAP_OUT,
+ ENABLE,
+ DISABLE;
Review Comment:
Had an offline discussion. We will keep `enable` and `disable` for backward compatibility. When user set HELIX_ENABLED from false to true, we will set OPERATION to be `enabled` if the field is non empty and vice vesa.
##########
helix-common/src/main/java/org/apache/helix/constants/InstanceConstants.java:
##########
@@ -8,4 +8,12 @@ public enum InstanceDisabledType {
USER_OPERATION,
DEFAULT_INSTANCE_DISABLE_TYPE
}
+
+ public enum InstanceOperation {
+ EVACUATE,
+ SWAP_IN,
+ SWAP_OUT,
+ ENABLE,
+ DISABLE;
Review Comment:
Had an offline discussion. We will keep `enable` and `disable` for backward compatibility. When user set HELIX_ENABLED from false to true, we will set OPERATION to be `enabled` if the field is non empty and vice versa.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] himanshukandwal commented on a diff in pull request #2588: Add API for Instance evacuation
Posted by "himanshukandwal (via GitHub)" <gi...@apache.org>.
himanshukandwal commented on code in PR #2588:
URL: https://github.com/apache/helix/pull/2588#discussion_r1293928702
##########
helix-common/src/main/java/org/apache/helix/constants/InstanceConstants.java:
##########
@@ -8,4 +8,12 @@ public enum InstanceDisabledType {
USER_OPERATION,
DEFAULT_INSTANCE_DISABLE_TYPE
}
+
+ public enum InstanceOperation {
+ EVACUATE,
+ SWAP_IN,
+ SWAP_OUT,
+ ENABLE,
Review Comment:
Just curious, do you think this might create confusion with Instance config HELIX_ENABLED.
I remember you mentioned SWAPED_IN, this transitioning to `instanceConfig.setInstanceEnabled` also looks good.
`Enable` is quite common term and maybe this way, from the name itself its clear that which state lifecycle we are talking about.
##########
helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixAdmin.java:
##########
@@ -374,6 +374,44 @@ public void enableInstance(String clusterName, List<String> instances, boolean e
//enableInstance(clusterName, instances, enabled, null, null);
}
+ @Override
+ // TODO: Name may change in future
+ public void setInstanceOperation(String clusterName, String instanceName,
+ InstanceConstants.InstanceOperation instanceOperation) {
+
+
Review Comment:
nit: extra lines.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] xyuanlu commented on pull request #2588: Add API for Instance evacuation
Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on PR #2588:
URL: https://github.com/apache/helix/pull/2588#issuecomment-1679789422
This change has been approved by @junkaixue // Thanks!!
Commit message: Add API for Instance evacuation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] xyuanlu commented on a diff in pull request #2588: Add API for Instance evacuation
Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2588:
URL: https://github.com/apache/helix/pull/2588#discussion_r1294027027
##########
helix-common/src/main/java/org/apache/helix/constants/InstanceConstants.java:
##########
@@ -8,4 +8,12 @@ public enum InstanceDisabledType {
USER_OPERATION,
DEFAULT_INSTANCE_DISABLE_TYPE
}
+
+ public enum InstanceOperation {
+ EVACUATE,
+ SWAP_IN,
+ SWAP_OUT,
+ ENABLE,
Review Comment:
This field is an Union of all operation. Including enable and disable. We will also update `HELIX_ENABLED ` when operation is set to enable/disable.
We decide not to introduce life cycle as it adds more complexity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] junkaixue commented on a diff in pull request #2588: Add API for Instance evacuation
Posted by "junkaixue (via GitHub)" <gi...@apache.org>.
junkaixue commented on code in PR #2588:
URL: https://github.com/apache/helix/pull/2588#discussion_r1294076992
##########
helix-core/src/main/java/org/apache/helix/model/InstanceConfig.java:
##########
@@ -329,6 +330,17 @@ public long getInstanceEnabledTime() {
return _record.getLongField(InstanceConfigProperty.HELIX_ENABLED_TIMESTAMP.name(), -1);
}
+ public void setInstanceOperation(InstanceConstants.InstanceOperation operation) {
+ // TODO: also setInstanceEnabled after sanity check.
+
+ _record.setSimpleField(InstanceConfigProperty.INSTANCE_OPERATION.toString(),
Review Comment:
use InstanceConfigProperty.INSTANCE_OPERATION.name().
Not use toString anymore. It is not consistent with line 341
##########
helix-common/src/main/java/org/apache/helix/constants/InstanceConstants.java:
##########
@@ -8,4 +8,12 @@ public enum InstanceDisabledType {
USER_OPERATION,
DEFAULT_INSTANCE_DISABLE_TYPE
}
+
+ public enum InstanceOperation {
+ EVACUATE,
+ SWAP_IN,
+ SWAP_OUT,
+ ENABLE,
+ DISABLE;
Review Comment:
Why we need both enable/disable... Suggest just have disable.
##########
helix-rest/src/test/java/org/apache/helix/rest/server/TestPerInstanceAccessor.java:
##########
@@ -486,6 +486,39 @@ public void updateInstance() throws IOException {
Assert.assertEquals(new HashSet<>(instanceConfig.getDisabledPartitionsMap()
.get(CLUSTER_NAME + dbName.substring(0, dbName.length() - 1))),
new HashSet<>(Arrays.asList(CLUSTER_NAME + dbName + "0", CLUSTER_NAME + dbName + "3")));
+
+ // test set instance operation
+ new JerseyUriRequestBuilder("clusters/{}/instances/{}?command=setInstanceOperation&instanceOperation=EVACUATE")
+ .format(CLUSTER_NAME, INSTANCE_NAME).post(this, entity);
+ instanceConfig = _configAccessor.getInstanceConfig(CLUSTER_NAME, INSTANCE_NAME);
+ Assert.assertEquals(
+ instanceConfig.getInstanceOperation(), InstanceConstants.InstanceOperation.EVACUATE.toString());
+ new JerseyUriRequestBuilder("clusters/{}/instances/{}?command=setInstanceOperation&instanceOperation=INVALIDOP")
+ .expectedReturnStatusCode(Response.Status.NOT_FOUND.getStatusCode()).format(CLUSTER_NAME, INSTANCE_NAME).post(this, entity);
+ new JerseyUriRequestBuilder("clusters/{}/instances/{}?command=setInstanceOperation")
+ .expectedReturnStatusCode(Response.Status.BAD_REQUEST.getStatusCode()).format(CLUSTER_NAME, INSTANCE_NAME).post(this, entity);
+ // TODO: enable the following test when we add sanity check.
+ /*
Review Comment:
Pls remove them if they are not be part of official code.
##########
helix-core/src/test/java/org/apache/helix/mock/MockHelixAdmin.java:
##########
@@ -306,6 +305,32 @@ public void enableInstance(String clusterName, List<String> instances, boolean e
}
+ @Override
+ public void setInstanceOperation(String clusterName, String instanceName,
+ InstanceConstants.InstanceOperation instanceOperation) {
+
+ String path = PropertyPathBuilder.instanceConfig(clusterName, instanceName);
+
+ if (!_baseDataAccessor.exists(path, 0)) {
+ throw new HelixException(
+ "Cluster " + clusterName + ", instance: " + instanceName + ", instance config does not exist");
+ }
+
+ _baseDataAccessor.update(path, new DataUpdater<ZNRecord>() {
+ @Override
+ public ZNRecord update(ZNRecord currentData) {
+ if (currentData == null) {
+ throw new HelixException(
+ "Cluster: " + clusterName + ", instance: " + instanceName + ", participant config is null");
+ }
+
+ InstanceConfig config = new InstanceConfig(currentData);
+ config.setInstanceOperation(instanceOperation); // we set instance enabled in instance config
+ return config.getRecord();
Review Comment:
nit: This is not used anywhere. If this function cannot be verified , why not just print a log instead of having this large code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] himanshukandwal commented on a diff in pull request #2588: Add API for Instance evacuation
Posted by "himanshukandwal (via GitHub)" <gi...@apache.org>.
himanshukandwal commented on code in PR #2588:
URL: https://github.com/apache/helix/pull/2588#discussion_r1294883243
##########
helix-common/src/main/java/org/apache/helix/constants/InstanceConstants.java:
##########
@@ -8,4 +8,12 @@ public enum InstanceDisabledType {
USER_OPERATION,
DEFAULT_INSTANCE_DISABLE_TYPE
}
+
+ public enum InstanceOperation {
+ EVACUATE,
+ SWAP_IN,
+ SWAP_OUT,
+ ENABLE,
Review Comment:
Sounds good, thanks @xyuanlu!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org