You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "hsato03 (via GitHub)" <gi...@apache.org> on 2024/08/29 23:11:32 UTC
[PR] API to validate Quota activation rule [cloudstack]
hsato03 opened a new pull request, #9605:
URL: https://github.com/apache/cloudstack/pull/9605
### Description
When entering an activation rule during the creation and editing of Quota tariffs, the defined rule is not validated. This way, errors are only identified while processing the tariff.
Because of this, the `quotaValidateActivationRule` API was created, which informs whether the activation rule is valid or not. Also, the activation rule field was added when creating and editing the tariff via UI.
#### API parameters
| Parameter | Description | Required |
| ------ | ------ | ------ |
| activationrule | The activation rule that will be validated. | Yes |
| quotaresourcetype | tariff usage type to validate the variables used in the _script_. | Yes |
#### Example
```
đą > quota validateactivationrule activationrule="if (account.name == 'admin') { 0 } else { 300 }" usagetype=6
{
"validactivationrule": {
"activationrule": "if (account.name == 'admin') { 0 } else { 300 }",
"isvalid": true,
"message": "The script has no syntax errors and all variables are compatible with the given usage type.",
"quotatype": "VOLUME"
}
}
```
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] build/CI
- [ ] test (unit or integration test code)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [ ] Major
- [X] Minor
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
### Screenshots (if appropriate):
### How Has This Been Tested?
| N° | Test | Valid activation rule |
| ------ | ------ | ------ |
| 1 | Activation rule with valid JS code and with variables compatible with the given usage type | Yes |
| 2 | Activation rule with valid JS code and with variables that are not compatible with the given usage type | No |
| 3 | Activation rule with invalid JS code | No |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] API to validate Quota activation rule [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #9605:
URL: https://github.com/apache/cloudstack/pull/9605#issuecomment-2319522152
Packaging result [SF]: âď¸ el8 âď¸ el9 âď¸ debian âď¸ suse15. SL-JID 10872
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] API to validate Quota activation rule [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #9605:
URL: https://github.com/apache/cloudstack/pull/9605#issuecomment-2319360176
@hsato03 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] API to validate Quota activation rule [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on PR #9605:
URL: https://github.com/apache/cloudstack/pull/9605#issuecomment-2515205146
Merging based on approvals and manual tests by @bernardodemarco. CI failures are not related to the PR (https://github.com/apache/cloudstack/pull/9605#issuecomment-2394982847)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] API to validate Quota activation rule [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #9605:
URL: https://github.com/apache/cloudstack/pull/9605#issuecomment-2380297184
@hsato03 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] API to validate Quota activation rule [cloudstack]
Posted by "hsato03 (via GitHub)" <gi...@apache.org>.
hsato03 commented on PR #9605:
URL: https://github.com/apache/cloudstack/pull/9605#issuecomment-2380296325
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] API to validate Quota activation rule [cloudstack]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #9605:
URL: https://github.com/apache/cloudstack/pull/9605#issuecomment-2319385551
## [Codecov](https://app.codecov.io/gh/apache/cloudstack/pull/9605?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 4.16%. Comparing base [(`1e12a80`)](https://app.codecov.io/gh/apache/cloudstack/commit/1e12a802104937228847f8cf5f4fa199530370e7?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`6b37e33`)](https://app.codecov.io/gh/apache/cloudstack/commit/6b37e3356919f0be2705d07ecf4ecf63a7a7e371?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 23 commits behind head on main.
> :exclamation: There is a different number of reports uploaded between BASE (1e12a80) and HEAD (6b37e33). Click for more details.
>
> <details><summary>HEAD has 1 upload less than BASE</summary>
>
>| Flag | BASE (1e12a80) | HEAD (6b37e33) |
>|------|------|------|
>|unittests|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #9605 +/- ##
============================================
- Coverage 15.55% 4.16% -11.40%
============================================
Files 5500 377 -5123
Lines 481889 30922 -450967
Branches 62240 5474 -56766
============================================
- Hits 74966 1288 -73678
+ Misses 398637 29488 -369149
+ Partials 8286 146 -8140
```
| [Flag](https://app.codecov.io/gh/apache/cloudstack/pull/9605/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Î | |
|---|---|---|
| [uitests](https://app.codecov.io/gh/apache/cloudstack/pull/9605/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `4.16% <ø> (-0.01%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/cloudstack/pull/9605/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/cloudstack/pull/9605?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] API to validate Quota activation rule [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9605:
URL: https://github.com/apache/cloudstack/pull/9605#issuecomment-2356084481
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] API to validate Quota activation rule [cloudstack]
Posted by "hsato03 (via GitHub)" <gi...@apache.org>.
hsato03 commented on PR #9605:
URL: https://github.com/apache/cloudstack/pull/9605#issuecomment-2319354018
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] API to validate Quota activation rule [cloudstack]
Posted by "BryanMLima (via GitHub)" <gi...@apache.org>.
BryanMLima commented on code in PR #9605:
URL: https://github.com/apache/cloudstack/pull/9605#discussion_r1771422822
##########
plugins/database/quota/src/main/java/org/apache/cloudstack/api/response/QuotaValidateActivationRuleResponse.java:
##########
@@ -0,0 +1,76 @@
+//Licensed to the Apache Software Foundation (ASF) under one
+//or more contributor license agreements. See the NOTICE file
+//distributed with this work for additional information
+//regarding copyright ownership. The ASF licenses this file
+//to you under the Apache License, Version 2.0 (the
+//"License"); you may not use this file except in compliance
+//with the License. You may obtain a copy of the License at
+//
+//http://www.apache.org/licenses/LICENSE-2.0
+//
+//Unless required by applicable law or agreed to in writing,
+//software distributed under the License is distributed on an
+//"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+//KIND, either express or implied. See the License for the
+//specific language governing permissions and limitations
+//under the License.
+package org.apache.cloudstack.api.response;
+
+import com.cloud.serializer.Param;
+import com.google.gson.annotations.SerializedName;
+import org.apache.cloudstack.api.BaseResponse;
+
+public class QuotaValidateActivationRuleResponse extends BaseResponse {
+
+ @SerializedName("activationrule")
+ @Param(description = "The validated activation rule.")
Review Comment:
```suggestion
@Param(description = "The activation rule to be validated.")
```
##########
server/src/main/java/org/apache/cloudstack/jsinterpreter/JsInterpreterHelper.java:
##########
@@ -0,0 +1,235 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.jsinterpreter;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+import org.openjdk.nashorn.api.scripting.ScriptUtils;
+import org.openjdk.nashorn.internal.runtime.Context;
+import org.openjdk.nashorn.internal.runtime.ErrorManager;
+import org.openjdk.nashorn.internal.runtime.options.Options;
+
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.Map;
+import java.util.Set;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+public class JsInterpreterHelper {
+ private final Logger logger = LogManager.getLogger(getClass());
+
+ private int callExpressions;
+
+ private StringBuilder variable;
+
+ private Set<String> variables;
+
+ /**
+ * Returns all variables from the given script.
+ *
+ * @param script the script to extract the variables.
+ * @return A {@link Set<String>} containing all variables in the script.
+ */
+ public Set<String> getScriptVariables(String script) {
+ String parseTree = getScriptAsJsonTree(script);
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode jsonNode = null;
+ variables = new HashSet<>();
+ variable = new StringBuilder();
+
+ try {
+ jsonNode = mapper.readTree(parseTree);
+ } catch (JsonProcessingException e) {
+ logger.error("Unable to create the script JSON tree due to: [{}].", e.getMessage(), e);
+ }
+
+ logger.trace("Searching script variables from [{}].", script);
+ iterateOverJsonTree(jsonNode.fields());
+
+ if (StringUtils.isNotBlank(variable.toString())) {
+ logger.trace("Adding variable [{}] into the variables set.", variable);
+ removeCallFunctionsFromVariable();
+ variables.add(variable.toString());
+ }
+
+ logger.trace("Found the following variables from the given script: [{}]", variables);
+ return variables;
+ }
+
+ private String getScriptAsJsonTree(String script) {
+ logger.trace("Creating JSON Tree for script [{}].", script);
+ Options options = new Options("nashorn");
+ options.set("anon.functions", true);
+ options.set("parse.only", true);
+ options.set("scripting", true);
+
+ ErrorManager errors = new ErrorManager();
+ Context context = new Context(options, errors, Thread.currentThread().getContextClassLoader());
+ Context.setGlobal(context.createGlobal());
+
+ return ScriptUtils.parse(script, "nashorn", false);
+ }
+
+ protected void iterateOverJsonTree(Iterator<Map.Entry<String, JsonNode>> iterator) {
+ while (iterator.hasNext()) {
+ iterateOverJsonTree(iterator.next());
+ }
+ }
+
+ protected void iterateOverJsonTree(Map.Entry<String, JsonNode> fields) {
+ JsonNode node = null;
+
+ if (fields.getValue().isArray()) {
+ iterateOverArrayNodes(fields);
+ } else {
+ node = fields.getValue();
+ }
+
+ String fieldName = searchIntoObjectNodes(node);
+
+ if (fieldName == null) {
+ String key = fields.getKey();
+ if ("type".equals(key) && "CallExpression".equals(node.textValue())) {
+ callExpressions++;
+ }
+
+ if ("name".equals(key) || "property".equals(key)) {
Review Comment:
You could extract the Strings constants to variable, as they are used in other methods of this class.
##########
plugins/database/quota/src/main/java/org/apache/cloudstack/api/response/QuotaResponseBuilderImpl.java:
##########
@@ -928,4 +937,82 @@ protected QuotaConfigureEmailResponse createQuotaConfigureEmailResponse(QuotaEma
return quotaConfigureEmailResponse;
}
+
+ @Override
+ public QuotaValidateActivationRuleResponse validateActivationRule(QuotaValidateActivationRuleCmd cmd) {
+ String message;
+ String activationRule = cmd.getActivationRule();
+ QuotaTypes quotaType = cmd.getQuotaType();
+ String quotaName = quotaType.getQuotaName();
+ List<Pair<String, String>> usageTypeVariablesAndDescriptions = new ArrayList<>();
+
+ addAllPresetVariables(PresetVariables.class, quotaType, usageTypeVariablesAndDescriptions, null);
+ List<String> usageTypeVariables = usageTypeVariablesAndDescriptions.stream().map(Pair::first).collect(Collectors.toList());
+
+ try (JsInterpreter jsInterpreter = new JsInterpreter(QuotaConfig.QuotaActivationRuleTimeout.value())) {
+ Map<String, String> newVariables = injectUsageTypeVariables(jsInterpreter, usageTypeVariables);
+ String scriptToExecute = jsInterpreterHelper.replaceScriptVariables(activationRule, newVariables);
+ jsInterpreter.executeScript(String.format("new Function(\"%s\")", scriptToExecute.replaceAll("\n", "")));
+ } catch (IOException | CloudRuntimeException e) {
+ logger.error("Unable to execute activation rule due to: [{}].", e.getMessage(), e);
+ message = "Error while executing activation rule. Check if there are no syntax errors and all variables are compatible with the given usage type.";
+ return createValidateActivationRuleResponse(activationRule, quotaName, false, message);
+ }
+
+ Set<String> scriptVariables = jsInterpreterHelper.getScriptVariables(activationRule);
+ if (isScriptVariablesValid(scriptVariables, usageTypeVariables)) {
+ message = "The script has no syntax errors and all variables are compatible with the given usage type.";
+ return createValidateActivationRuleResponse(activationRule, quotaName, true, message);
+ }
+
+ message = "Found variables that are not compatible with the given usage type.";
+ return createValidateActivationRuleResponse(activationRule, quotaName, false, message);
+ }
+
+ /**
+ * Checks whether script variables are compatible with the usage type. First, we remove all script variables that correspond to the script's usage type variables.
+ * Then, returns true if none of the remaining script variables match any usage types variables, and false otherwise.
+ *
+ * @param scriptVariables Script variables.
+ * @param scriptUsageTypeVariables Script usage type variables.
+ * @return True if the script variables are valid, false otherwise.
+ */
+ protected boolean isScriptVariablesValid(Set<String> scriptVariables, List<String> scriptUsageTypeVariables) {
+ List<Pair<String, String>> allUsageTypeVariablesAndDescriptions = new ArrayList<>();
+ addAllPresetVariables(PresetVariables.class, null, allUsageTypeVariablesAndDescriptions, null);
+ List<String> allUsageTypesVariables = allUsageTypeVariablesAndDescriptions.stream().map(Pair::first).collect(Collectors.toList());
+
+ List<String> matchVariables = scriptVariables.stream().filter(scriptUsageTypeVariables::contains).collect(Collectors.toList());
+ matchVariables.forEach(scriptVariables::remove);
+
+ return scriptVariables.stream().noneMatch(allUsageTypesVariables::contains);
+ }
+
+ /**
+ * Injects variables into JavaScript interpreter. It's necessary to remove all dots from the given variables for the interpreter
Review Comment:
Just a minor suggestion
```suggestion
* Injects variables into JavaScript interpreter. It's necessary to remove all dots from the given variables, as the interpreter
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] API to validate Quota activation rule [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre merged PR #9605:
URL: https://github.com/apache/cloudstack/pull/9605
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] API to validate Quota activation rule [cloudstack]
Posted by "BryanMLima (via GitHub)" <gi...@apache.org>.
BryanMLima commented on PR #9605:
URL: https://github.com/apache/cloudstack/pull/9605#issuecomment-2368284026
@hsato03, the build is failing due to the logger variable in the `VmwareHelper.java` class, it should be fixed by #9714.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] API to validate Quota activation rule [cloudstack]
Posted by "hsato03 (via GitHub)" <gi...@apache.org>.
hsato03 commented on code in PR #9605:
URL: https://github.com/apache/cloudstack/pull/9605#discussion_r1779309329
##########
plugins/database/quota/src/main/java/org/apache/cloudstack/api/response/QuotaValidateActivationRuleResponse.java:
##########
@@ -0,0 +1,76 @@
+//Licensed to the Apache Software Foundation (ASF) under one
+//or more contributor license agreements. See the NOTICE file
+//distributed with this work for additional information
+//regarding copyright ownership. The ASF licenses this file
+//to you under the Apache License, Version 2.0 (the
+//"License"); you may not use this file except in compliance
+//with the License. You may obtain a copy of the License at
+//
+//http://www.apache.org/licenses/LICENSE-2.0
+//
+//Unless required by applicable law or agreed to in writing,
+//software distributed under the License is distributed on an
+//"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+//KIND, either express or implied. See the License for the
+//specific language governing permissions and limitations
+//under the License.
+package org.apache.cloudstack.api.response;
+
+import com.cloud.serializer.Param;
+import com.google.gson.annotations.SerializedName;
+import org.apache.cloudstack.api.BaseResponse;
+
+public class QuotaValidateActivationRuleResponse extends BaseResponse {
+
+ @SerializedName("activationrule")
+ @Param(description = "The validated activation rule.")
Review Comment:
At this point, I think the rule has already been validated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] API to validate Quota activation rule [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #9605:
URL: https://github.com/apache/cloudstack/pull/9605#issuecomment-2380327766
Packaging result [SF]: âď¸ el8 âď¸ el9 âď¸ debian âď¸ suse15. SL-JID 11224
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org