You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/03/05 00:23:14 UTC
[GitHub] [incubator-pinot] akshayrai opened a new pull request #5113: [TE]
added a couple of validations to check if detection is subscribed & valid;
cleaned up other validations
akshayrai opened a new pull request #5113: [TE] added a couple of validations to check if detection is subscribed & valid; cleaned up other validations
URL: https://github.com/apache/incubator-pinot/pull/5113
Validations added:
* Make sure subscription group subscribes to the detectionName while creating an alert
* Make sure all the subscribed detections are valid
* recipients should be configured under the EMAIL scheme params and not at the root level
Validations removed:
* type is not compulsory in subscription group
* type is not compulsory for sub-entity alerts.
Added unit tests for newly added validations
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] akshayrai commented on a change in pull request
#5113: [TE] added a couple of validations to check if detection is
subscribed & valid; cleaned up other validations
Posted by GitBox <gi...@apache.org>.
akshayrai commented on a change in pull request #5113: [TE] added a couple of validations to check if detection is subscribed & valid; cleaned up other validations
URL: https://github.com/apache/incubator-pinot/pull/5113#discussion_r388436966
##########
File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##########
@@ -262,6 +263,23 @@ private Response processBadAuthorizationResponse(String type, String operation,
return Response.status(Response.Status.UNAUTHORIZED).entity(responseMessage).build();
}
+ /**
+ * Perform some basic validations on the create alert payload
+ */
+ public void validateCreateAlertYaml(Map<String, String> config) throws IOException {
+ Preconditions.checkArgument(config.containsKey(PROP_DETECTION), "Detection pipeline yaml is missing");
+ Preconditions.checkArgument(config.containsKey(PROP_SUBSCRIPTION), "Subscription group yaml is missing.");
+
+ // check if subscription group has subscribed to the detection
+ Map<String, Object> detectionConfigMap = new HashMap<>(ConfigUtils.getMap(this.yaml.load(config.get(PROP_DETECTION))));
+ String detectionName = MapUtils.getString(detectionConfigMap, PROP_DETECTION_NAME);
+ Map<String, Object> subscriptionConfigMap = new HashMap<>(ConfigUtils.getMap(this.yaml.load(config.get(PROP_SUBSCRIPTION))));
+ List<String> detectionNames = ConfigUtils.getList(subscriptionConfigMap.get(PROP_DETECTION_NAMES));
+ Preconditions.checkArgument(detectionNames.contains(detectionName),
+ "You have not subscribed to the alert. Please copy-paste the detectionName under the "
Review comment:
There is already a separate api to create just the detection.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] xiaohui-sun commented on issue #5113: [TE] added
a couple of validations to check if detection is subscribed & valid;
cleaned up other validations
Posted by GitBox <gi...@apache.org>.
xiaohui-sun commented on issue #5113: [TE] added a couple of validations to check if detection is subscribed & valid; cleaned up other validations
URL: https://github.com/apache/incubator-pinot/pull/5113#issuecomment-595321310
Thanks for the clean up. I am thinking of refactoring the validation module to:
1. Can expose multiple error messages together.
2. Easier to add more validation rules.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] akshayrai commented on issue #5113: [TE] added a
couple of validations to check if detection is subscribed & valid;
cleaned up other validations
Posted by GitBox <gi...@apache.org>.
akshayrai commented on issue #5113: [TE] added a couple of validations to check if detection is subscribed & valid; cleaned up other validations
URL: https://github.com/apache/incubator-pinot/pull/5113#issuecomment-595371758
> 1. Can expose multiple error messages together.
That might be better but I am sure how much value it will bring. We can club together some independent validations and show them together but we still won't be able to run validation on the complete config.
> 2. Easier to add more validation rules.
The validations are already abstracted out and written in a separate validation class. What are you thinking about?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] xiaohui-sun edited a comment on issue #5113: [TE]
added a couple of validations to check if detection is subscribed & valid;
cleaned up other validations
Posted by GitBox <gi...@apache.org>.
xiaohui-sun edited a comment on issue #5113: [TE] added a couple of validations to check if detection is subscribed & valid; cleaned up other validations
URL: https://github.com/apache/incubator-pinot/pull/5113#issuecomment-595374306
> > 1. Can expose multiple error messages together.
>
> That might be better but I am sure how much value it will bring. We can club together some independent validations and show them together but we still won't be able to run validation on the complete config.
We want to show all validation results to users. Currently any error will block the other validators.
>
> > 1. Easier to add more validation rules.
>
> The validations are already abstracted out and written in a separate validation class. What are you thinking about?
It is pretty good now. Some improvements we can do:
1. Pass the error message more clearly. Some of the error message was wrapped in exceptions. Some of them are in precondition checks.
2. The validators are coupled and states are shared. You can't run the validators separately.
3. Some validations are warnings and some are errors. We should differentiate them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] akshayrai commented on issue #5113: [TE] added a
couple of validations to check if detection is subscribed & valid;
cleaned up other validations
Posted by GitBox <gi...@apache.org>.
akshayrai commented on issue #5113: [TE] added a couple of validations to check if detection is subscribed & valid; cleaned up other validations
URL: https://github.com/apache/incubator-pinot/pull/5113#issuecomment-595394809
@xiaohui-sun these are great ideas. We can discuss them offline and take that up as a separate effort and not merge into this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] xiaohui-sun edited a comment on issue #5113: [TE]
added a couple of validations to check if detection is subscribed & valid;
cleaned up other validations
Posted by GitBox <gi...@apache.org>.
xiaohui-sun edited a comment on issue #5113: [TE] added a couple of validations to check if detection is subscribed & valid; cleaned up other validations
URL: https://github.com/apache/incubator-pinot/pull/5113#issuecomment-595374306
> > 1. Can expose multiple error messages together.
>
> That might be better but I am sure how much value it will bring. We can club together some independent validations and show them together but we still won't be able to run validation on the complete config.
We want to show all validation results to users. Currently any error will block the other validators.
>
> > 1. Easier to add more validation rules.
>
> The validations are already abstracted out and written in a separate validation class. What are you thinking about?
It is pretty good now. Some improvement we can do:
1. Pass the error message more clearly. Some of the error message was wrapped in exceptions. Some of them are in precondition checks.
2. The validators are coupled and states are shared. You can't run the validators separately.
3. Some validations are warnings and some are errors. We should differentiate them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] akshayrai commented on issue #5113: [TE] added a
couple of validations to check if detection is subscribed & valid;
cleaned up other validations
Posted by GitBox <gi...@apache.org>.
akshayrai commented on issue #5113: [TE] added a couple of validations to check if detection is subscribed & valid; cleaned up other validations
URL: https://github.com/apache/incubator-pinot/pull/5113#issuecomment-595882853
Validations added:
* Make sure subscription group subscribes to the detectionName while creating an alert
* Make sure all the subscribed detections are valid
* recipients should be configured under the EMAIL scheme params and not at the root level
Validations removed:
* type is not compulsory in subscription group
* type is not compulsory for sub-entity alerts.
Added unit tests for newly added validations
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] xiaohui-sun commented on issue #5113: [TE] added
a couple of validations to check if detection is subscribed & valid;
cleaned up other validations
Posted by GitBox <gi...@apache.org>.
xiaohui-sun commented on issue #5113: [TE] added a couple of validations to check if detection is subscribed & valid; cleaned up other validations
URL: https://github.com/apache/incubator-pinot/pull/5113#issuecomment-595374306
> > 1. Can expose multiple error messages together.
>
> That might be better but I am sure how much value it will bring. We can club together some independent validations and show them together but we still won't be able to run validation on the complete config.
We want to show all validation results to users. Currently any error will block the other validators.
>
> > 1. Easier to add more validation rules.
>
> The validations are already abstracted out and written in a separate validation class. What are you thinking about?
It is pretty good now. Some improvement we can do:
1. Pass the error message more clearly. Some of the error message was wrapped in exceptions. Some of them are in precondition checks.
2. The validators are coupled and states are shared. You can't run the validators separately.
3. Some validations are warnings and some are errors. We should differentiate them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] akshayrai removed a comment on issue #5113: [TE]
added a couple of validations to check if detection is subscribed & valid;
cleaned up other validations
Posted by GitBox <gi...@apache.org>.
akshayrai removed a comment on issue #5113: [TE] added a couple of validations to check if detection is subscribed & valid; cleaned up other validations
URL: https://github.com/apache/incubator-pinot/pull/5113#issuecomment-595882853
Validations added:
* Make sure subscription group subscribes to the detectionName while creating an alert
* Make sure all the subscribed detections are valid
* recipients should be configured under the EMAIL scheme params and not at the root level
Validations removed:
* type is not compulsory in subscription group
* type is not compulsory for sub-entity alerts.
Added unit tests for newly added validations
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] akshayrai merged pull request #5113: [TE] added a
couple of validations to check if detection is subscribed & valid;
cleaned up other validations
Posted by GitBox <gi...@apache.org>.
akshayrai merged pull request #5113: [TE] added a couple of validations to check if detection is subscribed & valid; cleaned up other validations
URL: https://github.com/apache/incubator-pinot/pull/5113
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] xiaohui-sun commented on a change in pull request
#5113: [TE] added a couple of validations to check if detection is
subscribed & valid; cleaned up other validations
Posted by GitBox <gi...@apache.org>.
xiaohui-sun commented on a change in pull request #5113: [TE] added a couple of validations to check if detection is subscribed & valid; cleaned up other validations
URL: https://github.com/apache/incubator-pinot/pull/5113#discussion_r388412999
##########
File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##########
@@ -262,6 +263,23 @@ private Response processBadAuthorizationResponse(String type, String operation,
return Response.status(Response.Status.UNAUTHORIZED).entity(responseMessage).build();
}
+ /**
+ * Perform some basic validations on the create alert payload
+ */
+ public void validateCreateAlertYaml(Map<String, String> config) throws IOException {
+ Preconditions.checkArgument(config.containsKey(PROP_DETECTION), "Detection pipeline yaml is missing");
+ Preconditions.checkArgument(config.containsKey(PROP_SUBSCRIPTION), "Subscription group yaml is missing.");
+
+ // check if subscription group has subscribed to the detection
+ Map<String, Object> detectionConfigMap = new HashMap<>(ConfigUtils.getMap(this.yaml.load(config.get(PROP_DETECTION))));
+ String detectionName = MapUtils.getString(detectionConfigMap, PROP_DETECTION_NAME);
+ Map<String, Object> subscriptionConfigMap = new HashMap<>(ConfigUtils.getMap(this.yaml.load(config.get(PROP_SUBSCRIPTION))));
+ List<String> detectionNames = ConfigUtils.getList(subscriptionConfigMap.get(PROP_DETECTION_NAMES));
+ Preconditions.checkArgument(detectionNames.contains(detectionName),
+ "You have not subscribed to the alert. Please copy-paste the detectionName under the "
Review comment:
There are some use cases (e.g, DHM) that they only want to create alert w/o subscription groups.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org