You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/03/11 19:35:36 UTC
[GitHub] [incubator-pinot] akshayrai opened a new pull request #5142: [TE]
Detection creation endpoints should trigger Replay & Tuning
akshayrai opened a new pull request #5142: [TE] Detection creation endpoints should trigger Replay & Tuning
URL: https://github.com/apache/incubator-pinot/pull/5142
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] xiaohui-sun commented on a change in pull request
#5142: [TE] Detection creation endpoints should trigger Replay & Tuning
Posted by GitBox <gi...@apache.org>.
xiaohui-sun commented on a change in pull request #5142: [TE] Detection creation endpoints should trigger Replay & Tuning
URL: https://github.com/apache/incubator-pinot/pull/5142#discussion_r391237311
##########
File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##########
@@ -380,6 +380,9 @@ long createDetectionConfig(@NotNull String payload, long startTime, long endTime
Long id = this.detectionConfigDAO.save(detectionConfig);
Preconditions.checkNotNull(id, "Error while saving the detection pipeline");
+ // create an yaml onboarding task to run replay and tuning
+ createYamlOnboardingTask(detectionConfig.getId(), startTime, endTime);
Review comment:
Will this cause lot of system burden if user bulk onboarding using API?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] akshayrai commented on a change in pull request
#5142: [TE] Detection creation endpoints should trigger Replay & Tuning
Posted by GitBox <gi...@apache.org>.
akshayrai commented on a change in pull request #5142: [TE] Detection creation endpoints should trigger Replay & Tuning
URL: https://github.com/apache/incubator-pinot/pull/5142#discussion_r391295289
##########
File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##########
@@ -380,6 +380,9 @@ long createDetectionConfig(@NotNull String payload, long startTime, long endTime
Long id = this.detectionConfigDAO.save(detectionConfig);
Preconditions.checkNotNull(id, "Error while saving the detection pipeline");
+ // create an yaml onboarding task to run replay and tuning
+ createYamlOnboardingTask(detectionConfig.getId(), startTime, endTime);
Review comment:
The onboarding jobs complete within half a minute on an average case and I don't think it should be a problem. In addition, I am not sure if bulk-onboarding of alerts is a common use-case or whether we should even encourage such use-cases except for some special cases like "onboard all UMP metrics" in which case we should anyways take precaution. Alerts are typically created by users on various portals and the throughput(requests/sec) is typically not that high.
On the other hand, if we do not trigger the onboarding job, then apart from the replay, even the tuning component isn't triggered.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] xiaohui-sun commented on a change in pull request
#5142: [TE] Detection creation endpoints should trigger Replay & Tuning
Posted by GitBox <gi...@apache.org>.
xiaohui-sun commented on a change in pull request #5142: [TE] Detection creation endpoints should trigger Replay & Tuning
URL: https://github.com/apache/incubator-pinot/pull/5142#discussion_r391408969
##########
File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##########
@@ -380,6 +380,9 @@ long createDetectionConfig(@NotNull String payload, long startTime, long endTime
Long id = this.detectionConfigDAO.save(detectionConfig);
Preconditions.checkNotNull(id, "Error while saving the detection pipeline");
+ // create an yaml onboarding task to run replay and tuning
+ createYamlOnboardingTask(detectionConfig.getId(), startTime, endTime);
Review comment:
> The onboarding jobs complete within half a minute on an average case and I don't think it should be a problem. In addition, I am not sure if bulk-onboarding of alerts is a common use-case or whether we should even encourage such use-cases except for some special cases like "onboard all UMP metrics" in which case we should anyways take precaution. Alerts are typically created by users on various portals and the throughput(requests/sec) is typically not that high.
>
> On the other hand, if we do not trigger the onboarding job, then apart from the replay, even the tuning component isn't triggered.
Definitely this is not encouraged. We need to have a fuse mechanism somewhere to avoid bringing down the whole site if it happens. Maybe we should throttle the replay tasks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] akshayrai merged pull request #5142: [TE]
Detection creation endpoints should trigger Replay & Tuning
Posted by GitBox <gi...@apache.org>.
akshayrai merged pull request #5142: [TE] Detection creation endpoints should trigger Replay & Tuning
URL: https://github.com/apache/incubator-pinot/pull/5142
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] xiaohui-sun commented on a change in pull request
#5142: [TE] Detection creation endpoints should trigger Replay & Tuning
Posted by GitBox <gi...@apache.org>.
xiaohui-sun commented on a change in pull request #5142: [TE] Detection creation endpoints should trigger Replay & Tuning
URL: https://github.com/apache/incubator-pinot/pull/5142#discussion_r392539891
##########
File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##########
@@ -380,6 +380,9 @@ long createDetectionConfig(@NotNull String payload, long startTime, long endTime
Long id = this.detectionConfigDAO.save(detectionConfig);
Preconditions.checkNotNull(id, "Error while saving the detection pipeline");
+ // create an yaml onboarding task to run replay and tuning
+ createYamlOnboardingTask(detectionConfig.getId(), startTime, endTime);
Review comment:
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] harleyjj commented on a change in pull request
#5142: [TE] Detection creation endpoints should trigger Replay & Tuning
Posted by GitBox <gi...@apache.org>.
harleyjj commented on a change in pull request #5142: [TE] Detection creation endpoints should trigger Replay & Tuning
URL: https://github.com/apache/incubator-pinot/pull/5142#discussion_r391297651
##########
File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##########
@@ -380,6 +380,9 @@ long createDetectionConfig(@NotNull String payload, long startTime, long endTime
Long id = this.detectionConfigDAO.save(detectionConfig);
Preconditions.checkNotNull(id, "Error while saving the detection pipeline");
+ // create an yaml onboarding task to run replay and tuning
+ createYamlOnboardingTask(detectionConfig.getId(), startTime, endTime);
Review comment:
One approach could be having a flag - like 'ui=true' - in the API, so that the frontend could get the error messages and non-UI clients can skip the messages.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] akshayrai commented on a change in pull request
#5142: [TE] Detection creation endpoints should trigger Replay & Tuning
Posted by GitBox <gi...@apache.org>.
akshayrai commented on a change in pull request #5142: [TE] Detection creation endpoints should trigger Replay & Tuning
URL: https://github.com/apache/incubator-pinot/pull/5142#discussion_r391295289
##########
File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##########
@@ -380,6 +380,9 @@ long createDetectionConfig(@NotNull String payload, long startTime, long endTime
Long id = this.detectionConfigDAO.save(detectionConfig);
Preconditions.checkNotNull(id, "Error while saving the detection pipeline");
+ // create an yaml onboarding task to run replay and tuning
+ createYamlOnboardingTask(detectionConfig.getId(), startTime, endTime);
Review comment:
The onboarding jobs complete within half a minute on an average case and I don't think it should be a problem. In addition, I am not sure if bulk-onboarding of alerts is a common use-case or whether we should even encourage such use-cases except for some special cases like "onboard all UMP metrics" in which case we should anyways take precaution. Alerts are typically created by users on various portals and the throughput(requests/sec) is typically not that high.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] akshayrai commented on a change in pull request
#5142: [TE] Detection creation endpoints should trigger Replay & Tuning
Posted by GitBox <gi...@apache.org>.
akshayrai commented on a change in pull request #5142: [TE] Detection creation endpoints should trigger Replay & Tuning
URL: https://github.com/apache/incubator-pinot/pull/5142#discussion_r392343210
##########
File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##########
@@ -380,6 +380,9 @@ long createDetectionConfig(@NotNull String payload, long startTime, long endTime
Long id = this.detectionConfigDAO.save(detectionConfig);
Preconditions.checkNotNull(id, "Error while saving the detection pipeline");
+ // create an yaml onboarding task to run replay and tuning
+ createYamlOnboardingTask(detectionConfig.getId(), startTime, endTime);
Review comment:
@xiaohui-sun +1, I have filed a separate ticket to track the fuse mechanism.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org