[GitHub] [incubator-pinot] xiaohui-sun commented on a change in pull request #4136: [TE] catch exceptions in dimension exploration

[GitBox <gi...@apache.org>]

xiaohui-sun commented on a change in pull request #4136: [TE] catch exceptions in dimension exploration
URL: https://github.com/apache/incubator-pinot/pull/4136#discussion_r276786512
 
 

 ##########
 File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/algorithm/DimensionWrapper.java
 ##########
 @@ -202,10 +202,14 @@ public DetectionPipelineResult run() throws Exception {
     for (MetricEntity metric : nestedMetrics) {
       for (Map<String, Object> properties : this.nestedProperties) {
         LOG.info("running detection for {}", metric.toString());
-        DetectionPipelineResult intermediate = this.runNested(metric, properties);
-        lastTimeStamps.add(intermediate.getLastTimestamp());
-        anomalies.addAll(intermediate.getAnomalies());
-        diagnostics.put(metric.getUrn(), intermediate.getDiagnostics());
+        try {
+          DetectionPipelineResult intermediate = this.runNested(metric, properties);
+          lastTimeStamps.add(intermediate.getLastTimestamp());
+          anomalies.addAll(intermediate.getAnomalies());
+          diagnostics.put(metric.getUrn(), intermediate.getDiagnostics());
+        } catch (Exception e) {
 
 Review comment:
   It is good we raised this issue.
   I suggest to do the following to make it more effective, like what we did for detection flow:
   1. Keep track of exceptions for each metric.
   2. If it failed for all of the first 10 metrics then "exit early", and then throw exception.
   3. If it failed for all metrics then throw exception.
   
   After that, modify to send exception out in detection flow.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org