You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/06/17 00:58:17 UTC
[GitHub] [superset] zhaorui2022 opened a new pull request, #20419: fix(embedded) Retry when executing alert queries to avoid sending transient errors to users as alert failure notifications
zhaorui2022 opened a new pull request, #20419:
URL: https://github.com/apache/superset/pull/20419
fix(embedded) Retry when executing alert queries to avoid sending transient errors to users as alert failure notifications
### SUMMARY
Details documented at https://github.com/apache/superset/issues/20418
When there are transient errors, alert owners will receive a notification saying the alert fails even though the underlying dataset is working properly. Adding retries to avoid users receiving redundant notifications.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A
### TESTING INSTRUCTIONS
This only happens when there are transient errors, so there is no stable way to reproduce the issue. Added unit tests to make sure execute_query function is retried when it should.
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [X] Has associated issue: #20418
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] bkyryliuk commented on a diff in pull request #20419: fix(embedded): Retry when executing alert queries to avoid sending transient errors to users as alert failure notifications
Posted by GitBox <gi...@apache.org>.
bkyryliuk commented on code in PR #20419:
URL: https://github.com/apache/superset/pull/20419#discussion_r907783151
##########
superset/reports/commands/alert.py:
##########
@@ -46,6 +47,10 @@
# to avoid heavy loads done by a user mistake
OPERATOR_FUNCTIONS = {">=": ge, ">": gt, "<=": le, "<": lt, "==": eq, "!=": ne}
+# Max tries to run queries to prevent false errors caused by transient errors
+# being returned to users
+EXECUTE_QUERY_MAX_TRIES = 2
Review Comment:
keep it at 1 to keep the default behavior
also let's move it to the config to let other to decide what retry policy they are looking for
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] bkyryliuk commented on a diff in pull request #20419: fix(embedded): Retry when executing alert queries to avoid sending transient errors to users as alert failure notifications
Posted by GitBox <gi...@apache.org>.
bkyryliuk commented on code in PR #20419:
URL: https://github.com/apache/superset/pull/20419#discussion_r910273070
##########
superset/reports/commands/alert.py:
##########
@@ -27,6 +27,7 @@
from superset import app, jinja_context, security_manager
from superset.commands.base import BaseCommand
+from superset.config import ALERT_REPORTS_QUERY_EXECUTION_MAX_TRIES
Review Comment:
it should be accessed in a different way: https://github.com/apache/superset/blob/4f77824e550bc70d2f6365221b9078004d49b78c/superset/reports/commands/execute.py#L388
##########
tests/unit_tests/reports/commands/alert_tests.py:
##########
@@ -0,0 +1,117 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name, unused-argument, import-outside-toplevel
+
+import pandas as pd
+from pytest_mock import MockFixture
+
+from superset.config import ALERT_REPORTS_QUERY_EXECUTION_MAX_TRIES
Review Comment:
let's overwrite it for the test to be set to 2, feel free to use a test config
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] bkyryliuk merged pull request #20419: fix(embedded): Retry when executing alert queries to avoid sending transient errors to users as alert failure notifications
Posted by GitBox <gi...@apache.org>.
bkyryliuk merged PR #20419:
URL: https://github.com/apache/superset/pull/20419
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] bkyryliuk commented on a diff in pull request #20419: fix(embedded): Retry when executing alert queries to avoid sending transient errors to users as alert failure notifications
Posted by GitBox <gi...@apache.org>.
bkyryliuk commented on code in PR #20419:
URL: https://github.com/apache/superset/pull/20419#discussion_r911259279
##########
tests/integration_tests/reports/alert_tests.py:
##########
@@ -0,0 +1,122 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name, unused-argument, import-outside-toplevel
+
+import pandas as pd
+from pytest_mock import MockFixture
+
+from superset import app
+
+
+def test_execute_query_succeeded_no_retry(
+ mocker: MockFixture, app_context: None
+) -> None:
+
+ from superset.reports.commands.alert import AlertCommand
+
+ execute_query_mock = mocker.patch(
+ "superset.reports.commands.alert.AlertCommand._execute_query",
+ side_effect=lambda: pd.DataFrame([{"sample_col": 0}]),
+ )
+
+ command = AlertCommand(report_schedule=mocker.Mock())
+
+ command.validate()
+
+ assert execute_query_mock.call_count == 1
+
+
+def test_execute_query_succeeded_with_retries(
+ mocker: MockFixture, app_context: None
+) -> None:
+ from superset.reports.commands.alert import AlertCommand, AlertQueryError
+
+ execute_query_mock = mocker.patch(
+ "superset.reports.commands.alert.AlertCommand._execute_query"
+ )
+
+ query_executed_count = 0
+ expected_max_retries = app.config["ALERT_REPORTS_QUERY_EXECUTION_MAX_TRIES"]
+
+ def _mocked_execute_query() -> pd.DataFrame:
+ nonlocal query_executed_count
+ query_executed_count += 1
+
+ if query_executed_count < expected_max_retries:
+ raise AlertQueryError()
+ else:
+ return pd.DataFrame([{"sample_col": 0}])
+
+ execute_query_mock.side_effect = _mocked_execute_query
+ execute_query_mock.__name__ = "mocked_execute_query"
+
+ command = AlertCommand(report_schedule=mocker.Mock())
+
+ command.validate()
+
+ assert execute_query_mock.call_count == expected_max_retries
Review Comment:
do
```assert execute_query_mock.call_count == 3```
instead, it will make sure that you are using a right config
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] commented on pull request #20419: fix(embedded) Retry when executing alert queries to avoid sending transient errors to users as alert failure notifications
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #20419:
URL: https://github.com/apache/superset/pull/20419#issuecomment-1158360918
# [Codecov](https://codecov.io/gh/apache/superset/pull/20419?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#20419](https://codecov.io/gh/apache/superset/pull/20419?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8bb45cc) into [master](https://codecov.io/gh/apache/superset/commit/467d8ef89ea11dcfc4cd67d88a514886254820d2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (467d8ef) will **decrease** coverage by `11.91%`.
> The diff coverage is `100.00%`.
```diff
@@ Coverage Diff @@
## master #20419 +/- ##
===========================================
- Coverage 66.70% 54.79% -11.92%
===========================================
Files 1739 1739
Lines 65135 65139 +4
Branches 6897 6897
===========================================
- Hits 43450 35693 -7757
- Misses 19932 27693 +7761
Partials 1753 1753
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `?` | |
| postgres | `?` | |
| presto | `53.56% <66.66%> (+<0.01%)` | :arrow_up: |
| python | `58.06% <100.00%> (-24.72%)` | :arrow_down: |
| sqlite | `?` | |
| unit | `50.70% <100.00%> (+0.55%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/20419?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/reports/commands/alert.py](https://codecov.io/gh/apache/superset/pull/20419/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvcmVwb3J0cy9jb21tYW5kcy9hbGVydC5weQ==) | `52.80% <100.00%> (-43.75%)` | :arrow_down: |
| [superset/utils/dashboard\_import\_export.py](https://codecov.io/gh/apache/superset/pull/20419/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdXRpbHMvZGFzaGJvYXJkX2ltcG9ydF9leHBvcnQucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [superset/key\_value/commands/upsert.py](https://codecov.io/gh/apache/superset/pull/20419/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3Vwc2VydC5weQ==) | `0.00% <0.00%> (-89.59%)` | :arrow_down: |
| [superset/key\_value/commands/update.py](https://codecov.io/gh/apache/superset/pull/20419/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3VwZGF0ZS5weQ==) | `0.00% <0.00%> (-89.37%)` | :arrow_down: |
| [superset/key\_value/commands/delete.py](https://codecov.io/gh/apache/superset/pull/20419/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZS5weQ==) | `0.00% <0.00%> (-85.30%)` | :arrow_down: |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/20419/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-85.19%)` | :arrow_down: |
| [superset/key\_value/commands/delete\_expired.py](https://codecov.io/gh/apache/superset/pull/20419/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZV9leHBpcmVkLnB5) | `0.00% <0.00%> (-80.77%)` | :arrow_down: |
| [superset/dashboards/commands/importers/v0.py](https://codecov.io/gh/apache/superset/pull/20419/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9jb21tYW5kcy9pbXBvcnRlcnMvdjAucHk=) | `15.62% <0.00%> (-76.25%)` | :arrow_down: |
| [superset/datasets/commands/update.py](https://codecov.io/gh/apache/superset/pull/20419/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGF0YXNldHMvY29tbWFuZHMvdXBkYXRlLnB5) | `25.88% <0.00%> (-68.24%)` | :arrow_down: |
| [superset/datasets/commands/create.py](https://codecov.io/gh/apache/superset/pull/20419/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGF0YXNldHMvY29tbWFuZHMvY3JlYXRlLnB5) | `30.18% <0.00%> (-67.93%)` | :arrow_down: |
| ... and [274 more](https://codecov.io/gh/apache/superset/pull/20419/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/20419?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/20419?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [467d8ef...8bb45cc](https://codecov.io/gh/apache/superset/pull/20419?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org