You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/05/02 12:08:08 UTC
[GitHub] [airflow] Jorricks opened a new pull request, #23402: Introduce sla_miss metric
Jorricks opened a new pull request, #23402:
URL: https://github.com/apache/airflow/pull/23402
We have a metric for both sla callback failures and sla email failures.
However, we are not tracking when an SLA is not met.
This is crucial to know how many SLAs we had in total.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragement file, named `{pr_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Jorricks commented on pull request #23402: Introduce sla_miss metric
Posted by GitBox <gi...@apache.org>.
Jorricks commented on PR #23402:
URL: https://github.com/apache/airflow/pull/23402#issuecomment-1193669790
Hey,
Sorry I dropped the ball on this one. Had some other projects going on.
Rebased and fixed conflict :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23402: Introduce sla_miss metric
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23402:
URL: https://github.com/apache/airflow/pull/23402#issuecomment-1195779152
Why not :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #23402: Introduce sla_miss metric
Posted by GitBox <gi...@apache.org>.
eladkal commented on PR #23402:
URL: https://github.com/apache/airflow/pull/23402#issuecomment-1193256563
@Jorricks can you rebase and resolve conflicts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] o-nikolas commented on pull request #23402: Introduce sla_miss metric
Posted by GitBox <gi...@apache.org>.
o-nikolas commented on PR #23402:
URL: https://github.com/apache/airflow/pull/23402#issuecomment-1168039931
Just looking through open PRs I've reviewed. Is there any development waiting for this one @Jorricks? Or is it just awaiting a review from a committer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23402: Introduce sla_miss metric
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23402:
URL: https://github.com/apache/airflow/pull/23402#issuecomment-1168675455
needs at least rebase to fix conflict
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Jorricks commented on pull request #23402: Introduce sla_miss metric
Posted by GitBox <gi...@apache.org>.
Jorricks commented on PR #23402:
URL: https://github.com/apache/airflow/pull/23402#issuecomment-1195303292
@potiuk could you please help me out here?
I don't think the error I saw was actually related to my PR but I am not sure. Could you check to be sure?
I rebased my branch in the meantime to see if that fixes it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Jorricks commented on a diff in pull request #23402: Introduce sla_miss metric
Posted by GitBox <gi...@apache.org>.
Jorricks commented on code in PR #23402:
URL: https://github.com/apache/airflow/pull/23402#discussion_r867465934
##########
tests/dag_processing/test_processor.py:
##########
@@ -231,6 +233,7 @@ def test_dag_file_processor_sla_miss_doesnot_raise_integrity_error(self, dag_mak
.count()
)
assert sla_miss_count == 1
+ mock_stats_incr.assert_called_with('sla_missed')
Review Comment:
Yes I thought about this but I choose this test on purpose as this is one of the only ones(if not the only one) that creates the SLA by the scheduler code (instead of creating it inside the testing code). \
Therefore I feel like this was the best way to do it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23402: Introduce sla_miss metric
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23402:
URL: https://github.com/apache/airflow/pull/23402#issuecomment-1195305348
MsSQL is not the most "stable" part of our CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Jorricks commented on pull request #23402: Introduce sla_miss metric
Posted by GitBox <gi...@apache.org>.
Jorricks commented on PR #23402:
URL: https://github.com/apache/airflow/pull/23402#issuecomment-1195595855
> MsSQL is not the most "stable" part of our CI.
So should we merge this then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] o-nikolas commented on a diff in pull request #23402: Introduce sla_miss metric
Posted by GitBox <gi...@apache.org>.
o-nikolas commented on code in PR #23402:
URL: https://github.com/apache/airflow/pull/23402#discussion_r865142884
##########
tests/dag_processing/test_processor.py:
##########
@@ -231,6 +233,7 @@ def test_dag_file_processor_sla_miss_doesnot_raise_integrity_error(self, dag_mak
.count()
)
assert sla_miss_count == 1
+ mock_stats_incr.assert_called_with('sla_missed')
Review Comment:
Super small nit: There are other tests that already mock the stats object (for testing other SLA related metrics) that you could add this line to, instead of adding a net-new mock object to this test. But it's not a big deal, still approving. Thanks for the commit :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #23402: Introduce sla_miss metric
Posted by GitBox <gi...@apache.org>.
potiuk merged PR #23402:
URL: https://github.com/apache/airflow/pull/23402
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23402: Introduce sla_miss metric
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23402:
URL: https://github.com/apache/airflow/pull/23402#issuecomment-1195304512
Likely it was intermittent failure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23402: Introduce sla_miss metric
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23402:
URL: https://github.com/apache/airflow/pull/23402#issuecomment-1194239143
Rebased again @Jorricks - we had some "Werkzeug" drama this morning. But I am afraid some tests that failed will fail again and will need to be fixed :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org