You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "thegunner157 (via GitHub)" <gi...@apache.org> on 2023/09/06 18:59:24 UTC

[GitHub] [airflow] thegunner157 opened a new issue, #34145: Airflow bug with SLA

thegunner157 opened a new issue, #34145:
URL: https://github.com/apache/airflow/issues/34145

   ### Apache Airflow version
   
   Other Airflow 2 version (please specify below)
   
   ### What happened
   
   There is a DAG which breaks its SLA every 20 mins. It was active for few days. Then DAG was paused and after few hours (about 6) resumed. Suddenly after a short while it started to send email alerts about breaking SLAs every minute and it couldn't be stopped even after disabling once again DAG. Deleting SLA Misses rows via GUI didn't help. It was stopped only by removing SMTP backend.     
   
   Implementation of custom sla_miss_callback function:
   
   `def sla_breach_callback(dag_name, sla_limit, email):
       subject = f"SLA Breach in {dag_name}"
       body = (
           f"DAG {dag_name} missed its SLA.\n"
           f"SLA details: \n"
           f"SLA limit is {sla_limit}\n"
       )
       send_email(
           to=email,
           subject=subject,
           html_content=body
       )`
   
   
   ### What you think should happen instead
   
   Mails shouldn't be sent so often and after stopping DAG mails shouldn't attempt to do that
   
   ### How to reproduce
   
   Create a DAG which always breaks its SLA and set up schedule every 20 mins. Let it be active for a day. Then pause DAG and after 6 hours resume it. Then mails should be sent very often. After that disable once again DAG. Use following sla_miss_callback function implementation: 
   
   `def sla_breach_callback(dag_name, sla_limit, email):
       subject = f"SLA Breach in {dag_name}"
       body = (
           f"DAG {dag_name} missed its SLA.\n"
           f"SLA details: \n"
           f"SLA limit is {sla_limit}\n"
       )
       send_email(
           to=email,
           subject=subject,
           html_content=body
       )`
   
   ### Operating System
   
   Debian GNU/Linux 11
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Official Apache Airflow Helm Chart
   
   ### Deployment details
   
   Deployed on AKS with official helm chart.
   
   
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] thegunner157 commented on issue #34145: Airflow bug with SLA

Posted by "thegunner157 (via GitHub)" <gi...@apache.org>.
thegunner157 commented on issue #34145:
URL: https://github.com/apache/airflow/issues/34145#issuecomment-1708950670

   Hi, thanks for your reply. However, could you help me with a connected question in the comment regarding storing logs for airflow.utils.email.send_email_smtp backend? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] thegunner157 commented on issue #34145: Airflow bug with SLA

Posted by "thegunner157 (via GitHub)" <gi...@apache.org>.
thegunner157 commented on issue #34145:
URL: https://github.com/apache/airflow/issues/34145#issuecomment-1708929408

   Also connected question to issue is can we somehow check where logs are stored for default backend: airflow.utils.email.send_email_smtp? And if not, is it possible to somehow set this up?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk closed issue #34145: Airflow bug with SLA

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #34145: Airflow bug with SLA 
URL: https://github.com/apache/airflow/issues/34145


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal commented on issue #34145: Airflow bug with SLA

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #34145:
URL: https://github.com/apache/airflow/issues/34145#issuecomment-1708931333

   SLA feature is currently under rewrite https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-57+Refactor+SLA+Feature
   
   It is very unlikely that anyone would invest time in fixing old SLA behavior.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org