You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "malthe (via GitHub)" <gi...@apache.org> on 2023/03/07 08:58:39 UTC
[GitHub] [airflow] malthe opened a new issue, #29949: UI-configurable email on failure
malthe opened a new issue, #29949:
URL: https://github.com/apache/airflow/issues/29949
### Description
This is about dynamically being able to configure email sending on failure per DAG, using the Airflow UI.
### Use case/motivation
We have a scenario where failures in our test environment are fairly common while production failures are important if not critical.
For this reason, we typically end up with a line such as:
```python
default_args["email_on_failure"] = os.environ["AIRFLOW_ENVIRONMENMT"] == "prod"
```
That works, but it would be nice and clean if we were able to set this to `None`, instead having this as a toggle in the Airflow UI – so that when we enable a DAG, we can also choose whether to receive email on failure or not, and even change this setting dynamically.
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] malthe commented on issue #29949: UI-configurable email on failure
Posted by "malthe (via GitHub)" <gi...@apache.org>.
malthe commented on issue #29949:
URL: https://github.com/apache/airflow/issues/29949#issuecomment-1459599229
Email notification is perhaps just one area where there is configuration logic that one might want out of DAG code, but perhaps the big picture here is that email notifications are just events that something happened.
It could be that rather than wanting more flexibility for email within Airflow, moving to something like the [OpenTelemetry Events API](https://opentelemetry.io/docs/reference/specification/logs/event-api/) makes more sense.
From there, an event listener can decide to send an email or post a message to a Slack channel etc depending on the event metadata (first and foremost the event name which in this case is something akin to "task failed" or "dag failed").
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] malthe closed issue #29949: UI-configurable email on failure
Posted by "malthe (via GitHub)" <gi...@apache.org>.
malthe closed issue #29949: UI-configurable email on failure
URL: https://github.com/apache/airflow/issues/29949
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #29949: UI-configurable email on failure
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #29949:
URL: https://github.com/apache/airflow/issues/29949#issuecomment-1463700073
> It could be that rather than wanting more flexibility for email within Airflow, moving to something like the [OpenTelemetry Events API](https://opentelemetry.io/docs/reference/specification/logs/event-api/) makes more sense.
Yes. That would follow very well our current effort to turn Airflow as an extensible platform where rather than implementing integrations and define our own ways of doing things we switch to existing "open" APIS - we already have OpenTelemetry (not events API) and OpenLineage approved. Adding OpenTelemetry Events API seems like a natural extension of it.
Rather than using that in UI, I'd expose options to configure it in a way the Events API would support, so that rather than having to configure it in DAG code or UI, to get it more confiigurable on deployment level.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #29949: UI-configurable email on failure
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #29949:
URL: https://github.com/apache/airflow/issues/29949#issuecomment-1459084821
> setting this using code (perhaps committing changes to the DAG sources, adding email addresses here) is not desirable.
I would say doing it for email only is rather strange. Why not for other fields? What makeds email different in this case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] malthe commented on issue #29949: UI-configurable email on failure
Posted by "malthe (via GitHub)" <gi...@apache.org>.
malthe commented on issue #29949:
URL: https://github.com/apache/airflow/issues/29949#issuecomment-1458979618
That's a good point – but perhaps another case here is a configurable `email` field – someone wants to get notified if a particular DAG fails, but for whatever reason, setting this using code (perhaps committing changes to the DAG sources, adding email addresses here) is not desirable.
This is somewhat similar to how we're not adding connection details to DAG code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #29949: UI-configurable email on failure
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #29949:
URL: https://github.com/apache/airflow/issues/29949#issuecomment-1458964946
Isn't that the question of just not configuring the email backend in production?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] malthe commented on issue #29949: UI-configurable email on failure
Posted by "malthe (via GitHub)" <gi...@apache.org>.
malthe commented on issue #29949:
URL: https://github.com/apache/airflow/issues/29949#issuecomment-1463713639
Decide to shift these ideas to an event-based approach; closing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org