You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2023/12/31 16:32:22 UTC
[I] Hook notification when you turn on/off (pause/unpause) a DAG from the UI [airflow]
potiuk opened a new issue, #36516:
URL: https://github.com/apache/airflow/issues/36516
### Discussed in https://github.com/apache/airflow/discussions/18511
<div type='discussions-op-text'>
<sup>Originally posted by **salvob41** September 22, 2021</sup>
### Description
It would be useful if when someone decide to turn a DAG on/off (pause/unpause) to get notified with a hook/notification that a particular DAG has been turn on (maybe knowing who made this action)
### Use case/motivation
it's extremely easy turn a DAG on/off. This might get dangerous when different people (from different team) use airflow as scheduler. Having a hook might give some controls on what is being executed or not.
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
</div>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Hook notification when you turn on/off (pause/unpause) a DAG from the UI [airflow]
Posted by "dirrao (via GitHub)" <gi...@apache.org>.
dirrao commented on issue #36516:
URL: https://github.com/apache/airflow/issues/36516#issuecomment-1873181641
Yes, it's a useful feature. A use case could be notify when the production DAGs are turned off/on. So, it will be easy to know and react to them quickly.
We have multiple sources to change the DAG state. I believe in building the mechanism based on the DB table column change. So that it will work irrespective of the source.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Hook notification when you turn on/off (pause/unpause) a DAG from the UI [airflow]
Posted by "avanish-garg (via GitHub)" <gi...@apache.org>.
avanish-garg commented on issue #36516:
URL: https://github.com/apache/airflow/issues/36516#issuecomment-5497180707
Hi @potiuk — saw this traces back to #18511 where you agreed extending the hook system was the
better fit than polling. Opened a draft PR with an implementation: https://github.com/apache/airflow/pull/72378.
**Approach**: new `on_dag_pause_status_change(dag, is_paused)` hookspec in
`airflow/listeners/spec/dag.py`, mirroring the existing pattern in `listeners/spec/dagrun.py`.
Fires from `patch_dag()` (REST API) and the `airflow dags pause`/`unpause` CLI commands when
`is_paused` actually changes. Scoped to those two user-initiated paths for now — the scheduler's
automatic `max_consecutive_failed_dag_runs` circuit breaker also flips `is_paused` but does it via
a bulk UPDATE without loading a `DagModel` instance, so wiring that one in would need more
restructuring; happy to add it as a follow-up if useful.
Opened as a draft to make this concrete rather than just talk about it — not asking for merge yet,
happy to rework based on feedback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Hook notification when you turn on/off (pause/unpause) a DAG from the UI [airflow]
Posted by "Harshith246 (via GitHub)" <gi...@apache.org>.
Harshith246 commented on issue #36516:
URL: https://github.com/apache/airflow/issues/36516#issuecomment-1980188495
@potiuk Did you manage to try the above option ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Hook notification when you turn on/off (pause/unpause) a DAG from the UI [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #36516:
URL: https://github.com/apache/airflow/issues/36516#issuecomment-1873240412
> One possible option could be to publish the DAG state metric. Consumers can monitor the DAG's last state from the metrics and create alerts around it. However, this option doesn't provide a reaction to the state change.
Correct - my initial proposal in the discussion is active checking via API - metrics is almost the same but we would have to have *something* gathering and publishing the metrics regularly (still you'd have to monitor the metrics and react). In order to implement this one you'd have to have something regularly checking state of DAGs.
One of the options would be to build it in DAGFileProcessor. It already parses all dags regularly, so if we store state observed during last parsing, DAGFileProcessor could fire a notification when the state changed from last parsing. A bit racy approach and would have to be carefully done to account for potential state changes while the file is being parsed, but generally doable I think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Hook notification when you turn on/off (pause/unpause) a DAG from the UI [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #36516:
URL: https://github.com/apache/airflow/issues/36516#issuecomment-1873769958
Yeah. Sure :). It's a bit more involved though as we need to likstly store "last seen" state of the DAG in the DAG table. But yeeah feel free.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Hook notification when you turn on/off (pause/unpause) a DAG from the UI [airflow]
Posted by "XtremAlRaven (via GitHub)" <gi...@apache.org>.
XtremAlRaven commented on issue #36516:
URL: https://github.com/apache/airflow/issues/36516#issuecomment-2247642548
Hi! Is this still in progress? We're anxiously awaiting for this feature. I see https://github.com/apache/airflow/pull/38683 is about to be closed in a few days due to becoming stale.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Hook notification when you turn on/off (pause/unpause) a DAG from the UI [airflow]
Posted by "Bowrna (via GitHub)" <gi...@apache.org>.
Bowrna commented on issue #36516:
URL: https://github.com/apache/airflow/issues/36516#issuecomment-1873626221
@potiuk how about trying this to implement it via listenerplugin?
I would like to pick up this issue and see if i can work on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Hook notification when you turn on/off (pause/unpause) a DAG from the UI [airflow]
Posted by "Bowrna (via GitHub)" <gi...@apache.org>.
Bowrna commented on issue #36516:
URL: https://github.com/apache/airflow/issues/36516#issuecomment-2249128769
yes, it is still in progress. I have added a few migrations and testing the code changes that I have added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Hook notification when you turn on/off (pause/unpause) a DAG from the UI [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #36516:
URL: https://github.com/apache/airflow/issues/36516#issuecomment-1872989976
Just to note - this might be rather complex because it requires a new way of communication. Webserver / API uses DB to communicate with whatever can produce notifications (Webserver shoudl not - on it's own reach out with any kind of custom notifications) - so we need **something** to be able to react on the fact that DAG has chaned from active to paused. We do not have that **something** currently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Hook notification when you turn on/off (pause/unpause) a DAG from the UI [airflow]
Posted by "Bowrna (via GitHub)" <gi...@apache.org>.
Bowrna commented on issue #36516:
URL: https://github.com/apache/airflow/issues/36516#issuecomment-2011325373
no i will have more time from this week and let me take a look at this @Harshith246
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Hook notification when you turn on/off (pause/unpause) a DAG from the UI [airflow]
Posted by "Bowrna (via GitHub)" <gi...@apache.org>.
Bowrna commented on issue #36516:
URL: https://github.com/apache/airflow/issues/36516#issuecomment-1874842101
ok @potiuk Let me start it this weekend and come up with initial idea and propose it here for review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Hook notification when you turn on/off (pause/unpause) a DAG from the UI [airflow]
Posted by "Bowrna (via GitHub)" <gi...@apache.org>.
Bowrna commented on issue #36516:
URL: https://github.com/apache/airflow/issues/36516#issuecomment-2014663145
> > One possible option could be to publish the DAG state metric. Consumers can monitor the DAG's last state from the metrics and create alerts around it. However, this option doesn't provide a reaction to the state change.
>
> Correct - my initial proposal in the discussion is active checking via API - metrics is almost the same but we would have to have _something_ gathering and publishing the metrics regularly (still you'd have to monitor the metrics and react). In order to implement this one you'd have to have something regularly checking state of DAGs.
>
> One of the options would be to build it in DAGFileProcessor. It already parses all dags regularly, so if we store state observed during last parsing, DAGFileProcessor could fire a notification when the state changed from last parsing. A bit racy approach and would have to be carefully done to account for potential state changes while the file is being parsed, but generally doable I think.
Approach:
1. Checking in the DAGFileProcessor about the state of the DAG and on noticing change of the state, hook a notification and maintain a state that its notified along with the time at which its notified( or say time at which the state change is seen). - this is active verifying in the DAGFileProcessor side and its like polling technique
2. ListenerPlugin approach - Watching when the DB gets a state change message and hooking a notification as a result of it. Though this seems like cleaner approach, have to check in depth on which one would be feasible.
Let me make a elaborate plan and present it with the findings I have collected based on the above 2 approach
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org