You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ram07eng (via GitHub)" <gi...@apache.org> on 2026/02/24 10:51:53 UTC
[PR] Add Airflow Watcher to ecosystem page [airflow-site]
ram07eng opened a new pull request, #1423:
URL: https://github.com/apache/airflow-site/pull/1423
Adding [airflow-watcher](https://github.com/ram07eng/airflow-watcher) to the ecosystem page under Third Party Airflow Plugins and Providers.
airflow-watcher is an Airflow UI plugin that provides:
- DAG failure and SLA miss monitoring with dashboard views
- Task health tracking (long-running, zombie, retry-heavy tasks)
- Scheduling lag analysis and cross-DAG dependency detection
- REST API endpoints for external monitoring integration
- Multi-channel alerting (Slack, PagerDuty, Email)
- StatsD/Datadog and Prometheus metrics export
Published on PyPI: https://pypi.org/project/airflow-watcher/
Licensed under Apache 2.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Airflow Watcher - dag monitoring plugin to ecosystem page [airflow-site]
Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on PR #1423:
URL: https://github.com/apache/airflow-site/pull/1423#issuecomment-3955161317
Yeah, I hear that. I think you could have some simple heuristics that would help a lot. Like the project must have existed for some number of days (7, 20, 30 something like this). And then maybe have any 2 of the following: X number of stars, X number of issues (open/closed), x number of PRs (open/closed), X number of previous releases, etc.
I think just a small handful of simple rules like this will help quite a bit. Adding new projects/tools that have existed for only a small number of days (or hours even after publishing) really leaves us vulnerable for a bait and switch like you described happening in the past. People use AI to blast together some project, get it linked here, then swap it for whatever they like.
But again, it was just a spare thought. Happy to stand down given the load you think it would put on PMC otherwise! And as always appreciate the engagement @potiuk!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Airflow Watcher - dag monitoring plugin to ecosystem page [airflow-site]
Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on PR #1423:
URL: https://github.com/apache/airflow-site/pull/1423#issuecomment-3954872062
This project looks cool! :star_struck:
With the rate of code, tools and apps being created these days (given what we all can do with AI) the ecosystem page is getting updated frequently these days (I think 5 or more items already this year?), and this will only increase.
This particular tool was released one day ago, has zero stars on GH, no issues, no PRs. I'm not at all saying we should revert this change, but I do wonder if we need a bit stricter criteria of what is included on the official Airflow ecosystem page. For example we have no idea if this project will take off or be found useful by the community and we can't just keep adding any tool people create as soon as they publish it (IMHO anyway). Curious to hear what others think! CC @potiuk @pierrejeambrun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Airflow Watcher - dag monitoring plugin to ecosystem page [airflow-site]
Posted by "ram07eng (via GitHub)" <gi...@apache.org>.
ram07eng commented on PR #1423:
URL: https://github.com/apache/airflow-site/pull/1423#issuecomment-3955874630
> This project looks cool! 🤩
>
>thank you so much for the feedback
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Airflow Watcher - dag monitoring plugin to ecosystem page [airflow-site]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #1423:
URL: https://github.com/apache/airflow-site/pull/1423
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Airflow Watcher - dag monitoring plugin to ecosystem page [airflow-site]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #1423:
URL: https://github.com/apache/airflow-site/pull/1423#issuecomment-3955124410
> Agreed ☺️ I'm not sure we'd need a committee, but some simple set of criteria to follow at least.
Depends on the criteria. Most criteria leave it to a discretion of "someone"- usualy in things like that it's very difficult to have a 0/1 criteria. Even in provider governance we have some numbers, but they are provisional and PMC has a "final vote". I can't imagine the whole PMC voting on a single entry in ecosystem, that would be adding up a lot of work for PMC - not only because of voting when added but also having some mechanism to evaluate if it has not changed.
Again - we had in the past a case where such link became a spam - https://github.com/apache/airflow-site/issues/1205 apparently an old link we added have been pointing to "escort" service. Happened. True story. Flagged by one of the users and removed.
Since we do not assume any responsibility and assume "sole discretion" to follow the links, such cases are not really our (PMC) responsibility - we should (and we do) remove them when flagged, but other than that, we do not take any responsibility in proacitvely checking and removing links that became "roque".
So once we assume some "vetting" responsiblity and criteria, we need to make sur they are kept - that's some work for whoever will be responsible for it, and while PMC might be the right body - then that body should accept that responsibility and act accordingly. This is why I thought of a comittee.
But whatever the decision body is - they will need to take things seriously if we start gating things that we accept there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Airflow Watcher - dag monitoring plugin to ecosystem page [airflow-site]
Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on PR #1423:
URL: https://github.com/apache/airflow-site/pull/1423#issuecomment-3955002963
> Look at the header of ecosystem page. We have exactly **0** decision making into what goes in there and exactly **0** responsibility for it as PMC. This is very clear at the ecosystem page:
Yes, I'm aware, I was thinking out loud on whether we should consider changing this :slightly_smiling_face:
> If we were to `vet` and decide on what we add and aprove here then:
>
> * we would have to agree on criteria
>
> * and the way to evaluate it
>
> * and have at least a commitee to decide what should be and what should not be linked here.
Agreed :relaxed: I'm not sure we'd need a committee, but some simple set of criteria to follow at least.
>
> If you have some concrete proposals @o-nikolas - I think starting a discussion on devlist is a good point - ideally wiht some ideas how to do it, but also this bears some responsibilities on the PMC. If we set some criteria, we need to execute those criteria and have some responsibility to verify those and keep them verified. Those pages can change any time and start linking to any content (say adult content) - so if we do it, we need to have some ways ot verify if they have not changed. IMHO any proposal to verify and approve such content should also include ways and processes how we keep them following it. And any proposal here is welcome - but whatever we write in the preamble of the page, we should stick to it.
>
> Currently it's "sole discretion". which is an easy solutioin and the only one that we can currently commit to. But if there are any other proposals, I think posting them on devlist is a good idea.
Yes of course, happy to post on the devlist, just wanted to test the waters and comment here beside an example when it was top of mind. Though, I get the sense that this wouldn't be received very well, so I'm happy to drop it. I just think times are changing, with AI we could find ourselves having a page with hundreds of tools that were each created in a few days or hours cluttering up this page. But we can see how it plays out, thanks for the thoughts @potiuk :smile:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Airflow Watcher - dag monitoring plugin to ecosystem page [airflow-site]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #1423:
URL: https://github.com/apache/airflow-site/pull/1423#issuecomment-3954974756
> This particular tool was released one day ago, has zero stars on GH, no issues, no PRs. I'm not at all saying we should revert this change, but I do wonder if we need a bit stricter criteria of what is included on the official Airflow ecosystem page. For example we have no idea if this project will take off or be found useful by the community and we can't just keep adding any tool people create as soon as they publish it (IMHO anyway). Curious to hear what others think! CC @potiuk @pierrejeambrun
Look at the header of ecosystem page. We have exactly **0** decision making into what goes in there and exactly **0** responsibility for it as PMC. This is very clear at the ecosystem page:
> These resources and services are not maintained, nor endorsed by the Apache Airflow® Community and Apache Airflow project (maintained by the Committers and the Airflow PMC). Use them at your sole discretion. The community does not verify the licences nor validity of those tools, so it’s your responsibility to verify them.
>
> If you would you like to be included on this page, please reach out to the [Apache Airflow dev or user mailing list](https://airflow.apache.org/community/) and let us know or simply open a Pull Request to that page.
We only review whether it's not an obvious scam (i.e. description is different than it points to) and whether it is even valid after we merge it. This is **entirely** the responsibility of users to decide if they can trust those links. We even had an old link pointig to a site that was repurposed for scam - and we only removed it because we got noitified by one of our users. We take **no responsibility** for those links being "valid" at any point in time according to any criteria - because we physically can't unless there are some ways we can do automatically and proactivelyl
If we were to `vet` and decide on what we add and aprove here then:
* we would have to agree on criteria
* and the way to evaluate it
* and have at least a commitee to decide what should be and what should not be linked here.
I think if we want to have some criteria (number of stars or whatever) then we should agree it on devlist, but currently the only criteria is "is it not an obvious scam".
If you have some concrete proposals @o-nikolas - I think starting a discussion on devlist is a good point - ideally wiht some ideas how to do it, but also this bears some responsibilities on the PMC. If we set some criteria, we need to execute those criteria and have some responsibility to verify those and keep them verified. Those pages can change any time and start linking to any content (say adult content) - so if we do it, we need to have some ways ot verify if they have not changed. IMHO any proposal to verify and approve such content should also include ways and processes how we keep them following it. And any proposal here is welcome - but whatever we write in the preamble of the page, we should keep.
Currently it's "sole discretion". which is an easy solutioin and the only one that we can currently commit to. But if there are any other proposals, I think posting them on devlist is a good idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org