You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Kaxil Naik <ka...@gmail.com> on 2026/02/20 17:08:47 UTC
[DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Hey all,
*tl;dr*: I'm proposing an official Provider Registry for the Airflow
project, deployed at https://airflow.apache.org/registry/.
Preview is up at https://airflow.staged.apache.org/registry/ -- take a look
and let me know what you think.
PR to Airflow repo incoming in a couple of hours :)
*Why now*
With AIP-95 approved, Airflow now has a formal provider lifecycle:
incubation, production, mature, and deprecated. That opens the door for
accepting more community-built providers and giving them an official home,
while setting clear expectations about maturity and support. But lifecycle
stages only work if users can actually see them.
Right now, there's no place on airflow.apache.org where someone can browse
providers, check their lifecycle stage, or discover what modules they ship.
This registry fills that gap. It gives the PMC a tool to communicate
provider maturity to users, and it gives the community an official way to
surface new providers -- clearly labelled with their lifecycle stage.
*What it does*
The registry currently catalogs 99 providers and 1,648 modules across all
11 module types (operators, hooks, sensors, triggers, transfers, executors,
notifiers, secret backends, logging handlers, Dag bundles, and decorators).
It's built with Eleventy <https://www.11ty.dev/> (thanks Ash, for the
suggestion and for prototyping an approach with it) and auto-generated
directly from the provider.yaml files in the repo -- no separate data
pipeline, no manual curation. When a provider is added or updated, the next
CI build picks it up automatically.
The entire registry is a static site (HTML, CSS, JS): no server, no
database, same deployment model as the existing Airflow docs. It's
generated at build time from the provider.yaml files and served from S3 via
CloudFront.
*A few things you can do with it*:
- Search across all providers and modules (Cmd+K, powered by Pagefind)
- Browse by category (Cloud, Databases, AI & ML, etc.)
- Filter/sort by lifecycle stage, downloads, module count
- Explore provider detail pages with per-version module listings,
connection types, parameters, and install commands
- Access JSON API endpoints (/api/providers.json, /api/modules.json) for
programmatic access -- useful for AI agents and tooling
The design is deliberately discovery-first: it links out to the API
reference docs and user guides rather than hosting everything itself. This
avoids duplicating content between provider docs and registry entries.
CI/CD is integrated with our existing docs pipeline and syncs to S3
automatically. Nothing in provider code, provider.yaml schemas, core
Airflow, or the docs build is changed by this.
*How it relates to the Astronomer Registry*
Many of you know the Astronomer Registry (https://registry.astronomer.io),
which has been the go-to for discovering Airflow providers for years. Big
thanks to Astronomer and Josh Fell for building and maintaining it.
This new registry is designed to be a community-owned successor on
airflow.apache.org, with the eventual goal of redirecting
registry.astronomer.io traffic here once it's stable.
*Remaining work*
Still to do after this lands:
- apache/airflow-site PR for .htaccess rewrite and a "Registry" nav link
- Redirect registry.astronomer.io traffic once the official one is stable
- A way to add third-party providers that are not in the Airflow repo,
like Great Expectations, Cosmos etc - I have a POC working on this.
*Future ideas (will create GH issues)*
- Explicit categories in provider.yaml (currently keyword-based matching)
- LLM-friendly exports (llms.txt, "Copy for AI" buttons etc.)
- Example DAGs for each Provider.
- and many more – but I think the current state is valuable enough
already
I'd appreciate feedback and reviews!
Regards,
Kaxil
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Jarek Potiuk <ja...@potiuk.com>.
Cool. This is a fantastic start to work on more tooling to help our
stewards to do their duties for providers they will "steward". We very much
need it :)
J.
On Sat, Mar 7, 2026 at 5:08 AM Kaxil Naik <ka...@gmail.com> wrote:
> Thank you for all the feedback, I really appreciate it.
>
> In the interest of ensuring reviews become manageable I merged the PR. If
> there are any issues in CI or around doc publishing, please let me know.
>
> Nothing would show up in the "live" (non-staging site) until I add
> `.htaccess`. I plan to work on incremental updates to handle some of the
> todos and review comments as separate PRs next week with a goal to get
> the *Airflow
> Registry* live around the same timing as Airflow 3.2 or by the week of 16th
> March.
>
> I hope to write some sort of blog post on the Airflow website about it at
> the time of launch.
>
> Regards,
> Kaxil
>
> On Thu, 5 Mar 2026 at 05:30, Srabasti Banerjee <
> srabasti.banerjee.c@gmail.com> wrote:
>
> > This is cool - all providers visually in one place - without scrolling
> > through pages of documentation - in dark mode - super helpful Kaxil:)
> >
> > Warm Regards,
> > Srabasti Banerjee
> >
> > On Tue, Feb 24, 2026 at 9:37 PM Rahul Vats <ra...@gmail.com>
> wrote:
> >
> > > This is great, thank you Kaxil!
> > >
> > >
> > > Regards,
> > > Rahul Vats
> > >
> > > On Tue, 24 Feb 2026 at 19:32, Dheeraj Turaga <dh...@gmail.com>
> > > wrote:
> > >
> > > > Oh wow! Looks amazing! Great work Kaxil!!!
> > > >
> > > > On Mon, Feb 23, 2026 at 3:12 PM Buğra Öztürk <
> ozturkbugra93@gmail.com>
> > > > wrote:
> > > >
> > > > > Looks really cool, thanks Kaxil! This is a great update, and it
> feels
> > > how
> > > > > it should exhibit in the scale of the Airflow providers :)
> > > > >
> > > > > On Mon, Feb 23, 2026 at 5:17 PM Vikram Koka via dev <
> > > > > dev@airflow.apache.org>
> > > > > wrote:
> > > > >
> > > > > > Excellent work Kaxil!
> > > > > > This is absolutely transformative for the Airflow user community
> > and
> > > > the
> > > > > > integrations effort
> > > > > >
> > > > > > Vikram
> > > > > >
> > > > > > On Sun, Feb 22, 2026 at 10:47 PM Amogh Desai <
> > amoghdesai@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Looks amazing, thanks for working on this, Kaxil.
> > > > > > >
> > > > > > > I already see it to be super useful instead of going into
> > > manhunting
> > > > > > > on the Airflow docs and google searches.
> > > > > > >
> > > > > > > Looking forward to see how this one will be adopted in the
> > future.
> > > > > > >
> > > > > > > Thanks & Regards,
> > > > > > > Amogh Desai
> > > > > > >
> > > > > > >
> > > > > > > On Sun, Feb 22, 2026 at 1:34 AM Jens Scheffler <
> > > jscheffl@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > +1 into "Very cool" category. The rest of Airflow site
> > > immediately
> > > > > > feels
> > > > > > > > like 1980 :-D
> > > > > > > >
> > > > > > > > On 20.02.26 23:37, Kaxil Naik wrote:
> > > > > > > > > PR: https://github.com/apache/airflow/pull/62261
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, 20 Feb 2026 at 21:50, Pavankumar Gopidesu <
> > > > > > > > gopidesupavan@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> Woohoo, what an update , thanks kaxil, it's really nice :)
> > > > > > > > >>
> > > > > > > > >> On Fri, Feb 20, 2026 at 9:12 PM Vincent Beck <
> > > > vincbeck@apache.org
> > > > > >
> > > > > > > > wrote:
> > > > > > > > >>
> > > > > > > > >>> Very cool indeed :) I can see it very useful for users!
> > > > > > > > >>>
> > > > > > > > >>> On 2026/02/20 20:04:58 Jarek Potiuk wrote:
> > > > > > > > >>>> Very cool :)
> > > > > > > > >>>>
> > > > > > > > >>>> J.
> > > > > > > > >>>>
> > > > > > > > >>>>
> > > > > > > > >>>> On Fri, Feb 20, 2026 at 6:22 PM Zhe-You Liu <
> > > > > zhu424.dev@gmail.com
> > > > > > >
> > > > > > > > >>> wrote:
> > > > > > > > >>>>> Hi Kaxil,
> > > > > > > > >>>>>
> > > > > > > > >>>>> I just checked out the staged site of the official
> > Provider
> > > > > > > Registry,
> > > > > > > > >>> and
> > > > > > > > >>>>> it looks super impressive! The UX is excellent, and
> it’s
> > > > > > > definitely a
> > > > > > > > >>> big
> > > > > > > > >>>>> plus for users to find existing community integrations
> > for
> > > > > their
> > > > > > > use
> > > > > > > > >>> cases
> > > > > > > > >>>>> and involve more Airflow users. I really appreciate
> your
> > > > > effort,
> > > > > > > and
> > > > > > > > >> I
> > > > > > > > >>> will
> > > > > > > > >>>>> check out the PR when it comes out.
> > > > > > > > >>>>>
> > > > > > > > >>>>> Best regards,
> > > > > > > > >>>>> Jason
> > > > > > > > >>>>>
> > > > > > > > >>>>> On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <
> > > > > kaxilnaik@gmail.com>
> > > > > > > > >>> wrote:
> > > > > > > > >>>>>> Hey all,
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> *tl;dr*: I'm proposing an official Provider Registry
> for
> > > the
> > > > > > > > >> Airflow
> > > > > > > > >>>>>> project, deployed at
> > https://airflow.apache.org/registry/
> > > .
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> Preview is up at
> > > > https://airflow.staged.apache.org/registry/
> > > > > --
> > > > > > > > >>> take a
> > > > > > > > >>>>>> look
> > > > > > > > >>>>>> and let me know what you think.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> PR to Airflow repo incoming in a couple of hours :)
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> *Why now*
> > > > > > > > >>>>>> With AIP-95 approved, Airflow now has a formal
> provider
> > > > > > lifecycle:
> > > > > > > > >>>>>> incubation, production, mature, and deprecated. That
> > opens
> > > > the
> > > > > > > door
> > > > > > > > >>> for
> > > > > > > > >>>>>> accepting more community-built providers and giving
> them
> > > an
> > > > > > > > >> official
> > > > > > > > >>>>> home,
> > > > > > > > >>>>>> while setting clear expectations about maturity and
> > > support.
> > > > > But
> > > > > > > > >>>>> lifecycle
> > > > > > > > >>>>>> stages only work if users can actually see them.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> Right now, there's no place on airflow.apache.org
> where
> > > > > someone
> > > > > > > > >> can
> > > > > > > > >>>>> browse
> > > > > > > > >>>>>> providers, check their lifecycle stage, or discover
> what
> > > > > modules
> > > > > > > > >> they
> > > > > > > > >>>>> ship.
> > > > > > > > >>>>>> This registry fills that gap. It gives the PMC a tool
> to
> > > > > > > > >> communicate
> > > > > > > > >>>>>> provider maturity to users, and it gives the community
> > an
> > > > > > official
> > > > > > > > >>> way to
> > > > > > > > >>>>>> surface new providers -- clearly labelled with their
> > > > lifecycle
> > > > > > > > >> stage.
> > > > > > > > >>>>>> *What it does*
> > > > > > > > >>>>>> The registry currently catalogs 99 providers and 1,648
> > > > modules
> > > > > > > > >>> across all
> > > > > > > > >>>>>> 11 module types (operators, hooks, sensors, triggers,
> > > > > transfers,
> > > > > > > > >>>>> executors,
> > > > > > > > >>>>>> notifiers, secret backends, logging handlers, Dag
> > bundles,
> > > > and
> > > > > > > > >>>>> decorators).
> > > > > > > > >>>>>> It's built with Eleventy <https://www.11ty.dev/>
> > (thanks
> > > > Ash,
> > > > > > for
> > > > > > > > >>> the
> > > > > > > > >>>>>> suggestion and for prototyping an approach with it)
> and
> > > > > > > > >>> auto-generated
> > > > > > > > >>>>>> directly from the provider.yaml files in the repo --
> no
> > > > > separate
> > > > > > > > >> data
> > > > > > > > >>>>>> pipeline, no manual curation. When a provider is added
> > or
> > > > > > updated,
> > > > > > > > >>> the
> > > > > > > > >>>>> next
> > > > > > > > >>>>>> CI build picks it up automatically.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> The entire registry is a static site (HTML, CSS, JS):
> no
> > > > > server,
> > > > > > > no
> > > > > > > > >>>>>> database, same deployment model as the existing
> Airflow
> > > > docs.
> > > > > > It's
> > > > > > > > >>>>>> generated at build time from the provider.yaml files
> and
> > > > > served
> > > > > > > > >> from
> > > > > > > > >>> S3
> > > > > > > > >>>>> via
> > > > > > > > >>>>>> CloudFront.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> *A few things you can do with it*:
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> - Search across all providers and modules (Cmd+K,
> > > > powered
> > > > > by
> > > > > > > > >>> Pagefind)
> > > > > > > > >>>>>> - Browse by category (Cloud, Databases, AI & ML,
> > etc.)
> > > > > > > > >>>>>> - Filter/sort by lifecycle stage, downloads,
> module
> > > > count
> > > > > > > > >>>>>> - Explore provider detail pages with per-version
> > > module
> > > > > > > > >> listings,
> > > > > > > > >>>>>> connection types, parameters, and install commands
> > > > > > > > >>>>>> - Access JSON API endpoints (/api/providers.json,
> > > > > > > > >>> /api/modules.json)
> > > > > > > > >>>>> for
> > > > > > > > >>>>>> programmatic access -- useful for AI agents and
> > > tooling
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> The design is deliberately discovery-first: it links
> out
> > > to
> > > > > the
> > > > > > > API
> > > > > > > > >>>>>> reference docs and user guides rather than hosting
> > > > everything
> > > > > > > > >> itself.
> > > > > > > > >>>>> This
> > > > > > > > >>>>>> avoids duplicating content between provider docs and
> > > > registry
> > > > > > > > >>> entries.
> > > > > > > > >>>>>> CI/CD is integrated with our existing docs pipeline
> and
> > > > syncs
> > > > > to
> > > > > > > S3
> > > > > > > > >>>>>> automatically. Nothing in provider code, provider.yaml
> > > > > schemas,
> > > > > > > > >> core
> > > > > > > > >>>>>> Airflow, or the docs build is changed by this.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> *How it relates to the Astronomer Registry*
> > > > > > > > >>>>>> Many of you know the Astronomer Registry (
> > > > > > > > >>> https://registry.astronomer.io
> > > > > > > > >>>>> ),
> > > > > > > > >>>>>> which has been the go-to for discovering Airflow
> > providers
> > > > for
> > > > > > > > >>> years. Big
> > > > > > > > >>>>>> thanks to Astronomer and Josh Fell for building and
> > > > > maintaining
> > > > > > > it.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> This new registry is designed to be a community-owned
> > > > > successor
> > > > > > on
> > > > > > > > >>>>>> airflow.apache.org, with the eventual goal of
> > redirecting
> > > > > > > > >>>>>> registry.astronomer.io traffic here once it's stable.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> *Remaining work*
> > > > > > > > >>>>>> Still to do after this lands:
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> - apache/airflow-site PR for .htaccess rewrite
> and a
> > > > > > > "Registry"
> > > > > > > > >>> nav
> > > > > > > > >>>>> link
> > > > > > > > >>>>>> - Redirect registry.astronomer.io traffic once
> the
> > > > > official
> > > > > > > one
> > > > > > > > >>> is
> > > > > > > > >>>>>> stable
> > > > > > > > >>>>>> - A way to add third-party providers that are not
> in
> > > the
> > > > > > > Airflow
> > > > > > > > >>> repo,
> > > > > > > > >>>>>> like Great Expectations, Cosmos etc - I have a POC
> > > > working
> > > > > > on
> > > > > > > > >>> this.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> *Future ideas (will create GH issues)*
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> - Explicit categories in provider.yaml (currently
> > > > > > > keyword-based
> > > > > > > > >>>>>> matching)
> > > > > > > > >>>>>> - LLM-friendly exports (llms.txt, "Copy for AI"
> > > buttons
> > > > > > etc.)
> > > > > > > > >>>>>> - Example DAGs for each Provider.
> > > > > > > > >>>>>> - and many more – but I think the current state is
> > > > > valuable
> > > > > > > > >> enough
> > > > > > > > >>>>>> already
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> I'd appreciate feedback and reviews!
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> Regards,
> > > > > > > > >>>>>> Kaxil
> > > > > > > > >>>>>>
> > > > > > > > >>>
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > >>> To unsubscribe, e-mail:
> dev-unsubscribe@airflow.apache.org
> > > > > > > > >>> For additional commands, e-mail:
> > dev-help@airflow.apache.org
> > > > > > > > >>>
> > > > > > > > >>>
> > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@airflow.apache.org
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Bugra Ozturk
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Kaxil Naik <ka...@gmail.com>.
Thank you for all the feedback, I really appreciate it.
In the interest of ensuring reviews become manageable I merged the PR. If
there are any issues in CI or around doc publishing, please let me know.
Nothing would show up in the "live" (non-staging site) until I add
`.htaccess`. I plan to work on incremental updates to handle some of the
todos and review comments as separate PRs next week with a goal to get
the *Airflow
Registry* live around the same timing as Airflow 3.2 or by the week of 16th
March.
I hope to write some sort of blog post on the Airflow website about it at
the time of launch.
Regards,
Kaxil
On Thu, 5 Mar 2026 at 05:30, Srabasti Banerjee <
srabasti.banerjee.c@gmail.com> wrote:
> This is cool - all providers visually in one place - without scrolling
> through pages of documentation - in dark mode - super helpful Kaxil:)
>
> Warm Regards,
> Srabasti Banerjee
>
> On Tue, Feb 24, 2026 at 9:37 PM Rahul Vats <ra...@gmail.com> wrote:
>
> > This is great, thank you Kaxil!
> >
> >
> > Regards,
> > Rahul Vats
> >
> > On Tue, 24 Feb 2026 at 19:32, Dheeraj Turaga <dh...@gmail.com>
> > wrote:
> >
> > > Oh wow! Looks amazing! Great work Kaxil!!!
> > >
> > > On Mon, Feb 23, 2026 at 3:12 PM Buğra Öztürk <oz...@gmail.com>
> > > wrote:
> > >
> > > > Looks really cool, thanks Kaxil! This is a great update, and it feels
> > how
> > > > it should exhibit in the scale of the Airflow providers :)
> > > >
> > > > On Mon, Feb 23, 2026 at 5:17 PM Vikram Koka via dev <
> > > > dev@airflow.apache.org>
> > > > wrote:
> > > >
> > > > > Excellent work Kaxil!
> > > > > This is absolutely transformative for the Airflow user community
> and
> > > the
> > > > > integrations effort
> > > > >
> > > > > Vikram
> > > > >
> > > > > On Sun, Feb 22, 2026 at 10:47 PM Amogh Desai <
> amoghdesai@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Looks amazing, thanks for working on this, Kaxil.
> > > > > >
> > > > > > I already see it to be super useful instead of going into
> > manhunting
> > > > > > on the Airflow docs and google searches.
> > > > > >
> > > > > > Looking forward to see how this one will be adopted in the
> future.
> > > > > >
> > > > > > Thanks & Regards,
> > > > > > Amogh Desai
> > > > > >
> > > > > >
> > > > > > On Sun, Feb 22, 2026 at 1:34 AM Jens Scheffler <
> > jscheffl@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > +1 into "Very cool" category. The rest of Airflow site
> > immediately
> > > > > feels
> > > > > > > like 1980 :-D
> > > > > > >
> > > > > > > On 20.02.26 23:37, Kaxil Naik wrote:
> > > > > > > > PR: https://github.com/apache/airflow/pull/62261
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, 20 Feb 2026 at 21:50, Pavankumar Gopidesu <
> > > > > > > gopidesupavan@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Woohoo, what an update , thanks kaxil, it's really nice :)
> > > > > > > >>
> > > > > > > >> On Fri, Feb 20, 2026 at 9:12 PM Vincent Beck <
> > > vincbeck@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > > >>
> > > > > > > >>> Very cool indeed :) I can see it very useful for users!
> > > > > > > >>>
> > > > > > > >>> On 2026/02/20 20:04:58 Jarek Potiuk wrote:
> > > > > > > >>>> Very cool :)
> > > > > > > >>>>
> > > > > > > >>>> J.
> > > > > > > >>>>
> > > > > > > >>>>
> > > > > > > >>>> On Fri, Feb 20, 2026 at 6:22 PM Zhe-You Liu <
> > > > zhu424.dev@gmail.com
> > > > > >
> > > > > > > >>> wrote:
> > > > > > > >>>>> Hi Kaxil,
> > > > > > > >>>>>
> > > > > > > >>>>> I just checked out the staged site of the official
> Provider
> > > > > > Registry,
> > > > > > > >>> and
> > > > > > > >>>>> it looks super impressive! The UX is excellent, and it’s
> > > > > > definitely a
> > > > > > > >>> big
> > > > > > > >>>>> plus for users to find existing community integrations
> for
> > > > their
> > > > > > use
> > > > > > > >>> cases
> > > > > > > >>>>> and involve more Airflow users. I really appreciate your
> > > > effort,
> > > > > > and
> > > > > > > >> I
> > > > > > > >>> will
> > > > > > > >>>>> check out the PR when it comes out.
> > > > > > > >>>>>
> > > > > > > >>>>> Best regards,
> > > > > > > >>>>> Jason
> > > > > > > >>>>>
> > > > > > > >>>>> On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <
> > > > kaxilnaik@gmail.com>
> > > > > > > >>> wrote:
> > > > > > > >>>>>> Hey all,
> > > > > > > >>>>>>
> > > > > > > >>>>>> *tl;dr*: I'm proposing an official Provider Registry for
> > the
> > > > > > > >> Airflow
> > > > > > > >>>>>> project, deployed at
> https://airflow.apache.org/registry/
> > .
> > > > > > > >>>>>>
> > > > > > > >>>>>> Preview is up at
> > > https://airflow.staged.apache.org/registry/
> > > > --
> > > > > > > >>> take a
> > > > > > > >>>>>> look
> > > > > > > >>>>>> and let me know what you think.
> > > > > > > >>>>>>
> > > > > > > >>>>>> PR to Airflow repo incoming in a couple of hours :)
> > > > > > > >>>>>>
> > > > > > > >>>>>> *Why now*
> > > > > > > >>>>>> With AIP-95 approved, Airflow now has a formal provider
> > > > > lifecycle:
> > > > > > > >>>>>> incubation, production, mature, and deprecated. That
> opens
> > > the
> > > > > > door
> > > > > > > >>> for
> > > > > > > >>>>>> accepting more community-built providers and giving them
> > an
> > > > > > > >> official
> > > > > > > >>>>> home,
> > > > > > > >>>>>> while setting clear expectations about maturity and
> > support.
> > > > But
> > > > > > > >>>>> lifecycle
> > > > > > > >>>>>> stages only work if users can actually see them.
> > > > > > > >>>>>>
> > > > > > > >>>>>> Right now, there's no place on airflow.apache.org where
> > > > someone
> > > > > > > >> can
> > > > > > > >>>>> browse
> > > > > > > >>>>>> providers, check their lifecycle stage, or discover what
> > > > modules
> > > > > > > >> they
> > > > > > > >>>>> ship.
> > > > > > > >>>>>> This registry fills that gap. It gives the PMC a tool to
> > > > > > > >> communicate
> > > > > > > >>>>>> provider maturity to users, and it gives the community
> an
> > > > > official
> > > > > > > >>> way to
> > > > > > > >>>>>> surface new providers -- clearly labelled with their
> > > lifecycle
> > > > > > > >> stage.
> > > > > > > >>>>>> *What it does*
> > > > > > > >>>>>> The registry currently catalogs 99 providers and 1,648
> > > modules
> > > > > > > >>> across all
> > > > > > > >>>>>> 11 module types (operators, hooks, sensors, triggers,
> > > > transfers,
> > > > > > > >>>>> executors,
> > > > > > > >>>>>> notifiers, secret backends, logging handlers, Dag
> bundles,
> > > and
> > > > > > > >>>>> decorators).
> > > > > > > >>>>>> It's built with Eleventy <https://www.11ty.dev/>
> (thanks
> > > Ash,
> > > > > for
> > > > > > > >>> the
> > > > > > > >>>>>> suggestion and for prototyping an approach with it) and
> > > > > > > >>> auto-generated
> > > > > > > >>>>>> directly from the provider.yaml files in the repo -- no
> > > > separate
> > > > > > > >> data
> > > > > > > >>>>>> pipeline, no manual curation. When a provider is added
> or
> > > > > updated,
> > > > > > > >>> the
> > > > > > > >>>>> next
> > > > > > > >>>>>> CI build picks it up automatically.
> > > > > > > >>>>>>
> > > > > > > >>>>>> The entire registry is a static site (HTML, CSS, JS): no
> > > > server,
> > > > > > no
> > > > > > > >>>>>> database, same deployment model as the existing Airflow
> > > docs.
> > > > > It's
> > > > > > > >>>>>> generated at build time from the provider.yaml files and
> > > > served
> > > > > > > >> from
> > > > > > > >>> S3
> > > > > > > >>>>> via
> > > > > > > >>>>>> CloudFront.
> > > > > > > >>>>>>
> > > > > > > >>>>>> *A few things you can do with it*:
> > > > > > > >>>>>>
> > > > > > > >>>>>> - Search across all providers and modules (Cmd+K,
> > > powered
> > > > by
> > > > > > > >>> Pagefind)
> > > > > > > >>>>>> - Browse by category (Cloud, Databases, AI & ML,
> etc.)
> > > > > > > >>>>>> - Filter/sort by lifecycle stage, downloads, module
> > > count
> > > > > > > >>>>>> - Explore provider detail pages with per-version
> > module
> > > > > > > >> listings,
> > > > > > > >>>>>> connection types, parameters, and install commands
> > > > > > > >>>>>> - Access JSON API endpoints (/api/providers.json,
> > > > > > > >>> /api/modules.json)
> > > > > > > >>>>> for
> > > > > > > >>>>>> programmatic access -- useful for AI agents and
> > tooling
> > > > > > > >>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>> The design is deliberately discovery-first: it links out
> > to
> > > > the
> > > > > > API
> > > > > > > >>>>>> reference docs and user guides rather than hosting
> > > everything
> > > > > > > >> itself.
> > > > > > > >>>>> This
> > > > > > > >>>>>> avoids duplicating content between provider docs and
> > > registry
> > > > > > > >>> entries.
> > > > > > > >>>>>> CI/CD is integrated with our existing docs pipeline and
> > > syncs
> > > > to
> > > > > > S3
> > > > > > > >>>>>> automatically. Nothing in provider code, provider.yaml
> > > > schemas,
> > > > > > > >> core
> > > > > > > >>>>>> Airflow, or the docs build is changed by this.
> > > > > > > >>>>>>
> > > > > > > >>>>>> *How it relates to the Astronomer Registry*
> > > > > > > >>>>>> Many of you know the Astronomer Registry (
> > > > > > > >>> https://registry.astronomer.io
> > > > > > > >>>>> ),
> > > > > > > >>>>>> which has been the go-to for discovering Airflow
> providers
> > > for
> > > > > > > >>> years. Big
> > > > > > > >>>>>> thanks to Astronomer and Josh Fell for building and
> > > > maintaining
> > > > > > it.
> > > > > > > >>>>>>
> > > > > > > >>>>>> This new registry is designed to be a community-owned
> > > > successor
> > > > > on
> > > > > > > >>>>>> airflow.apache.org, with the eventual goal of
> redirecting
> > > > > > > >>>>>> registry.astronomer.io traffic here once it's stable.
> > > > > > > >>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>> *Remaining work*
> > > > > > > >>>>>> Still to do after this lands:
> > > > > > > >>>>>>
> > > > > > > >>>>>> - apache/airflow-site PR for .htaccess rewrite and a
> > > > > > "Registry"
> > > > > > > >>> nav
> > > > > > > >>>>> link
> > > > > > > >>>>>> - Redirect registry.astronomer.io traffic once the
> > > > official
> > > > > > one
> > > > > > > >>> is
> > > > > > > >>>>>> stable
> > > > > > > >>>>>> - A way to add third-party providers that are not in
> > the
> > > > > > Airflow
> > > > > > > >>> repo,
> > > > > > > >>>>>> like Great Expectations, Cosmos etc - I have a POC
> > > working
> > > > > on
> > > > > > > >>> this.
> > > > > > > >>>>>>
> > > > > > > >>>>>> *Future ideas (will create GH issues)*
> > > > > > > >>>>>>
> > > > > > > >>>>>> - Explicit categories in provider.yaml (currently
> > > > > > keyword-based
> > > > > > > >>>>>> matching)
> > > > > > > >>>>>> - LLM-friendly exports (llms.txt, "Copy for AI"
> > buttons
> > > > > etc.)
> > > > > > > >>>>>> - Example DAGs for each Provider.
> > > > > > > >>>>>> - and many more – but I think the current state is
> > > > valuable
> > > > > > > >> enough
> > > > > > > >>>>>> already
> > > > > > > >>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>> I'd appreciate feedback and reviews!
> > > > > > > >>>>>>
> > > > > > > >>>>>> Regards,
> > > > > > > >>>>>> Kaxil
> > > > > > > >>>>>>
> > > > > > > >>>
> > > > >
> ---------------------------------------------------------------------
> > > > > > > >>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > > > > > > >>> For additional commands, e-mail:
> dev-help@airflow.apache.org
> > > > > > > >>>
> > > > > > > >>>
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > > > > > > For additional commands, e-mail: dev-help@airflow.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Bugra Ozturk
> > > >
> > >
> >
>
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Srabasti Banerjee <sr...@gmail.com>.
This is cool - all providers visually in one place - without scrolling
through pages of documentation - in dark mode - super helpful Kaxil:)
Warm Regards,
Srabasti Banerjee
On Tue, Feb 24, 2026 at 9:37 PM Rahul Vats <ra...@gmail.com> wrote:
> This is great, thank you Kaxil!
>
>
> Regards,
> Rahul Vats
>
> On Tue, 24 Feb 2026 at 19:32, Dheeraj Turaga <dh...@gmail.com>
> wrote:
>
> > Oh wow! Looks amazing! Great work Kaxil!!!
> >
> > On Mon, Feb 23, 2026 at 3:12 PM Buğra Öztürk <oz...@gmail.com>
> > wrote:
> >
> > > Looks really cool, thanks Kaxil! This is a great update, and it feels
> how
> > > it should exhibit in the scale of the Airflow providers :)
> > >
> > > On Mon, Feb 23, 2026 at 5:17 PM Vikram Koka via dev <
> > > dev@airflow.apache.org>
> > > wrote:
> > >
> > > > Excellent work Kaxil!
> > > > This is absolutely transformative for the Airflow user community and
> > the
> > > > integrations effort
> > > >
> > > > Vikram
> > > >
> > > > On Sun, Feb 22, 2026 at 10:47 PM Amogh Desai <am...@apache.org>
> > > > wrote:
> > > >
> > > > > Looks amazing, thanks for working on this, Kaxil.
> > > > >
> > > > > I already see it to be super useful instead of going into
> manhunting
> > > > > on the Airflow docs and google searches.
> > > > >
> > > > > Looking forward to see how this one will be adopted in the future.
> > > > >
> > > > > Thanks & Regards,
> > > > > Amogh Desai
> > > > >
> > > > >
> > > > > On Sun, Feb 22, 2026 at 1:34 AM Jens Scheffler <
> jscheffl@apache.org>
> > > > > wrote:
> > > > >
> > > > > > +1 into "Very cool" category. The rest of Airflow site
> immediately
> > > > feels
> > > > > > like 1980 :-D
> > > > > >
> > > > > > On 20.02.26 23:37, Kaxil Naik wrote:
> > > > > > > PR: https://github.com/apache/airflow/pull/62261
> > > > > > >
> > > > > > >
> > > > > > > On Fri, 20 Feb 2026 at 21:50, Pavankumar Gopidesu <
> > > > > > gopidesupavan@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Woohoo, what an update , thanks kaxil, it's really nice :)
> > > > > > >>
> > > > > > >> On Fri, Feb 20, 2026 at 9:12 PM Vincent Beck <
> > vincbeck@apache.org
> > > >
> > > > > > wrote:
> > > > > > >>
> > > > > > >>> Very cool indeed :) I can see it very useful for users!
> > > > > > >>>
> > > > > > >>> On 2026/02/20 20:04:58 Jarek Potiuk wrote:
> > > > > > >>>> Very cool :)
> > > > > > >>>>
> > > > > > >>>> J.
> > > > > > >>>>
> > > > > > >>>>
> > > > > > >>>> On Fri, Feb 20, 2026 at 6:22 PM Zhe-You Liu <
> > > zhu424.dev@gmail.com
> > > > >
> > > > > > >>> wrote:
> > > > > > >>>>> Hi Kaxil,
> > > > > > >>>>>
> > > > > > >>>>> I just checked out the staged site of the official Provider
> > > > > Registry,
> > > > > > >>> and
> > > > > > >>>>> it looks super impressive! The UX is excellent, and it’s
> > > > > definitely a
> > > > > > >>> big
> > > > > > >>>>> plus for users to find existing community integrations for
> > > their
> > > > > use
> > > > > > >>> cases
> > > > > > >>>>> and involve more Airflow users. I really appreciate your
> > > effort,
> > > > > and
> > > > > > >> I
> > > > > > >>> will
> > > > > > >>>>> check out the PR when it comes out.
> > > > > > >>>>>
> > > > > > >>>>> Best regards,
> > > > > > >>>>> Jason
> > > > > > >>>>>
> > > > > > >>>>> On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <
> > > kaxilnaik@gmail.com>
> > > > > > >>> wrote:
> > > > > > >>>>>> Hey all,
> > > > > > >>>>>>
> > > > > > >>>>>> *tl;dr*: I'm proposing an official Provider Registry for
> the
> > > > > > >> Airflow
> > > > > > >>>>>> project, deployed at https://airflow.apache.org/registry/
> .
> > > > > > >>>>>>
> > > > > > >>>>>> Preview is up at
> > https://airflow.staged.apache.org/registry/
> > > --
> > > > > > >>> take a
> > > > > > >>>>>> look
> > > > > > >>>>>> and let me know what you think.
> > > > > > >>>>>>
> > > > > > >>>>>> PR to Airflow repo incoming in a couple of hours :)
> > > > > > >>>>>>
> > > > > > >>>>>> *Why now*
> > > > > > >>>>>> With AIP-95 approved, Airflow now has a formal provider
> > > > lifecycle:
> > > > > > >>>>>> incubation, production, mature, and deprecated. That opens
> > the
> > > > > door
> > > > > > >>> for
> > > > > > >>>>>> accepting more community-built providers and giving them
> an
> > > > > > >> official
> > > > > > >>>>> home,
> > > > > > >>>>>> while setting clear expectations about maturity and
> support.
> > > But
> > > > > > >>>>> lifecycle
> > > > > > >>>>>> stages only work if users can actually see them.
> > > > > > >>>>>>
> > > > > > >>>>>> Right now, there's no place on airflow.apache.org where
> > > someone
> > > > > > >> can
> > > > > > >>>>> browse
> > > > > > >>>>>> providers, check their lifecycle stage, or discover what
> > > modules
> > > > > > >> they
> > > > > > >>>>> ship.
> > > > > > >>>>>> This registry fills that gap. It gives the PMC a tool to
> > > > > > >> communicate
> > > > > > >>>>>> provider maturity to users, and it gives the community an
> > > > official
> > > > > > >>> way to
> > > > > > >>>>>> surface new providers -- clearly labelled with their
> > lifecycle
> > > > > > >> stage.
> > > > > > >>>>>> *What it does*
> > > > > > >>>>>> The registry currently catalogs 99 providers and 1,648
> > modules
> > > > > > >>> across all
> > > > > > >>>>>> 11 module types (operators, hooks, sensors, triggers,
> > > transfers,
> > > > > > >>>>> executors,
> > > > > > >>>>>> notifiers, secret backends, logging handlers, Dag bundles,
> > and
> > > > > > >>>>> decorators).
> > > > > > >>>>>> It's built with Eleventy <https://www.11ty.dev/> (thanks
> > Ash,
> > > > for
> > > > > > >>> the
> > > > > > >>>>>> suggestion and for prototyping an approach with it) and
> > > > > > >>> auto-generated
> > > > > > >>>>>> directly from the provider.yaml files in the repo -- no
> > > separate
> > > > > > >> data
> > > > > > >>>>>> pipeline, no manual curation. When a provider is added or
> > > > updated,
> > > > > > >>> the
> > > > > > >>>>> next
> > > > > > >>>>>> CI build picks it up automatically.
> > > > > > >>>>>>
> > > > > > >>>>>> The entire registry is a static site (HTML, CSS, JS): no
> > > server,
> > > > > no
> > > > > > >>>>>> database, same deployment model as the existing Airflow
> > docs.
> > > > It's
> > > > > > >>>>>> generated at build time from the provider.yaml files and
> > > served
> > > > > > >> from
> > > > > > >>> S3
> > > > > > >>>>> via
> > > > > > >>>>>> CloudFront.
> > > > > > >>>>>>
> > > > > > >>>>>> *A few things you can do with it*:
> > > > > > >>>>>>
> > > > > > >>>>>> - Search across all providers and modules (Cmd+K,
> > powered
> > > by
> > > > > > >>> Pagefind)
> > > > > > >>>>>> - Browse by category (Cloud, Databases, AI & ML, etc.)
> > > > > > >>>>>> - Filter/sort by lifecycle stage, downloads, module
> > count
> > > > > > >>>>>> - Explore provider detail pages with per-version
> module
> > > > > > >> listings,
> > > > > > >>>>>> connection types, parameters, and install commands
> > > > > > >>>>>> - Access JSON API endpoints (/api/providers.json,
> > > > > > >>> /api/modules.json)
> > > > > > >>>>> for
> > > > > > >>>>>> programmatic access -- useful for AI agents and
> tooling
> > > > > > >>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>> The design is deliberately discovery-first: it links out
> to
> > > the
> > > > > API
> > > > > > >>>>>> reference docs and user guides rather than hosting
> > everything
> > > > > > >> itself.
> > > > > > >>>>> This
> > > > > > >>>>>> avoids duplicating content between provider docs and
> > registry
> > > > > > >>> entries.
> > > > > > >>>>>> CI/CD is integrated with our existing docs pipeline and
> > syncs
> > > to
> > > > > S3
> > > > > > >>>>>> automatically. Nothing in provider code, provider.yaml
> > > schemas,
> > > > > > >> core
> > > > > > >>>>>> Airflow, or the docs build is changed by this.
> > > > > > >>>>>>
> > > > > > >>>>>> *How it relates to the Astronomer Registry*
> > > > > > >>>>>> Many of you know the Astronomer Registry (
> > > > > > >>> https://registry.astronomer.io
> > > > > > >>>>> ),
> > > > > > >>>>>> which has been the go-to for discovering Airflow providers
> > for
> > > > > > >>> years. Big
> > > > > > >>>>>> thanks to Astronomer and Josh Fell for building and
> > > maintaining
> > > > > it.
> > > > > > >>>>>>
> > > > > > >>>>>> This new registry is designed to be a community-owned
> > > successor
> > > > on
> > > > > > >>>>>> airflow.apache.org, with the eventual goal of redirecting
> > > > > > >>>>>> registry.astronomer.io traffic here once it's stable.
> > > > > > >>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>> *Remaining work*
> > > > > > >>>>>> Still to do after this lands:
> > > > > > >>>>>>
> > > > > > >>>>>> - apache/airflow-site PR for .htaccess rewrite and a
> > > > > "Registry"
> > > > > > >>> nav
> > > > > > >>>>> link
> > > > > > >>>>>> - Redirect registry.astronomer.io traffic once the
> > > official
> > > > > one
> > > > > > >>> is
> > > > > > >>>>>> stable
> > > > > > >>>>>> - A way to add third-party providers that are not in
> the
> > > > > Airflow
> > > > > > >>> repo,
> > > > > > >>>>>> like Great Expectations, Cosmos etc - I have a POC
> > working
> > > > on
> > > > > > >>> this.
> > > > > > >>>>>>
> > > > > > >>>>>> *Future ideas (will create GH issues)*
> > > > > > >>>>>>
> > > > > > >>>>>> - Explicit categories in provider.yaml (currently
> > > > > keyword-based
> > > > > > >>>>>> matching)
> > > > > > >>>>>> - LLM-friendly exports (llms.txt, "Copy for AI"
> buttons
> > > > etc.)
> > > > > > >>>>>> - Example DAGs for each Provider.
> > > > > > >>>>>> - and many more – but I think the current state is
> > > valuable
> > > > > > >> enough
> > > > > > >>>>>> already
> > > > > > >>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>> I'd appreciate feedback and reviews!
> > > > > > >>>>>>
> > > > > > >>>>>> Regards,
> > > > > > >>>>>> Kaxil
> > > > > > >>>>>>
> > > > > > >>>
> > > > ---------------------------------------------------------------------
> > > > > > >>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > > > > > >>> For additional commands, e-mail: dev-help@airflow.apache.org
> > > > > > >>>
> > > > > > >>>
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > > > > > For additional commands, e-mail: dev-help@airflow.apache.org
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Bugra Ozturk
> > >
> >
>
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Rahul Vats <ra...@gmail.com>.
This is great, thank you Kaxil!
Regards,
Rahul Vats
On Tue, 24 Feb 2026 at 19:32, Dheeraj Turaga <dh...@gmail.com>
wrote:
> Oh wow! Looks amazing! Great work Kaxil!!!
>
> On Mon, Feb 23, 2026 at 3:12 PM Buğra Öztürk <oz...@gmail.com>
> wrote:
>
> > Looks really cool, thanks Kaxil! This is a great update, and it feels how
> > it should exhibit in the scale of the Airflow providers :)
> >
> > On Mon, Feb 23, 2026 at 5:17 PM Vikram Koka via dev <
> > dev@airflow.apache.org>
> > wrote:
> >
> > > Excellent work Kaxil!
> > > This is absolutely transformative for the Airflow user community and
> the
> > > integrations effort
> > >
> > > Vikram
> > >
> > > On Sun, Feb 22, 2026 at 10:47 PM Amogh Desai <am...@apache.org>
> > > wrote:
> > >
> > > > Looks amazing, thanks for working on this, Kaxil.
> > > >
> > > > I already see it to be super useful instead of going into manhunting
> > > > on the Airflow docs and google searches.
> > > >
> > > > Looking forward to see how this one will be adopted in the future.
> > > >
> > > > Thanks & Regards,
> > > > Amogh Desai
> > > >
> > > >
> > > > On Sun, Feb 22, 2026 at 1:34 AM Jens Scheffler <js...@apache.org>
> > > > wrote:
> > > >
> > > > > +1 into "Very cool" category. The rest of Airflow site immediately
> > > feels
> > > > > like 1980 :-D
> > > > >
> > > > > On 20.02.26 23:37, Kaxil Naik wrote:
> > > > > > PR: https://github.com/apache/airflow/pull/62261
> > > > > >
> > > > > >
> > > > > > On Fri, 20 Feb 2026 at 21:50, Pavankumar Gopidesu <
> > > > > gopidesupavan@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > >> Woohoo, what an update , thanks kaxil, it's really nice :)
> > > > > >>
> > > > > >> On Fri, Feb 20, 2026 at 9:12 PM Vincent Beck <
> vincbeck@apache.org
> > >
> > > > > wrote:
> > > > > >>
> > > > > >>> Very cool indeed :) I can see it very useful for users!
> > > > > >>>
> > > > > >>> On 2026/02/20 20:04:58 Jarek Potiuk wrote:
> > > > > >>>> Very cool :)
> > > > > >>>>
> > > > > >>>> J.
> > > > > >>>>
> > > > > >>>>
> > > > > >>>> On Fri, Feb 20, 2026 at 6:22 PM Zhe-You Liu <
> > zhu424.dev@gmail.com
> > > >
> > > > > >>> wrote:
> > > > > >>>>> Hi Kaxil,
> > > > > >>>>>
> > > > > >>>>> I just checked out the staged site of the official Provider
> > > > Registry,
> > > > > >>> and
> > > > > >>>>> it looks super impressive! The UX is excellent, and it’s
> > > > definitely a
> > > > > >>> big
> > > > > >>>>> plus for users to find existing community integrations for
> > their
> > > > use
> > > > > >>> cases
> > > > > >>>>> and involve more Airflow users. I really appreciate your
> > effort,
> > > > and
> > > > > >> I
> > > > > >>> will
> > > > > >>>>> check out the PR when it comes out.
> > > > > >>>>>
> > > > > >>>>> Best regards,
> > > > > >>>>> Jason
> > > > > >>>>>
> > > > > >>>>> On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <
> > kaxilnaik@gmail.com>
> > > > > >>> wrote:
> > > > > >>>>>> Hey all,
> > > > > >>>>>>
> > > > > >>>>>> *tl;dr*: I'm proposing an official Provider Registry for the
> > > > > >> Airflow
> > > > > >>>>>> project, deployed at https://airflow.apache.org/registry/.
> > > > > >>>>>>
> > > > > >>>>>> Preview is up at
> https://airflow.staged.apache.org/registry/
> > --
> > > > > >>> take a
> > > > > >>>>>> look
> > > > > >>>>>> and let me know what you think.
> > > > > >>>>>>
> > > > > >>>>>> PR to Airflow repo incoming in a couple of hours :)
> > > > > >>>>>>
> > > > > >>>>>> *Why now*
> > > > > >>>>>> With AIP-95 approved, Airflow now has a formal provider
> > > lifecycle:
> > > > > >>>>>> incubation, production, mature, and deprecated. That opens
> the
> > > > door
> > > > > >>> for
> > > > > >>>>>> accepting more community-built providers and giving them an
> > > > > >> official
> > > > > >>>>> home,
> > > > > >>>>>> while setting clear expectations about maturity and support.
> > But
> > > > > >>>>> lifecycle
> > > > > >>>>>> stages only work if users can actually see them.
> > > > > >>>>>>
> > > > > >>>>>> Right now, there's no place on airflow.apache.org where
> > someone
> > > > > >> can
> > > > > >>>>> browse
> > > > > >>>>>> providers, check their lifecycle stage, or discover what
> > modules
> > > > > >> they
> > > > > >>>>> ship.
> > > > > >>>>>> This registry fills that gap. It gives the PMC a tool to
> > > > > >> communicate
> > > > > >>>>>> provider maturity to users, and it gives the community an
> > > official
> > > > > >>> way to
> > > > > >>>>>> surface new providers -- clearly labelled with their
> lifecycle
> > > > > >> stage.
> > > > > >>>>>> *What it does*
> > > > > >>>>>> The registry currently catalogs 99 providers and 1,648
> modules
> > > > > >>> across all
> > > > > >>>>>> 11 module types (operators, hooks, sensors, triggers,
> > transfers,
> > > > > >>>>> executors,
> > > > > >>>>>> notifiers, secret backends, logging handlers, Dag bundles,
> and
> > > > > >>>>> decorators).
> > > > > >>>>>> It's built with Eleventy <https://www.11ty.dev/> (thanks
> Ash,
> > > for
> > > > > >>> the
> > > > > >>>>>> suggestion and for prototyping an approach with it) and
> > > > > >>> auto-generated
> > > > > >>>>>> directly from the provider.yaml files in the repo -- no
> > separate
> > > > > >> data
> > > > > >>>>>> pipeline, no manual curation. When a provider is added or
> > > updated,
> > > > > >>> the
> > > > > >>>>> next
> > > > > >>>>>> CI build picks it up automatically.
> > > > > >>>>>>
> > > > > >>>>>> The entire registry is a static site (HTML, CSS, JS): no
> > server,
> > > > no
> > > > > >>>>>> database, same deployment model as the existing Airflow
> docs.
> > > It's
> > > > > >>>>>> generated at build time from the provider.yaml files and
> > served
> > > > > >> from
> > > > > >>> S3
> > > > > >>>>> via
> > > > > >>>>>> CloudFront.
> > > > > >>>>>>
> > > > > >>>>>> *A few things you can do with it*:
> > > > > >>>>>>
> > > > > >>>>>> - Search across all providers and modules (Cmd+K,
> powered
> > by
> > > > > >>> Pagefind)
> > > > > >>>>>> - Browse by category (Cloud, Databases, AI & ML, etc.)
> > > > > >>>>>> - Filter/sort by lifecycle stage, downloads, module
> count
> > > > > >>>>>> - Explore provider detail pages with per-version module
> > > > > >> listings,
> > > > > >>>>>> connection types, parameters, and install commands
> > > > > >>>>>> - Access JSON API endpoints (/api/providers.json,
> > > > > >>> /api/modules.json)
> > > > > >>>>> for
> > > > > >>>>>> programmatic access -- useful for AI agents and tooling
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>> The design is deliberately discovery-first: it links out to
> > the
> > > > API
> > > > > >>>>>> reference docs and user guides rather than hosting
> everything
> > > > > >> itself.
> > > > > >>>>> This
> > > > > >>>>>> avoids duplicating content between provider docs and
> registry
> > > > > >>> entries.
> > > > > >>>>>> CI/CD is integrated with our existing docs pipeline and
> syncs
> > to
> > > > S3
> > > > > >>>>>> automatically. Nothing in provider code, provider.yaml
> > schemas,
> > > > > >> core
> > > > > >>>>>> Airflow, or the docs build is changed by this.
> > > > > >>>>>>
> > > > > >>>>>> *How it relates to the Astronomer Registry*
> > > > > >>>>>> Many of you know the Astronomer Registry (
> > > > > >>> https://registry.astronomer.io
> > > > > >>>>> ),
> > > > > >>>>>> which has been the go-to for discovering Airflow providers
> for
> > > > > >>> years. Big
> > > > > >>>>>> thanks to Astronomer and Josh Fell for building and
> > maintaining
> > > > it.
> > > > > >>>>>>
> > > > > >>>>>> This new registry is designed to be a community-owned
> > successor
> > > on
> > > > > >>>>>> airflow.apache.org, with the eventual goal of redirecting
> > > > > >>>>>> registry.astronomer.io traffic here once it's stable.
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>> *Remaining work*
> > > > > >>>>>> Still to do after this lands:
> > > > > >>>>>>
> > > > > >>>>>> - apache/airflow-site PR for .htaccess rewrite and a
> > > > "Registry"
> > > > > >>> nav
> > > > > >>>>> link
> > > > > >>>>>> - Redirect registry.astronomer.io traffic once the
> > official
> > > > one
> > > > > >>> is
> > > > > >>>>>> stable
> > > > > >>>>>> - A way to add third-party providers that are not in the
> > > > Airflow
> > > > > >>> repo,
> > > > > >>>>>> like Great Expectations, Cosmos etc - I have a POC
> working
> > > on
> > > > > >>> this.
> > > > > >>>>>>
> > > > > >>>>>> *Future ideas (will create GH issues)*
> > > > > >>>>>>
> > > > > >>>>>> - Explicit categories in provider.yaml (currently
> > > > keyword-based
> > > > > >>>>>> matching)
> > > > > >>>>>> - LLM-friendly exports (llms.txt, "Copy for AI" buttons
> > > etc.)
> > > > > >>>>>> - Example DAGs for each Provider.
> > > > > >>>>>> - and many more – but I think the current state is
> > valuable
> > > > > >> enough
> > > > > >>>>>> already
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>> I'd appreciate feedback and reviews!
> > > > > >>>>>>
> > > > > >>>>>> Regards,
> > > > > >>>>>> Kaxil
> > > > > >>>>>>
> > > > > >>>
> > > ---------------------------------------------------------------------
> > > > > >>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > > > > >>> For additional commands, e-mail: dev-help@airflow.apache.org
> > > > > >>>
> > > > > >>>
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > > > > For additional commands, e-mail: dev-help@airflow.apache.org
> > > > >
> > > > >
> > > >
> > >
> >
> >
> > --
> > Bugra Ozturk
> >
>
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Dheeraj Turaga <dh...@gmail.com>.
Oh wow! Looks amazing! Great work Kaxil!!!
On Mon, Feb 23, 2026 at 3:12 PM Buğra Öztürk <oz...@gmail.com>
wrote:
> Looks really cool, thanks Kaxil! This is a great update, and it feels how
> it should exhibit in the scale of the Airflow providers :)
>
> On Mon, Feb 23, 2026 at 5:17 PM Vikram Koka via dev <
> dev@airflow.apache.org>
> wrote:
>
> > Excellent work Kaxil!
> > This is absolutely transformative for the Airflow user community and the
> > integrations effort
> >
> > Vikram
> >
> > On Sun, Feb 22, 2026 at 10:47 PM Amogh Desai <am...@apache.org>
> > wrote:
> >
> > > Looks amazing, thanks for working on this, Kaxil.
> > >
> > > I already see it to be super useful instead of going into manhunting
> > > on the Airflow docs and google searches.
> > >
> > > Looking forward to see how this one will be adopted in the future.
> > >
> > > Thanks & Regards,
> > > Amogh Desai
> > >
> > >
> > > On Sun, Feb 22, 2026 at 1:34 AM Jens Scheffler <js...@apache.org>
> > > wrote:
> > >
> > > > +1 into "Very cool" category. The rest of Airflow site immediately
> > feels
> > > > like 1980 :-D
> > > >
> > > > On 20.02.26 23:37, Kaxil Naik wrote:
> > > > > PR: https://github.com/apache/airflow/pull/62261
> > > > >
> > > > >
> > > > > On Fri, 20 Feb 2026 at 21:50, Pavankumar Gopidesu <
> > > > gopidesupavan@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> Woohoo, what an update , thanks kaxil, it's really nice :)
> > > > >>
> > > > >> On Fri, Feb 20, 2026 at 9:12 PM Vincent Beck <vincbeck@apache.org
> >
> > > > wrote:
> > > > >>
> > > > >>> Very cool indeed :) I can see it very useful for users!
> > > > >>>
> > > > >>> On 2026/02/20 20:04:58 Jarek Potiuk wrote:
> > > > >>>> Very cool :)
> > > > >>>>
> > > > >>>> J.
> > > > >>>>
> > > > >>>>
> > > > >>>> On Fri, Feb 20, 2026 at 6:22 PM Zhe-You Liu <
> zhu424.dev@gmail.com
> > >
> > > > >>> wrote:
> > > > >>>>> Hi Kaxil,
> > > > >>>>>
> > > > >>>>> I just checked out the staged site of the official Provider
> > > Registry,
> > > > >>> and
> > > > >>>>> it looks super impressive! The UX is excellent, and it’s
> > > definitely a
> > > > >>> big
> > > > >>>>> plus for users to find existing community integrations for
> their
> > > use
> > > > >>> cases
> > > > >>>>> and involve more Airflow users. I really appreciate your
> effort,
> > > and
> > > > >> I
> > > > >>> will
> > > > >>>>> check out the PR when it comes out.
> > > > >>>>>
> > > > >>>>> Best regards,
> > > > >>>>> Jason
> > > > >>>>>
> > > > >>>>> On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <
> kaxilnaik@gmail.com>
> > > > >>> wrote:
> > > > >>>>>> Hey all,
> > > > >>>>>>
> > > > >>>>>> *tl;dr*: I'm proposing an official Provider Registry for the
> > > > >> Airflow
> > > > >>>>>> project, deployed at https://airflow.apache.org/registry/.
> > > > >>>>>>
> > > > >>>>>> Preview is up at https://airflow.staged.apache.org/registry/
> --
> > > > >>> take a
> > > > >>>>>> look
> > > > >>>>>> and let me know what you think.
> > > > >>>>>>
> > > > >>>>>> PR to Airflow repo incoming in a couple of hours :)
> > > > >>>>>>
> > > > >>>>>> *Why now*
> > > > >>>>>> With AIP-95 approved, Airflow now has a formal provider
> > lifecycle:
> > > > >>>>>> incubation, production, mature, and deprecated. That opens the
> > > door
> > > > >>> for
> > > > >>>>>> accepting more community-built providers and giving them an
> > > > >> official
> > > > >>>>> home,
> > > > >>>>>> while setting clear expectations about maturity and support.
> But
> > > > >>>>> lifecycle
> > > > >>>>>> stages only work if users can actually see them.
> > > > >>>>>>
> > > > >>>>>> Right now, there's no place on airflow.apache.org where
> someone
> > > > >> can
> > > > >>>>> browse
> > > > >>>>>> providers, check their lifecycle stage, or discover what
> modules
> > > > >> they
> > > > >>>>> ship.
> > > > >>>>>> This registry fills that gap. It gives the PMC a tool to
> > > > >> communicate
> > > > >>>>>> provider maturity to users, and it gives the community an
> > official
> > > > >>> way to
> > > > >>>>>> surface new providers -- clearly labelled with their lifecycle
> > > > >> stage.
> > > > >>>>>> *What it does*
> > > > >>>>>> The registry currently catalogs 99 providers and 1,648 modules
> > > > >>> across all
> > > > >>>>>> 11 module types (operators, hooks, sensors, triggers,
> transfers,
> > > > >>>>> executors,
> > > > >>>>>> notifiers, secret backends, logging handlers, Dag bundles, and
> > > > >>>>> decorators).
> > > > >>>>>> It's built with Eleventy <https://www.11ty.dev/> (thanks Ash,
> > for
> > > > >>> the
> > > > >>>>>> suggestion and for prototyping an approach with it) and
> > > > >>> auto-generated
> > > > >>>>>> directly from the provider.yaml files in the repo -- no
> separate
> > > > >> data
> > > > >>>>>> pipeline, no manual curation. When a provider is added or
> > updated,
> > > > >>> the
> > > > >>>>> next
> > > > >>>>>> CI build picks it up automatically.
> > > > >>>>>>
> > > > >>>>>> The entire registry is a static site (HTML, CSS, JS): no
> server,
> > > no
> > > > >>>>>> database, same deployment model as the existing Airflow docs.
> > It's
> > > > >>>>>> generated at build time from the provider.yaml files and
> served
> > > > >> from
> > > > >>> S3
> > > > >>>>> via
> > > > >>>>>> CloudFront.
> > > > >>>>>>
> > > > >>>>>> *A few things you can do with it*:
> > > > >>>>>>
> > > > >>>>>> - Search across all providers and modules (Cmd+K, powered
> by
> > > > >>> Pagefind)
> > > > >>>>>> - Browse by category (Cloud, Databases, AI & ML, etc.)
> > > > >>>>>> - Filter/sort by lifecycle stage, downloads, module count
> > > > >>>>>> - Explore provider detail pages with per-version module
> > > > >> listings,
> > > > >>>>>> connection types, parameters, and install commands
> > > > >>>>>> - Access JSON API endpoints (/api/providers.json,
> > > > >>> /api/modules.json)
> > > > >>>>> for
> > > > >>>>>> programmatic access -- useful for AI agents and tooling
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> The design is deliberately discovery-first: it links out to
> the
> > > API
> > > > >>>>>> reference docs and user guides rather than hosting everything
> > > > >> itself.
> > > > >>>>> This
> > > > >>>>>> avoids duplicating content between provider docs and registry
> > > > >>> entries.
> > > > >>>>>> CI/CD is integrated with our existing docs pipeline and syncs
> to
> > > S3
> > > > >>>>>> automatically. Nothing in provider code, provider.yaml
> schemas,
> > > > >> core
> > > > >>>>>> Airflow, or the docs build is changed by this.
> > > > >>>>>>
> > > > >>>>>> *How it relates to the Astronomer Registry*
> > > > >>>>>> Many of you know the Astronomer Registry (
> > > > >>> https://registry.astronomer.io
> > > > >>>>> ),
> > > > >>>>>> which has been the go-to for discovering Airflow providers for
> > > > >>> years. Big
> > > > >>>>>> thanks to Astronomer and Josh Fell for building and
> maintaining
> > > it.
> > > > >>>>>>
> > > > >>>>>> This new registry is designed to be a community-owned
> successor
> > on
> > > > >>>>>> airflow.apache.org, with the eventual goal of redirecting
> > > > >>>>>> registry.astronomer.io traffic here once it's stable.
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> *Remaining work*
> > > > >>>>>> Still to do after this lands:
> > > > >>>>>>
> > > > >>>>>> - apache/airflow-site PR for .htaccess rewrite and a
> > > "Registry"
> > > > >>> nav
> > > > >>>>> link
> > > > >>>>>> - Redirect registry.astronomer.io traffic once the
> official
> > > one
> > > > >>> is
> > > > >>>>>> stable
> > > > >>>>>> - A way to add third-party providers that are not in the
> > > Airflow
> > > > >>> repo,
> > > > >>>>>> like Great Expectations, Cosmos etc - I have a POC working
> > on
> > > > >>> this.
> > > > >>>>>>
> > > > >>>>>> *Future ideas (will create GH issues)*
> > > > >>>>>>
> > > > >>>>>> - Explicit categories in provider.yaml (currently
> > > keyword-based
> > > > >>>>>> matching)
> > > > >>>>>> - LLM-friendly exports (llms.txt, "Copy for AI" buttons
> > etc.)
> > > > >>>>>> - Example DAGs for each Provider.
> > > > >>>>>> - and many more – but I think the current state is
> valuable
> > > > >> enough
> > > > >>>>>> already
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> I'd appreciate feedback and reviews!
> > > > >>>>>>
> > > > >>>>>> Regards,
> > > > >>>>>> Kaxil
> > > > >>>>>>
> > > > >>>
> > ---------------------------------------------------------------------
> > > > >>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > > > >>> For additional commands, e-mail: dev-help@airflow.apache.org
> > > > >>>
> > > > >>>
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > > > For additional commands, e-mail: dev-help@airflow.apache.org
> > > >
> > > >
> > >
> >
>
>
> --
> Bugra Ozturk
>
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Buğra Öztürk <oz...@gmail.com>.
Looks really cool, thanks Kaxil! This is a great update, and it feels how
it should exhibit in the scale of the Airflow providers :)
On Mon, Feb 23, 2026 at 5:17 PM Vikram Koka via dev <de...@airflow.apache.org>
wrote:
> Excellent work Kaxil!
> This is absolutely transformative for the Airflow user community and the
> integrations effort
>
> Vikram
>
> On Sun, Feb 22, 2026 at 10:47 PM Amogh Desai <am...@apache.org>
> wrote:
>
> > Looks amazing, thanks for working on this, Kaxil.
> >
> > I already see it to be super useful instead of going into manhunting
> > on the Airflow docs and google searches.
> >
> > Looking forward to see how this one will be adopted in the future.
> >
> > Thanks & Regards,
> > Amogh Desai
> >
> >
> > On Sun, Feb 22, 2026 at 1:34 AM Jens Scheffler <js...@apache.org>
> > wrote:
> >
> > > +1 into "Very cool" category. The rest of Airflow site immediately
> feels
> > > like 1980 :-D
> > >
> > > On 20.02.26 23:37, Kaxil Naik wrote:
> > > > PR: https://github.com/apache/airflow/pull/62261
> > > >
> > > >
> > > > On Fri, 20 Feb 2026 at 21:50, Pavankumar Gopidesu <
> > > gopidesupavan@gmail.com>
> > > > wrote:
> > > >
> > > >> Woohoo, what an update , thanks kaxil, it's really nice :)
> > > >>
> > > >> On Fri, Feb 20, 2026 at 9:12 PM Vincent Beck <vi...@apache.org>
> > > wrote:
> > > >>
> > > >>> Very cool indeed :) I can see it very useful for users!
> > > >>>
> > > >>> On 2026/02/20 20:04:58 Jarek Potiuk wrote:
> > > >>>> Very cool :)
> > > >>>>
> > > >>>> J.
> > > >>>>
> > > >>>>
> > > >>>> On Fri, Feb 20, 2026 at 6:22 PM Zhe-You Liu <zhu424.dev@gmail.com
> >
> > > >>> wrote:
> > > >>>>> Hi Kaxil,
> > > >>>>>
> > > >>>>> I just checked out the staged site of the official Provider
> > Registry,
> > > >>> and
> > > >>>>> it looks super impressive! The UX is excellent, and it’s
> > definitely a
> > > >>> big
> > > >>>>> plus for users to find existing community integrations for their
> > use
> > > >>> cases
> > > >>>>> and involve more Airflow users. I really appreciate your effort,
> > and
> > > >> I
> > > >>> will
> > > >>>>> check out the PR when it comes out.
> > > >>>>>
> > > >>>>> Best regards,
> > > >>>>> Jason
> > > >>>>>
> > > >>>>> On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <ka...@gmail.com>
> > > >>> wrote:
> > > >>>>>> Hey all,
> > > >>>>>>
> > > >>>>>> *tl;dr*: I'm proposing an official Provider Registry for the
> > > >> Airflow
> > > >>>>>> project, deployed at https://airflow.apache.org/registry/.
> > > >>>>>>
> > > >>>>>> Preview is up at https://airflow.staged.apache.org/registry/ --
> > > >>> take a
> > > >>>>>> look
> > > >>>>>> and let me know what you think.
> > > >>>>>>
> > > >>>>>> PR to Airflow repo incoming in a couple of hours :)
> > > >>>>>>
> > > >>>>>> *Why now*
> > > >>>>>> With AIP-95 approved, Airflow now has a formal provider
> lifecycle:
> > > >>>>>> incubation, production, mature, and deprecated. That opens the
> > door
> > > >>> for
> > > >>>>>> accepting more community-built providers and giving them an
> > > >> official
> > > >>>>> home,
> > > >>>>>> while setting clear expectations about maturity and support. But
> > > >>>>> lifecycle
> > > >>>>>> stages only work if users can actually see them.
> > > >>>>>>
> > > >>>>>> Right now, there's no place on airflow.apache.org where someone
> > > >> can
> > > >>>>> browse
> > > >>>>>> providers, check their lifecycle stage, or discover what modules
> > > >> they
> > > >>>>> ship.
> > > >>>>>> This registry fills that gap. It gives the PMC a tool to
> > > >> communicate
> > > >>>>>> provider maturity to users, and it gives the community an
> official
> > > >>> way to
> > > >>>>>> surface new providers -- clearly labelled with their lifecycle
> > > >> stage.
> > > >>>>>> *What it does*
> > > >>>>>> The registry currently catalogs 99 providers and 1,648 modules
> > > >>> across all
> > > >>>>>> 11 module types (operators, hooks, sensors, triggers, transfers,
> > > >>>>> executors,
> > > >>>>>> notifiers, secret backends, logging handlers, Dag bundles, and
> > > >>>>> decorators).
> > > >>>>>> It's built with Eleventy <https://www.11ty.dev/> (thanks Ash,
> for
> > > >>> the
> > > >>>>>> suggestion and for prototyping an approach with it) and
> > > >>> auto-generated
> > > >>>>>> directly from the provider.yaml files in the repo -- no separate
> > > >> data
> > > >>>>>> pipeline, no manual curation. When a provider is added or
> updated,
> > > >>> the
> > > >>>>> next
> > > >>>>>> CI build picks it up automatically.
> > > >>>>>>
> > > >>>>>> The entire registry is a static site (HTML, CSS, JS): no server,
> > no
> > > >>>>>> database, same deployment model as the existing Airflow docs.
> It's
> > > >>>>>> generated at build time from the provider.yaml files and served
> > > >> from
> > > >>> S3
> > > >>>>> via
> > > >>>>>> CloudFront.
> > > >>>>>>
> > > >>>>>> *A few things you can do with it*:
> > > >>>>>>
> > > >>>>>> - Search across all providers and modules (Cmd+K, powered by
> > > >>> Pagefind)
> > > >>>>>> - Browse by category (Cloud, Databases, AI & ML, etc.)
> > > >>>>>> - Filter/sort by lifecycle stage, downloads, module count
> > > >>>>>> - Explore provider detail pages with per-version module
> > > >> listings,
> > > >>>>>> connection types, parameters, and install commands
> > > >>>>>> - Access JSON API endpoints (/api/providers.json,
> > > >>> /api/modules.json)
> > > >>>>> for
> > > >>>>>> programmatic access -- useful for AI agents and tooling
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> The design is deliberately discovery-first: it links out to the
> > API
> > > >>>>>> reference docs and user guides rather than hosting everything
> > > >> itself.
> > > >>>>> This
> > > >>>>>> avoids duplicating content between provider docs and registry
> > > >>> entries.
> > > >>>>>> CI/CD is integrated with our existing docs pipeline and syncs to
> > S3
> > > >>>>>> automatically. Nothing in provider code, provider.yaml schemas,
> > > >> core
> > > >>>>>> Airflow, or the docs build is changed by this.
> > > >>>>>>
> > > >>>>>> *How it relates to the Astronomer Registry*
> > > >>>>>> Many of you know the Astronomer Registry (
> > > >>> https://registry.astronomer.io
> > > >>>>> ),
> > > >>>>>> which has been the go-to for discovering Airflow providers for
> > > >>> years. Big
> > > >>>>>> thanks to Astronomer and Josh Fell for building and maintaining
> > it.
> > > >>>>>>
> > > >>>>>> This new registry is designed to be a community-owned successor
> on
> > > >>>>>> airflow.apache.org, with the eventual goal of redirecting
> > > >>>>>> registry.astronomer.io traffic here once it's stable.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> *Remaining work*
> > > >>>>>> Still to do after this lands:
> > > >>>>>>
> > > >>>>>> - apache/airflow-site PR for .htaccess rewrite and a
> > "Registry"
> > > >>> nav
> > > >>>>> link
> > > >>>>>> - Redirect registry.astronomer.io traffic once the official
> > one
> > > >>> is
> > > >>>>>> stable
> > > >>>>>> - A way to add third-party providers that are not in the
> > Airflow
> > > >>> repo,
> > > >>>>>> like Great Expectations, Cosmos etc - I have a POC working
> on
> > > >>> this.
> > > >>>>>>
> > > >>>>>> *Future ideas (will create GH issues)*
> > > >>>>>>
> > > >>>>>> - Explicit categories in provider.yaml (currently
> > keyword-based
> > > >>>>>> matching)
> > > >>>>>> - LLM-friendly exports (llms.txt, "Copy for AI" buttons
> etc.)
> > > >>>>>> - Example DAGs for each Provider.
> > > >>>>>> - and many more – but I think the current state is valuable
> > > >> enough
> > > >>>>>> already
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> I'd appreciate feedback and reviews!
> > > >>>>>>
> > > >>>>>> Regards,
> > > >>>>>> Kaxil
> > > >>>>>>
> > > >>>
> ---------------------------------------------------------------------
> > > >>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > > >>> For additional commands, e-mail: dev-help@airflow.apache.org
> > > >>>
> > > >>>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > > For additional commands, e-mail: dev-help@airflow.apache.org
> > >
> > >
> >
>
--
Bugra Ozturk
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Vikram Koka via dev <de...@airflow.apache.org>.
Excellent work Kaxil!
This is absolutely transformative for the Airflow user community and the
integrations effort
Vikram
On Sun, Feb 22, 2026 at 10:47 PM Amogh Desai <am...@apache.org> wrote:
> Looks amazing, thanks for working on this, Kaxil.
>
> I already see it to be super useful instead of going into manhunting
> on the Airflow docs and google searches.
>
> Looking forward to see how this one will be adopted in the future.
>
> Thanks & Regards,
> Amogh Desai
>
>
> On Sun, Feb 22, 2026 at 1:34 AM Jens Scheffler <js...@apache.org>
> wrote:
>
> > +1 into "Very cool" category. The rest of Airflow site immediately feels
> > like 1980 :-D
> >
> > On 20.02.26 23:37, Kaxil Naik wrote:
> > > PR: https://github.com/apache/airflow/pull/62261
> > >
> > >
> > > On Fri, 20 Feb 2026 at 21:50, Pavankumar Gopidesu <
> > gopidesupavan@gmail.com>
> > > wrote:
> > >
> > >> Woohoo, what an update , thanks kaxil, it's really nice :)
> > >>
> > >> On Fri, Feb 20, 2026 at 9:12 PM Vincent Beck <vi...@apache.org>
> > wrote:
> > >>
> > >>> Very cool indeed :) I can see it very useful for users!
> > >>>
> > >>> On 2026/02/20 20:04:58 Jarek Potiuk wrote:
> > >>>> Very cool :)
> > >>>>
> > >>>> J.
> > >>>>
> > >>>>
> > >>>> On Fri, Feb 20, 2026 at 6:22 PM Zhe-You Liu <zh...@gmail.com>
> > >>> wrote:
> > >>>>> Hi Kaxil,
> > >>>>>
> > >>>>> I just checked out the staged site of the official Provider
> Registry,
> > >>> and
> > >>>>> it looks super impressive! The UX is excellent, and it’s
> definitely a
> > >>> big
> > >>>>> plus for users to find existing community integrations for their
> use
> > >>> cases
> > >>>>> and involve more Airflow users. I really appreciate your effort,
> and
> > >> I
> > >>> will
> > >>>>> check out the PR when it comes out.
> > >>>>>
> > >>>>> Best regards,
> > >>>>> Jason
> > >>>>>
> > >>>>> On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <ka...@gmail.com>
> > >>> wrote:
> > >>>>>> Hey all,
> > >>>>>>
> > >>>>>> *tl;dr*: I'm proposing an official Provider Registry for the
> > >> Airflow
> > >>>>>> project, deployed at https://airflow.apache.org/registry/.
> > >>>>>>
> > >>>>>> Preview is up at https://airflow.staged.apache.org/registry/ --
> > >>> take a
> > >>>>>> look
> > >>>>>> and let me know what you think.
> > >>>>>>
> > >>>>>> PR to Airflow repo incoming in a couple of hours :)
> > >>>>>>
> > >>>>>> *Why now*
> > >>>>>> With AIP-95 approved, Airflow now has a formal provider lifecycle:
> > >>>>>> incubation, production, mature, and deprecated. That opens the
> door
> > >>> for
> > >>>>>> accepting more community-built providers and giving them an
> > >> official
> > >>>>> home,
> > >>>>>> while setting clear expectations about maturity and support. But
> > >>>>> lifecycle
> > >>>>>> stages only work if users can actually see them.
> > >>>>>>
> > >>>>>> Right now, there's no place on airflow.apache.org where someone
> > >> can
> > >>>>> browse
> > >>>>>> providers, check their lifecycle stage, or discover what modules
> > >> they
> > >>>>> ship.
> > >>>>>> This registry fills that gap. It gives the PMC a tool to
> > >> communicate
> > >>>>>> provider maturity to users, and it gives the community an official
> > >>> way to
> > >>>>>> surface new providers -- clearly labelled with their lifecycle
> > >> stage.
> > >>>>>> *What it does*
> > >>>>>> The registry currently catalogs 99 providers and 1,648 modules
> > >>> across all
> > >>>>>> 11 module types (operators, hooks, sensors, triggers, transfers,
> > >>>>> executors,
> > >>>>>> notifiers, secret backends, logging handlers, Dag bundles, and
> > >>>>> decorators).
> > >>>>>> It's built with Eleventy <https://www.11ty.dev/> (thanks Ash, for
> > >>> the
> > >>>>>> suggestion and for prototyping an approach with it) and
> > >>> auto-generated
> > >>>>>> directly from the provider.yaml files in the repo -- no separate
> > >> data
> > >>>>>> pipeline, no manual curation. When a provider is added or updated,
> > >>> the
> > >>>>> next
> > >>>>>> CI build picks it up automatically.
> > >>>>>>
> > >>>>>> The entire registry is a static site (HTML, CSS, JS): no server,
> no
> > >>>>>> database, same deployment model as the existing Airflow docs. It's
> > >>>>>> generated at build time from the provider.yaml files and served
> > >> from
> > >>> S3
> > >>>>> via
> > >>>>>> CloudFront.
> > >>>>>>
> > >>>>>> *A few things you can do with it*:
> > >>>>>>
> > >>>>>> - Search across all providers and modules (Cmd+K, powered by
> > >>> Pagefind)
> > >>>>>> - Browse by category (Cloud, Databases, AI & ML, etc.)
> > >>>>>> - Filter/sort by lifecycle stage, downloads, module count
> > >>>>>> - Explore provider detail pages with per-version module
> > >> listings,
> > >>>>>> connection types, parameters, and install commands
> > >>>>>> - Access JSON API endpoints (/api/providers.json,
> > >>> /api/modules.json)
> > >>>>> for
> > >>>>>> programmatic access -- useful for AI agents and tooling
> > >>>>>>
> > >>>>>>
> > >>>>>> The design is deliberately discovery-first: it links out to the
> API
> > >>>>>> reference docs and user guides rather than hosting everything
> > >> itself.
> > >>>>> This
> > >>>>>> avoids duplicating content between provider docs and registry
> > >>> entries.
> > >>>>>> CI/CD is integrated with our existing docs pipeline and syncs to
> S3
> > >>>>>> automatically. Nothing in provider code, provider.yaml schemas,
> > >> core
> > >>>>>> Airflow, or the docs build is changed by this.
> > >>>>>>
> > >>>>>> *How it relates to the Astronomer Registry*
> > >>>>>> Many of you know the Astronomer Registry (
> > >>> https://registry.astronomer.io
> > >>>>> ),
> > >>>>>> which has been the go-to for discovering Airflow providers for
> > >>> years. Big
> > >>>>>> thanks to Astronomer and Josh Fell for building and maintaining
> it.
> > >>>>>>
> > >>>>>> This new registry is designed to be a community-owned successor on
> > >>>>>> airflow.apache.org, with the eventual goal of redirecting
> > >>>>>> registry.astronomer.io traffic here once it's stable.
> > >>>>>>
> > >>>>>>
> > >>>>>> *Remaining work*
> > >>>>>> Still to do after this lands:
> > >>>>>>
> > >>>>>> - apache/airflow-site PR for .htaccess rewrite and a
> "Registry"
> > >>> nav
> > >>>>> link
> > >>>>>> - Redirect registry.astronomer.io traffic once the official
> one
> > >>> is
> > >>>>>> stable
> > >>>>>> - A way to add third-party providers that are not in the
> Airflow
> > >>> repo,
> > >>>>>> like Great Expectations, Cosmos etc - I have a POC working on
> > >>> this.
> > >>>>>>
> > >>>>>> *Future ideas (will create GH issues)*
> > >>>>>>
> > >>>>>> - Explicit categories in provider.yaml (currently
> keyword-based
> > >>>>>> matching)
> > >>>>>> - LLM-friendly exports (llms.txt, "Copy for AI" buttons etc.)
> > >>>>>> - Example DAGs for each Provider.
> > >>>>>> - and many more – but I think the current state is valuable
> > >> enough
> > >>>>>> already
> > >>>>>>
> > >>>>>>
> > >>>>>> I'd appreciate feedback and reviews!
> > >>>>>>
> > >>>>>> Regards,
> > >>>>>> Kaxil
> > >>>>>>
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > >>> For additional commands, e-mail: dev-help@airflow.apache.org
> > >>>
> > >>>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > For additional commands, e-mail: dev-help@airflow.apache.org
> >
> >
>
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Amogh Desai <am...@apache.org>.
Looks amazing, thanks for working on this, Kaxil.
I already see it to be super useful instead of going into manhunting
on the Airflow docs and google searches.
Looking forward to see how this one will be adopted in the future.
Thanks & Regards,
Amogh Desai
On Sun, Feb 22, 2026 at 1:34 AM Jens Scheffler <js...@apache.org> wrote:
> +1 into "Very cool" category. The rest of Airflow site immediately feels
> like 1980 :-D
>
> On 20.02.26 23:37, Kaxil Naik wrote:
> > PR: https://github.com/apache/airflow/pull/62261
> >
> >
> > On Fri, 20 Feb 2026 at 21:50, Pavankumar Gopidesu <
> gopidesupavan@gmail.com>
> > wrote:
> >
> >> Woohoo, what an update , thanks kaxil, it's really nice :)
> >>
> >> On Fri, Feb 20, 2026 at 9:12 PM Vincent Beck <vi...@apache.org>
> wrote:
> >>
> >>> Very cool indeed :) I can see it very useful for users!
> >>>
> >>> On 2026/02/20 20:04:58 Jarek Potiuk wrote:
> >>>> Very cool :)
> >>>>
> >>>> J.
> >>>>
> >>>>
> >>>> On Fri, Feb 20, 2026 at 6:22 PM Zhe-You Liu <zh...@gmail.com>
> >>> wrote:
> >>>>> Hi Kaxil,
> >>>>>
> >>>>> I just checked out the staged site of the official Provider Registry,
> >>> and
> >>>>> it looks super impressive! The UX is excellent, and it’s definitely a
> >>> big
> >>>>> plus for users to find existing community integrations for their use
> >>> cases
> >>>>> and involve more Airflow users. I really appreciate your effort, and
> >> I
> >>> will
> >>>>> check out the PR when it comes out.
> >>>>>
> >>>>> Best regards,
> >>>>> Jason
> >>>>>
> >>>>> On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <ka...@gmail.com>
> >>> wrote:
> >>>>>> Hey all,
> >>>>>>
> >>>>>> *tl;dr*: I'm proposing an official Provider Registry for the
> >> Airflow
> >>>>>> project, deployed at https://airflow.apache.org/registry/.
> >>>>>>
> >>>>>> Preview is up at https://airflow.staged.apache.org/registry/ --
> >>> take a
> >>>>>> look
> >>>>>> and let me know what you think.
> >>>>>>
> >>>>>> PR to Airflow repo incoming in a couple of hours :)
> >>>>>>
> >>>>>> *Why now*
> >>>>>> With AIP-95 approved, Airflow now has a formal provider lifecycle:
> >>>>>> incubation, production, mature, and deprecated. That opens the door
> >>> for
> >>>>>> accepting more community-built providers and giving them an
> >> official
> >>>>> home,
> >>>>>> while setting clear expectations about maturity and support. But
> >>>>> lifecycle
> >>>>>> stages only work if users can actually see them.
> >>>>>>
> >>>>>> Right now, there's no place on airflow.apache.org where someone
> >> can
> >>>>> browse
> >>>>>> providers, check their lifecycle stage, or discover what modules
> >> they
> >>>>> ship.
> >>>>>> This registry fills that gap. It gives the PMC a tool to
> >> communicate
> >>>>>> provider maturity to users, and it gives the community an official
> >>> way to
> >>>>>> surface new providers -- clearly labelled with their lifecycle
> >> stage.
> >>>>>> *What it does*
> >>>>>> The registry currently catalogs 99 providers and 1,648 modules
> >>> across all
> >>>>>> 11 module types (operators, hooks, sensors, triggers, transfers,
> >>>>> executors,
> >>>>>> notifiers, secret backends, logging handlers, Dag bundles, and
> >>>>> decorators).
> >>>>>> It's built with Eleventy <https://www.11ty.dev/> (thanks Ash, for
> >>> the
> >>>>>> suggestion and for prototyping an approach with it) and
> >>> auto-generated
> >>>>>> directly from the provider.yaml files in the repo -- no separate
> >> data
> >>>>>> pipeline, no manual curation. When a provider is added or updated,
> >>> the
> >>>>> next
> >>>>>> CI build picks it up automatically.
> >>>>>>
> >>>>>> The entire registry is a static site (HTML, CSS, JS): no server, no
> >>>>>> database, same deployment model as the existing Airflow docs. It's
> >>>>>> generated at build time from the provider.yaml files and served
> >> from
> >>> S3
> >>>>> via
> >>>>>> CloudFront.
> >>>>>>
> >>>>>> *A few things you can do with it*:
> >>>>>>
> >>>>>> - Search across all providers and modules (Cmd+K, powered by
> >>> Pagefind)
> >>>>>> - Browse by category (Cloud, Databases, AI & ML, etc.)
> >>>>>> - Filter/sort by lifecycle stage, downloads, module count
> >>>>>> - Explore provider detail pages with per-version module
> >> listings,
> >>>>>> connection types, parameters, and install commands
> >>>>>> - Access JSON API endpoints (/api/providers.json,
> >>> /api/modules.json)
> >>>>> for
> >>>>>> programmatic access -- useful for AI agents and tooling
> >>>>>>
> >>>>>>
> >>>>>> The design is deliberately discovery-first: it links out to the API
> >>>>>> reference docs and user guides rather than hosting everything
> >> itself.
> >>>>> This
> >>>>>> avoids duplicating content between provider docs and registry
> >>> entries.
> >>>>>> CI/CD is integrated with our existing docs pipeline and syncs to S3
> >>>>>> automatically. Nothing in provider code, provider.yaml schemas,
> >> core
> >>>>>> Airflow, or the docs build is changed by this.
> >>>>>>
> >>>>>> *How it relates to the Astronomer Registry*
> >>>>>> Many of you know the Astronomer Registry (
> >>> https://registry.astronomer.io
> >>>>> ),
> >>>>>> which has been the go-to for discovering Airflow providers for
> >>> years. Big
> >>>>>> thanks to Astronomer and Josh Fell for building and maintaining it.
> >>>>>>
> >>>>>> This new registry is designed to be a community-owned successor on
> >>>>>> airflow.apache.org, with the eventual goal of redirecting
> >>>>>> registry.astronomer.io traffic here once it's stable.
> >>>>>>
> >>>>>>
> >>>>>> *Remaining work*
> >>>>>> Still to do after this lands:
> >>>>>>
> >>>>>> - apache/airflow-site PR for .htaccess rewrite and a "Registry"
> >>> nav
> >>>>> link
> >>>>>> - Redirect registry.astronomer.io traffic once the official one
> >>> is
> >>>>>> stable
> >>>>>> - A way to add third-party providers that are not in the Airflow
> >>> repo,
> >>>>>> like Great Expectations, Cosmos etc - I have a POC working on
> >>> this.
> >>>>>>
> >>>>>> *Future ideas (will create GH issues)*
> >>>>>>
> >>>>>> - Explicit categories in provider.yaml (currently keyword-based
> >>>>>> matching)
> >>>>>> - LLM-friendly exports (llms.txt, "Copy for AI" buttons etc.)
> >>>>>> - Example DAGs for each Provider.
> >>>>>> - and many more – but I think the current state is valuable
> >> enough
> >>>>>> already
> >>>>>>
> >>>>>>
> >>>>>> I'd appreciate feedback and reviews!
> >>>>>>
> >>>>>> Regards,
> >>>>>> Kaxil
> >>>>>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> >>> For additional commands, e-mail: dev-help@airflow.apache.org
> >>>
> >>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> For additional commands, e-mail: dev-help@airflow.apache.org
>
>
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Jens Scheffler <js...@apache.org>.
+1 into "Very cool" category. The rest of Airflow site immediately feels
like 1980 :-D
On 20.02.26 23:37, Kaxil Naik wrote:
> PR: https://github.com/apache/airflow/pull/62261
>
>
> On Fri, 20 Feb 2026 at 21:50, Pavankumar Gopidesu <go...@gmail.com>
> wrote:
>
>> Woohoo, what an update , thanks kaxil, it's really nice :)
>>
>> On Fri, Feb 20, 2026 at 9:12 PM Vincent Beck <vi...@apache.org> wrote:
>>
>>> Very cool indeed :) I can see it very useful for users!
>>>
>>> On 2026/02/20 20:04:58 Jarek Potiuk wrote:
>>>> Very cool :)
>>>>
>>>> J.
>>>>
>>>>
>>>> On Fri, Feb 20, 2026 at 6:22 PM Zhe-You Liu <zh...@gmail.com>
>>> wrote:
>>>>> Hi Kaxil,
>>>>>
>>>>> I just checked out the staged site of the official Provider Registry,
>>> and
>>>>> it looks super impressive! The UX is excellent, and it’s definitely a
>>> big
>>>>> plus for users to find existing community integrations for their use
>>> cases
>>>>> and involve more Airflow users. I really appreciate your effort, and
>> I
>>> will
>>>>> check out the PR when it comes out.
>>>>>
>>>>> Best regards,
>>>>> Jason
>>>>>
>>>>> On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <ka...@gmail.com>
>>> wrote:
>>>>>> Hey all,
>>>>>>
>>>>>> *tl;dr*: I'm proposing an official Provider Registry for the
>> Airflow
>>>>>> project, deployed at https://airflow.apache.org/registry/.
>>>>>>
>>>>>> Preview is up at https://airflow.staged.apache.org/registry/ --
>>> take a
>>>>>> look
>>>>>> and let me know what you think.
>>>>>>
>>>>>> PR to Airflow repo incoming in a couple of hours :)
>>>>>>
>>>>>> *Why now*
>>>>>> With AIP-95 approved, Airflow now has a formal provider lifecycle:
>>>>>> incubation, production, mature, and deprecated. That opens the door
>>> for
>>>>>> accepting more community-built providers and giving them an
>> official
>>>>> home,
>>>>>> while setting clear expectations about maturity and support. But
>>>>> lifecycle
>>>>>> stages only work if users can actually see them.
>>>>>>
>>>>>> Right now, there's no place on airflow.apache.org where someone
>> can
>>>>> browse
>>>>>> providers, check their lifecycle stage, or discover what modules
>> they
>>>>> ship.
>>>>>> This registry fills that gap. It gives the PMC a tool to
>> communicate
>>>>>> provider maturity to users, and it gives the community an official
>>> way to
>>>>>> surface new providers -- clearly labelled with their lifecycle
>> stage.
>>>>>> *What it does*
>>>>>> The registry currently catalogs 99 providers and 1,648 modules
>>> across all
>>>>>> 11 module types (operators, hooks, sensors, triggers, transfers,
>>>>> executors,
>>>>>> notifiers, secret backends, logging handlers, Dag bundles, and
>>>>> decorators).
>>>>>> It's built with Eleventy <https://www.11ty.dev/> (thanks Ash, for
>>> the
>>>>>> suggestion and for prototyping an approach with it) and
>>> auto-generated
>>>>>> directly from the provider.yaml files in the repo -- no separate
>> data
>>>>>> pipeline, no manual curation. When a provider is added or updated,
>>> the
>>>>> next
>>>>>> CI build picks it up automatically.
>>>>>>
>>>>>> The entire registry is a static site (HTML, CSS, JS): no server, no
>>>>>> database, same deployment model as the existing Airflow docs. It's
>>>>>> generated at build time from the provider.yaml files and served
>> from
>>> S3
>>>>> via
>>>>>> CloudFront.
>>>>>>
>>>>>> *A few things you can do with it*:
>>>>>>
>>>>>> - Search across all providers and modules (Cmd+K, powered by
>>> Pagefind)
>>>>>> - Browse by category (Cloud, Databases, AI & ML, etc.)
>>>>>> - Filter/sort by lifecycle stage, downloads, module count
>>>>>> - Explore provider detail pages with per-version module
>> listings,
>>>>>> connection types, parameters, and install commands
>>>>>> - Access JSON API endpoints (/api/providers.json,
>>> /api/modules.json)
>>>>> for
>>>>>> programmatic access -- useful for AI agents and tooling
>>>>>>
>>>>>>
>>>>>> The design is deliberately discovery-first: it links out to the API
>>>>>> reference docs and user guides rather than hosting everything
>> itself.
>>>>> This
>>>>>> avoids duplicating content between provider docs and registry
>>> entries.
>>>>>> CI/CD is integrated with our existing docs pipeline and syncs to S3
>>>>>> automatically. Nothing in provider code, provider.yaml schemas,
>> core
>>>>>> Airflow, or the docs build is changed by this.
>>>>>>
>>>>>> *How it relates to the Astronomer Registry*
>>>>>> Many of you know the Astronomer Registry (
>>> https://registry.astronomer.io
>>>>> ),
>>>>>> which has been the go-to for discovering Airflow providers for
>>> years. Big
>>>>>> thanks to Astronomer and Josh Fell for building and maintaining it.
>>>>>>
>>>>>> This new registry is designed to be a community-owned successor on
>>>>>> airflow.apache.org, with the eventual goal of redirecting
>>>>>> registry.astronomer.io traffic here once it's stable.
>>>>>>
>>>>>>
>>>>>> *Remaining work*
>>>>>> Still to do after this lands:
>>>>>>
>>>>>> - apache/airflow-site PR for .htaccess rewrite and a "Registry"
>>> nav
>>>>> link
>>>>>> - Redirect registry.astronomer.io traffic once the official one
>>> is
>>>>>> stable
>>>>>> - A way to add third-party providers that are not in the Airflow
>>> repo,
>>>>>> like Great Expectations, Cosmos etc - I have a POC working on
>>> this.
>>>>>>
>>>>>> *Future ideas (will create GH issues)*
>>>>>>
>>>>>> - Explicit categories in provider.yaml (currently keyword-based
>>>>>> matching)
>>>>>> - LLM-friendly exports (llms.txt, "Copy for AI" buttons etc.)
>>>>>> - Example DAGs for each Provider.
>>>>>> - and many more – but I think the current state is valuable
>> enough
>>>>>> already
>>>>>>
>>>>>>
>>>>>> I'd appreciate feedback and reviews!
>>>>>>
>>>>>> Regards,
>>>>>> Kaxil
>>>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
>>> For additional commands, e-mail: dev-help@airflow.apache.org
>>>
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
For additional commands, e-mail: dev-help@airflow.apache.org
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Kaxil Naik <ka...@gmail.com>.
PR: https://github.com/apache/airflow/pull/62261
On Fri, 20 Feb 2026 at 21:50, Pavankumar Gopidesu <go...@gmail.com>
wrote:
> Woohoo, what an update , thanks kaxil, it's really nice :)
>
> On Fri, Feb 20, 2026 at 9:12 PM Vincent Beck <vi...@apache.org> wrote:
>
> > Very cool indeed :) I can see it very useful for users!
> >
> > On 2026/02/20 20:04:58 Jarek Potiuk wrote:
> > > Very cool :)
> > >
> > > J.
> > >
> > >
> > > On Fri, Feb 20, 2026 at 6:22 PM Zhe-You Liu <zh...@gmail.com>
> > wrote:
> > >
> > > > Hi Kaxil,
> > > >
> > > > I just checked out the staged site of the official Provider Registry,
> > and
> > > > it looks super impressive! The UX is excellent, and it’s definitely a
> > big
> > > > plus for users to find existing community integrations for their use
> > cases
> > > > and involve more Airflow users. I really appreciate your effort, and
> I
> > will
> > > > check out the PR when it comes out.
> > > >
> > > > Best regards,
> > > > Jason
> > > >
> > > > On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <ka...@gmail.com>
> > wrote:
> > > >
> > > > > Hey all,
> > > > >
> > > > > *tl;dr*: I'm proposing an official Provider Registry for the
> Airflow
> > > > > project, deployed at https://airflow.apache.org/registry/.
> > > > >
> > > > > Preview is up at https://airflow.staged.apache.org/registry/ --
> > take a
> > > > > look
> > > > > and let me know what you think.
> > > > >
> > > > > PR to Airflow repo incoming in a couple of hours :)
> > > > >
> > > > > *Why now*
> > > > > With AIP-95 approved, Airflow now has a formal provider lifecycle:
> > > > > incubation, production, mature, and deprecated. That opens the door
> > for
> > > > > accepting more community-built providers and giving them an
> official
> > > > home,
> > > > > while setting clear expectations about maturity and support. But
> > > > lifecycle
> > > > > stages only work if users can actually see them.
> > > > >
> > > > > Right now, there's no place on airflow.apache.org where someone
> can
> > > > browse
> > > > > providers, check their lifecycle stage, or discover what modules
> they
> > > > ship.
> > > > >
> > > > > This registry fills that gap. It gives the PMC a tool to
> communicate
> > > > > provider maturity to users, and it gives the community an official
> > way to
> > > > > surface new providers -- clearly labelled with their lifecycle
> stage.
> > > > >
> > > > > *What it does*
> > > > > The registry currently catalogs 99 providers and 1,648 modules
> > across all
> > > > > 11 module types (operators, hooks, sensors, triggers, transfers,
> > > > executors,
> > > > > notifiers, secret backends, logging handlers, Dag bundles, and
> > > > decorators).
> > > > >
> > > > > It's built with Eleventy <https://www.11ty.dev/> (thanks Ash, for
> > the
> > > > > suggestion and for prototyping an approach with it) and
> > auto-generated
> > > > > directly from the provider.yaml files in the repo -- no separate
> data
> > > > > pipeline, no manual curation. When a provider is added or updated,
> > the
> > > > next
> > > > > CI build picks it up automatically.
> > > > >
> > > > > The entire registry is a static site (HTML, CSS, JS): no server, no
> > > > > database, same deployment model as the existing Airflow docs. It's
> > > > > generated at build time from the provider.yaml files and served
> from
> > S3
> > > > via
> > > > > CloudFront.
> > > > >
> > > > > *A few things you can do with it*:
> > > > >
> > > > > - Search across all providers and modules (Cmd+K, powered by
> > Pagefind)
> > > > > - Browse by category (Cloud, Databases, AI & ML, etc.)
> > > > > - Filter/sort by lifecycle stage, downloads, module count
> > > > > - Explore provider detail pages with per-version module
> listings,
> > > > > connection types, parameters, and install commands
> > > > > - Access JSON API endpoints (/api/providers.json,
> > /api/modules.json)
> > > > for
> > > > > programmatic access -- useful for AI agents and tooling
> > > > >
> > > > >
> > > > > The design is deliberately discovery-first: it links out to the API
> > > > > reference docs and user guides rather than hosting everything
> itself.
> > > > This
> > > > > avoids duplicating content between provider docs and registry
> > entries.
> > > > >
> > > > > CI/CD is integrated with our existing docs pipeline and syncs to S3
> > > > > automatically. Nothing in provider code, provider.yaml schemas,
> core
> > > > > Airflow, or the docs build is changed by this.
> > > > >
> > > > > *How it relates to the Astronomer Registry*
> > > > > Many of you know the Astronomer Registry (
> > https://registry.astronomer.io
> > > > ),
> > > > > which has been the go-to for discovering Airflow providers for
> > years. Big
> > > > > thanks to Astronomer and Josh Fell for building and maintaining it.
> > > > >
> > > > > This new registry is designed to be a community-owned successor on
> > > > > airflow.apache.org, with the eventual goal of redirecting
> > > > > registry.astronomer.io traffic here once it's stable.
> > > > >
> > > > >
> > > > > *Remaining work*
> > > > > Still to do after this lands:
> > > > >
> > > > > - apache/airflow-site PR for .htaccess rewrite and a "Registry"
> > nav
> > > > link
> > > > > - Redirect registry.astronomer.io traffic once the official one
> > is
> > > > > stable
> > > > > - A way to add third-party providers that are not in the Airflow
> > repo,
> > > > > like Great Expectations, Cosmos etc - I have a POC working on
> > this.
> > > > >
> > > > >
> > > > > *Future ideas (will create GH issues)*
> > > > >
> > > > > - Explicit categories in provider.yaml (currently keyword-based
> > > > > matching)
> > > > > - LLM-friendly exports (llms.txt, "Copy for AI" buttons etc.)
> > > > > - Example DAGs for each Provider.
> > > > > - and many more – but I think the current state is valuable
> enough
> > > > > already
> > > > >
> > > > >
> > > > > I'd appreciate feedback and reviews!
> > > > >
> > > > > Regards,
> > > > > Kaxil
> > > > >
> > > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > For additional commands, e-mail: dev-help@airflow.apache.org
> >
> >
>
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Pavankumar Gopidesu <go...@gmail.com>.
Woohoo, what an update , thanks kaxil, it's really nice :)
On Fri, Feb 20, 2026 at 9:12 PM Vincent Beck <vi...@apache.org> wrote:
> Very cool indeed :) I can see it very useful for users!
>
> On 2026/02/20 20:04:58 Jarek Potiuk wrote:
> > Very cool :)
> >
> > J.
> >
> >
> > On Fri, Feb 20, 2026 at 6:22 PM Zhe-You Liu <zh...@gmail.com>
> wrote:
> >
> > > Hi Kaxil,
> > >
> > > I just checked out the staged site of the official Provider Registry,
> and
> > > it looks super impressive! The UX is excellent, and it’s definitely a
> big
> > > plus for users to find existing community integrations for their use
> cases
> > > and involve more Airflow users. I really appreciate your effort, and I
> will
> > > check out the PR when it comes out.
> > >
> > > Best regards,
> > > Jason
> > >
> > > On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <ka...@gmail.com>
> wrote:
> > >
> > > > Hey all,
> > > >
> > > > *tl;dr*: I'm proposing an official Provider Registry for the Airflow
> > > > project, deployed at https://airflow.apache.org/registry/.
> > > >
> > > > Preview is up at https://airflow.staged.apache.org/registry/ --
> take a
> > > > look
> > > > and let me know what you think.
> > > >
> > > > PR to Airflow repo incoming in a couple of hours :)
> > > >
> > > > *Why now*
> > > > With AIP-95 approved, Airflow now has a formal provider lifecycle:
> > > > incubation, production, mature, and deprecated. That opens the door
> for
> > > > accepting more community-built providers and giving them an official
> > > home,
> > > > while setting clear expectations about maturity and support. But
> > > lifecycle
> > > > stages only work if users can actually see them.
> > > >
> > > > Right now, there's no place on airflow.apache.org where someone can
> > > browse
> > > > providers, check their lifecycle stage, or discover what modules they
> > > ship.
> > > >
> > > > This registry fills that gap. It gives the PMC a tool to communicate
> > > > provider maturity to users, and it gives the community an official
> way to
> > > > surface new providers -- clearly labelled with their lifecycle stage.
> > > >
> > > > *What it does*
> > > > The registry currently catalogs 99 providers and 1,648 modules
> across all
> > > > 11 module types (operators, hooks, sensors, triggers, transfers,
> > > executors,
> > > > notifiers, secret backends, logging handlers, Dag bundles, and
> > > decorators).
> > > >
> > > > It's built with Eleventy <https://www.11ty.dev/> (thanks Ash, for
> the
> > > > suggestion and for prototyping an approach with it) and
> auto-generated
> > > > directly from the provider.yaml files in the repo -- no separate data
> > > > pipeline, no manual curation. When a provider is added or updated,
> the
> > > next
> > > > CI build picks it up automatically.
> > > >
> > > > The entire registry is a static site (HTML, CSS, JS): no server, no
> > > > database, same deployment model as the existing Airflow docs. It's
> > > > generated at build time from the provider.yaml files and served from
> S3
> > > via
> > > > CloudFront.
> > > >
> > > > *A few things you can do with it*:
> > > >
> > > > - Search across all providers and modules (Cmd+K, powered by
> Pagefind)
> > > > - Browse by category (Cloud, Databases, AI & ML, etc.)
> > > > - Filter/sort by lifecycle stage, downloads, module count
> > > > - Explore provider detail pages with per-version module listings,
> > > > connection types, parameters, and install commands
> > > > - Access JSON API endpoints (/api/providers.json,
> /api/modules.json)
> > > for
> > > > programmatic access -- useful for AI agents and tooling
> > > >
> > > >
> > > > The design is deliberately discovery-first: it links out to the API
> > > > reference docs and user guides rather than hosting everything itself.
> > > This
> > > > avoids duplicating content between provider docs and registry
> entries.
> > > >
> > > > CI/CD is integrated with our existing docs pipeline and syncs to S3
> > > > automatically. Nothing in provider code, provider.yaml schemas, core
> > > > Airflow, or the docs build is changed by this.
> > > >
> > > > *How it relates to the Astronomer Registry*
> > > > Many of you know the Astronomer Registry (
> https://registry.astronomer.io
> > > ),
> > > > which has been the go-to for discovering Airflow providers for
> years. Big
> > > > thanks to Astronomer and Josh Fell for building and maintaining it.
> > > >
> > > > This new registry is designed to be a community-owned successor on
> > > > airflow.apache.org, with the eventual goal of redirecting
> > > > registry.astronomer.io traffic here once it's stable.
> > > >
> > > >
> > > > *Remaining work*
> > > > Still to do after this lands:
> > > >
> > > > - apache/airflow-site PR for .htaccess rewrite and a "Registry"
> nav
> > > link
> > > > - Redirect registry.astronomer.io traffic once the official one
> is
> > > > stable
> > > > - A way to add third-party providers that are not in the Airflow
> repo,
> > > > like Great Expectations, Cosmos etc - I have a POC working on
> this.
> > > >
> > > >
> > > > *Future ideas (will create GH issues)*
> > > >
> > > > - Explicit categories in provider.yaml (currently keyword-based
> > > > matching)
> > > > - LLM-friendly exports (llms.txt, "Copy for AI" buttons etc.)
> > > > - Example DAGs for each Provider.
> > > > - and many more – but I think the current state is valuable enough
> > > > already
> > > >
> > > >
> > > > I'd appreciate feedback and reviews!
> > > >
> > > > Regards,
> > > > Kaxil
> > > >
> > >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> For additional commands, e-mail: dev-help@airflow.apache.org
>
>
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Vincent Beck <vi...@apache.org>.
Very cool indeed :) I can see it very useful for users!
On 2026/02/20 20:04:58 Jarek Potiuk wrote:
> Very cool :)
>
> J.
>
>
> On Fri, Feb 20, 2026 at 6:22 PM Zhe-You Liu <zh...@gmail.com> wrote:
>
> > Hi Kaxil,
> >
> > I just checked out the staged site of the official Provider Registry, and
> > it looks super impressive! The UX is excellent, and it’s definitely a big
> > plus for users to find existing community integrations for their use cases
> > and involve more Airflow users. I really appreciate your effort, and I will
> > check out the PR when it comes out.
> >
> > Best regards,
> > Jason
> >
> > On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <ka...@gmail.com> wrote:
> >
> > > Hey all,
> > >
> > > *tl;dr*: I'm proposing an official Provider Registry for the Airflow
> > > project, deployed at https://airflow.apache.org/registry/.
> > >
> > > Preview is up at https://airflow.staged.apache.org/registry/ -- take a
> > > look
> > > and let me know what you think.
> > >
> > > PR to Airflow repo incoming in a couple of hours :)
> > >
> > > *Why now*
> > > With AIP-95 approved, Airflow now has a formal provider lifecycle:
> > > incubation, production, mature, and deprecated. That opens the door for
> > > accepting more community-built providers and giving them an official
> > home,
> > > while setting clear expectations about maturity and support. But
> > lifecycle
> > > stages only work if users can actually see them.
> > >
> > > Right now, there's no place on airflow.apache.org where someone can
> > browse
> > > providers, check their lifecycle stage, or discover what modules they
> > ship.
> > >
> > > This registry fills that gap. It gives the PMC a tool to communicate
> > > provider maturity to users, and it gives the community an official way to
> > > surface new providers -- clearly labelled with their lifecycle stage.
> > >
> > > *What it does*
> > > The registry currently catalogs 99 providers and 1,648 modules across all
> > > 11 module types (operators, hooks, sensors, triggers, transfers,
> > executors,
> > > notifiers, secret backends, logging handlers, Dag bundles, and
> > decorators).
> > >
> > > It's built with Eleventy <https://www.11ty.dev/> (thanks Ash, for the
> > > suggestion and for prototyping an approach with it) and auto-generated
> > > directly from the provider.yaml files in the repo -- no separate data
> > > pipeline, no manual curation. When a provider is added or updated, the
> > next
> > > CI build picks it up automatically.
> > >
> > > The entire registry is a static site (HTML, CSS, JS): no server, no
> > > database, same deployment model as the existing Airflow docs. It's
> > > generated at build time from the provider.yaml files and served from S3
> > via
> > > CloudFront.
> > >
> > > *A few things you can do with it*:
> > >
> > > - Search across all providers and modules (Cmd+K, powered by Pagefind)
> > > - Browse by category (Cloud, Databases, AI & ML, etc.)
> > > - Filter/sort by lifecycle stage, downloads, module count
> > > - Explore provider detail pages with per-version module listings,
> > > connection types, parameters, and install commands
> > > - Access JSON API endpoints (/api/providers.json, /api/modules.json)
> > for
> > > programmatic access -- useful for AI agents and tooling
> > >
> > >
> > > The design is deliberately discovery-first: it links out to the API
> > > reference docs and user guides rather than hosting everything itself.
> > This
> > > avoids duplicating content between provider docs and registry entries.
> > >
> > > CI/CD is integrated with our existing docs pipeline and syncs to S3
> > > automatically. Nothing in provider code, provider.yaml schemas, core
> > > Airflow, or the docs build is changed by this.
> > >
> > > *How it relates to the Astronomer Registry*
> > > Many of you know the Astronomer Registry (https://registry.astronomer.io
> > ),
> > > which has been the go-to for discovering Airflow providers for years. Big
> > > thanks to Astronomer and Josh Fell for building and maintaining it.
> > >
> > > This new registry is designed to be a community-owned successor on
> > > airflow.apache.org, with the eventual goal of redirecting
> > > registry.astronomer.io traffic here once it's stable.
> > >
> > >
> > > *Remaining work*
> > > Still to do after this lands:
> > >
> > > - apache/airflow-site PR for .htaccess rewrite and a "Registry" nav
> > link
> > > - Redirect registry.astronomer.io traffic once the official one is
> > > stable
> > > - A way to add third-party providers that are not in the Airflow repo,
> > > like Great Expectations, Cosmos etc - I have a POC working on this.
> > >
> > >
> > > *Future ideas (will create GH issues)*
> > >
> > > - Explicit categories in provider.yaml (currently keyword-based
> > > matching)
> > > - LLM-friendly exports (llms.txt, "Copy for AI" buttons etc.)
> > > - Example DAGs for each Provider.
> > > - and many more – but I think the current state is valuable enough
> > > already
> > >
> > >
> > > I'd appreciate feedback and reviews!
> > >
> > > Regards,
> > > Kaxil
> > >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
For additional commands, e-mail: dev-help@airflow.apache.org
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Jarek Potiuk <ja...@potiuk.com>.
Very cool :)
J.
On Fri, Feb 20, 2026 at 6:22 PM Zhe-You Liu <zh...@gmail.com> wrote:
> Hi Kaxil,
>
> I just checked out the staged site of the official Provider Registry, and
> it looks super impressive! The UX is excellent, and it’s definitely a big
> plus for users to find existing community integrations for their use cases
> and involve more Airflow users. I really appreciate your effort, and I will
> check out the PR when it comes out.
>
> Best regards,
> Jason
>
> On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <ka...@gmail.com> wrote:
>
> > Hey all,
> >
> > *tl;dr*: I'm proposing an official Provider Registry for the Airflow
> > project, deployed at https://airflow.apache.org/registry/.
> >
> > Preview is up at https://airflow.staged.apache.org/registry/ -- take a
> > look
> > and let me know what you think.
> >
> > PR to Airflow repo incoming in a couple of hours :)
> >
> > *Why now*
> > With AIP-95 approved, Airflow now has a formal provider lifecycle:
> > incubation, production, mature, and deprecated. That opens the door for
> > accepting more community-built providers and giving them an official
> home,
> > while setting clear expectations about maturity and support. But
> lifecycle
> > stages only work if users can actually see them.
> >
> > Right now, there's no place on airflow.apache.org where someone can
> browse
> > providers, check their lifecycle stage, or discover what modules they
> ship.
> >
> > This registry fills that gap. It gives the PMC a tool to communicate
> > provider maturity to users, and it gives the community an official way to
> > surface new providers -- clearly labelled with their lifecycle stage.
> >
> > *What it does*
> > The registry currently catalogs 99 providers and 1,648 modules across all
> > 11 module types (operators, hooks, sensors, triggers, transfers,
> executors,
> > notifiers, secret backends, logging handlers, Dag bundles, and
> decorators).
> >
> > It's built with Eleventy <https://www.11ty.dev/> (thanks Ash, for the
> > suggestion and for prototyping an approach with it) and auto-generated
> > directly from the provider.yaml files in the repo -- no separate data
> > pipeline, no manual curation. When a provider is added or updated, the
> next
> > CI build picks it up automatically.
> >
> > The entire registry is a static site (HTML, CSS, JS): no server, no
> > database, same deployment model as the existing Airflow docs. It's
> > generated at build time from the provider.yaml files and served from S3
> via
> > CloudFront.
> >
> > *A few things you can do with it*:
> >
> > - Search across all providers and modules (Cmd+K, powered by Pagefind)
> > - Browse by category (Cloud, Databases, AI & ML, etc.)
> > - Filter/sort by lifecycle stage, downloads, module count
> > - Explore provider detail pages with per-version module listings,
> > connection types, parameters, and install commands
> > - Access JSON API endpoints (/api/providers.json, /api/modules.json)
> for
> > programmatic access -- useful for AI agents and tooling
> >
> >
> > The design is deliberately discovery-first: it links out to the API
> > reference docs and user guides rather than hosting everything itself.
> This
> > avoids duplicating content between provider docs and registry entries.
> >
> > CI/CD is integrated with our existing docs pipeline and syncs to S3
> > automatically. Nothing in provider code, provider.yaml schemas, core
> > Airflow, or the docs build is changed by this.
> >
> > *How it relates to the Astronomer Registry*
> > Many of you know the Astronomer Registry (https://registry.astronomer.io
> ),
> > which has been the go-to for discovering Airflow providers for years. Big
> > thanks to Astronomer and Josh Fell for building and maintaining it.
> >
> > This new registry is designed to be a community-owned successor on
> > airflow.apache.org, with the eventual goal of redirecting
> > registry.astronomer.io traffic here once it's stable.
> >
> >
> > *Remaining work*
> > Still to do after this lands:
> >
> > - apache/airflow-site PR for .htaccess rewrite and a "Registry" nav
> link
> > - Redirect registry.astronomer.io traffic once the official one is
> > stable
> > - A way to add third-party providers that are not in the Airflow repo,
> > like Great Expectations, Cosmos etc - I have a POC working on this.
> >
> >
> > *Future ideas (will create GH issues)*
> >
> > - Explicit categories in provider.yaml (currently keyword-based
> > matching)
> > - LLM-friendly exports (llms.txt, "Copy for AI" buttons etc.)
> > - Example DAGs for each Provider.
> > - and many more – but I think the current state is valuable enough
> > already
> >
> >
> > I'd appreciate feedback and reviews!
> >
> > Regards,
> > Kaxil
> >
>
Re: [DISCUSS] Apache Airflow Provider Registry: a searchable catalog of all providers and modules
Posted by Zhe-You Liu <zh...@gmail.com>.
Hi Kaxil,
I just checked out the staged site of the official Provider Registry, and
it looks super impressive! The UX is excellent, and it’s definitely a big
plus for users to find existing community integrations for their use cases
and involve more Airflow users. I really appreciate your effort, and I will
check out the PR when it comes out.
Best regards,
Jason
On Sat, Feb 21, 2026 at 1:09 AM Kaxil Naik <ka...@gmail.com> wrote:
> Hey all,
>
> *tl;dr*: I'm proposing an official Provider Registry for the Airflow
> project, deployed at https://airflow.apache.org/registry/.
>
> Preview is up at https://airflow.staged.apache.org/registry/ -- take a
> look
> and let me know what you think.
>
> PR to Airflow repo incoming in a couple of hours :)
>
> *Why now*
> With AIP-95 approved, Airflow now has a formal provider lifecycle:
> incubation, production, mature, and deprecated. That opens the door for
> accepting more community-built providers and giving them an official home,
> while setting clear expectations about maturity and support. But lifecycle
> stages only work if users can actually see them.
>
> Right now, there's no place on airflow.apache.org where someone can browse
> providers, check their lifecycle stage, or discover what modules they ship.
>
> This registry fills that gap. It gives the PMC a tool to communicate
> provider maturity to users, and it gives the community an official way to
> surface new providers -- clearly labelled with their lifecycle stage.
>
> *What it does*
> The registry currently catalogs 99 providers and 1,648 modules across all
> 11 module types (operators, hooks, sensors, triggers, transfers, executors,
> notifiers, secret backends, logging handlers, Dag bundles, and decorators).
>
> It's built with Eleventy <https://www.11ty.dev/> (thanks Ash, for the
> suggestion and for prototyping an approach with it) and auto-generated
> directly from the provider.yaml files in the repo -- no separate data
> pipeline, no manual curation. When a provider is added or updated, the next
> CI build picks it up automatically.
>
> The entire registry is a static site (HTML, CSS, JS): no server, no
> database, same deployment model as the existing Airflow docs. It's
> generated at build time from the provider.yaml files and served from S3 via
> CloudFront.
>
> *A few things you can do with it*:
>
> - Search across all providers and modules (Cmd+K, powered by Pagefind)
> - Browse by category (Cloud, Databases, AI & ML, etc.)
> - Filter/sort by lifecycle stage, downloads, module count
> - Explore provider detail pages with per-version module listings,
> connection types, parameters, and install commands
> - Access JSON API endpoints (/api/providers.json, /api/modules.json) for
> programmatic access -- useful for AI agents and tooling
>
>
> The design is deliberately discovery-first: it links out to the API
> reference docs and user guides rather than hosting everything itself. This
> avoids duplicating content between provider docs and registry entries.
>
> CI/CD is integrated with our existing docs pipeline and syncs to S3
> automatically. Nothing in provider code, provider.yaml schemas, core
> Airflow, or the docs build is changed by this.
>
> *How it relates to the Astronomer Registry*
> Many of you know the Astronomer Registry (https://registry.astronomer.io),
> which has been the go-to for discovering Airflow providers for years. Big
> thanks to Astronomer and Josh Fell for building and maintaining it.
>
> This new registry is designed to be a community-owned successor on
> airflow.apache.org, with the eventual goal of redirecting
> registry.astronomer.io traffic here once it's stable.
>
>
> *Remaining work*
> Still to do after this lands:
>
> - apache/airflow-site PR for .htaccess rewrite and a "Registry" nav link
> - Redirect registry.astronomer.io traffic once the official one is
> stable
> - A way to add third-party providers that are not in the Airflow repo,
> like Great Expectations, Cosmos etc - I have a POC working on this.
>
>
> *Future ideas (will create GH issues)*
>
> - Explicit categories in provider.yaml (currently keyword-based
> matching)
> - LLM-friendly exports (llms.txt, "Copy for AI" buttons etc.)
> - Example DAGs for each Provider.
> - and many more – but I think the current state is valuable enough
> already
>
>
> I'd appreciate feedback and reviews!
>
> Regards,
> Kaxil
>