You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Jarek Potiuk <ja...@potiuk.com> on 2025/10/30 10:41:45 UTC
[ANNOUNCE] Consistent and "More ASF compliant" https://airflow.apache.org (and yet another "helpful AI/LLM" case
Hello everyone,
*TL;DR; Just wanted to make you aware that we have now some "refresh" of
the https://airflow.apache.org <https://airflow.apache.org> to make it more
in-line with the policies and expectations of the ASF*
If you want to read more about it and some details - feel free to dive
deeper:
# Changes deployed recently
* Important change ! We only show the latest patchlevel version of airflow,
providers, task0-sdk, helm-chart - you can still get to any version with
version in URL or when you have links to old documentation.
Version selector is **finally** usable.
* We are linking directly to the Privacy Policy of the ASF from the footer
(as required by the ASF)
* We are linking to "https://events.apache.org" - also as expected by the
ASF policies
* Our footers are now (hopefully) fully consistent across all 250K (!!!)
pages we have - including all versions of packages
* We are using CSP builder for Content-Security-Policy
https://infra.apache.org/tools/csp.html - which is a new way how we can
embed external sources (youtube videos) - I am still in the process of
verifying if particular use of it in 'Meetups" page is OK With ASF privacy
policy, but I think we are pretty close if not "perfect" when it comes to
compliance with ASF expectations.
* We will also have Airflow CTL documentation linked once we release the
1.0.0 version (the voting is in progress)
# Comment on doc infra tooling
Just a comment - that was all rather easy with the "infrastructure" we
worked on with Pavan mostly - and it has proven to be working really well.
We have a Git-archive where we can manipulate all historical versions where
we can manipulate all the "versioned" docs, we have workflows in CI that
sync changes back/forth and finally I could close the issue
https://github.com/apache/airflow/issues/44373 - which has 17 subtasks
completed since 26 th Novemeber last year where I raised the "we need to
finally modernise our docs infra".
Now I think we are fully ready to support anyone who would like to take any
steps with design changes or structure changes to the docs of ours - it
should be easier than ever and we have the right tooling in place to make
even very complex changes. We have fully automated workflows that are used
by release managers, including staging site generation for RC candidates,
automated cache invalidation when needed etc. etc. If anyone would like to
take a stab at restructuring some of our docs, improving the content,
navigation etc, we are ready to support that rather easily from the
"infrastructure" side - including testing things on staging end-2-end
before releasing it to the public.
This is thanks to many people - Pavan, Amogh, Shahar, Ankit, Ash, Kaxil.
myself - and likely a number of other people with smaller changes - spanned
over quite a long time - but done in parallel to all our Airflow 3.0 and
3.1 efforts without disrupting those. You can see the result of it in our
documentation building architecture
https://github.com/apache/airflow/tree/main/docs
# LLM support:
Also another shout-out to AI and LLMS helping in such efforts. For the big
"consistency" change I used (while in train to Vienna) GH Copilot + Claude
Sonet 4.5 to make the bulk changes on all historical variants of the
footers we had for 250K files. It turned out to be really helpful when
asked to synchronize all the footers - it took almost "0" effort from my
side.
I literally just overlooked what it was doing when it was plowing through
250K pages to fix after getting the right instructions (the AI even wrote a
temporary python script that I reviewed to apply the fix that it deleted
afterwards).
Copilot / Claude even used interesting ways of verifying if there is more
work to do, It did not only run all the changes but it actually attempted
to find and fix all other similar cases - all on its own - including
various ways of checking it for 250K files - like getting some subselection
of files to look at in various sequences an random sampling of pages.
Pretty neat when you consider that it took me just a few minutes to write
(long and comprehensive though) prompt/instructions.
I hope it worked well ;) but from what I checked, it did. The diff of the
change is so huge that it does not display nicely in GitHub but if you are
courious -
https://github.com/apache/airflow-site-archive/commit/64ff61087317802f76cbb4e3f44789a557d5e331
J.
Re: [ANNOUNCE] Consistent and "More ASF compliant" https://airflow.apache.org (and yet another "helpful AI/LLM" case
Posted by Jarek Potiuk <ja...@potiuk.com>.
https://whimsy.apache.org/site.cgi/project/airflow -> CSP check is "green"
now :). Just waiting for the events fix :)
[image: Screenshot 2025-11-02 at 11.40.01.png]
J.
On Sun, Nov 2, 2025 at 10:49 AM Jarek Potiuk <ja...@potiuk.com> wrote:
> I solved one **LAST** issue that I discussed at users@infra.apache.org
> and I finally removed all the CSP altogether - our "meetup" page had the
> youtube videos embedded and they were not entirely compliant - thanks to
> helpful comments by Arnout (thanks!) here
> https://lists.apache.org/thread/l1c84nryh8lywj1fr8s435qb24dlrgot, I
> applied the same approach we had on the "use cases" page - with embedded
> youtube photos and "click to play" links, so we should be finally
> compliant.
>
> You can see the commit here:
> https://github.com/apache/airflow-site/commit/0b83ae1fba4af3bebf4efeceddaf8880449f9842
>
> There is still a small issue that our website is still reported as **not**
> having the right link to events (which we do)
> https://whimsy.apache.org/site/project/airflow (the CSP at the bottom is
> already solved, just the site needs to be crawled) - but there is an open
> issue for it https://github.com/apache/whimsy/issues/280 and
> discussion with marketing / VP Conferences team that Sebb leads - hopefully
> all the questions will be sorted out and whimsy team will make a proper fix
> to make our website status "green"
>
> J.
>
>
> On Thu, Oct 30, 2025 at 6:43 PM Damian Shaw <ds...@striketechnologies.com>
> wrote:
>
>> Great news on the version selector! That has been a long annoyance of
>> mine that I can't select the version of Airflow my team currently uses!
>>
>> Damian
>>
>> -----Original Message-----
>> From: Ferruzzi, Dennis <fe...@amazon.com>
>> Sent: Thursday, October 30, 2025 1:11 PM
>> To: dev@airflow.apache.org
>> Cc: sebb AT ASF <se...@apache.org>
>> Subject: Re: [ANNOUNCE] Consistent and "More ASF compliant"
>> https://airflow.apache.org (and yet another "helpful AI/LLM" case
>>
>> It has been a while since I've been to our main page, I usually jump
>> right to the docs page or use a bookmark to go to specific pages. It's
>> looking good. Thanks for the effort!
>>
>>
>> - ferruzzi
>>
>>
>> ________________________________
>> From: Jarek Potiuk <ja...@potiuk.com>
>> Sent: Thursday, October 30, 2025 3:41 AM
>> To: dev@airflow.apache.org
>> Cc: sebb AT ASF
>> Subject: [EXT] [ANNOUNCE] Consistent and "More ASF compliant"
>> https://airflow.apache.org (and yet another "helpful AI/LLM" case
>>
>> CAUTION: This email originated from outside of the organization. Do not
>> click links or open attachments unless you can confirm the sender and know
>> the content is safe.
>>
>>
>>
>> AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe.
>> Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez
>> pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que
>> le contenu ne présente aucun risque.
>>
>>
>>
>> Hello everyone,
>>
>> *TL;DR; Just wanted to make you aware that we have now some "refresh" of
>> the https://airflow.apache.org <https://airflow.apache.org> to make it
>> more in-line with the policies and expectations of the ASF*
>>
>> If you want to read more about it and some details - feel free to dive
>> deeper:
>>
>> # Changes deployed recently
>>
>> * Important change ! We only show the latest patchlevel version of
>> airflow, providers, task0-sdk, helm-chart - you can still get to any
>> version with version in URL or when you have links to old documentation.
>> Version selector is **finally** usable.
>>
>> * We are linking directly to the Privacy Policy of the ASF from the
>> footer (as required by the ASF)
>>
>> * We are linking to "https://events.apache.org" - also as expected by
>> the ASF policies
>>
>> * Our footers are now (hopefully) fully consistent across all 250K (!!!)
>> pages we have - including all versions of packages
>>
>> * We are using CSP builder for Content-Security-Policy
>> https://infra.apache.org/tools/csp.html - which is a new way how we can
>> embed external sources (youtube videos) - I am still in the process of
>> verifying if particular use of it in 'Meetups" page is OK With ASF privacy
>> policy, but I think we are pretty close if not "perfect" when it comes to
>> compliance with ASF expectations.
>>
>> * We will also have Airflow CTL documentation linked once we release the
>> 1.0.0 version (the voting is in progress)
>>
>> # Comment on doc infra tooling
>>
>> Just a comment - that was all rather easy with the "infrastructure" we
>> worked on with Pavan mostly - and it has proven to be working really well.
>> We have a Git-archive where we can manipulate all historical versions
>> where we can manipulate all the "versioned" docs, we have workflows in CI
>> that sync changes back/forth and finally I could close the issue
>> https://github.com/apache/airflow/issues/44373 - which has 17 subtasks
>> completed since 26 th Novemeber last year where I raised the "we need to
>> finally modernise our docs infra".
>>
>> Now I think we are fully ready to support anyone who would like to take
>> any steps with design changes or structure changes to the docs of ours - it
>> should be easier than ever and we have the right tooling in place to make
>> even very complex changes. We have fully automated workflows that are used
>> by release managers, including staging site generation for RC candidates,
>> automated cache invalidation when needed etc. etc. If anyone would like to
>> take a stab at restructuring some of our docs, improving the content,
>> navigation etc, we are ready to support that rather easily from the
>> "infrastructure" side - including testing things on staging end-2-end
>> before releasing it to the public.
>>
>> This is thanks to many people - Pavan, Amogh, Shahar, Ankit, Ash, Kaxil.
>> myself - and likely a number of other people with smaller changes -
>> spanned over quite a long time - but done in parallel to all our Airflow
>> 3.0 and
>> 3.1 efforts without disrupting those. You can see the result of it in our
>> documentation building architecture
>> https://github.com/apache/airflow/tree/main/docs
>>
>> # LLM support:
>>
>> Also another shout-out to AI and LLMS helping in such efforts. For the
>> big "consistency" change I used (while in train to Vienna) GH Copilot +
>> Claude Sonet 4.5 to make the bulk changes on all historical variants of the
>> footers we had for 250K files. It turned out to be really helpful when
>> asked to synchronize all the footers - it took almost "0" effort from my
>> side.
>>
>> I literally just overlooked what it was doing when it was plowing through
>> 250K pages to fix after getting the right instructions (the AI even wrote a
>> temporary python script that I reviewed to apply the fix that it deleted
>> afterwards).
>>
>> Copilot / Claude even used interesting ways of verifying if there is more
>> work to do, It did not only run all the changes but it actually attempted
>> to find and fix all other similar cases - all on its own - including
>> various ways of checking it for 250K files - like getting some subselection
>> of files to look at in various sequences an random sampling of pages.
>> Pretty neat when you consider that it took me just a few minutes to write
>> (long and comprehensive though) prompt/instructions.
>>
>> I hope it worked well ;) but from what I checked, it did. The diff of the
>> change is so huge that it does not display nicely in GitHub but if you are
>> courious -
>>
>> https://github.com/apache/airflow-site-archive/commit/64ff61087317802f76cbb4e3f44789a557d5e331
>>
>>
>> J.
>> ________________________________
>> Strike Technologies, LLC (“Strike”) is part of the GTS family of
>> companies. Strike is a technology solutions provider, and is not a broker
>> or dealer and does not transact any securities related business directly
>> whatsoever. This communication is the property of Strike and its
>> affiliates, and does not constitute an offer to sell or the solicitation of
>> an offer to buy any security in any jurisdiction. It is intended only for
>> the person to whom it is addressed and may contain information that is
>> privileged, confidential, or otherwise protected from disclosure.
>> Distribution or copying of this communication, or the information contained
>> herein, by anyone other than the intended recipient is prohibited. If you
>> have received this communication in error, please immediately notify Strike
>> at info@striketechnologies.com, and delete and destroy any copies hereof.
>> ________________________________
>>
>> CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments
>> are intended solely for the addressee. This transmission is covered by the
>> Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The
>> information contained in this transmission is confidential in nature and
>> protected from further use or disclosure under U.S. Pub. L. 106-102, 113
>> U.S. Stat. 1338 (1999), and may be subject to attorney-client or other
>> legal privilege. Your use or disclosure of this information for any purpose
>> other than that intended by its transmittal is strictly prohibited, and may
>> subject you to fines and/or penalties under federal and state law. If you
>> are not the intended recipient of this transmission, please DESTROY ALL
>> COPIES RECEIVED and confirm destruction to the sender via return
>> transmittal.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
>> For additional commands, e-mail: dev-help@airflow.apache.org
>>
>
Re: [ANNOUNCE] Consistent and "More ASF compliant" https://airflow.apache.org (and yet another "helpful AI/LLM" case
Posted by Jarek Potiuk <ja...@potiuk.com>.
I solved one **LAST** issue that I discussed at users@infra.apache.org and
I finally removed all the CSP altogether - our "meetup" page had the
youtube videos embedded and they were not entirely compliant - thanks to
helpful comments by Arnout (thanks!) here
https://lists.apache.org/thread/l1c84nryh8lywj1fr8s435qb24dlrgot, I applied
the same approach we had on the "use cases" page - with embedded youtube
photos and "click to play" links, so we should be finally compliant.
You can see the commit here:
https://github.com/apache/airflow-site/commit/0b83ae1fba4af3bebf4efeceddaf8880449f9842
There is still a small issue that our website is still reported as **not**
having the right link to events (which we do)
https://whimsy.apache.org/site/project/airflow (the CSP at the bottom is
already solved, just the site needs to be crawled) - but there is an open
issue for it https://github.com/apache/whimsy/issues/280 and
discussion with marketing / VP Conferences team that Sebb leads - hopefully
all the questions will be sorted out and whimsy team will make a proper fix
to make our website status "green"
J.
On Thu, Oct 30, 2025 at 6:43 PM Damian Shaw <ds...@striketechnologies.com>
wrote:
> Great news on the version selector! That has been a long annoyance of mine
> that I can't select the version of Airflow my team currently uses!
>
> Damian
>
> -----Original Message-----
> From: Ferruzzi, Dennis <fe...@amazon.com>
> Sent: Thursday, October 30, 2025 1:11 PM
> To: dev@airflow.apache.org
> Cc: sebb AT ASF <se...@apache.org>
> Subject: Re: [ANNOUNCE] Consistent and "More ASF compliant"
> https://airflow.apache.org (and yet another "helpful AI/LLM" case
>
> It has been a while since I've been to our main page, I usually jump right
> to the docs page or use a bookmark to go to specific pages. It's looking
> good. Thanks for the effort!
>
>
> - ferruzzi
>
>
> ________________________________
> From: Jarek Potiuk <ja...@potiuk.com>
> Sent: Thursday, October 30, 2025 3:41 AM
> To: dev@airflow.apache.org
> Cc: sebb AT ASF
> Subject: [EXT] [ANNOUNCE] Consistent and "More ASF compliant"
> https://airflow.apache.org (and yet another "helpful AI/LLM" case
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
>
>
>
> AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe.
> Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez
> pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que
> le contenu ne présente aucun risque.
>
>
>
> Hello everyone,
>
> *TL;DR; Just wanted to make you aware that we have now some "refresh" of
> the https://airflow.apache.org <https://airflow.apache.org> to make it
> more in-line with the policies and expectations of the ASF*
>
> If you want to read more about it and some details - feel free to dive
> deeper:
>
> # Changes deployed recently
>
> * Important change ! We only show the latest patchlevel version of
> airflow, providers, task0-sdk, helm-chart - you can still get to any
> version with version in URL or when you have links to old documentation.
> Version selector is **finally** usable.
>
> * We are linking directly to the Privacy Policy of the ASF from the footer
> (as required by the ASF)
>
> * We are linking to "https://events.apache.org" - also as expected by the
> ASF policies
>
> * Our footers are now (hopefully) fully consistent across all 250K (!!!)
> pages we have - including all versions of packages
>
> * We are using CSP builder for Content-Security-Policy
> https://infra.apache.org/tools/csp.html - which is a new way how we can
> embed external sources (youtube videos) - I am still in the process of
> verifying if particular use of it in 'Meetups" page is OK With ASF privacy
> policy, but I think we are pretty close if not "perfect" when it comes to
> compliance with ASF expectations.
>
> * We will also have Airflow CTL documentation linked once we release the
> 1.0.0 version (the voting is in progress)
>
> # Comment on doc infra tooling
>
> Just a comment - that was all rather easy with the "infrastructure" we
> worked on with Pavan mostly - and it has proven to be working really well.
> We have a Git-archive where we can manipulate all historical versions
> where we can manipulate all the "versioned" docs, we have workflows in CI
> that sync changes back/forth and finally I could close the issue
> https://github.com/apache/airflow/issues/44373 - which has 17 subtasks
> completed since 26 th Novemeber last year where I raised the "we need to
> finally modernise our docs infra".
>
> Now I think we are fully ready to support anyone who would like to take
> any steps with design changes or structure changes to the docs of ours - it
> should be easier than ever and we have the right tooling in place to make
> even very complex changes. We have fully automated workflows that are used
> by release managers, including staging site generation for RC candidates,
> automated cache invalidation when needed etc. etc. If anyone would like to
> take a stab at restructuring some of our docs, improving the content,
> navigation etc, we are ready to support that rather easily from the
> "infrastructure" side - including testing things on staging end-2-end
> before releasing it to the public.
>
> This is thanks to many people - Pavan, Amogh, Shahar, Ankit, Ash, Kaxil.
> myself - and likely a number of other people with smaller changes -
> spanned over quite a long time - but done in parallel to all our Airflow
> 3.0 and
> 3.1 efforts without disrupting those. You can see the result of it in our
> documentation building architecture
> https://github.com/apache/airflow/tree/main/docs
>
> # LLM support:
>
> Also another shout-out to AI and LLMS helping in such efforts. For the big
> "consistency" change I used (while in train to Vienna) GH Copilot + Claude
> Sonet 4.5 to make the bulk changes on all historical variants of the
> footers we had for 250K files. It turned out to be really helpful when
> asked to synchronize all the footers - it took almost "0" effort from my
> side.
>
> I literally just overlooked what it was doing when it was plowing through
> 250K pages to fix after getting the right instructions (the AI even wrote a
> temporary python script that I reviewed to apply the fix that it deleted
> afterwards).
>
> Copilot / Claude even used interesting ways of verifying if there is more
> work to do, It did not only run all the changes but it actually attempted
> to find and fix all other similar cases - all on its own - including
> various ways of checking it for 250K files - like getting some subselection
> of files to look at in various sequences an random sampling of pages.
> Pretty neat when you consider that it took me just a few minutes to write
> (long and comprehensive though) prompt/instructions.
>
> I hope it worked well ;) but from what I checked, it did. The diff of the
> change is so huge that it does not display nicely in GitHub but if you are
> courious -
>
> https://github.com/apache/airflow-site-archive/commit/64ff61087317802f76cbb4e3f44789a557d5e331
>
>
> J.
> ________________________________
> Strike Technologies, LLC (“Strike”) is part of the GTS family of
> companies. Strike is a technology solutions provider, and is not a broker
> or dealer and does not transact any securities related business directly
> whatsoever. This communication is the property of Strike and its
> affiliates, and does not constitute an offer to sell or the solicitation of
> an offer to buy any security in any jurisdiction. It is intended only for
> the person to whom it is addressed and may contain information that is
> privileged, confidential, or otherwise protected from disclosure.
> Distribution or copying of this communication, or the information contained
> herein, by anyone other than the intended recipient is prohibited. If you
> have received this communication in error, please immediately notify Strike
> at info@striketechnologies.com, and delete and destroy any copies hereof.
> ________________________________
>
> CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments
> are intended solely for the addressee. This transmission is covered by the
> Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The
> information contained in this transmission is confidential in nature and
> protected from further use or disclosure under U.S. Pub. L. 106-102, 113
> U.S. Stat. 1338 (1999), and may be subject to attorney-client or other
> legal privilege. Your use or disclosure of this information for any purpose
> other than that intended by its transmittal is strictly prohibited, and may
> subject you to fines and/or penalties under federal and state law. If you
> are not the intended recipient of this transmission, please DESTROY ALL
> COPIES RECEIVED and confirm destruction to the sender via return
> transmittal.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> For additional commands, e-mail: dev-help@airflow.apache.org
>
RE: [ANNOUNCE] Consistent and "More ASF compliant" https://airflow.apache.org (and yet another "helpful AI/LLM" case
Posted by Damian Shaw <ds...@striketechnologies.com>.
Great news on the version selector! That has been a long annoyance of mine that I can't select the version of Airflow my team currently uses!
Damian
-----Original Message-----
From: Ferruzzi, Dennis <fe...@amazon.com>
Sent: Thursday, October 30, 2025 1:11 PM
To: dev@airflow.apache.org
Cc: sebb AT ASF <se...@apache.org>
Subject: Re: [ANNOUNCE] Consistent and "More ASF compliant" https://airflow.apache.org (and yet another "helpful AI/LLM" case
It has been a while since I've been to our main page, I usually jump right to the docs page or use a bookmark to go to specific pages. It's looking good. Thanks for the effort!
- ferruzzi
________________________________
From: Jarek Potiuk <ja...@potiuk.com>
Sent: Thursday, October 30, 2025 3:41 AM
To: dev@airflow.apache.org
Cc: sebb AT ASF
Subject: [EXT] [ANNOUNCE] Consistent and "More ASF compliant" https://airflow.apache.org (and yet another "helpful AI/LLM" case
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que le contenu ne présente aucun risque.
Hello everyone,
*TL;DR; Just wanted to make you aware that we have now some "refresh" of the https://airflow.apache.org <https://airflow.apache.org> to make it more in-line with the policies and expectations of the ASF*
If you want to read more about it and some details - feel free to dive
deeper:
# Changes deployed recently
* Important change ! We only show the latest patchlevel version of airflow, providers, task0-sdk, helm-chart - you can still get to any version with version in URL or when you have links to old documentation.
Version selector is **finally** usable.
* We are linking directly to the Privacy Policy of the ASF from the footer (as required by the ASF)
* We are linking to "https://events.apache.org" - also as expected by the ASF policies
* Our footers are now (hopefully) fully consistent across all 250K (!!!) pages we have - including all versions of packages
* We are using CSP builder for Content-Security-Policy https://infra.apache.org/tools/csp.html - which is a new way how we can embed external sources (youtube videos) - I am still in the process of verifying if particular use of it in 'Meetups" page is OK With ASF privacy policy, but I think we are pretty close if not "perfect" when it comes to compliance with ASF expectations.
* We will also have Airflow CTL documentation linked once we release the
1.0.0 version (the voting is in progress)
# Comment on doc infra tooling
Just a comment - that was all rather easy with the "infrastructure" we worked on with Pavan mostly - and it has proven to be working really well.
We have a Git-archive where we can manipulate all historical versions where we can manipulate all the "versioned" docs, we have workflows in CI that sync changes back/forth and finally I could close the issue
https://github.com/apache/airflow/issues/44373 - which has 17 subtasks completed since 26 th Novemeber last year where I raised the "we need to finally modernise our docs infra".
Now I think we are fully ready to support anyone who would like to take any steps with design changes or structure changes to the docs of ours - it should be easier than ever and we have the right tooling in place to make even very complex changes. We have fully automated workflows that are used by release managers, including staging site generation for RC candidates, automated cache invalidation when needed etc. etc. If anyone would like to take a stab at restructuring some of our docs, improving the content, navigation etc, we are ready to support that rather easily from the "infrastructure" side - including testing things on staging end-2-end before releasing it to the public.
This is thanks to many people - Pavan, Amogh, Shahar, Ankit, Ash, Kaxil.
myself - and likely a number of other people with smaller changes - spanned over quite a long time - but done in parallel to all our Airflow 3.0 and
3.1 efforts without disrupting those. You can see the result of it in our documentation building architecture https://github.com/apache/airflow/tree/main/docs
# LLM support:
Also another shout-out to AI and LLMS helping in such efforts. For the big "consistency" change I used (while in train to Vienna) GH Copilot + Claude Sonet 4.5 to make the bulk changes on all historical variants of the footers we had for 250K files. It turned out to be really helpful when asked to synchronize all the footers - it took almost "0" effort from my side.
I literally just overlooked what it was doing when it was plowing through 250K pages to fix after getting the right instructions (the AI even wrote a temporary python script that I reviewed to apply the fix that it deleted afterwards).
Copilot / Claude even used interesting ways of verifying if there is more work to do, It did not only run all the changes but it actually attempted to find and fix all other similar cases - all on its own - including various ways of checking it for 250K files - like getting some subselection of files to look at in various sequences an random sampling of pages.
Pretty neat when you consider that it took me just a few minutes to write (long and comprehensive though) prompt/instructions.
I hope it worked well ;) but from what I checked, it did. The diff of the change is so huge that it does not display nicely in GitHub but if you are courious -
https://github.com/apache/airflow-site-archive/commit/64ff61087317802f76cbb4e3f44789a557d5e331
J.
________________________________
Strike Technologies, LLC (“Strike”) is part of the GTS family of companies. Strike is a technology solutions provider, and is not a broker or dealer and does not transact any securities related business directly whatsoever. This communication is the property of Strike and its affiliates, and does not constitute an offer to sell or the solicitation of an offer to buy any security in any jurisdiction. It is intended only for the person to whom it is addressed and may contain information that is privileged, confidential, or otherwise protected from disclosure. Distribution or copying of this communication, or the information contained herein, by anyone other than the intended recipient is prohibited. If you have received this communication in error, please immediately notify Strike at info@striketechnologies.com, and delete and destroy any copies hereof.
________________________________
CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments are intended solely for the addressee. This transmission is covered by the Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The information contained in this transmission is confidential in nature and protected from further use or disclosure under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be subject to attorney-client or other legal privilege. Your use or disclosure of this information for any purpose other than that intended by its transmittal is strictly prohibited, and may subject you to fines and/or penalties under federal and state law. If you are not the intended recipient of this transmission, please DESTROY ALL COPIES RECEIVED and confirm destruction to the sender via return transmittal.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
For additional commands, e-mail: dev-help@airflow.apache.org
Re: [ANNOUNCE] Consistent and "More ASF compliant" https://airflow.apache.org (and yet another "helpful AI/LLM" case
Posted by "Ferruzzi, Dennis" <fe...@amazon.com>.
It has been a while since I've been to our main page, I usually jump right to the docs page or use a bookmark to go to specific pages. It's looking good. Thanks for the effort!
- ferruzzi
________________________________
From: Jarek Potiuk <ja...@potiuk.com>
Sent: Thursday, October 30, 2025 3:41 AM
To: dev@airflow.apache.org
Cc: sebb AT ASF
Subject: [EXT] [ANNOUNCE] Consistent and "More ASF compliant" https://airflow.apache.org (and yet another "helpful AI/LLM" case
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que le contenu ne présente aucun risque.
Hello everyone,
*TL;DR; Just wanted to make you aware that we have now some "refresh" of
the https://airflow.apache.org <https://airflow.apache.org> to make it more
in-line with the policies and expectations of the ASF*
If you want to read more about it and some details - feel free to dive
deeper:
# Changes deployed recently
* Important change ! We only show the latest patchlevel version of airflow,
providers, task0-sdk, helm-chart - you can still get to any version with
version in URL or when you have links to old documentation.
Version selector is **finally** usable.
* We are linking directly to the Privacy Policy of the ASF from the footer
(as required by the ASF)
* We are linking to "https://events.apache.org" - also as expected by the
ASF policies
* Our footers are now (hopefully) fully consistent across all 250K (!!!)
pages we have - including all versions of packages
* We are using CSP builder for Content-Security-Policy
https://infra.apache.org/tools/csp.html - which is a new way how we can
embed external sources (youtube videos) - I am still in the process of
verifying if particular use of it in 'Meetups" page is OK With ASF privacy
policy, but I think we are pretty close if not "perfect" when it comes to
compliance with ASF expectations.
* We will also have Airflow CTL documentation linked once we release the
1.0.0 version (the voting is in progress)
# Comment on doc infra tooling
Just a comment - that was all rather easy with the "infrastructure" we
worked on with Pavan mostly - and it has proven to be working really well.
We have a Git-archive where we can manipulate all historical versions where
we can manipulate all the "versioned" docs, we have workflows in CI that
sync changes back/forth and finally I could close the issue
https://github.com/apache/airflow/issues/44373 - which has 17 subtasks
completed since 26 th Novemeber last year where I raised the "we need to
finally modernise our docs infra".
Now I think we are fully ready to support anyone who would like to take any
steps with design changes or structure changes to the docs of ours - it
should be easier than ever and we have the right tooling in place to make
even very complex changes. We have fully automated workflows that are used
by release managers, including staging site generation for RC candidates,
automated cache invalidation when needed etc. etc. If anyone would like to
take a stab at restructuring some of our docs, improving the content,
navigation etc, we are ready to support that rather easily from the
"infrastructure" side - including testing things on staging end-2-end
before releasing it to the public.
This is thanks to many people - Pavan, Amogh, Shahar, Ankit, Ash, Kaxil.
myself - and likely a number of other people with smaller changes - spanned
over quite a long time - but done in parallel to all our Airflow 3.0 and
3.1 efforts without disrupting those. You can see the result of it in our
documentation building architecture
https://github.com/apache/airflow/tree/main/docs
# LLM support:
Also another shout-out to AI and LLMS helping in such efforts. For the big
"consistency" change I used (while in train to Vienna) GH Copilot + Claude
Sonet 4.5 to make the bulk changes on all historical variants of the
footers we had for 250K files. It turned out to be really helpful when
asked to synchronize all the footers - it took almost "0" effort from my
side.
I literally just overlooked what it was doing when it was plowing through
250K pages to fix after getting the right instructions (the AI even wrote a
temporary python script that I reviewed to apply the fix that it deleted
afterwards).
Copilot / Claude even used interesting ways of verifying if there is more
work to do, It did not only run all the changes but it actually attempted
to find and fix all other similar cases - all on its own - including
various ways of checking it for 250K files - like getting some subselection
of files to look at in various sequences an random sampling of pages.
Pretty neat when you consider that it took me just a few minutes to write
(long and comprehensive though) prompt/instructions.
I hope it worked well ;) but from what I checked, it did. The diff of the
change is so huge that it does not display nicely in GitHub but if you are
courious -
https://github.com/apache/airflow-site-archive/commit/64ff61087317802f76cbb4e3f44789a557d5e331
J.