You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@airflow.apache.org by Julia Wrzosińska <ju...@astronomer.io> on 2022/02/24 20:37:31 UTC
Apache Airflow Newsletter / February 2022
Apache Airflow newsletter for anyone too busy to read the devlist.
View email in your browser
<https://viewstripo.email/template/9e9300ba-bc20-4de4-b8da-7c15d2759339>
January-February 2022
*The Apache Airflow **Newsletter *
*Airflow Community News & Events*
*✍️ Editors' Desk ✍️*
Only two months in and our community is already having a noteworthy
year: engagement with Airflow has reached a new high! See below for the
details.
We’d also like to remind you that registration <https://hubs.ly/Q014QFVM0>
and the CFP <https://hubs.ly/Q014QG2p0> for Airflow Summit are still open
and waiting for *you* to act. Don’t miss out!
And that’s not all... January saw a new Helm chart release, February saw
the release of Airflow 2.2.4, and the NYC Apache Airflow Meetup group will
be hosting an event soon. Meanwhile, our community continues to develop new
Airflow functionalities, Dynamic Task Mapping
<https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-42+Dynamic+Task+Mapping>
AIP is under active development and slated for inclusion in 2.3. See the
progress on Github <https://github.com/apache/airflow/projects/12>.
Finally, the newsletter has new co-editors — hello there! 👋
- Julia & Michael
<https://github.com/apache/airflow>
*🚀 Releases & Documentation Improvements 🚀*
<https://airflow.apache.org/announcements/>
On February 22, we released Apache *Airflow 2.2.4*.
📦 PyPI: https://pypi.org/project/apache-airflow/2.2.4/
📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.2.4/
🛠️ Changelog:
https://airflow.apache.org/docs/apache-airflow/2.2.4/changelog.html
🪶 Sources:
https://airflow.apache.org/docs/apache-airflow/2.2.4/installation/installing-from-sources.html
Congratulations to the entire crew working on this release!
We are getting within sight of launching Airflow 2.3.
On January 10, we released Apache Airflow Helm chart 1.4.0.
📦 ArtifactHub:
*https://artifacthub.io/packages/helm/apache-airflow/airflow*
<https://artifacthub.io/packages/helm/apache-airflow/airflow>
📚 Docs: *https://airflow.apache.org/docs/helm-chart/1.4.0/*
<https://airflow.apache.org/docs/helm-chart/1.4.0/>
🛠️ Changelog:
*https://airflow.apache.org/docs/helm-chart/1.4.0/changelog.html*
<https://airflow.apache.org/docs/helm-chart/1.4.0/changelog.html>
🪶 Sources:
*https://airflow.apache.org/docs/helm-chart/1.4.0/installing-helm-chart-from-sources.html*
<https://airflow.apache.org/docs/helm-chart/1.4.0/installing-helm-chart-from-sources.html>
The regular monthly release of new provider packages was slightly delayed,
but this month's release was the first that reached RC3. Some new versions
<https://github.com/apache/airflow/issues/21443> were released on February
13th, while others
<https://lists.apache.org/thread/3214r1s3z8lcgbm0fzlygkovx8gls4o0> on
February 18. Be sure to check them out. Thanks to everyone who helped with
testing!
*🗓️ Upcoming Events 🗓️*
We are always looking for speakers for meetups. If you want to present your
Airflow contribution, demo your project, or share your experience at our
Community Meetups this year, let us know by filling out this *form*
<http://bit.ly/3lhp1aR>. You could win some excellent swag!
-
There are only three months left before the Airflow Summit! View the
call for papers and submit a proposal on *Sessionize*
<https://sessionize.com/airflow-summit-2022/>, or register
<https://www.crowdcast.io/e/airflowsummit2022/register> to attend.
-
Join your fellow NYC-based community members on Wednesday, March 9 at 12
pm EST for the next NYC Apache Airflow Meetup
<https://www.meetup.com/NYC-Apache-Airflow-Meetup/events/283717174/>.
You’ll learn how to add steps to your DAGs that do data quality checks
using the Great Expectations Airflow operator and how to make the specs for
those checks live with your code on Github.
*✔️ Recent Events ✔️*
-
Did you miss Jarek Potiuk’s talk on Building Airflow Pipelines the
Modern Way
<https://www.meetup.com/NYC-Apache-Airflow-Meetup/events/283559034/> on
February 2? Not to worry — the video and slides are available
<https://photos.google.com/share/AF1QipOq-ZgKbF23tjYxX8K2A9yOI2FOoLviQmmv_M7SZJIzWTTlgRBgwNT3DIaUrCrIRg/photo/AF1QipMJer8Ijs2xIG0FUg1Uio1QaAq0gRZ7VaH04b2L?key=aDF0Q1E3bzlMZGlfdjlDUkhhSUFOSGJscVdicnBn>
.
-
On February 16, Leah Cole, Developer Programs Engineer at Google,
presented a webinar entitled "*Happy DAGs + Happier Teammates: How a
little CICD can go a long way*" at the first Apache Airflow Community
Meetup of 2022. Catch the recording!
<https://www.youtube.com/watch?v=MfiEhlDhEI0>
-
On February 22, Benji Lampel, Enterprise Platform Architect at
Astronomer, and Tal Gluck, Software Engineer at Superconductive, presented
a webinar entitled "*Advanced Data Quality Use Cases with Airflow and
Great Expectations.*" The recap <https://hubs.ly/Q014QG6R0> is available
on demand.
*👏 New Committers & PMC Members 👏 *
On January 4, Airflow PMC welcomed Jed Cunningham (*@jedcunningham*
<https://github.com/jedcunningham>) as the newest addition to the Airflow
PMC, while Malthe Borch (@malthe <https://github.com/malthe>) and Josh Fell
(@josh-fell <https://github.com/josh-fell>) were recognized as new Airflow
Committers on February 19. Congratulations to all of them!
*💎 PR of the month 💎 *
*Update Information about Rebasing after Enabling Rebase on Github (#21340)
<https://github.com/apache/airflow/pull/21340>* addresses a challenge
contributors face when trying to complete the rebase workflow on Github.
*"The problem this PR solves is the big struggle new contributors had —
namely, using the ‘rebase workflow’ when contributing. This workflow, which
Airflow has adopted, has a much better outcome than the usual ‘merge
workflow,’ but it is far more challenging to learn for people who are used
to merging.Recently, GitHub introduced a change that enabled the rebase
workflow to work in most cases directly from the web UI of GitHub. The PR I
worked on allows the UI update with the rebase action on all PRs (by
default, it is only available in some cases) and adds instructions to
first-time contributors on using it. We’re lowering the entry threshold for
new contributors!"*
*👀 New Feature: Contributor Spotlight 👀*
We’re eager to recognize the incredible people in the community for their
work on Apache Airflow. Not just those who code — everyone. Each month,
starting in March, a randomly selected community member will get the
spotlight and a brief introduction. Watch this space!
*📢 Communications Digest 📢*
-
*Merging Python Modules*
<https://bhavaniravi.com/blog/merge-python-module-script/> by Bhavani
Ravi explains the “humongous” task Bhavani undertook to merge Python
modules from AWS providers that were not complying with the latest
conventions.
-
*PowerShell Remoting on Windows using Airflow*
<https://maltheborch.com/#powershell-remoting-on-windows-using-airflow>
by Malthe Borch covers the use of PSRPOperator, which has recently been
updated and is currently available as a release candidate.
-
In Ten best practices for Airflow users <https://hubs.ly/Q014QGk50>,
Julia Wrzosińska shares essential tips for ensuring your pipelines run
smoothly and efficiently.
-
Head to Ula Rydiger's article and discover how Astronomer drives the
Apache Airflow project <https://hubs.ly/Q014QGsH0> together with the
community—from 600K to 4M downloads per month.
*🖋 Introducing the New Co-editors 🖋*
Starting with this issue, the newsletter is being edited by Julia
Wrzosińska, Content and Research Manager at Astronomer, and Michael
Robinson, Developer Relations Engineer at Datakin.
Please reach out to us with your Airflow-related news and events! Julia can
be reached at *julia@astronomer.io* <ju...@astronomer.io> and Michael at
*michael@datakin.com* <mi...@datakin.com>.
*💻 In-development 💻*
*2.3.0* <https://github.com/apache/airflow/milestone/36>: new features
including Dynamic Task Mapping.
*Helm Chart 1.5.0* <https://github.com/apache/airflow/milestone/49>: bug
fixes and new features.
For more info about upcoming releases and in-development projects, visit
the current project's issue (*#10176*
<https://github.com/apache/airflow/issues/10176>) in the Airflow repo.
The vote on AIP-43 (DAG processor separation) has taken place, and the new
feature is now in development. Mateusz Henc wrote the initial proposal,
which separates the DAG Processor from the Scheduler and runs it as an
independent component. This is part of the AIP-1 Airflow multi-tenancy
effort.
That's all, Air-folks! See you in March 👋
Prepared by mschickensoup <https://github.com/mschickensoup>
Re: Apache Airflow Newsletter / February 2022
Posted by Kaxil Naik <ka...@gmail.com>.
Great work, a warm welcome to our new co-editors: Julia & Michael
On Thu, 24 Feb 2022 at 20:50, Jarek Potiuk <ja...@potiuk.com> wrote:
> Cool!
>
> On Thu, Feb 24, 2022 at 9:37 PM Julia Wrzosińska <ju...@astronomer.io>
> wrote:
>
>> Apache Airflow newsletter for anyone too busy to read the devlist.
>>
>> View email in your browser
>> <https://viewstripo.email/template/9e9300ba-bc20-4de4-b8da-7c15d2759339>
>>
>> January-February 2022
>> *The Apache Airflow **Newsletter *
>>
>> *Airflow Community News & Events*
>>
>> *✍️ Editors' Desk ✍️*
>>
>> Only two months in and our community is already having a noteworthy
>> year: engagement with Airflow has reached a new high! See below for the
>> details.
>>
>> We’d also like to remind you that registration
>> <https://hubs.ly/Q014QFVM0> and the CFP <https://hubs.ly/Q014QG2p0> for
>> Airflow Summit are still open and waiting for *you* to act. Don’t miss out!
>>
>> And that’s not all... January saw a new Helm chart release, February saw
>> the release of Airflow 2.2.4, and the NYC Apache Airflow Meetup group will
>> be hosting an event soon. Meanwhile, our community continues to develop new
>> Airflow functionalities, Dynamic Task Mapping
>> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-42+Dynamic+Task+Mapping>
>> AIP is under active development and slated for inclusion in 2.3. See the
>> progress on Github <https://github.com/apache/airflow/projects/12>.
>>
>> Finally, the newsletter has new co-editors — hello there! 👋
>>
>> - Julia & Michael
>> <https://github.com/apache/airflow>
>>
>>
>> *🚀 Releases & Documentation Improvements 🚀*
>> <https://airflow.apache.org/announcements/>
>>
>>
>> On February 22, we released Apache *Airflow 2.2.4*.
>>
>> 📦 PyPI: https://pypi.org/project/apache-airflow/2.2.4/
>> 📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.2.4/
>> 🛠️ Changelog:
>> https://airflow.apache.org/docs/apache-airflow/2.2.4/changelog.html
>> 🪶 Sources:
>> https://airflow.apache.org/docs/apache-airflow/2.2.4/installation/installing-from-sources.html
>>
>> Congratulations to the entire crew working on this release!
>> We are getting within sight of launching Airflow 2.3.
>>
>>
>> On January 10, we released Apache Airflow Helm chart 1.4.0.
>>
>> 📦 ArtifactHub:
>> *https://artifacthub.io/packages/helm/apache-airflow/airflow*
>> <https://artifacthub.io/packages/helm/apache-airflow/airflow>
>>
>> 📚 Docs: *https://airflow.apache.org/docs/helm-chart/1.4.0/*
>> <https://airflow.apache.org/docs/helm-chart/1.4.0/>
>>
>> 🛠️ Changelog:
>> *https://airflow.apache.org/docs/helm-chart/1.4.0/changelog.html*
>> <https://airflow.apache.org/docs/helm-chart/1.4.0/changelog.html>
>>
>> 🪶 Sources:
>> *https://airflow.apache.org/docs/helm-chart/1.4.0/installing-helm-chart-from-sources.html*
>> <https://airflow.apache.org/docs/helm-chart/1.4.0/installing-helm-chart-from-sources.html>
>>
>> The regular monthly release of new provider packages was slightly
>> delayed, but this month's release was the first that reached RC3. Some
>> new versions <https://github.com/apache/airflow/issues/21443> were
>> released on February 13th, while others
>> <https://lists.apache.org/thread/3214r1s3z8lcgbm0fzlygkovx8gls4o0> on
>> February 18. Be sure to check them out. Thanks to everyone who helped with
>> testing!
>>
>> *🗓️ Upcoming Events 🗓️*
>>
>> We are always looking for speakers for meetups. If you want to present
>> your Airflow contribution, demo your project, or share your experience at
>> our Community Meetups this year, let us know by filling out this *form*
>> <http://bit.ly/3lhp1aR>. You could win some excellent swag!
>>
>> -
>>
>> There are only three months left before the Airflow Summit! View the
>> call for papers and submit a proposal on *Sessionize*
>> <https://sessionize.com/airflow-summit-2022/>, or register
>> <https://www.crowdcast.io/e/airflowsummit2022/register> to attend.
>> -
>>
>> Join your fellow NYC-based community members on Wednesday, March 9 at
>> 12 pm EST for the next NYC Apache Airflow Meetup
>> <https://www.meetup.com/NYC-Apache-Airflow-Meetup/events/283717174/>.
>> You’ll learn how to add steps to your DAGs that do data quality checks
>> using the Great Expectations Airflow operator and how to make the specs for
>> those checks live with your code on Github.
>>
>> *✔️ Recent Events ✔️*
>>
>> -
>>
>> Did you miss Jarek Potiuk’s talk on Building Airflow Pipelines the
>> Modern Way
>> <https://www.meetup.com/NYC-Apache-Airflow-Meetup/events/283559034/> on
>> February 2? Not to worry — the video and slides are available
>> <https://photos.google.com/share/AF1QipOq-ZgKbF23tjYxX8K2A9yOI2FOoLviQmmv_M7SZJIzWTTlgRBgwNT3DIaUrCrIRg/photo/AF1QipMJer8Ijs2xIG0FUg1Uio1QaAq0gRZ7VaH04b2L?key=aDF0Q1E3bzlMZGlfdjlDUkhhSUFOSGJscVdicnBn>
>> .
>> -
>>
>> On February 16, Leah Cole, Developer Programs Engineer at Google,
>> presented a webinar entitled "*Happy DAGs + Happier Teammates: How a
>> little CICD can go a long way*" at the first Apache Airflow Community
>> Meetup of 2022. Catch the recording!
>> <https://www.youtube.com/watch?v=MfiEhlDhEI0>
>> -
>>
>> On February 22, Benji Lampel, Enterprise Platform Architect at
>> Astronomer, and Tal Gluck, Software Engineer at Superconductive, presented
>> a webinar entitled "*Advanced Data Quality Use Cases with Airflow and
>> Great Expectations.*" The recap <https://hubs.ly/Q014QG6R0> is
>> available on demand.
>>
>> *👏 New Committers & PMC Members 👏 *
>>
>> On January 4, Airflow PMC welcomed Jed Cunningham (*@jedcunningham*
>> <https://github.com/jedcunningham>) as the newest addition to the
>> Airflow PMC, while Malthe Borch (@malthe <https://github.com/malthe>)
>> and Josh Fell (@josh-fell <https://github.com/josh-fell>) were
>> recognized as new Airflow Committers on February 19. Congratulations to all
>> of them!
>>
>> *💎 PR of the month 💎 *
>>
>> *Update Information about Rebasing after Enabling Rebase on
>> Github (#21340) <https://github.com/apache/airflow/pull/21340>* addresses
>> a challenge contributors face when trying to complete the rebase workflow
>> on Github.
>>
>>
>>
>> *"The problem this PR solves is the big struggle new contributors had —
>> namely, using the ‘rebase workflow’ when contributing. This workflow, which
>> Airflow has adopted, has a much better outcome than the usual ‘merge
>> workflow,’ but it is far more challenging to learn for people who are used
>> to merging.Recently, GitHub introduced a change that enabled the rebase
>> workflow to work in most cases directly from the web UI of GitHub. The PR I
>> worked on allows the UI update with the rebase action on all PRs (by
>> default, it is only available in some cases) and adds instructions to
>> first-time contributors on using it. We’re lowering the entry threshold for
>> new contributors!"*
>>
>> *👀 New Feature: Contributor Spotlight 👀*
>>
>> We’re eager to recognize the incredible people in the community for their
>> work on Apache Airflow. Not just those who code — everyone. Each month,
>> starting in March, a randomly selected community member will get the
>> spotlight and a brief introduction. Watch this space!
>> *📢 Communications Digest 📢*
>>
>>
>>
>> -
>>
>> *Merging Python Modules*
>> <https://bhavaniravi.com/blog/merge-python-module-script/> by Bhavani
>> Ravi explains the “humongous” task Bhavani undertook to merge Python
>> modules from AWS providers that were not complying with the latest
>> conventions.
>> -
>>
>> *PowerShell Remoting on Windows using Airflow*
>> <https://maltheborch.com/#powershell-remoting-on-windows-using-airflow>
>> by Malthe Borch covers the use of PSRPOperator, which has recently been
>> updated and is currently available as a release candidate.
>> -
>>
>> In Ten best practices for Airflow users <https://hubs.ly/Q014QGk50>,
>> Julia Wrzosińska shares essential tips for ensuring your pipelines run
>> smoothly and efficiently.
>> -
>>
>> Head to Ula Rydiger's article and discover how Astronomer drives the
>> Apache Airflow project <https://hubs.ly/Q014QGsH0> together with the
>> community—from 600K to 4M downloads per month.
>>
>> *🖋 Introducing the New Co-editors 🖋*
>>
>> Starting with this issue, the newsletter is being edited by Julia
>> Wrzosińska, Content and Research Manager at Astronomer, and Michael
>> Robinson, Developer Relations Engineer at Datakin.
>>
>> Please reach out to us with your Airflow-related news and events! Julia
>> can be reached at *julia@astronomer.io* <ju...@astronomer.io> and
>> Michael at *michael@datakin.com* <mi...@datakin.com>.
>>
>> *💻 In-development 💻*
>>
>> *2.3.0* <https://github.com/apache/airflow/milestone/36>: new features
>> including Dynamic Task Mapping.
>>
>> *Helm Chart 1.5.0* <https://github.com/apache/airflow/milestone/49>: bug
>> fixes and new features.
>>
>> For more info about upcoming releases and in-development projects, visit
>> the current project's issue (*#10176*
>> <https://github.com/apache/airflow/issues/10176>) in the Airflow repo.
>>
>> The vote on AIP-43 (DAG processor separation) has taken place, and the
>> new feature is now in development. Mateusz Henc wrote the initial proposal,
>> which separates the DAG Processor from the Scheduler and runs it as an
>> independent component. This is part of the AIP-1 Airflow multi-tenancy
>> effort.
>>
>> That's all, Air-folks! See you in March 👋
>>
>> Prepared by mschickensoup <https://github.com/mschickensoup>
>>
>
Re: Apache Airflow Newsletter / February 2022
Posted by Kaxil Naik <ka...@gmail.com>.
Great work, a warm welcome to our new co-editors: Julia & Michael
On Thu, 24 Feb 2022 at 20:50, Jarek Potiuk <ja...@potiuk.com> wrote:
> Cool!
>
> On Thu, Feb 24, 2022 at 9:37 PM Julia Wrzosińska <ju...@astronomer.io>
> wrote:
>
>> Apache Airflow newsletter for anyone too busy to read the devlist.
>>
>> View email in your browser
>> <https://viewstripo.email/template/9e9300ba-bc20-4de4-b8da-7c15d2759339>
>>
>> January-February 2022
>> *The Apache Airflow **Newsletter *
>>
>> *Airflow Community News & Events*
>>
>> *✍️ Editors' Desk ✍️*
>>
>> Only two months in and our community is already having a noteworthy
>> year: engagement with Airflow has reached a new high! See below for the
>> details.
>>
>> We’d also like to remind you that registration
>> <https://hubs.ly/Q014QFVM0> and the CFP <https://hubs.ly/Q014QG2p0> for
>> Airflow Summit are still open and waiting for *you* to act. Don’t miss out!
>>
>> And that’s not all... January saw a new Helm chart release, February saw
>> the release of Airflow 2.2.4, and the NYC Apache Airflow Meetup group will
>> be hosting an event soon. Meanwhile, our community continues to develop new
>> Airflow functionalities, Dynamic Task Mapping
>> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-42+Dynamic+Task+Mapping>
>> AIP is under active development and slated for inclusion in 2.3. See the
>> progress on Github <https://github.com/apache/airflow/projects/12>.
>>
>> Finally, the newsletter has new co-editors — hello there! 👋
>>
>> - Julia & Michael
>> <https://github.com/apache/airflow>
>>
>>
>> *🚀 Releases & Documentation Improvements 🚀*
>> <https://airflow.apache.org/announcements/>
>>
>>
>> On February 22, we released Apache *Airflow 2.2.4*.
>>
>> 📦 PyPI: https://pypi.org/project/apache-airflow/2.2.4/
>> 📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.2.4/
>> 🛠️ Changelog:
>> https://airflow.apache.org/docs/apache-airflow/2.2.4/changelog.html
>> 🪶 Sources:
>> https://airflow.apache.org/docs/apache-airflow/2.2.4/installation/installing-from-sources.html
>>
>> Congratulations to the entire crew working on this release!
>> We are getting within sight of launching Airflow 2.3.
>>
>>
>> On January 10, we released Apache Airflow Helm chart 1.4.0.
>>
>> 📦 ArtifactHub:
>> *https://artifacthub.io/packages/helm/apache-airflow/airflow*
>> <https://artifacthub.io/packages/helm/apache-airflow/airflow>
>>
>> 📚 Docs: *https://airflow.apache.org/docs/helm-chart/1.4.0/*
>> <https://airflow.apache.org/docs/helm-chart/1.4.0/>
>>
>> 🛠️ Changelog:
>> *https://airflow.apache.org/docs/helm-chart/1.4.0/changelog.html*
>> <https://airflow.apache.org/docs/helm-chart/1.4.0/changelog.html>
>>
>> 🪶 Sources:
>> *https://airflow.apache.org/docs/helm-chart/1.4.0/installing-helm-chart-from-sources.html*
>> <https://airflow.apache.org/docs/helm-chart/1.4.0/installing-helm-chart-from-sources.html>
>>
>> The regular monthly release of new provider packages was slightly
>> delayed, but this month's release was the first that reached RC3. Some
>> new versions <https://github.com/apache/airflow/issues/21443> were
>> released on February 13th, while others
>> <https://lists.apache.org/thread/3214r1s3z8lcgbm0fzlygkovx8gls4o0> on
>> February 18. Be sure to check them out. Thanks to everyone who helped with
>> testing!
>>
>> *🗓️ Upcoming Events 🗓️*
>>
>> We are always looking for speakers for meetups. If you want to present
>> your Airflow contribution, demo your project, or share your experience at
>> our Community Meetups this year, let us know by filling out this *form*
>> <http://bit.ly/3lhp1aR>. You could win some excellent swag!
>>
>> -
>>
>> There are only three months left before the Airflow Summit! View the
>> call for papers and submit a proposal on *Sessionize*
>> <https://sessionize.com/airflow-summit-2022/>, or register
>> <https://www.crowdcast.io/e/airflowsummit2022/register> to attend.
>> -
>>
>> Join your fellow NYC-based community members on Wednesday, March 9 at
>> 12 pm EST for the next NYC Apache Airflow Meetup
>> <https://www.meetup.com/NYC-Apache-Airflow-Meetup/events/283717174/>.
>> You’ll learn how to add steps to your DAGs that do data quality checks
>> using the Great Expectations Airflow operator and how to make the specs for
>> those checks live with your code on Github.
>>
>> *✔️ Recent Events ✔️*
>>
>> -
>>
>> Did you miss Jarek Potiuk’s talk on Building Airflow Pipelines the
>> Modern Way
>> <https://www.meetup.com/NYC-Apache-Airflow-Meetup/events/283559034/> on
>> February 2? Not to worry — the video and slides are available
>> <https://photos.google.com/share/AF1QipOq-ZgKbF23tjYxX8K2A9yOI2FOoLviQmmv_M7SZJIzWTTlgRBgwNT3DIaUrCrIRg/photo/AF1QipMJer8Ijs2xIG0FUg1Uio1QaAq0gRZ7VaH04b2L?key=aDF0Q1E3bzlMZGlfdjlDUkhhSUFOSGJscVdicnBn>
>> .
>> -
>>
>> On February 16, Leah Cole, Developer Programs Engineer at Google,
>> presented a webinar entitled "*Happy DAGs + Happier Teammates: How a
>> little CICD can go a long way*" at the first Apache Airflow Community
>> Meetup of 2022. Catch the recording!
>> <https://www.youtube.com/watch?v=MfiEhlDhEI0>
>> -
>>
>> On February 22, Benji Lampel, Enterprise Platform Architect at
>> Astronomer, and Tal Gluck, Software Engineer at Superconductive, presented
>> a webinar entitled "*Advanced Data Quality Use Cases with Airflow and
>> Great Expectations.*" The recap <https://hubs.ly/Q014QG6R0> is
>> available on demand.
>>
>> *👏 New Committers & PMC Members 👏 *
>>
>> On January 4, Airflow PMC welcomed Jed Cunningham (*@jedcunningham*
>> <https://github.com/jedcunningham>) as the newest addition to the
>> Airflow PMC, while Malthe Borch (@malthe <https://github.com/malthe>)
>> and Josh Fell (@josh-fell <https://github.com/josh-fell>) were
>> recognized as new Airflow Committers on February 19. Congratulations to all
>> of them!
>>
>> *💎 PR of the month 💎 *
>>
>> *Update Information about Rebasing after Enabling Rebase on
>> Github (#21340) <https://github.com/apache/airflow/pull/21340>* addresses
>> a challenge contributors face when trying to complete the rebase workflow
>> on Github.
>>
>>
>>
>> *"The problem this PR solves is the big struggle new contributors had —
>> namely, using the ‘rebase workflow’ when contributing. This workflow, which
>> Airflow has adopted, has a much better outcome than the usual ‘merge
>> workflow,’ but it is far more challenging to learn for people who are used
>> to merging.Recently, GitHub introduced a change that enabled the rebase
>> workflow to work in most cases directly from the web UI of GitHub. The PR I
>> worked on allows the UI update with the rebase action on all PRs (by
>> default, it is only available in some cases) and adds instructions to
>> first-time contributors on using it. We’re lowering the entry threshold for
>> new contributors!"*
>>
>> *👀 New Feature: Contributor Spotlight 👀*
>>
>> We’re eager to recognize the incredible people in the community for their
>> work on Apache Airflow. Not just those who code — everyone. Each month,
>> starting in March, a randomly selected community member will get the
>> spotlight and a brief introduction. Watch this space!
>> *📢 Communications Digest 📢*
>>
>>
>>
>> -
>>
>> *Merging Python Modules*
>> <https://bhavaniravi.com/blog/merge-python-module-script/> by Bhavani
>> Ravi explains the “humongous” task Bhavani undertook to merge Python
>> modules from AWS providers that were not complying with the latest
>> conventions.
>> -
>>
>> *PowerShell Remoting on Windows using Airflow*
>> <https://maltheborch.com/#powershell-remoting-on-windows-using-airflow>
>> by Malthe Borch covers the use of PSRPOperator, which has recently been
>> updated and is currently available as a release candidate.
>> -
>>
>> In Ten best practices for Airflow users <https://hubs.ly/Q014QGk50>,
>> Julia Wrzosińska shares essential tips for ensuring your pipelines run
>> smoothly and efficiently.
>> -
>>
>> Head to Ula Rydiger's article and discover how Astronomer drives the
>> Apache Airflow project <https://hubs.ly/Q014QGsH0> together with the
>> community—from 600K to 4M downloads per month.
>>
>> *🖋 Introducing the New Co-editors 🖋*
>>
>> Starting with this issue, the newsletter is being edited by Julia
>> Wrzosińska, Content and Research Manager at Astronomer, and Michael
>> Robinson, Developer Relations Engineer at Datakin.
>>
>> Please reach out to us with your Airflow-related news and events! Julia
>> can be reached at *julia@astronomer.io* <ju...@astronomer.io> and
>> Michael at *michael@datakin.com* <mi...@datakin.com>.
>>
>> *💻 In-development 💻*
>>
>> *2.3.0* <https://github.com/apache/airflow/milestone/36>: new features
>> including Dynamic Task Mapping.
>>
>> *Helm Chart 1.5.0* <https://github.com/apache/airflow/milestone/49>: bug
>> fixes and new features.
>>
>> For more info about upcoming releases and in-development projects, visit
>> the current project's issue (*#10176*
>> <https://github.com/apache/airflow/issues/10176>) in the Airflow repo.
>>
>> The vote on AIP-43 (DAG processor separation) has taken place, and the
>> new feature is now in development. Mateusz Henc wrote the initial proposal,
>> which separates the DAG Processor from the Scheduler and runs it as an
>> independent component. This is part of the AIP-1 Airflow multi-tenancy
>> effort.
>>
>> That's all, Air-folks! See you in March 👋
>>
>> Prepared by mschickensoup <https://github.com/mschickensoup>
>>
>
Re: Apache Airflow Newsletter / February 2022
Posted by Jarek Potiuk <ja...@potiuk.com>.
Cool!
On Thu, Feb 24, 2022 at 9:37 PM Julia Wrzosińska <ju...@astronomer.io>
wrote:
> Apache Airflow newsletter for anyone too busy to read the devlist.
>
> View email in your browser
> <https://viewstripo.email/template/9e9300ba-bc20-4de4-b8da-7c15d2759339>
>
> January-February 2022
> *The Apache Airflow **Newsletter *
>
> *Airflow Community News & Events*
>
> *✍️ Editors' Desk ✍️*
>
> Only two months in and our community is already having a noteworthy
> year: engagement with Airflow has reached a new high! See below for the
> details.
>
> We’d also like to remind you that registration <https://hubs.ly/Q014QFVM0>
> and the CFP <https://hubs.ly/Q014QG2p0> for Airflow Summit are still open
> and waiting for *you* to act. Don’t miss out!
>
> And that’s not all... January saw a new Helm chart release, February saw
> the release of Airflow 2.2.4, and the NYC Apache Airflow Meetup group will
> be hosting an event soon. Meanwhile, our community continues to develop new
> Airflow functionalities, Dynamic Task Mapping
> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-42+Dynamic+Task+Mapping>
> AIP is under active development and slated for inclusion in 2.3. See the
> progress on Github <https://github.com/apache/airflow/projects/12>.
>
> Finally, the newsletter has new co-editors — hello there! 👋
>
> - Julia & Michael
> <https://github.com/apache/airflow>
>
>
> *🚀 Releases & Documentation Improvements 🚀*
> <https://airflow.apache.org/announcements/>
>
>
> On February 22, we released Apache *Airflow 2.2.4*.
>
> 📦 PyPI: https://pypi.org/project/apache-airflow/2.2.4/
> 📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.2.4/
> 🛠️ Changelog:
> https://airflow.apache.org/docs/apache-airflow/2.2.4/changelog.html
> 🪶 Sources:
> https://airflow.apache.org/docs/apache-airflow/2.2.4/installation/installing-from-sources.html
>
> Congratulations to the entire crew working on this release!
> We are getting within sight of launching Airflow 2.3.
>
>
> On January 10, we released Apache Airflow Helm chart 1.4.0.
>
> 📦 ArtifactHub:
> *https://artifacthub.io/packages/helm/apache-airflow/airflow*
> <https://artifacthub.io/packages/helm/apache-airflow/airflow>
>
> 📚 Docs: *https://airflow.apache.org/docs/helm-chart/1.4.0/*
> <https://airflow.apache.org/docs/helm-chart/1.4.0/>
>
> 🛠️ Changelog:
> *https://airflow.apache.org/docs/helm-chart/1.4.0/changelog.html*
> <https://airflow.apache.org/docs/helm-chart/1.4.0/changelog.html>
>
> 🪶 Sources:
> *https://airflow.apache.org/docs/helm-chart/1.4.0/installing-helm-chart-from-sources.html*
> <https://airflow.apache.org/docs/helm-chart/1.4.0/installing-helm-chart-from-sources.html>
>
> The regular monthly release of new provider packages was slightly delayed,
> but this month's release was the first that reached RC3. Some new
> versions <https://github.com/apache/airflow/issues/21443> were released
> on February 13th, while others
> <https://lists.apache.org/thread/3214r1s3z8lcgbm0fzlygkovx8gls4o0> on
> February 18. Be sure to check them out. Thanks to everyone who helped with
> testing!
>
> *🗓️ Upcoming Events 🗓️*
>
> We are always looking for speakers for meetups. If you want to present
> your Airflow contribution, demo your project, or share your experience at
> our Community Meetups this year, let us know by filling out this *form*
> <http://bit.ly/3lhp1aR>. You could win some excellent swag!
>
> -
>
> There are only three months left before the Airflow Summit! View the
> call for papers and submit a proposal on *Sessionize*
> <https://sessionize.com/airflow-summit-2022/>, or register
> <https://www.crowdcast.io/e/airflowsummit2022/register> to attend.
> -
>
> Join your fellow NYC-based community members on Wednesday, March 9 at
> 12 pm EST for the next NYC Apache Airflow Meetup
> <https://www.meetup.com/NYC-Apache-Airflow-Meetup/events/283717174/>.
> You’ll learn how to add steps to your DAGs that do data quality checks
> using the Great Expectations Airflow operator and how to make the specs for
> those checks live with your code on Github.
>
> *✔️ Recent Events ✔️*
>
> -
>
> Did you miss Jarek Potiuk’s talk on Building Airflow Pipelines the
> Modern Way
> <https://www.meetup.com/NYC-Apache-Airflow-Meetup/events/283559034/> on
> February 2? Not to worry — the video and slides are available
> <https://photos.google.com/share/AF1QipOq-ZgKbF23tjYxX8K2A9yOI2FOoLviQmmv_M7SZJIzWTTlgRBgwNT3DIaUrCrIRg/photo/AF1QipMJer8Ijs2xIG0FUg1Uio1QaAq0gRZ7VaH04b2L?key=aDF0Q1E3bzlMZGlfdjlDUkhhSUFOSGJscVdicnBn>
> .
> -
>
> On February 16, Leah Cole, Developer Programs Engineer at Google,
> presented a webinar entitled "*Happy DAGs + Happier Teammates: How a
> little CICD can go a long way*" at the first Apache Airflow Community
> Meetup of 2022. Catch the recording!
> <https://www.youtube.com/watch?v=MfiEhlDhEI0>
> -
>
> On February 22, Benji Lampel, Enterprise Platform Architect at
> Astronomer, and Tal Gluck, Software Engineer at Superconductive, presented
> a webinar entitled "*Advanced Data Quality Use Cases with Airflow and
> Great Expectations.*" The recap <https://hubs.ly/Q014QG6R0> is
> available on demand.
>
> *👏 New Committers & PMC Members 👏 *
>
> On January 4, Airflow PMC welcomed Jed Cunningham (*@jedcunningham*
> <https://github.com/jedcunningham>) as the newest addition to the Airflow
> PMC, while Malthe Borch (@malthe <https://github.com/malthe>) and Josh
> Fell (@josh-fell <https://github.com/josh-fell>) were recognized as new
> Airflow Committers on February 19. Congratulations to all of them!
>
> *💎 PR of the month 💎 *
>
> *Update Information about Rebasing after Enabling Rebase on
> Github (#21340) <https://github.com/apache/airflow/pull/21340>* addresses
> a challenge contributors face when trying to complete the rebase workflow
> on Github.
>
>
>
> *"The problem this PR solves is the big struggle new contributors had —
> namely, using the ‘rebase workflow’ when contributing. This workflow, which
> Airflow has adopted, has a much better outcome than the usual ‘merge
> workflow,’ but it is far more challenging to learn for people who are used
> to merging.Recently, GitHub introduced a change that enabled the rebase
> workflow to work in most cases directly from the web UI of GitHub. The PR I
> worked on allows the UI update with the rebase action on all PRs (by
> default, it is only available in some cases) and adds instructions to
> first-time contributors on using it. We’re lowering the entry threshold for
> new contributors!"*
>
> *👀 New Feature: Contributor Spotlight 👀*
>
> We’re eager to recognize the incredible people in the community for their
> work on Apache Airflow. Not just those who code — everyone. Each month,
> starting in March, a randomly selected community member will get the
> spotlight and a brief introduction. Watch this space!
> *📢 Communications Digest 📢*
>
>
>
> -
>
> *Merging Python Modules*
> <https://bhavaniravi.com/blog/merge-python-module-script/> by Bhavani
> Ravi explains the “humongous” task Bhavani undertook to merge Python
> modules from AWS providers that were not complying with the latest
> conventions.
> -
>
> *PowerShell Remoting on Windows using Airflow*
> <https://maltheborch.com/#powershell-remoting-on-windows-using-airflow>
> by Malthe Borch covers the use of PSRPOperator, which has recently been
> updated and is currently available as a release candidate.
> -
>
> In Ten best practices for Airflow users <https://hubs.ly/Q014QGk50>,
> Julia Wrzosińska shares essential tips for ensuring your pipelines run
> smoothly and efficiently.
> -
>
> Head to Ula Rydiger's article and discover how Astronomer drives the
> Apache Airflow project <https://hubs.ly/Q014QGsH0> together with the
> community—from 600K to 4M downloads per month.
>
> *🖋 Introducing the New Co-editors 🖋*
>
> Starting with this issue, the newsletter is being edited by Julia
> Wrzosińska, Content and Research Manager at Astronomer, and Michael
> Robinson, Developer Relations Engineer at Datakin.
>
> Please reach out to us with your Airflow-related news and events! Julia
> can be reached at *julia@astronomer.io* <ju...@astronomer.io> and Michael
> at *michael@datakin.com* <mi...@datakin.com>.
>
> *💻 In-development 💻*
>
> *2.3.0* <https://github.com/apache/airflow/milestone/36>: new features
> including Dynamic Task Mapping.
>
> *Helm Chart 1.5.0* <https://github.com/apache/airflow/milestone/49>: bug
> fixes and new features.
>
> For more info about upcoming releases and in-development projects, visit
> the current project's issue (*#10176*
> <https://github.com/apache/airflow/issues/10176>) in the Airflow repo.
>
> The vote on AIP-43 (DAG processor separation) has taken place, and the new
> feature is now in development. Mateusz Henc wrote the initial proposal,
> which separates the DAG Processor from the Scheduler and runs it as an
> independent component. This is part of the AIP-1 Airflow multi-tenancy
> effort.
>
> That's all, Air-folks! See you in March 👋
>
> Prepared by mschickensoup <https://github.com/mschickensoup>
>
Re: Apache Airflow Newsletter / February 2022
Posted by Jarek Potiuk <ja...@potiuk.com>.
Cool!
On Thu, Feb 24, 2022 at 9:37 PM Julia Wrzosińska <ju...@astronomer.io>
wrote:
> Apache Airflow newsletter for anyone too busy to read the devlist.
>
> View email in your browser
> <https://viewstripo.email/template/9e9300ba-bc20-4de4-b8da-7c15d2759339>
>
> January-February 2022
> *The Apache Airflow **Newsletter *
>
> *Airflow Community News & Events*
>
> *✍️ Editors' Desk ✍️*
>
> Only two months in and our community is already having a noteworthy
> year: engagement with Airflow has reached a new high! See below for the
> details.
>
> We’d also like to remind you that registration <https://hubs.ly/Q014QFVM0>
> and the CFP <https://hubs.ly/Q014QG2p0> for Airflow Summit are still open
> and waiting for *you* to act. Don’t miss out!
>
> And that’s not all... January saw a new Helm chart release, February saw
> the release of Airflow 2.2.4, and the NYC Apache Airflow Meetup group will
> be hosting an event soon. Meanwhile, our community continues to develop new
> Airflow functionalities, Dynamic Task Mapping
> <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-42+Dynamic+Task+Mapping>
> AIP is under active development and slated for inclusion in 2.3. See the
> progress on Github <https://github.com/apache/airflow/projects/12>.
>
> Finally, the newsletter has new co-editors — hello there! 👋
>
> - Julia & Michael
> <https://github.com/apache/airflow>
>
>
> *🚀 Releases & Documentation Improvements 🚀*
> <https://airflow.apache.org/announcements/>
>
>
> On February 22, we released Apache *Airflow 2.2.4*.
>
> 📦 PyPI: https://pypi.org/project/apache-airflow/2.2.4/
> 📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.2.4/
> 🛠️ Changelog:
> https://airflow.apache.org/docs/apache-airflow/2.2.4/changelog.html
> 🪶 Sources:
> https://airflow.apache.org/docs/apache-airflow/2.2.4/installation/installing-from-sources.html
>
> Congratulations to the entire crew working on this release!
> We are getting within sight of launching Airflow 2.3.
>
>
> On January 10, we released Apache Airflow Helm chart 1.4.0.
>
> 📦 ArtifactHub:
> *https://artifacthub.io/packages/helm/apache-airflow/airflow*
> <https://artifacthub.io/packages/helm/apache-airflow/airflow>
>
> 📚 Docs: *https://airflow.apache.org/docs/helm-chart/1.4.0/*
> <https://airflow.apache.org/docs/helm-chart/1.4.0/>
>
> 🛠️ Changelog:
> *https://airflow.apache.org/docs/helm-chart/1.4.0/changelog.html*
> <https://airflow.apache.org/docs/helm-chart/1.4.0/changelog.html>
>
> 🪶 Sources:
> *https://airflow.apache.org/docs/helm-chart/1.4.0/installing-helm-chart-from-sources.html*
> <https://airflow.apache.org/docs/helm-chart/1.4.0/installing-helm-chart-from-sources.html>
>
> The regular monthly release of new provider packages was slightly delayed,
> but this month's release was the first that reached RC3. Some new
> versions <https://github.com/apache/airflow/issues/21443> were released
> on February 13th, while others
> <https://lists.apache.org/thread/3214r1s3z8lcgbm0fzlygkovx8gls4o0> on
> February 18. Be sure to check them out. Thanks to everyone who helped with
> testing!
>
> *🗓️ Upcoming Events 🗓️*
>
> We are always looking for speakers for meetups. If you want to present
> your Airflow contribution, demo your project, or share your experience at
> our Community Meetups this year, let us know by filling out this *form*
> <http://bit.ly/3lhp1aR>. You could win some excellent swag!
>
> -
>
> There are only three months left before the Airflow Summit! View the
> call for papers and submit a proposal on *Sessionize*
> <https://sessionize.com/airflow-summit-2022/>, or register
> <https://www.crowdcast.io/e/airflowsummit2022/register> to attend.
> -
>
> Join your fellow NYC-based community members on Wednesday, March 9 at
> 12 pm EST for the next NYC Apache Airflow Meetup
> <https://www.meetup.com/NYC-Apache-Airflow-Meetup/events/283717174/>.
> You’ll learn how to add steps to your DAGs that do data quality checks
> using the Great Expectations Airflow operator and how to make the specs for
> those checks live with your code on Github.
>
> *✔️ Recent Events ✔️*
>
> -
>
> Did you miss Jarek Potiuk’s talk on Building Airflow Pipelines the
> Modern Way
> <https://www.meetup.com/NYC-Apache-Airflow-Meetup/events/283559034/> on
> February 2? Not to worry — the video and slides are available
> <https://photos.google.com/share/AF1QipOq-ZgKbF23tjYxX8K2A9yOI2FOoLviQmmv_M7SZJIzWTTlgRBgwNT3DIaUrCrIRg/photo/AF1QipMJer8Ijs2xIG0FUg1Uio1QaAq0gRZ7VaH04b2L?key=aDF0Q1E3bzlMZGlfdjlDUkhhSUFOSGJscVdicnBn>
> .
> -
>
> On February 16, Leah Cole, Developer Programs Engineer at Google,
> presented a webinar entitled "*Happy DAGs + Happier Teammates: How a
> little CICD can go a long way*" at the first Apache Airflow Community
> Meetup of 2022. Catch the recording!
> <https://www.youtube.com/watch?v=MfiEhlDhEI0>
> -
>
> On February 22, Benji Lampel, Enterprise Platform Architect at
> Astronomer, and Tal Gluck, Software Engineer at Superconductive, presented
> a webinar entitled "*Advanced Data Quality Use Cases with Airflow and
> Great Expectations.*" The recap <https://hubs.ly/Q014QG6R0> is
> available on demand.
>
> *👏 New Committers & PMC Members 👏 *
>
> On January 4, Airflow PMC welcomed Jed Cunningham (*@jedcunningham*
> <https://github.com/jedcunningham>) as the newest addition to the Airflow
> PMC, while Malthe Borch (@malthe <https://github.com/malthe>) and Josh
> Fell (@josh-fell <https://github.com/josh-fell>) were recognized as new
> Airflow Committers on February 19. Congratulations to all of them!
>
> *💎 PR of the month 💎 *
>
> *Update Information about Rebasing after Enabling Rebase on
> Github (#21340) <https://github.com/apache/airflow/pull/21340>* addresses
> a challenge contributors face when trying to complete the rebase workflow
> on Github.
>
>
>
> *"The problem this PR solves is the big struggle new contributors had —
> namely, using the ‘rebase workflow’ when contributing. This workflow, which
> Airflow has adopted, has a much better outcome than the usual ‘merge
> workflow,’ but it is far more challenging to learn for people who are used
> to merging.Recently, GitHub introduced a change that enabled the rebase
> workflow to work in most cases directly from the web UI of GitHub. The PR I
> worked on allows the UI update with the rebase action on all PRs (by
> default, it is only available in some cases) and adds instructions to
> first-time contributors on using it. We’re lowering the entry threshold for
> new contributors!"*
>
> *👀 New Feature: Contributor Spotlight 👀*
>
> We’re eager to recognize the incredible people in the community for their
> work on Apache Airflow. Not just those who code — everyone. Each month,
> starting in March, a randomly selected community member will get the
> spotlight and a brief introduction. Watch this space!
> *📢 Communications Digest 📢*
>
>
>
> -
>
> *Merging Python Modules*
> <https://bhavaniravi.com/blog/merge-python-module-script/> by Bhavani
> Ravi explains the “humongous” task Bhavani undertook to merge Python
> modules from AWS providers that were not complying with the latest
> conventions.
> -
>
> *PowerShell Remoting on Windows using Airflow*
> <https://maltheborch.com/#powershell-remoting-on-windows-using-airflow>
> by Malthe Borch covers the use of PSRPOperator, which has recently been
> updated and is currently available as a release candidate.
> -
>
> In Ten best practices for Airflow users <https://hubs.ly/Q014QGk50>,
> Julia Wrzosińska shares essential tips for ensuring your pipelines run
> smoothly and efficiently.
> -
>
> Head to Ula Rydiger's article and discover how Astronomer drives the
> Apache Airflow project <https://hubs.ly/Q014QGsH0> together with the
> community—from 600K to 4M downloads per month.
>
> *🖋 Introducing the New Co-editors 🖋*
>
> Starting with this issue, the newsletter is being edited by Julia
> Wrzosińska, Content and Research Manager at Astronomer, and Michael
> Robinson, Developer Relations Engineer at Datakin.
>
> Please reach out to us with your Airflow-related news and events! Julia
> can be reached at *julia@astronomer.io* <ju...@astronomer.io> and Michael
> at *michael@datakin.com* <mi...@datakin.com>.
>
> *💻 In-development 💻*
>
> *2.3.0* <https://github.com/apache/airflow/milestone/36>: new features
> including Dynamic Task Mapping.
>
> *Helm Chart 1.5.0* <https://github.com/apache/airflow/milestone/49>: bug
> fixes and new features.
>
> For more info about upcoming releases and in-development projects, visit
> the current project's issue (*#10176*
> <https://github.com/apache/airflow/issues/10176>) in the Airflow repo.
>
> The vote on AIP-43 (DAG processor separation) has taken place, and the new
> feature is now in development. Mateusz Henc wrote the initial proposal,
> which separates the DAG Processor from the Scheduler and runs it as an
> independent component. This is part of the AIP-1 Airflow multi-tenancy
> effort.
>
> That's all, Air-folks! See you in March 👋
>
> Prepared by mschickensoup <https://github.com/mschickensoup>
>