Apache Airflow Newsletter | May 2022

[Airflow Newsletter <ju...@astronomer.io.INVALID>]

Airflow community news in May 2022

View this email in your browser (https://mailchi.mp/5933052f2f46/apache-airflow-newsletter-april-9028981?e=9963ef1d25)

Airflow 2.3.1 is in the wild – thanks to 2151 commits, 302,350 file changes (207,983 additions and 94,367 deletions) to 3412 files, and the work of 68 contributors since the release of 2.2.0! Over the next 12 months, we plan to add support for mapping dynamic tasks to variables, DAG configurations, and more.

We added a new committer, Ping Zhang (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9b5c4183dc&e=9963ef1d25) from Airbnb. Welcome and congratulations, Ping!

Airflow continues to grow at an exceptional rate. In fact, 30% of all Airflow commits ever made have occurred in just the last 16 months! The future is bright, and it’s all thanks to you!
🚨 Important news 🚨

The 2022 Airflow user survey (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a477b42803&e=9963ef1d25) is now LIVE. If you can take 10-15 minutes to fill out a response, it would be greatly appreciated. But don't delay -- the survey will close on Friday, June 3.

The survey has 36 questions in 5 categories, and we're aiming for at least 300 responses. It helps guide the project's future development, so your participation matters. The results will be published on the Airflow website (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a7715cb613&e=9963ef1d25) .

The 2022 event was outstanding! A huge shoutout goes to the entire Airflow Summit team (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cbff640b26&e=9963ef1d25) : Bolke de Bruin, Joanna Kasprzak-Kajder, Jarek Potiuk, Aizhamal Nurmamatkyzy, John Jackson, Kaxil Naik, Leah Cole, Rafal Biegacz and Ry Walker. Additional thanks to Ewa Mikusek for refreshing the branding (we love the summit T-shirts) and to the entire SoftwareGuru team for handling the technicals. You folks are incredible!

The 2022 Airflow Summit team:
* Created a fresh, new design and branding
* Gathered over 7500 participants
* Organized 4 workshops
* Delivered 53 talks
* Led many valuable Q&A discussions
* Organized 19 live in-person events worldwide
* Gave us a video of Bolke stand-up-paddleboarding 🤟🏽
* Provided the Airflow Community with a fun yet knowledge-packed event
* Gave us something to look forward to in 2023…

FYI, last year the number of the Airflow Summit participants grew by 30% during and after the summit. So if you missed this year’s sessions, join the afterparty! You won’t be able to attend live meetings or socialize, but on the summit website (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e55e601f9a&e=9963ef1d25) you can still view much of the brilliant content our speakers and participants put together. (Registration (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ae813f70ae&e=9963ef1d25) is required.)

We released Apache Airflow 2.3.1!

:package: PyPI: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=00e286e32f&e=9963ef1d25 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b8bdfcc80a&e=9963ef1d25)

:books: Docs: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=f5f8e8e341&e=9963ef1d25

:hammer_and_wrench: Changelog: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ecfa7d045e&e=9963ef1d25

:ship: Docker Image: “docker pull apache/airflow:2.3.1”

:busstop: Constraints: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=24ed8dd275&e=9963ef1d25

We released Apache Airflow Helm Chart 1.6.0!

:books: Docs: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=93321764aa&e=9963ef1d25

:hammer_and_wrench: Release Notes: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=382b4af7ed&e=9963ef1d25

:busstop: Tag: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a1cc9db18c&e=9963ef1d25

We can always use speakers for meetups. If you want to present your Airflow contribution, demo your project, or share your experience at a Community Meetup this year, let us know by filling out this form (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0571720ea1&e=9963ef1d25) . You could win some excellent swag!
* “Essential Airflow Tips for Beginners (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=656b5cd344&e=9963ef1d25) ” webinar by Kenten Danas, May 31
* “Data Pipeline Tutorial: Automatically Summarize Podcasts with Airflow” (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e232c18164&e=9963ef1d25) webinar by Vik Paruchuri from Dataquest, Jun 1, 2022
* “Airflow 101” (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0f613178a5&e=9963ef1d25) webinar by Marc Lamberti, June 2 (all funds raised will be donated to Code to Inspire (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4a93b49408&e=9963ef1d25) )
* “Apache Airflow Technical Essentials (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ed882a1155&e=9963ef1d25) ” online course by Lee Gaines from O’Reilly, June 21
* “Data + AI Summit” (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e5f74dff0b&e=9963ef1d25) conference by Databricks, June 27-30

If you missed any of these, you can catch up with a recap, watch on-demand, or dig into the slides:
*  Airflow Summit 2022 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=8c353e4886&e=9963ef1d25) , May 23-27
* Astronomer Providers (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=665ac10bce&e=9963ef1d25) webinar, May 17
* “What’s New in Airflow 2.3 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=55a8a90979&e=9963ef1d25) ” webinar, May 3

Add Databricks Deferrable Operators (#19736 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7a201baa64&e=9963ef1d25) )

Author: Eugene Karimov (e (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=878209acce&e=9963ef1d25) skarimov (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=14184a37bc&e=9963ef1d25) )

This PR added deferrable versions of DatabricksSubmitRunOperator and DatabricksRunNowOperator, which are using new Airflow functionality introduced with version 2.2.0 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7733751598&e=9963ef1d25) .

This month we’re excited to introduce Edith Puclla (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2c190b8856&e=9963ef1d25) ! Read our short interview and get to know one of your fellow contributors better.
1. Where are you based?

I'm from Ayacucho, Peru, in South America
2. What is your current role (& industry, if not tech)?

Open to Work as a DevOps/Infrastructure Developer
3. Why did you start contributing to Airflow?

When Outreachy accepted me into their program to contribute to an Open Source project, I didn't know how to make my first contribution. To my good luck, the Apache Airflow mentors organized a workshop for beginners. I received a warm welcome, and they created a safe space to answer our questions regarding the project. The empathy that they transmit as a community and the constant support of all the members attracted me to be part of Apache Airflow and keep contributing.
4. What do you use it for?

I use Apache Airflow as a learning tool to get to know its architecture, improve my programming skills and, learn about Airflow's implementation flow, delivery, and above all improve my communication skills.
5. When was your first PR, and what was it?

My first contribution to Apache Airflow was in November 2021. I was following the instructions to install Apache Airflow with Docker. I saw the need to improve the documentation so I added more details about Docker Compose to improve the installation guide and make it more user-friendly.
6. What was your latest PR?

My latest PR would add a regular expression on pre-commits to improve the inclusiveness of projects and raise awareness of conscious language use.
7. What do you like or enjoy about working on OSS projects?

I really enjoy interacting with the community. Apache Airflow is huge and knowing a little more about this ecosystem is gratifying. It is also nice to learn from other developers by reading their PRs and of course closing my own!
* “Apache Airflow 2.3: Everything You Need to Know” (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=553edb8b7a&e=9963ef1d25) by Steve Swoyer
* “Airflow 2.3.0 is Here” (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6ccb6aaeef&e=9963ef1d25) by Ephraim Anierobi
* “Lessons Learned from Running Apache Airflow at Scale” (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2011e3f751&e=9963ef1d25) by Megan Parker and Sam Wheating (Spotify)
* “ (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=874ac9cf05&e=9963ef1d25) Orchestrate data pipelines with Apache Airflow! (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2a6478b3b5&e=9963ef1d25) ” (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6b12de0992&e=9963ef1d25) by Ginelle D’Souza
* “OpenLineage and Airflow” (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=23e3fb2e07&e=9963ef1d25) , an Astronomer guide
* “Is it beneficial to use Apache Airflow in 2022” (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2f69b55bb1&e=9963ef1d25) by Priyanshi Bajaj
* “How to Use Apache Airflow to Schedule and Manage Workflows” (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=12aa34e81a&e=9963ef1d25) by Sameer Shukla
* “Apache Airflow on Docker With AWS S3” (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=544b9c278b&e=9963ef1d25) by Narotam Aggarwal

* Airflow 2.3.2 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=5fbe38eae5&e=9963ef1d25) : bug fixes
* A (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7019100942&e=9963ef1d25) irflow 2.4.0 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e09c624cc7&e=9963ef1d25) : new features
* Airflow Helm Chart 1.7.0 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a14f4e624a&e=9963ef1d25) : bug fixes and new features

For more info about upcoming releases and in-development projects, visit the current projects issue (#10176 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=630050ec1e&e=9963ef1d25) ).

The newsletter is moving to a new platform, Mailchimp. Please subscribe to ensure you always receive the latest issue: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=07cc1e9037&e=9963ef1d25.

Know of an upcoming event or publication the community should know about? Drop us a line at michael.robinson@astronomer.io (mailto:michae.robinson@astronomer.io) or julia@astronomer.io (mailto:julia@astronomer.io) .

That’s all, Air-folks. See you in June! 👋

============================================================
** https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c2b221844f&e=9963ef1d25 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7be0e904b7&e=9963ef1d25)
** Website (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=fc83f9c03a&e=9963ef1d25)
** GitHub (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9ea1a2ba5d&e=9963ef1d25)

This email was sent to dev@airflow.apache.org (mailto:dev@airflow.apache.org)
why did I get this? (https://apache.us14.list-manage.com/about?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=9963ef1d25&c=018c277cf1)     unsubscribe from this list (https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=9963ef1d25&c=018c277cf1)     update subscription preferences (https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=9963ef1d25&c=018c277cf1)
Apache Airflow . 1000 N West Street . Suite 1200 . Wilmington, DE 19801 . USA

Email Marketing Powered by Mailchimp
http://www.mailchimp.com/email-referral/?utm_source=freemium_newsletter&utm_medium=email&utm_campaign=referral_marketing&aid=fe7ef7a8dbb32933f30a10466&afl=1