Fwd: Airflow Newsletter | June 2022

[Michael Robinson via users <us...@airflow.apache.org>]

Hi everyone, here’s the latest issue of the Airflow Newsletter! Please subscribe using this link: http://eepurl.com/hXUA3r <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c7e2d07421&e=ce16eef4ef>.

-Michael

> 
> View this email in your browser <https://mailchi.mp/e9626a0b81b2/airflow-newsletter-june-2022?e=ce16eef4ef>
> 
> June was another productive month in the Airflow community! We released Airflow 2.3.2, featuring a faster grid view and many bug fixes, and we conducted our annual user survey. Over on the dev list, we voted on data-driven scheduling, OpenTelemetry support, and a providers packages release.
> 
> Also this month we're excited to introduce a new feature in the newsletter -- the Dev List Digest! There you'll get a high-level view of what we voted on and discussed this month on the list. 
> 
> 
> The results of this year’s Airflow user survey are in! Thanks to all the respondents. Your participation helps set the course for the project. This year, the survey contained 36 questions in 5 categories, and we received just over 200 responses. For the results and in-depth analysis, read this blog post <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b60a2e1d7d&e=ce16eef4ef> by John Thomas and Ewa Tatarczak on the Airflow website.
> 
> 
> We released Apache Airflow 2.3.2!
> 
>  PyPI: https://pypi.org/project/apache-airflow/2.3.2/ <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=dab54da7da&e=ce16eef4ef>
>  Docs: https://airflow.apache.org/docs/apache-airflow/2.3.2/ <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=d09ec8a851&e=ce16eef4ef>
>  Release Notes: https://airflow.apache.org/docs/apache-airflow/2.3.2/release_notes.html <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4a513d0338&e=ce16eef4ef>
>  Docker Image: “docker pull apache/airflow:2.3.2”
> 
>  Constraints: https://github.com/apache/airflow/tree/constraints-2.3.2 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7a312882ef&e=ce16eef4ef>
>  
> 
> We released Apache Airflow Providers packages!
> 
> Google:
> 
>  PyPI: https://pypi.org/project/apache-airflow-providers-google/ <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e17fe4530c&e=ce16eef4ef>
>  Docs: https://airflow.apache.org/docs/apache-airflow-providers-google/8.1.0/ <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=850f4d9b83&e=ce16eef4ef>
>  Changelog: https://airflow.apache.org/docs/apache-airflow-providers-google/8.1.0/#changelog <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=222c6f0fd5&e=ce16eef4ef>
> Oracle:
> 
>  PyPI: https://pypi.org/project/apache-airflow-providers-oracle/ <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b4733a37de&e=ce16eef4ef>
>  Docs: https://airflow.apache.org/docs/apache-airflow-providers-oracle/3.1.0/ <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=bec5199eca&e=ce16eef4ef>
>  Changelog: https://airflow.apache.org/docs/apache-airflow-providers-oracle/stable/index.html#changelog <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7e36cd9c25&e=ce16eef4ef>
> 
> If you missed any of these, you can catch a recap, watch on-demand, or dig into the slides:
> 
> “Data Pipeline Tutorial: Automatically Summarize Podcasts with Airflow” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=61f1b8becd&e=ce16eef4ef>webinar by Vik Paruchuri from Dataquest, Jun 1
> 
> “Airflow 101” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=f2593934ab&e=ce16eef4ef> webinar by Marc Lamberti, June 2 (all funds raised donated to Code to Inspire <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=634408653f&e=ce16eef4ef>)
> 
> “ML in Production with Airflow” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=260fbc8ca6&e=ce16eef4ef> webinar by Kenten Danas and Santona Tuli from Astronomer, June 14
> 
> “Airflow + Selenium: The Perfect Toolkit for Web-scraping” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7f752ac3da&e=ce16eef4ef> meetup with Alvaro Avila from Moonshot Partners, June 15th
> 
> “Apache Airflow Technical Essentials <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b2a9207859&e=ce16eef4ef>” online course by Lee Gaines from O’Reilly, June 21 
> 
> “The Python Task Decorator” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=059155c03e&e=ce16eef4ef> webinar by Kenten Danas and Viraj Parekh from Astronomer, June 21
> 
> “Writing Functional DAGs with Decorators” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=d3503207ca&e=ce16eef4ef> webinar by Kenten Danas from Astronomer, June 28
> 
> “Data + AI Summit” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=fe2cb19b4e&e=ce16eef4ef> conference by Databricks, June 27-30
> 
> We can always use speakers for meetups. If you want to present your Airflow contribution, demo your project, or share your experience at a Community Meetup this year, let us know by filling out this form <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=20b13666ad&e=ce16eef4ef>. You could win some excellent swag! 
> 
> July 7: Airflow Day 2022 on Google Open Source Live: 
> https://www.meetup.com/google-open-source/events/285970398/ <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=91e543b102&e=ce16eef4ef>
> July 12: “Anatomy of an Operator” webinar: 
> https://www.astronomer.io/events/webinars/anatomy-of-an-operator <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4499662e1f&e=ce16eef4ef>
> July 15: “Data Engineering: Bulk Up Your Data Eng Skills using Airflow, Spark & DASK”: https://www.meetup.com/at-wix/events/286306470/ <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0d9c1388b9&e=ce16eef4ef>
> 
> Add Task Logs to Grid Details Panel (#24249 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cd811eb98e&e=ce16eef4ef>)
> 
> Author: Pierre Jeambrun (pierrejeambraun <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=fcde6c520f&e=ce16eef4ef>)
> 
> This PR added task logs to the details panel to reduce the number of steps required to find out what's happening in an instance.
> 
> Before this addition, users who spotted a failing task instance in the tree/grid view had to navigate to a separate page to figure out what went wrong. 
> 
> 
> 
> Bowrna Prabhakaran (bowrna <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4feb53241f&e=ce16eef4ef>)
> Where are you based? I am based out of Tamil Nadu, India.
> 
> What is your current role? I am currently not working. Previously I was working as a Backend Engineer in vue.ai <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b743eff2a0&e=ce16eef4ef>. After my maternity break, I had to leave my job due to burnout.
> 
> Why did you start contributing to the project? After leaving my job, I got an opportunity to intern with Outreachy for Airflow. I worked on a "Rewriting Breeze from Bash to Python" project and I had an amazing experience. That way I keep contributing to Airflow and learning a lot.
> 
> What do you use it for? I haven't used Airflow in production in any of my previous jobs. I wrote my first DAG as an example DAG for MSSQL. I am learning more about Airflow by contributing to it and how others in the community use Airflow.
> 
> When was your first PR and what was it? My first PR was in the old breeze to auto-reload the zshrc file to enable autocomplete of breeze commands. It happened in Oct 2021.
> 
> What was your latest? My latest PR includes making tests run parallel in the breeze (with a lot of help from Jarek Potiuk), clear specific DAG run task instance API, and moving a few of Airflow CLI commands to use Python rich.
> 
> What do you like about working on OSS projects? I really love the vibrant community and how people from diverse locations converse and make the project better together. I am happy to be part of the Airflow community.
> 
> “Using Airflow Decorators to Author DAGs,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=3eafba2694&e=ce16eef4ef> by Giorgos Myrianthous, Towards Data Science Blog
> “The State of Data Engineering 2022,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=f586364de3&e=ce16eef4ef> by Einat Orr, LakeFS Blog
> 
> “Introducing New Astro CLI Commands to Make DAG Testing Easier,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1019f802b8&e=ce16eef4ef> by David Koenitzer, Astronomer Blog
> 
> “Apache Airflow Summit 2022 – Trends and Topics,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=19823e93a9&e=ce16eef4ef> by Markus, NextLytics Blog
> 
> Votes: 
> 
> Airflow Providers packages releases (approved)
> 
> AIP-48 Data Driven Scheduling (approved)
> Airflow 2.3.2 release (approved)
> 
> AIP-49 OpenTelemetry Support (approved)
> 
> Provider’s mixed governance model (in-process)
> 
> Remove “Label when reviewed” workflow (approved)
> 
> Discuss:
> 
> Templated fields and dynamic task mapping
> 
> User survey planning
> 
> Proposed: Use "server" time from default.cfg for template engine variables
> Hiding/archiving the JIRA project for Airflow
> 
> Airflow scheduling delay metric definition
> 
> 
> We're moving the newsletter to a new platform, Mailchimp. Please subscribe to ensure you always receive the latest issue: http://eepurl.com/hXUA3r <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c7e2d07421&e=ce16eef4ef>.
> 
> That’s all, Air-folks. See you in July! 👋
>  <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=98065a1eb2&e=ce16eef4ef>
>  <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6509912e34&e=ce16eef4ef>
>  <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6f701b9196&e=ce16eef4ef>
> Want to change how you receive these emails?
> You can update your preferences <https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=2809f8cd3a> or unsubscribe from this list <https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=2809f8cd3a>.
> 
> 
> 
> 
> 
> 
> This email was sent to michael.robinson@astronomer.io <ma...@astronomer.io> 
> why did I get this? <https://apache.us14.list-manage.com/about?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=2809f8cd3a>    unsubscribe from this list <https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=2809f8cd3a>    update subscription preferences <https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=2809f8cd3a> 
> Apache Airflow · 1000 N West Street · Suite 1200 · Wilmington, DE 19801 · USA 
> 
>  <http://www.mailchimp.com/email-referral/?utm_source=freemium_newsletter&utm_medium=email&utm_campaign=referral_marketing&aid=fe7ef7a8dbb32933f30a10466&afl=1>