You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Michael Robinson <mi...@astronomer.io.INVALID> on 2022/07/29 19:31:49 UTC
Fwd: Apache Airflow Newsletter | July 2022
> View this email in your browser <https://mailchi.mp/627a8910cc73/apache-airflow-newsletter-july-2022?e=ce16eef4ef>
>
>
> This month in the Airflow community, we released Airflow 2.3.3 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1bd27974f1&e=ce16eef4ef> and over 40 Airflow Providers packages <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aaec9074c7&e=ce16eef4ef>, including the new Common SQL shared provider and the new Tabular provider. We learned and networked at an Airflow Meetup in Tel Aviv <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=51d4c8cc3e&e=ce16eef4ef> and Google’s Airflow Day 2022 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2f955a2dfd&e=ce16eef4ef>, and over on the dev list we discussed Dynamic Yield’s “Magic Loop” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7cf95cdc1d&e=ce16eef4ef> for minimizing delays caused by many concurrent dynamic DAGs, among other topics.
>
> The newsletter is moving to a new platform, Mailchimp. Please subscribe <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=096e67c8c1&e=ce16eef4ef> to ensure you always receive the latest issue.
>
>
> We can always use speakers for meetups. If you want to present your Airflow contribution, demo your project, or share your experience at a Community Meetup this year, let us know <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e7fc2b5028&e=ce16eef4ef>. You could win some excellent swag!
>
> August 9: “Airflow 101: Essential Tips for Beginners” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=166096c917&e=ce16eef4ef>
> August 10: “Build a Robust Data Pipeline with Airflow, dbt, and Great Expectations – with Interactivity” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7dbc8d0b3e&e=ce16eef4ef>
> August 17: “Using Plugins for Webserver” with Faizan Qazi (details TBA on Clowder <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=88a3d50cba&e=ce16eef4ef>)
>
>
> We released Airflow 2.3.3. You can find the build artifacts using these links:
> PyPI <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0d09446543&e=ce16eef4ef> | Notes <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e2c70b8cbd&e=ce16eef4ef> | Constraints <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=901d8a64b3&e=ce16eef4ef>
> Docker image: apache/airflow:2.3.3
> We released Airflow Providers packages. You can find them using these links:
> Sources <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=22c14f066e&e=ce16eef4ef> | Guide <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=63b31b2c5a&e=ce16eef4ef>
>
>
> @uranusjr <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=5143e93b2b&e=ce16eef4ef>
> #24989 Implement expand_kwargs() <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=f4d747431b&e=ce16eef4ef>
> Implements a new ExpandInput type, "list-of-dicts", to support this new shape of mapped kwargs.
>
> To nominate a PR <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d9a79856a&e=ce16eef4ef> for a future issue, flag it by adding #protm to a comment.
>
> If you missed any of these, you can catch a recap, watch on-demand, or dig into the slides:
>
> July 28: “Apache Airflow TLV Meetup Summer Edition” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=48722ecbf9&e=ce16eef4ef>
> July 27: “Airflow, Spark, Trinio and Data Engineering Best Practices” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa33f35f0&e=ce16eef4ef>
>
> July 26: “Using Airflow with Tensorflow and MLFlow” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=31433f5b8d&e=ce16eef4ef>
> July 15: “Data Engineering: Bulk Up Your Data Eng Skills using Airflow, Spark & DASK” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=bc6157ff5d&e=ce16eef4ef>
> July 14: “Apache Airflow – Future of Data Workflows” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b776db73d0&e=ce16eef4ef>
> July 12: “Anatomy of an Operator” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa8403602&e=ce16eef4ef>
>
> July 7: "Airflow Day 2022 on Google Open Source Live" <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=052c3897b0&e=ce16eef4ef>
>
> July 6: “Introduction to Building Pipelines in GCP using Airflow” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ea6a2f2893&e=ce16eef4ef>
>
> “Is Airflow the Right Choice for Machine Learning, Too?,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6213fd2ecd&e=ce16eef4ef> by Samhita Alla, BetterProgramming/Medium
>
> “Deutsche Bank uses Cloud Composer Workload Automation,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9e495b03f8&e=ce16eef4ef>cnxtoday.com <http://cnxtoday.com/>
> “Shopify’s Practical Guidelines from Running Airflow for ML and Data Workflows at Scale,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=143cd0347e&e=ce16eef4ef> by Reza Rahimi, infoq.com <http://infoq.com/>
> “Magic Loop in Airflow – Reloaded,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c3257fbba3&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
>
> “Shared Volumes in Airflow – the Good, the Bad, and the Ugly,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9046873bce&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
>
> “Running Airflow on ARM M1/M2? Hell Yes, but Upgrade to 2.3+,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a782285305&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
>
> “Generic Airflow Transfers Made Easy,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=eeb6b8d1cb&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
>
> “Airflow’s Magic Loop,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cac6eafbf4&e=ce16eef4ef> by Itay Bittan, Towards Data Science/Medium
>
> “Load Data from Postgres to BigQuery with Airflow,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d83f3800a&e=ce16eef4ef> by Giorgos Myrianthous, Towards Data Science/Medium
>
> “Automated Alerts for Airflow from Slack,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aa3832265c&e=ce16eef4ef> by Chris Young, Towards Data Science/Medium
>
>
> Votes:
>
> Airflow 2.3.3rc1 release (approved)
>
> Airflow 2.3.3 release (approved)
>
> Airflow Providers (RC1) (approved)
>
> July’s PR of the Month (approved)
>
> Release Airflow Python client 2.3.0 based on 2.3.0rc1 (pending)
>
> Lazy consensus:
>
> Deprecate “dependency detector” pluggability (passed)
>
> Discussed:
>
> “Magic Loop” as first-class citizen in Airflow
>
> Airflow scheduling delay metric definition
>
> Grouping of concerns
>
> Migrate Airflow code to use PEP 563 (postponed evaluation of annotations)
>
> Standard way of getting provider version?
>
> Trimming down Airflow versions in issues
>
> Deprecating the SLA feature?
>
> Implementing a testing framework for Airflow
>
> Consolidate DAG scheduling params
>
> Proposed:
>
> Provider’s mixed governance model
>
> Simplification of WWW asset compilation for Breeze/dev env
>
>
> Airflow 2.4.0 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=d0f3d39055&e=ce16eef4ef>
> This release will focus on new features.
>
> Airflow Helm Chart 1.7.0 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cfff66719b&e=ce16eef4ef>
> The next release for the Airflow Helm Chart, fixing bugs and adding more features.
>
> For more info about upcoming releases and in-development projects, visit the current projects issue (#10176 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4c64a96866&e=ce16eef4ef>).
>
> The newsletter is moving to a new platform, Mailchimp. Please subscribe <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=05b5c570d8&e=ce16eef4ef> to ensure you always receive the latest issue.
>
> Is there an event or publication the community should know about? Let us know at michael.robinson@astronomer.io <ma...@astronomer.io> or julia@astronomer.io <ma...@astronomer.io>.
> Want to change how you receive these emails?
> You can update your preferences <https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4> or unsubscribe from this list <https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4>.
>
Re: Apache Airflow Newsletter | July 2022
Posted by Tobiasz Kędzierski <to...@gmail.com>.
Awesome!
Thanks Michael :)
pt., 29 lip 2022 o 21:34 Jarek Potiuk <ja...@potiuk.com> napisał(a):
> Nice!
>
> On Fri, Jul 29, 2022 at 9:32 PM Michael Robinson via users <
> users@airflow.apache.org> wrote:
>
>> View this email in your browser
>> <https://mailchi.mp/627a8910cc73/apache-airflow-newsletter-july-2022?e=ce16eef4ef>
>>
>> This month in the Airflow community, we released Airflow 2.3.3
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1bd27974f1&e=ce16eef4ef>
>> and over 40 Airflow Providers packages
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aaec9074c7&e=ce16eef4ef>,
>> including the new Common SQL shared provider and the new Tabular provider.
>> We learned and networked at an Airflow Meetup in Tel Aviv
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=51d4c8cc3e&e=ce16eef4ef>
>> and Google’s Airflow Day 2022
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2f955a2dfd&e=ce16eef4ef>,
>> and over on the dev list we discussed Dynamic Yield’s “Magic Loop”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7cf95cdc1d&e=ce16eef4ef>
>> for minimizing delays caused by many concurrent dynamic DAGs, among
>> other topics.
>>
>> The newsletter is moving to a new platform, Mailchimp. Please subscribe
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=096e67c8c1&e=ce16eef4ef>
>> to ensure you always receive the latest issue.
>>
>> We can always use speakers for meetups. If you want to present your
>> Airflow contribution, demo your project, or share your experience at a
>> Community Meetup this year, let us know
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e7fc2b5028&e=ce16eef4ef>.
>> You could win some excellent swag!
>>
>> -
>>
>> August 9: “Airflow 101: Essential Tips for Beginners”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=166096c917&e=ce16eef4ef>
>> -
>>
>> August 10: “Build a Robust Data Pipeline with Airflow, dbt, and Great
>> Expectations – with Interactivity”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7dbc8d0b3e&e=ce16eef4ef>
>> -
>>
>> August 17: “Using Plugins for Webserver” with Faizan Qazi (details
>> TBA on Clowder
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=88a3d50cba&e=ce16eef4ef>
>> )
>>
>> We released Airflow 2.3.3. You can find the build artifacts using these
>> links:
>>
>> - PyPI
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0d09446543&e=ce16eef4ef>
>> | Notes
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e2c70b8cbd&e=ce16eef4ef>
>> | Constraints
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=901d8a64b3&e=ce16eef4ef>
>> - Docker image: apache/airflow:2.3.3
>>
>> We released Airflow Providers packages. You can find them using these
>> links:
>>
>> - Sources
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=22c14f066e&e=ce16eef4ef>
>> | Guide
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=63b31b2c5a&e=ce16eef4ef>
>>
>> *@uranusjr
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=5143e93b2b&e=ce16eef4ef>*
>>
>> *#24989 Implement expand_kwargs()
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=f4d747431b&e=ce16eef4ef>*
>> Implements a new ExpandInput type, "list-of-dicts", to support this new
>> shape of mapped kwargs.
>> To nominate a PR
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d9a79856a&e=ce16eef4ef>
>> for a future issue, flag it by adding *#protm* to a comment.
>>
>> If you missed any of these, you can catch a recap, watch on-demand, or
>> dig into the slides:
>>
>> -
>>
>> July 28: “Apache Airflow TLV Meetup Summer Edition”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=48722ecbf9&e=ce16eef4ef>
>> -
>>
>> July 27: “Airflow, Spark, Trinio and Data Engineering Best Practices”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa33f35f0&e=ce16eef4ef>
>>
>> -
>>
>> July 26: “Using Airflow with Tensorflow and MLFlow”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=31433f5b8d&e=ce16eef4ef>
>> -
>>
>> July 15: “Data Engineering: Bulk Up Your Data Eng Skills using
>> Airflow, Spark & DASK”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=bc6157ff5d&e=ce16eef4ef>
>> -
>>
>> July 14: “Apache Airflow – Future of Data Workflows”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b776db73d0&e=ce16eef4ef>
>> -
>>
>> July 12: “Anatomy of an Operator”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa8403602&e=ce16eef4ef>
>>
>> -
>>
>> July 7: "Airflow Day 2022 on Google Open Source Live"
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=052c3897b0&e=ce16eef4ef>
>>
>>
>>
>> -
>>
>> July 6: “Introduction to Building Pipelines in GCP using Airflow”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ea6a2f2893&e=ce16eef4ef>
>>
>>
>> -
>>
>> “Is Airflow the Right Choice for Machine Learning, Too?,”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6213fd2ecd&e=ce16eef4ef>
>> by Samhita Alla, BetterProgramming/Medium
>> -
>>
>> “Deutsche Bank uses Cloud Composer Workload Automation,”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9e495b03f8&e=ce16eef4ef>
>> cnxtoday.com
>> -
>>
>> “Shopify’s Practical Guidelines from Running Airflow for ML and Data
>> Workflows at Scale,”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=143cd0347e&e=ce16eef4ef>
>> by Reza Rahimi, infoq.com
>> -
>>
>> “Magic Loop in Airflow – Reloaded,”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c3257fbba3&e=ce16eef4ef>
>> by Jarek Potiuk, Apache Airflow/Medium
>> -
>>
>> “Shared Volumes in Airflow – the Good, the Bad, and the Ugly,”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9046873bce&e=ce16eef4ef>
>> by Jarek Potiuk, Apache Airflow/Medium
>> -
>>
>> “Running Airflow on ARM M1/M2? Hell Yes, but Upgrade to 2.3+,”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a782285305&e=ce16eef4ef>
>> by Jarek Potiuk, Apache Airflow/Medium
>> -
>>
>> “Generic Airflow Transfers Made Easy,”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=eeb6b8d1cb&e=ce16eef4ef>
>> by Jarek Potiuk, Apache Airflow/Medium
>> -
>>
>> “Airflow’s Magic Loop,”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cac6eafbf4&e=ce16eef4ef>
>> by Itay Bittan, Towards Data Science/Medium
>> -
>>
>> “Load Data from Postgres to BigQuery with Airflow,”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d83f3800a&e=ce16eef4ef>
>> by Giorgos Myrianthous, Towards Data Science/Medium
>> -
>>
>> “Automated Alerts for Airflow from Slack,”
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aa3832265c&e=ce16eef4ef>
>> by Chris Young, Towards Data Science/Medium
>>
>>
>> -
>>
>> *Votes: *
>> -
>>
>> Airflow 2.3.3rc1 release (approved)
>> -
>>
>> Airflow 2.3.3 release (approved)
>> -
>>
>> Airflow Providers (RC1) (approved)
>> -
>>
>> July’s PR of the Month (approved)
>> -
>>
>> Release Airflow Python client 2.3.0 based on 2.3.0rc1 (pending)
>> -
>>
>> *Lazy consensus:*
>> -
>>
>> Deprecate “dependency detector” pluggability (passed)
>> -
>>
>> *Discussed:*
>> -
>>
>> “Magic Loop” as first-class citizen in Airflow
>> -
>>
>> Airflow scheduling delay metric definition
>> -
>>
>> Grouping of concerns
>> -
>>
>> Migrate Airflow code to use PEP 563 (postponed evaluation of
>> annotations)
>> -
>>
>> Standard way of getting provider version?
>> -
>>
>> Trimming down Airflow versions in issues
>> -
>>
>> Deprecating the SLA feature?
>> -
>>
>> Implementing a testing framework for Airflow
>> -
>>
>> Consolidate DAG scheduling params
>> -
>>
>> *Proposed:*
>> -
>>
>> Provider’s mixed governance model
>> -
>>
>> Simplification of WWW asset compilation for Breeze/dev env
>>
>>
>> -
>>
>> Airflow 2.4.0
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=d0f3d39055&e=ce16eef4ef>
>>
>> This release will focus on new features.
>>
>> -
>>
>> Airflow Helm Chart 1.7.0
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cfff66719b&e=ce16eef4ef>
>>
>> The next release for the Airflow Helm Chart, fixing bugs and adding more
>> features.
>> For more info about upcoming releases and in-development projects, visit
>> the current projects issue (#10176
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4c64a96866&e=ce16eef4ef>
>> ).
>>
>> The newsletter is moving to a new platform, Mailchimp. Please subscribe
>> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=05b5c570d8&e=ce16eef4ef>
>> to ensure you always receive the latest issue.
>> Is there an event or publication the community should know about? Let us
>> know at michael.robinson@astronomer.io <mi...@astronomer.io> or
>> julia@astronomer.io.
>> Want to change how you receive these emails?
>> You can update your preferences
>> <https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4>
>> or unsubscribe from this list
>> <https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4>
>> .
>>
>>
>>
RE: Apache Airflow Newsletter | July 2022
Posted by Adelaide Fleming <ad...@dosoffy.info>.
Let me know if there is a particular reason that this was scrapped? If I need to work around with my CFO on any pricing or other aspect I can push that to make it viable.
Just let me know the budget to get the approval from my CFO?
From: Jarek Potiuk <ja...@potiuk.com>
Sent: Saturday, July 30, 2022 1:04 AM
To: users@airflow.apache.org; Michael Robinson <mi...@astronomer.io>
Cc: dev@airflow.apache.org
Subject: Re: Apache Airflow Newsletter | July 2022
Nice!
On Fri, Jul 29, 2022 at 9:32 PM Michael Robinson via users <us...@airflow.apache.org>> wrote:
View this email in your browser<https://mailchi.mp/627a8910cc73/apache-airflow-newsletter-july-2022?e=ce16eef4ef>
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/940aaeeb-7d19-6946-b8e6-9c0d1f18a65c.png]
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/f09424c7-7b34-7252-53cb-e1ce49f98d4a.png]
This month in the Airflow community, we released Airflow 2.3.3<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1bd27974f1&e=ce16eef4ef> and over 40 Airflow Providers packages<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aaec9074c7&e=ce16eef4ef>, including the new Common SQL shared provider and the new Tabular provider. We learned and networked at an Airflow Meetup in Tel Aviv<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=51d4c8cc3e&e=ce16eef4ef> and Google’s Airflow Day 2022<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2f955a2dfd&e=ce16eef4ef>, and over on the dev list we discussed Dynamic Yield’s “Magic Loop”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7cf95cdc1d&e=ce16eef4ef> for minimizing delays caused by many concurrent dynamic DAGs, among other topics.
The newsletter is moving to a new platform, Mailchimp. Please subscribe<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=096e67c8c1&e=ce16eef4ef> to ensure you always receive the latest issue.
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/54e74767-a43b-478f-6d8b-4ccf389313cb.png]
We can always use speakers for meetups. If you want to present your Airflow contribution, demo your project, or share your experience at a Community Meetup this year, let us know<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e7fc2b5028&e=ce16eef4ef>. You could win some excellent swag!
* August 9: “Airflow 101: Essential Tips for Beginners”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=166096c917&e=ce16eef4ef>
* August 10: “Build a Robust Data Pipeline with Airflow, dbt, and Great Expectations – with Interactivity”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7dbc8d0b3e&e=ce16eef4ef>
* August 17: “Using Plugins for Webserver” with Faizan Qazi (details TBA on Clowder<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=88a3d50cba&e=ce16eef4ef>)
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/d3859580-85f5-1273-93e0-8e8ff1bdf4f8.png]
We released Airflow 2.3.3. You can find the build artifacts using these links:
* PyPI<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0d09446543&e=ce16eef4ef> | Notes<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e2c70b8cbd&e=ce16eef4ef> | Constraints<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=901d8a64b3&e=ce16eef4ef>
* Docker image: apache/airflow:2.3.3
We released Airflow Providers packages. You can find them using these links:
* Sources<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=22c14f066e&e=ce16eef4ef> | Guide<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=63b31b2c5a&e=ce16eef4ef>
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/9ded1a56-059e-3e72-3e4e-844bc636848b.png]
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/c92ef16e-00a0-de14-2552-97dd9bb47832.png]
@uranusjr <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=5143e93b2b&e=ce16eef4ef>
#24989 Implement expand_kwargs()<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=f4d747431b&e=ce16eef4ef>
Implements a new ExpandInput type, "list-of-dicts", to support this new shape of mapped kwargs.
To nominate a PR<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d9a79856a&e=ce16eef4ef> for a future issue, flag it by adding #protm to a comment.
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/7fc2ffcd-03ed-bcc8-b6d9-806670c45b90.png]
If you missed any of these, you can catch a recap, watch on-demand, or dig into the slides:
* July 28: “Apache Airflow TLV Meetup Summer Edition”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=48722ecbf9&e=ce16eef4ef>
* July 27: “Airflow, Spark, Trinio and Data Engineering Best Practices”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa33f35f0&e=ce16eef4ef>
* July 26: “Using Airflow with Tensorflow and MLFlow”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=31433f5b8d&e=ce16eef4ef>
* July 15: “Data Engineering: Bulk Up Your Data Eng Skills using Airflow, Spark & DASK”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=bc6157ff5d&e=ce16eef4ef>
* July 14: “Apache Airflow – Future of Data Workflows”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b776db73d0&e=ce16eef4ef>
* July 12: “Anatomy of an Operator”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa8403602&e=ce16eef4ef>
* July 7: "Airflow Day 2022 on Google Open Source Live"<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=052c3897b0&e=ce16eef4ef>
* July 6: “Introduction to Building Pipelines in GCP using Airflow”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ea6a2f2893&e=ce16eef4ef>
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/24fdcda5-566b-5d29-b4f0-ae039ef20802.png]
* “Is Airflow the Right Choice for Machine Learning, Too?,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6213fd2ecd&e=ce16eef4ef> by Samhita Alla, BetterProgramming/Medium
* “Deutsche Bank uses Cloud Composer Workload Automation,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9e495b03f8&e=ce16eef4ef>cnxtoday.com<http://cnxtoday.com/>
* “Shopify’s Practical Guidelines from Running Airflow for ML and Data Workflows at Scale,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=143cd0347e&e=ce16eef4ef> by Reza Rahimi, infoq.com<http://infoq.com/>
* “Magic Loop in Airflow – Reloaded,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c3257fbba3&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
* “Shared Volumes in Airflow – the Good, the Bad, and the Ugly,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9046873bce&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
* “Running Airflow on ARM M1/M2? Hell Yes, but Upgrade to 2.3+,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a782285305&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
* “Generic Airflow Transfers Made Easy,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=eeb6b8d1cb&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
* “Airflow’s Magic Loop,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cac6eafbf4&e=ce16eef4ef> by Itay Bittan, Towards Data Science/Medium
* “Load Data from Postgres to BigQuery with Airflow,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d83f3800a&e=ce16eef4ef> by Giorgos Myrianthous, Towards Data Science/Medium
* “Automated Alerts for Airflow from Slack,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aa3832265c&e=ce16eef4ef> by Chris Young, Towards Data Science/Medium
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/a5ea0304-201d-0f05-c808-243b63e92a28.png]
* Votes:
* Airflow 2.3.3rc1 release (approved)
* Airflow 2.3.3 release (approved)
* Airflow Providers (RC1) (approved)
* July’s PR of the Month (approved)
* Release Airflow Python client 2.3.0 based on 2.3.0rc1 (pending)
* Lazy consensus:
* Deprecate “dependency detector” pluggability (passed)
* Discussed:
* “Magic Loop” as first-class citizen in Airflow
* Airflow scheduling delay metric definition
* Grouping of concerns
* Migrate Airflow code to use PEP 563 (postponed evaluation of annotations)
* Standard way of getting provider version?
* Trimming down Airflow versions in issues
* Deprecating the SLA feature?
* Implementing a testing framework for Airflow
* Consolidate DAG scheduling params
* Proposed:
* Provider’s mixed governance model
* Simplification of WWW asset compilation for Breeze/dev env
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/7c616a13-898d-6342-a079-bd860959d6ac.png]
* Airflow 2.4.0<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=d0f3d39055&e=ce16eef4ef>
This release will focus on new features.
* Airflow Helm Chart 1.7.0<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cfff66719b&e=ce16eef4ef>
The next release for the Airflow Helm Chart, fixing bugs and adding more features.
For more info about upcoming releases and in-development projects, visit the current projects issue (#10176<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4c64a96866&e=ce16eef4ef>).
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/671f8dfd-92e0-ed40-9b4c-1ace301b8c4f.png]
The newsletter is moving to a new platform, Mailchimp. Please subscribe<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=05b5c570d8&e=ce16eef4ef> to ensure you always receive the latest issue.
Is there an event or publication the community should know about? Let us know at michael.robinson@astronomer.io<ma...@astronomer.io> or julia@astronomer.io<ma...@astronomer.io>.
Want to change how you receive these emails?
You can update your preferences<https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4> or unsubscribe from this list<https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4>.
Re: Apache Airflow Newsletter | July 2022
Posted by Curtis Stutz <cu...@gmail.com>.
Thank you
On Wed, Aug 3, 2022, 8:14 AM Valentin Nedkov <va...@viasattech.com>
wrote:
> This is a reply :)
>
> On 3 Aug 2022, at 16:07 , Adelaide Fleming <ad...@dosoffy.info>
> wrote:
>
> Hi, I have list please give me reply
>
> *From:* Jarek Potiuk <ja...@potiuk.com>
> *Sent:* Saturday, July 30, 2022 1:04 AM
> *To:* users@airflow.apache.org; Michael Robinson <
> michael.robinson@astronomer.io>
> *Cc:* dev@airflow.apache.org
> *Subject:* Re: Apache Airflow Newsletter | July 2022
>
> Nice!
>
> On Fri, Jul 29, 2022 at 9:32 PM Michael Robinson via users <
> users@airflow.apache.org> wrote:
>
> View this email in your browser
> <https://mailchi.mp/627a8910cc73/apache-airflow-newsletter-july-2022?e=ce16eef4ef>
>
>
>
>
> This month in the Airflow community, we released *Airflow 2.3.3*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1bd27974f1&e=ce16eef4ef> and
> over 40 *Airflow Providers packages*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aaec9074c7&e=ce16eef4ef>,
> including the new Common SQL shared provider and the new Tabular provider.
> We learned and networked at an *Airflow Meetup in Tel Aviv*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=51d4c8cc3e&e=ce16eef4ef> and
> Google’s *Airflow Day 2022*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2f955a2dfd&e=ce16eef4ef>,
> and over on the dev list we discussed *Dynamic Yield’s “Magic Loop”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7cf95cdc1d&e=ce16eef4ef> for
> minimizing delays caused by many concurrent dynamic DAGs, among other
> topics.
>
> The newsletter is moving to a new platform, Mailchimp. Please *subscribe*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=096e67c8c1&e=ce16eef4ef> to
> ensure you always receive the latest issue.
>
>
>
>
> We can always use speakers for meetups. If you want to present your
> Airflow contribution, demo your project, or share your experience at a
> Community Meetup this year, *let us know*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e7fc2b5028&e=ce16eef4ef>.
> You could win some excellent swag!
>
> - August 9: *“Airflow 101: Essential Tips for Beginners”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=166096c917&e=ce16eef4ef>
> - August 10: *“Build a Robust Data Pipeline with Airflow, dbt, and
> Great Expectations – with Interactivity”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7dbc8d0b3e&e=ce16eef4ef>
> - August 17: “Using Plugins for Webserver” with Faizan Qazi (details
> TBA on *Clowder*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=88a3d50cba&e=ce16eef4ef>
> )
>
>
>
> We released Airflow 2.3.3. You can find the build artifacts using these
> links:
>
> - *PyPI*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0d09446543&e=ce16eef4ef>
> | *Notes*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e2c70b8cbd&e=ce16eef4ef>
> | *Constraints*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=901d8a64b3&e=ce16eef4ef>
> - Docker image: apache/airflow:2.3.3
>
> We released Airflow Providers packages. You can find them using these
> links:
>
> - *Sources*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=22c14f066e&e=ce16eef4ef>
> | *Guide*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=63b31b2c5a&e=ce16eef4ef>
>
>
>
>
> *@uranusjr
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=5143e93b2b&e=ce16eef4ef>*
>
> *#24989 Implement expand_kwargs()
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=f4d747431b&e=ce16eef4ef>*
> Implements a new ExpandInput type, "list-of-dicts", to support this new
> shape of mapped kwargs.
>
> To nominate a *PR*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d9a79856a&e=ce16eef4ef> for
> a future issue, flag it by adding *#protm* to a comment.
>
>
>
> If you missed any of these, you can catch a recap, watch on-demand, or dig
> into the slides:
>
> - July 28: *“Apache Airflow TLV Meetup Summer Edition”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=48722ecbf9&e=ce16eef4ef>
> - July 27: *“Airflow, Spark, Trinio and Data Engineering Best
> Practices”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa33f35f0&e=ce16eef4ef>
>
> - July 26: *“Using Airflow with Tensorflow and MLFlow”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=31433f5b8d&e=ce16eef4ef>
> - July 15: *“Data Engineering: Bulk Up Your Data Eng Skills using
> Airflow, Spark & DASK”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=bc6157ff5d&e=ce16eef4ef>
> - July 14: *“Apache Airflow – Future of Data Workflows”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b776db73d0&e=ce16eef4ef>
> - July 12: *“Anatomy of an Operator”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa8403602&e=ce16eef4ef>
>
> - July 7: *"Airflow Day 2022 on Google Open Source Live"*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=052c3897b0&e=ce16eef4ef>
>
>
>
> - July 6: *“Introduction to Building Pipelines in GCP using Airflow”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ea6a2f2893&e=ce16eef4ef>
>
>
>
>
> - *“Is Airflow the Right Choice for Machine Learning, Too?,”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6213fd2ecd&e=ce16eef4ef> by
> Samhita Alla, BetterProgramming/Medium
> - *“Deutsche Bank uses Cloud Composer Workload Automation,”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9e495b03f8&e=ce16eef4ef>
> *cnxtoday.com* <http://cnxtoday.com/>
> - *“Shopify’s Practical Guidelines from Running Airflow for ML and
> Data Workflows at Scale,”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=143cd0347e&e=ce16eef4ef> by
> Reza Rahimi, *infoq.com* <http://infoq.com/>
> - *“Magic Loop in Airflow – Reloaded,”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c3257fbba3&e=ce16eef4ef> by
> Jarek Potiuk, Apache Airflow/Medium
> - *“Shared Volumes in Airflow – the Good, the Bad, and the Ugly,”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9046873bce&e=ce16eef4ef> by
> Jarek Potiuk, Apache Airflow/Medium
> - *“Running Airflow on ARM M1/M2? Hell Yes, but Upgrade to 2.3+,”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a782285305&e=ce16eef4ef> by
> Jarek Potiuk, Apache Airflow/Medium
> - *“Generic Airflow Transfers Made Easy,”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=eeb6b8d1cb&e=ce16eef4ef> by
> Jarek Potiuk, Apache Airflow/Medium
> - *“Airflow’s Magic Loop,”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cac6eafbf4&e=ce16eef4ef> by
> Itay Bittan, Towards Data Science/Medium
> - *“Load Data from Postgres to BigQuery with Airflow,”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d83f3800a&e=ce16eef4ef> by
> Giorgos Myrianthous, Towards Data Science/Medium
> - *“Automated Alerts for Airflow from Slack,”*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aa3832265c&e=ce16eef4ef> by
> Chris Young, Towards Data Science/Medium
>
>
>
>
> - *Votes: *
> - Airflow 2.3.3rc1 release (approved)
> - Airflow 2.3.3 release (approved)
> - Airflow Providers (RC1) (approved)
> - July’s PR of the Month (approved)
> - Release Airflow Python client 2.3.0 based on 2.3.0rc1 (pending)
> - *Lazy consensus:*
> - Deprecate “dependency detector” pluggability (passed)
> - *Discussed:*
> - “Magic Loop” as first-class citizen in Airflow
> - Airflow scheduling delay metric definition
> - Grouping of concerns
> - Migrate Airflow code to use PEP 563 (postponed evaluation of
> annotations)
> - Standard way of getting provider version?
> - Trimming down Airflow versions in issues
> - Deprecating the SLA feature?
> - Implementing a testing framework for Airflow
> - Consolidate DAG scheduling params
> - *Proposed:*
> - Provider’s mixed governance model
> - Simplification of WWW asset compilation for Breeze/dev env
>
>
>
>
> - *Airflow 2.4.0*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=d0f3d39055&e=ce16eef4ef>
>
> This release will focus on new features.
>
> - *Airflow Helm Chart 1.7.0*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cfff66719b&e=ce16eef4ef>
>
> The next release for the Airflow Helm Chart, fixing bugs and adding more
> features.
> For more info about upcoming releases and in-development projects, visit
> the current projects issue (*#10176*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4c64a96866&e=ce16eef4ef>
> ).
>
>
>
> The newsletter is moving to a new platform, Mailchimp. Please *subscribe*
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=05b5c570d8&e=ce16eef4ef> to
> ensure you always receive the latest issue.
> Is there an event or publication the community should know about? Let us
> know at *michael.robinson@astronomer.io* <mi...@astronomer.io>
> or *julia@astronomer.io* <ju...@astronomer.io>.
> Want to change how you receive these emails?
> You can update your preferences
> <https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4>
> or unsubscribe from this list
> <https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4>
> .
>
>
>
Re: Apache Airflow Newsletter | July 2022
Posted by Valentin Nedkov <va...@viasattech.com>.
This is a reply :)
> On 3 Aug 2022, at 16:07 , Adelaide Fleming <ad...@dosoffy.info> wrote:
>
> Hi, I have list please give me reply
>
> From: Jarek Potiuk <jarek@potiuk.com <ma...@potiuk.com>>
> Sent: Saturday, July 30, 2022 1:04 AM
> To: users@airflow.apache.org <ma...@airflow.apache.org>; Michael Robinson <michael.robinson@astronomer.io <ma...@astronomer.io>>
> Cc: dev@airflow.apache.org <ma...@airflow.apache.org>
> Subject: Re: Apache Airflow Newsletter | July 2022
>
> Nice!
>
> On Fri, Jul 29, 2022 at 9:32 PM Michael Robinson via users <users@airflow.apache.org <ma...@airflow.apache.org>> wrote:
> View this email in your browser <https://mailchi.mp/627a8910cc73/apache-airflow-newsletter-july-2022?e=ce16eef4ef>
>
>
>
>
>
> This month in the Airflow community, we released Airflow 2.3.3 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1bd27974f1&e=ce16eef4ef> and over 40 Airflow Providers packages <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aaec9074c7&e=ce16eef4ef>, including the new Common SQL shared provider and the new Tabular provider. We learned and networked at an Airflow Meetup in Tel Aviv <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=51d4c8cc3e&e=ce16eef4ef> and Google’s Airflow Day 2022 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2f955a2dfd&e=ce16eef4ef>, and over on the dev list we discussed Dynamic Yield’s “Magic Loop” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7cf95cdc1d&e=ce16eef4ef> for minimizing delays caused by many concurrent dynamic DAGs, among other topics.
>
> The newsletter is moving to a new platform, Mailchimp. Please subscribe <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=096e67c8c1&e=ce16eef4ef> to ensure you always receive the latest issue.
>
>
>
>
>
> We can always use speakers for meetups. If you want to present your Airflow contribution, demo your project, or share your experience at a Community Meetup this year, let us know <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e7fc2b5028&e=ce16eef4ef>. You could win some excellent swag!
>
> August 9: “Airflow 101: Essential Tips for Beginners” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=166096c917&e=ce16eef4ef>
> August 10: “Build a Robust Data Pipeline with Airflow, dbt, and Great Expectations – with Interactivity” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7dbc8d0b3e&e=ce16eef4ef>
> August 17: “Using Plugins for Webserver” with Faizan Qazi (details TBA on Clowder <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=88a3d50cba&e=ce16eef4ef>)
>
>
>
> We released Airflow 2.3.3. You can find the build artifacts using these links:
> PyPI <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0d09446543&e=ce16eef4ef> | Notes <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e2c70b8cbd&e=ce16eef4ef> | Constraints <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=901d8a64b3&e=ce16eef4ef>
> Docker image: apache/airflow:2.3.3
> We released Airflow Providers packages. You can find them using these links:
> Sources <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=22c14f066e&e=ce16eef4ef> | Guide <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=63b31b2c5a&e=ce16eef4ef>
>
>
>
>
> @uranusjr <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=5143e93b2b&e=ce16eef4ef>
> #24989 Implement expand_kwargs() <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=f4d747431b&e=ce16eef4ef>
> Implements a new ExpandInput type, "list-of-dicts", to support this new shape of mapped kwargs.
>
>
> To nominate a PR <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d9a79856a&e=ce16eef4ef> for a future issue, flag it by adding #protm to a comment.
>
>
>
> If you missed any of these, you can catch a recap, watch on-demand, or dig into the slides:
>
> July 28: “Apache Airflow TLV Meetup Summer Edition” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=48722ecbf9&e=ce16eef4ef>
> July 27: “Airflow, Spark, Trinio and Data Engineering Best Practices” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa33f35f0&e=ce16eef4ef>
> July 26: “Using Airflow with Tensorflow and MLFlow” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=31433f5b8d&e=ce16eef4ef>
> July 15: “Data Engineering: Bulk Up Your Data Eng Skills using Airflow, Spark & DASK” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=bc6157ff5d&e=ce16eef4ef>
> July 14: “Apache Airflow – Future of Data Workflows” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b776db73d0&e=ce16eef4ef>
> July 12: “Anatomy of an Operator” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa8403602&e=ce16eef4ef>
> July 7: "Airflow Day 2022 on Google Open Source Live" <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=052c3897b0&e=ce16eef4ef>
> July 6: “Introduction to Building Pipelines in GCP using Airflow” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ea6a2f2893&e=ce16eef4ef>
>
>
>
> “Is Airflow the Right Choice for Machine Learning, Too?,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6213fd2ecd&e=ce16eef4ef> by Samhita Alla, BetterProgramming/Medium
> “Deutsche Bank uses Cloud Composer Workload Automation,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9e495b03f8&e=ce16eef4ef>cnxtoday.com <http://cnxtoday.com/>
> “Shopify’s Practical Guidelines from Running Airflow for ML and Data Workflows at Scale,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=143cd0347e&e=ce16eef4ef> by Reza Rahimi, infoq.com <http://infoq.com/>
> “Magic Loop in Airflow – Reloaded,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c3257fbba3&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
> “Shared Volumes in Airflow – the Good, the Bad, and the Ugly,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9046873bce&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
> “Running Airflow on ARM M1/M2? Hell Yes, but Upgrade to 2.3+,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a782285305&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
> “Generic Airflow Transfers Made Easy,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=eeb6b8d1cb&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
> “Airflow’s Magic Loop,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cac6eafbf4&e=ce16eef4ef> by Itay Bittan, Towards Data Science/Medium
> “Load Data from Postgres to BigQuery with Airflow,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d83f3800a&e=ce16eef4ef> by Giorgos Myrianthous, Towards Data Science/Medium
> “Automated Alerts for Airflow from Slack,” <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aa3832265c&e=ce16eef4ef> by Chris Young, Towards Data Science/Medium
>
>
>
> Votes:
> Airflow 2.3.3rc1 release (approved)
> Airflow 2.3.3 release (approved)
> Airflow Providers (RC1) (approved)
> July’s PR of the Month (approved)
> Release Airflow Python client 2.3.0 based on 2.3.0rc1 (pending)
> Lazy consensus:
> Deprecate “dependency detector” pluggability (passed)
> Discussed:
> “Magic Loop” as first-class citizen in Airflow
> Airflow scheduling delay metric definition
> Grouping of concerns
> Migrate Airflow code to use PEP 563 (postponed evaluation of annotations)
> Standard way of getting provider version?
> Trimming down Airflow versions in issues
> Deprecating the SLA feature?
> Implementing a testing framework for Airflow
> Consolidate DAG scheduling params
> Proposed:
> Provider’s mixed governance model
> Simplification of WWW asset compilation for Breeze/dev env
>
>
>
> Airflow 2.4.0 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=d0f3d39055&e=ce16eef4ef>
> This release will focus on new features.
>
> Airflow Helm Chart 1.7.0 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cfff66719b&e=ce16eef4ef>
> The next release for the Airflow Helm Chart, fixing bugs and adding more features.
>
> For more info about upcoming releases and in-development projects, visit the current projects issue (#10176 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4c64a96866&e=ce16eef4ef>).
>
>
>
> The newsletter is moving to a new platform, Mailchimp. Please subscribe <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=05b5c570d8&e=ce16eef4ef> to ensure you always receive the latest issue.
>
> Is there an event or publication the community should know about? Let us know at michael.robinson@astronomer.io <ma...@astronomer.io> or julia@astronomer.io <ma...@astronomer.io>.
> Want to change how you receive these emails?
> You can update your preferences <https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4> or unsubscribe from this list <https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4>.
>
RE: Apache Airflow Newsletter | July 2022
Posted by Adelaide Fleming <ad...@dosoffy.info>.
Hi, I have list please give me reply
From: Jarek Potiuk <ja...@potiuk.com>
Sent: Saturday, July 30, 2022 1:04 AM
To: users@airflow.apache.org; Michael Robinson <mi...@astronomer.io>
Cc: dev@airflow.apache.org
Subject: Re: Apache Airflow Newsletter | July 2022
Nice!
On Fri, Jul 29, 2022 at 9:32 PM Michael Robinson via users <us...@airflow.apache.org>> wrote:
View this email in your browser<https://mailchi.mp/627a8910cc73/apache-airflow-newsletter-july-2022?e=ce16eef4ef>
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/940aaeeb-7d19-6946-b8e6-9c0d1f18a65c.png]
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/f09424c7-7b34-7252-53cb-e1ce49f98d4a.png]
This month in the Airflow community, we released Airflow 2.3.3<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1bd27974f1&e=ce16eef4ef> and over 40 Airflow Providers packages<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aaec9074c7&e=ce16eef4ef>, including the new Common SQL shared provider and the new Tabular provider. We learned and networked at an Airflow Meetup in Tel Aviv<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=51d4c8cc3e&e=ce16eef4ef> and Google’s Airflow Day 2022<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2f955a2dfd&e=ce16eef4ef>, and over on the dev list we discussed Dynamic Yield’s “Magic Loop”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7cf95cdc1d&e=ce16eef4ef> for minimizing delays caused by many concurrent dynamic DAGs, among other topics.
The newsletter is moving to a new platform, Mailchimp. Please subscribe<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=096e67c8c1&e=ce16eef4ef> to ensure you always receive the latest issue.
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/54e74767-a43b-478f-6d8b-4ccf389313cb.png]
We can always use speakers for meetups. If you want to present your Airflow contribution, demo your project, or share your experience at a Community Meetup this year, let us know<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e7fc2b5028&e=ce16eef4ef>. You could win some excellent swag!
* August 9: “Airflow 101: Essential Tips for Beginners”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=166096c917&e=ce16eef4ef>
* August 10: “Build a Robust Data Pipeline with Airflow, dbt, and Great Expectations – with Interactivity”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7dbc8d0b3e&e=ce16eef4ef>
* August 17: “Using Plugins for Webserver” with Faizan Qazi (details TBA on Clowder<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=88a3d50cba&e=ce16eef4ef>)
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/d3859580-85f5-1273-93e0-8e8ff1bdf4f8.png]
We released Airflow 2.3.3. You can find the build artifacts using these links:
* PyPI<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0d09446543&e=ce16eef4ef> | Notes<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e2c70b8cbd&e=ce16eef4ef> | Constraints<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=901d8a64b3&e=ce16eef4ef>
* Docker image: apache/airflow:2.3.3
We released Airflow Providers packages. You can find them using these links:
* Sources<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=22c14f066e&e=ce16eef4ef> | Guide<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=63b31b2c5a&e=ce16eef4ef>
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/9ded1a56-059e-3e72-3e4e-844bc636848b.png]
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/c92ef16e-00a0-de14-2552-97dd9bb47832.png]
@uranusjr <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=5143e93b2b&e=ce16eef4ef>
#24989 Implement expand_kwargs()<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=f4d747431b&e=ce16eef4ef>
Implements a new ExpandInput type, "list-of-dicts", to support this new shape of mapped kwargs.
To nominate a PR<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d9a79856a&e=ce16eef4ef> for a future issue, flag it by adding #protm to a comment.
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/7fc2ffcd-03ed-bcc8-b6d9-806670c45b90.png]
If you missed any of these, you can catch a recap, watch on-demand, or dig into the slides:
* July 28: “Apache Airflow TLV Meetup Summer Edition”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=48722ecbf9&e=ce16eef4ef>
* July 27: “Airflow, Spark, Trinio and Data Engineering Best Practices”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa33f35f0&e=ce16eef4ef>
* July 26: “Using Airflow with Tensorflow and MLFlow”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=31433f5b8d&e=ce16eef4ef>
* July 15: “Data Engineering: Bulk Up Your Data Eng Skills using Airflow, Spark & DASK”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=bc6157ff5d&e=ce16eef4ef>
* July 14: “Apache Airflow – Future of Data Workflows”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b776db73d0&e=ce16eef4ef>
* July 12: “Anatomy of an Operator”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa8403602&e=ce16eef4ef>
* July 7: "Airflow Day 2022 on Google Open Source Live"<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=052c3897b0&e=ce16eef4ef>
* July 6: “Introduction to Building Pipelines in GCP using Airflow”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ea6a2f2893&e=ce16eef4ef>
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/24fdcda5-566b-5d29-b4f0-ae039ef20802.png]
* “Is Airflow the Right Choice for Machine Learning, Too?,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6213fd2ecd&e=ce16eef4ef> by Samhita Alla, BetterProgramming/Medium
* “Deutsche Bank uses Cloud Composer Workload Automation,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9e495b03f8&e=ce16eef4ef>cnxtoday.com<http://cnxtoday.com/>
* “Shopify’s Practical Guidelines from Running Airflow for ML and Data Workflows at Scale,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=143cd0347e&e=ce16eef4ef> by Reza Rahimi, infoq.com<http://infoq.com/>
* “Magic Loop in Airflow – Reloaded,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c3257fbba3&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
* “Shared Volumes in Airflow – the Good, the Bad, and the Ugly,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9046873bce&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
* “Running Airflow on ARM M1/M2? Hell Yes, but Upgrade to 2.3+,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a782285305&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
* “Generic Airflow Transfers Made Easy,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=eeb6b8d1cb&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium
* “Airflow’s Magic Loop,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cac6eafbf4&e=ce16eef4ef> by Itay Bittan, Towards Data Science/Medium
* “Load Data from Postgres to BigQuery with Airflow,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d83f3800a&e=ce16eef4ef> by Giorgos Myrianthous, Towards Data Science/Medium
* “Automated Alerts for Airflow from Slack,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aa3832265c&e=ce16eef4ef> by Chris Young, Towards Data Science/Medium
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/a5ea0304-201d-0f05-c808-243b63e92a28.png]
* Votes:
* Airflow 2.3.3rc1 release (approved)
* Airflow 2.3.3 release (approved)
* Airflow Providers (RC1) (approved)
* July’s PR of the Month (approved)
* Release Airflow Python client 2.3.0 based on 2.3.0rc1 (pending)
* Lazy consensus:
* Deprecate “dependency detector” pluggability (passed)
* Discussed:
* “Magic Loop” as first-class citizen in Airflow
* Airflow scheduling delay metric definition
* Grouping of concerns
* Migrate Airflow code to use PEP 563 (postponed evaluation of annotations)
* Standard way of getting provider version?
* Trimming down Airflow versions in issues
* Deprecating the SLA feature?
* Implementing a testing framework for Airflow
* Consolidate DAG scheduling params
* Proposed:
* Provider’s mixed governance model
* Simplification of WWW asset compilation for Breeze/dev env
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/7c616a13-898d-6342-a079-bd860959d6ac.png]
* Airflow 2.4.0<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=d0f3d39055&e=ce16eef4ef>
This release will focus on new features.
* Airflow Helm Chart 1.7.0<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cfff66719b&e=ce16eef4ef>
The next release for the Airflow Helm Chart, fixing bugs and adding more features.
For more info about upcoming releases and in-development projects, visit the current projects issue (#10176<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4c64a96866&e=ce16eef4ef>).
[https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/671f8dfd-92e0-ed40-9b4c-1ace301b8c4f.png]
The newsletter is moving to a new platform, Mailchimp. Please subscribe<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=05b5c570d8&e=ce16eef4ef> to ensure you always receive the latest issue.
Is there an event or publication the community should know about? Let us know at michael.robinson@astronomer.io<ma...@astronomer.io> or julia@astronomer.io<ma...@astronomer.io>.
Want to change how you receive these emails?
You can update your preferences<https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4> or unsubscribe from this list<https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4>.
Re: Apache Airflow Newsletter | July 2022
Posted by Jarek Potiuk <ja...@potiuk.com>.
Nice!
On Fri, Jul 29, 2022 at 9:32 PM Michael Robinson via users <
users@airflow.apache.org> wrote:
> View this email in your browser
> <https://mailchi.mp/627a8910cc73/apache-airflow-newsletter-july-2022?e=ce16eef4ef>
>
> This month in the Airflow community, we released Airflow 2.3.3
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1bd27974f1&e=ce16eef4ef>
> and over 40 Airflow Providers packages
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aaec9074c7&e=ce16eef4ef>,
> including the new Common SQL shared provider and the new Tabular provider.
> We learned and networked at an Airflow Meetup in Tel Aviv
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=51d4c8cc3e&e=ce16eef4ef>
> and Google’s Airflow Day 2022
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2f955a2dfd&e=ce16eef4ef>,
> and over on the dev list we discussed Dynamic Yield’s “Magic Loop”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7cf95cdc1d&e=ce16eef4ef>
> for minimizing delays caused by many concurrent dynamic DAGs, among
> other topics.
>
> The newsletter is moving to a new platform, Mailchimp. Please subscribe
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=096e67c8c1&e=ce16eef4ef>
> to ensure you always receive the latest issue.
>
> We can always use speakers for meetups. If you want to present your
> Airflow contribution, demo your project, or share your experience at a
> Community Meetup this year, let us know
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e7fc2b5028&e=ce16eef4ef>.
> You could win some excellent swag!
>
> -
>
> August 9: “Airflow 101: Essential Tips for Beginners”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=166096c917&e=ce16eef4ef>
> -
>
> August 10: “Build a Robust Data Pipeline with Airflow, dbt, and Great
> Expectations – with Interactivity”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7dbc8d0b3e&e=ce16eef4ef>
> -
>
> August 17: “Using Plugins for Webserver” with Faizan Qazi (details TBA
> on Clowder
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=88a3d50cba&e=ce16eef4ef>
> )
>
> We released Airflow 2.3.3. You can find the build artifacts using these
> links:
>
> - PyPI
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0d09446543&e=ce16eef4ef>
> | Notes
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e2c70b8cbd&e=ce16eef4ef>
> | Constraints
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=901d8a64b3&e=ce16eef4ef>
> - Docker image: apache/airflow:2.3.3
>
> We released Airflow Providers packages. You can find them using these
> links:
>
> - Sources
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=22c14f066e&e=ce16eef4ef>
> | Guide
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=63b31b2c5a&e=ce16eef4ef>
>
> *@uranusjr
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=5143e93b2b&e=ce16eef4ef>*
>
> *#24989 Implement expand_kwargs()
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=f4d747431b&e=ce16eef4ef>*
> Implements a new ExpandInput type, "list-of-dicts", to support this new
> shape of mapped kwargs.
> To nominate a PR
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d9a79856a&e=ce16eef4ef>
> for a future issue, flag it by adding *#protm* to a comment.
>
> If you missed any of these, you can catch a recap, watch on-demand, or dig
> into the slides:
>
> -
>
> July 28: “Apache Airflow TLV Meetup Summer Edition”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=48722ecbf9&e=ce16eef4ef>
> -
>
> July 27: “Airflow, Spark, Trinio and Data Engineering Best Practices”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa33f35f0&e=ce16eef4ef>
>
> -
>
> July 26: “Using Airflow with Tensorflow and MLFlow”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=31433f5b8d&e=ce16eef4ef>
> -
>
> July 15: “Data Engineering: Bulk Up Your Data Eng Skills using
> Airflow, Spark & DASK”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=bc6157ff5d&e=ce16eef4ef>
> -
>
> July 14: “Apache Airflow – Future of Data Workflows”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b776db73d0&e=ce16eef4ef>
> -
>
> July 12: “Anatomy of an Operator”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa8403602&e=ce16eef4ef>
>
> -
>
> July 7: "Airflow Day 2022 on Google Open Source Live"
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=052c3897b0&e=ce16eef4ef>
>
>
>
> -
>
> July 6: “Introduction to Building Pipelines in GCP using Airflow”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ea6a2f2893&e=ce16eef4ef>
>
>
> -
>
> “Is Airflow the Right Choice for Machine Learning, Too?,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6213fd2ecd&e=ce16eef4ef>
> by Samhita Alla, BetterProgramming/Medium
> -
>
> “Deutsche Bank uses Cloud Composer Workload Automation,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9e495b03f8&e=ce16eef4ef>
> cnxtoday.com
> -
>
> “Shopify’s Practical Guidelines from Running Airflow for ML and Data
> Workflows at Scale,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=143cd0347e&e=ce16eef4ef>
> by Reza Rahimi, infoq.com
> -
>
> “Magic Loop in Airflow – Reloaded,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c3257fbba3&e=ce16eef4ef>
> by Jarek Potiuk, Apache Airflow/Medium
> -
>
> “Shared Volumes in Airflow – the Good, the Bad, and the Ugly,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9046873bce&e=ce16eef4ef>
> by Jarek Potiuk, Apache Airflow/Medium
> -
>
> “Running Airflow on ARM M1/M2? Hell Yes, but Upgrade to 2.3+,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a782285305&e=ce16eef4ef>
> by Jarek Potiuk, Apache Airflow/Medium
> -
>
> “Generic Airflow Transfers Made Easy,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=eeb6b8d1cb&e=ce16eef4ef>
> by Jarek Potiuk, Apache Airflow/Medium
> -
>
> “Airflow’s Magic Loop,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cac6eafbf4&e=ce16eef4ef>
> by Itay Bittan, Towards Data Science/Medium
> -
>
> “Load Data from Postgres to BigQuery with Airflow,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d83f3800a&e=ce16eef4ef>
> by Giorgos Myrianthous, Towards Data Science/Medium
> -
>
> “Automated Alerts for Airflow from Slack,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aa3832265c&e=ce16eef4ef>
> by Chris Young, Towards Data Science/Medium
>
>
> -
>
> *Votes: *
> -
>
> Airflow 2.3.3rc1 release (approved)
> -
>
> Airflow 2.3.3 release (approved)
> -
>
> Airflow Providers (RC1) (approved)
> -
>
> July’s PR of the Month (approved)
> -
>
> Release Airflow Python client 2.3.0 based on 2.3.0rc1 (pending)
> -
>
> *Lazy consensus:*
> -
>
> Deprecate “dependency detector” pluggability (passed)
> -
>
> *Discussed:*
> -
>
> “Magic Loop” as first-class citizen in Airflow
> -
>
> Airflow scheduling delay metric definition
> -
>
> Grouping of concerns
> -
>
> Migrate Airflow code to use PEP 563 (postponed evaluation of
> annotations)
> -
>
> Standard way of getting provider version?
> -
>
> Trimming down Airflow versions in issues
> -
>
> Deprecating the SLA feature?
> -
>
> Implementing a testing framework for Airflow
> -
>
> Consolidate DAG scheduling params
> -
>
> *Proposed:*
> -
>
> Provider’s mixed governance model
> -
>
> Simplification of WWW asset compilation for Breeze/dev env
>
>
> -
>
> Airflow 2.4.0
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=d0f3d39055&e=ce16eef4ef>
>
> This release will focus on new features.
>
> -
>
> Airflow Helm Chart 1.7.0
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cfff66719b&e=ce16eef4ef>
>
> The next release for the Airflow Helm Chart, fixing bugs and adding more
> features.
> For more info about upcoming releases and in-development projects, visit
> the current projects issue (#10176
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4c64a96866&e=ce16eef4ef>
> ).
>
> The newsletter is moving to a new platform, Mailchimp. Please subscribe
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=05b5c570d8&e=ce16eef4ef>
> to ensure you always receive the latest issue.
> Is there an event or publication the community should know about? Let us
> know at michael.robinson@astronomer.io <mi...@astronomer.io> or
> julia@astronomer.io.
> Want to change how you receive these emails?
> You can update your preferences
> <https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4>
> or unsubscribe from this list
> <https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4>
> .
>
>
>
Re: Apache Airflow Newsletter | July 2022
Posted by Jarek Potiuk <ja...@potiuk.com>.
Nice!
On Fri, Jul 29, 2022 at 9:32 PM Michael Robinson via users <
users@airflow.apache.org> wrote:
> View this email in your browser
> <https://mailchi.mp/627a8910cc73/apache-airflow-newsletter-july-2022?e=ce16eef4ef>
>
> This month in the Airflow community, we released Airflow 2.3.3
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1bd27974f1&e=ce16eef4ef>
> and over 40 Airflow Providers packages
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aaec9074c7&e=ce16eef4ef>,
> including the new Common SQL shared provider and the new Tabular provider.
> We learned and networked at an Airflow Meetup in Tel Aviv
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=51d4c8cc3e&e=ce16eef4ef>
> and Google’s Airflow Day 2022
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2f955a2dfd&e=ce16eef4ef>,
> and over on the dev list we discussed Dynamic Yield’s “Magic Loop”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7cf95cdc1d&e=ce16eef4ef>
> for minimizing delays caused by many concurrent dynamic DAGs, among
> other topics.
>
> The newsletter is moving to a new platform, Mailchimp. Please subscribe
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=096e67c8c1&e=ce16eef4ef>
> to ensure you always receive the latest issue.
>
> We can always use speakers for meetups. If you want to present your
> Airflow contribution, demo your project, or share your experience at a
> Community Meetup this year, let us know
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e7fc2b5028&e=ce16eef4ef>.
> You could win some excellent swag!
>
> -
>
> August 9: “Airflow 101: Essential Tips for Beginners”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=166096c917&e=ce16eef4ef>
> -
>
> August 10: “Build a Robust Data Pipeline with Airflow, dbt, and Great
> Expectations – with Interactivity”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7dbc8d0b3e&e=ce16eef4ef>
> -
>
> August 17: “Using Plugins for Webserver” with Faizan Qazi (details TBA
> on Clowder
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=88a3d50cba&e=ce16eef4ef>
> )
>
> We released Airflow 2.3.3. You can find the build artifacts using these
> links:
>
> - PyPI
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0d09446543&e=ce16eef4ef>
> | Notes
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e2c70b8cbd&e=ce16eef4ef>
> | Constraints
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=901d8a64b3&e=ce16eef4ef>
> - Docker image: apache/airflow:2.3.3
>
> We released Airflow Providers packages. You can find them using these
> links:
>
> - Sources
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=22c14f066e&e=ce16eef4ef>
> | Guide
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=63b31b2c5a&e=ce16eef4ef>
>
> *@uranusjr
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=5143e93b2b&e=ce16eef4ef>*
>
> *#24989 Implement expand_kwargs()
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=f4d747431b&e=ce16eef4ef>*
> Implements a new ExpandInput type, "list-of-dicts", to support this new
> shape of mapped kwargs.
> To nominate a PR
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d9a79856a&e=ce16eef4ef>
> for a future issue, flag it by adding *#protm* to a comment.
>
> If you missed any of these, you can catch a recap, watch on-demand, or dig
> into the slides:
>
> -
>
> July 28: “Apache Airflow TLV Meetup Summer Edition”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=48722ecbf9&e=ce16eef4ef>
> -
>
> July 27: “Airflow, Spark, Trinio and Data Engineering Best Practices”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa33f35f0&e=ce16eef4ef>
>
> -
>
> July 26: “Using Airflow with Tensorflow and MLFlow”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=31433f5b8d&e=ce16eef4ef>
> -
>
> July 15: “Data Engineering: Bulk Up Your Data Eng Skills using
> Airflow, Spark & DASK”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=bc6157ff5d&e=ce16eef4ef>
> -
>
> July 14: “Apache Airflow – Future of Data Workflows”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b776db73d0&e=ce16eef4ef>
> -
>
> July 12: “Anatomy of an Operator”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa8403602&e=ce16eef4ef>
>
> -
>
> July 7: "Airflow Day 2022 on Google Open Source Live"
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=052c3897b0&e=ce16eef4ef>
>
>
>
> -
>
> July 6: “Introduction to Building Pipelines in GCP using Airflow”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ea6a2f2893&e=ce16eef4ef>
>
>
> -
>
> “Is Airflow the Right Choice for Machine Learning, Too?,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6213fd2ecd&e=ce16eef4ef>
> by Samhita Alla, BetterProgramming/Medium
> -
>
> “Deutsche Bank uses Cloud Composer Workload Automation,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9e495b03f8&e=ce16eef4ef>
> cnxtoday.com
> -
>
> “Shopify’s Practical Guidelines from Running Airflow for ML and Data
> Workflows at Scale,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=143cd0347e&e=ce16eef4ef>
> by Reza Rahimi, infoq.com
> -
>
> “Magic Loop in Airflow – Reloaded,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c3257fbba3&e=ce16eef4ef>
> by Jarek Potiuk, Apache Airflow/Medium
> -
>
> “Shared Volumes in Airflow – the Good, the Bad, and the Ugly,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9046873bce&e=ce16eef4ef>
> by Jarek Potiuk, Apache Airflow/Medium
> -
>
> “Running Airflow on ARM M1/M2? Hell Yes, but Upgrade to 2.3+,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a782285305&e=ce16eef4ef>
> by Jarek Potiuk, Apache Airflow/Medium
> -
>
> “Generic Airflow Transfers Made Easy,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=eeb6b8d1cb&e=ce16eef4ef>
> by Jarek Potiuk, Apache Airflow/Medium
> -
>
> “Airflow’s Magic Loop,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cac6eafbf4&e=ce16eef4ef>
> by Itay Bittan, Towards Data Science/Medium
> -
>
> “Load Data from Postgres to BigQuery with Airflow,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d83f3800a&e=ce16eef4ef>
> by Giorgos Myrianthous, Towards Data Science/Medium
> -
>
> “Automated Alerts for Airflow from Slack,”
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aa3832265c&e=ce16eef4ef>
> by Chris Young, Towards Data Science/Medium
>
>
> -
>
> *Votes: *
> -
>
> Airflow 2.3.3rc1 release (approved)
> -
>
> Airflow 2.3.3 release (approved)
> -
>
> Airflow Providers (RC1) (approved)
> -
>
> July’s PR of the Month (approved)
> -
>
> Release Airflow Python client 2.3.0 based on 2.3.0rc1 (pending)
> -
>
> *Lazy consensus:*
> -
>
> Deprecate “dependency detector” pluggability (passed)
> -
>
> *Discussed:*
> -
>
> “Magic Loop” as first-class citizen in Airflow
> -
>
> Airflow scheduling delay metric definition
> -
>
> Grouping of concerns
> -
>
> Migrate Airflow code to use PEP 563 (postponed evaluation of
> annotations)
> -
>
> Standard way of getting provider version?
> -
>
> Trimming down Airflow versions in issues
> -
>
> Deprecating the SLA feature?
> -
>
> Implementing a testing framework for Airflow
> -
>
> Consolidate DAG scheduling params
> -
>
> *Proposed:*
> -
>
> Provider’s mixed governance model
> -
>
> Simplification of WWW asset compilation for Breeze/dev env
>
>
> -
>
> Airflow 2.4.0
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=d0f3d39055&e=ce16eef4ef>
>
> This release will focus on new features.
>
> -
>
> Airflow Helm Chart 1.7.0
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cfff66719b&e=ce16eef4ef>
>
> The next release for the Airflow Helm Chart, fixing bugs and adding more
> features.
> For more info about upcoming releases and in-development projects, visit
> the current projects issue (#10176
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4c64a96866&e=ce16eef4ef>
> ).
>
> The newsletter is moving to a new platform, Mailchimp. Please subscribe
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=05b5c570d8&e=ce16eef4ef>
> to ensure you always receive the latest issue.
> Is there an event or publication the community should know about? Let us
> know at michael.robinson@astronomer.io <mi...@astronomer.io> or
> julia@astronomer.io.
> Want to change how you receive these emails?
> You can update your preferences
> <https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4>
> or unsubscribe from this list
> <https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4>
> .
>
>
>