You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Ash Berlin-Taylor <as...@apache.org> on 2020/11/09 23:00:02 UTC
Apache Airflow 2.0.0beta1 and providers 1.0.0beta1 available
for testing.
I'm proud to announce the availability of Apache Airlow 2.0.0.beta1 for testing.
This now marks us being "feature complete" for core Airflow -- from now on any significant new features to core Airflow will have to wait until 2.1 -- but thanks to the split out of provider packages those can continue to be developed.
This "snapshot" is intended for members of the Airflow developer community to test the build and get an early start on testing 2.0.0. For clarity, this is not an official release of Apache Airflow either - that doesn't happen until we make a release candidate and then vote on that, and based on the expected timelines on the Airflow 2.0 planning page (https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0+-+Planning), we expect that to happen the week of 7th December, 2020.
Apache Airflow
The beta snapshot is available at:
https://dist.apache.org/repos/dist/dev/airflow/2.0.0b1 (https://dist.apache.org/repos/dist/dev/airflow/2.0.0b1/)/ (https://dist.apache.org/repos/dist/dev/airflow/2.0.0a1/)
*apache-airflow-2.0.0b1-source.tar.gz* is a source release that comes with INSTALL instructions.
*apache-airflow-2.0.0b1-bin.tar.gz* is the binary Python "sdist" snapshot.
*apache_airflow-2.0.0b1-py3-none-any.whl* is the binary Python wheel snapshot.
This snapshot has been pushed to PyPi.
Major features since alpha2:
* "Decorated Flow" renamed to "TaskFlow API" (we should be sticking with this name now)
* Further significant performance improvements to CeleryExecutors task launching speed (down from 2-3 seconds per task to 0.2seconds!)
* UI home page has been overhauled to make a better use of space.
The full changelog for core airflow (excluding providers and a few other internal things) since alpha2 is below.
Provider Packages
We have also released the 1.0.0b1 of the providers (not backports!) which can be found at
https://dist.apache.org/repos/dist/dev/airflow/providers/1.0.0b1/
We also made those versions available on PyPi for convenience ('pip install --pre apache-airflow-providers-*'):
https://pypi.org/search/?q=apache-airflow-providers
The documentation and changelogs are available in the readme of the PyPI packages -- see the end of this email for links to individual PyPi packages
Public keys for all releases are available at: https://www.apache.org/dist/airflow/KEYS
Airflow Changelog:
New Features
* Adds automated installation of dependent packages
* Render k8s yaml for tasks via the Airflow UI (#11815)
* Remove the ability to add hooks to airflow.hooks namespace (#12108)
* Add ability to specify pod_template_file in executor_config (#11784)
* Add SIGUSR2 handler to Scheduler to dump executor state (#12107)
* Remove the ability to import operators and sensors from plugins (#12072)
* Change Decorated Flow to TaskFlow API (#11895)
* Add Kubernetes cleanup-pods CLI command for Helm Chart (#11802)
* Perform "mini scheduling run" after task has finished (#11589)
* Allow airflow.providers to be installed in multiple python folders (#10806)
* Add Flower Authentication to Helm Chart (#11836)
* Disable XCom pickling by default (#11991)
* Fix: Responsive layout of DAGs (Home) view (#11958)
* Speed up task execution in Celery by pre-loading "expensive" modules (#11956)
* Fix oversized width of DAGs table with hide/reveal of "links" (#11866)
* Use resource permissions for Airflow view access (#11362)
* Make Dag Serialization a hard requirement (#11335)
Improvments
* Extend the same keyword args callable support in PythonOperator to some other sensors/operators (#11922)
* Filter dags by owner (#11121)
* Remove BaseDag and BaseDagBag classes (#12195)
* Don't include provider datafiles in the apache-airflow sdist (#12196)
* Call scheduler "book-keeping" operations less frequently. (#12139)
* Fix broken 'Blocked Highlight' feature in UI (#12183)
* Add authentication to AWS with Google credentials (#12079)
* Proper title for XCom List View page (#12169)
* Make doc_md field nullable and raise json for non-existing dag in dag detail endpoint (#12142)
* Retry Publishing Task to Celery Broker (#12140)
* Update to new helm stable repo (#12137)
* Override FAB table views where table width extends beyond parent containers (#12048)
* Update doc images to reflect latest UI (#11984)
* Allow fractional seconds for timeout values (#11966)
* Log instead of raise an Error for unregistered OperatorLinks (#11959)
* Upgrade FAB to 3.1.1 (#11884)
Bug Fixes
* Include data files in (backport) provider packages (#12200)
* Sync FAB Permissions for all base views (#12162)
* fix spacing between table and pagination (#12160)
* Fix default values for Helm Chart (#12153)
* airflow info fixed for python 3.8+ (#12132)
* Correct failure message in sql_sensor.py. (#12057)
* Avoid unnecessary IF checks when generate Duration & Landing Time views (#12075)
* Retry Dagbag.sync_to_db to avoid Deadlocks (#12046)
* Refine request check in api_connextion Pool endpoints (#12019)
* fix helm scheduler deployment / scheduler logs (#11685)
* fix helm chart worker deployment without kerberos (#11681)
* Unpin werkzeug & set default cookie_samesite to Lax (#11873)
Docs
* Fixed path of the test_core.py file in docs (#12191)
* Move metrics configuration to new section - metrics (#12165)
* Added 1.10.x section to Upgrading to 2.0 doc (#12173)
* Allow Connection Edit View to handle entries with NULL 'extra' (#12149)
* Move docs for max_db_retries option to core (#12167)
* Changed tutorial file to reflect name change to TaskFlow API (#12099)
* Fix typo in docker-context-files/README.md (#12078)
* Delete an environment-dependent value from CLI documentation (#12055)
* Adds updating note to ariflowignore fix (#12043)
* Fix incorrect .airflowignore behavior with multiple nested directories (#11994)
* Replace Docs GIF with updated UI screenshots (#12044)
* Ignore the basepath when ignoring files via .airflowignore (#11993)
* Adds documentation about the optimized PR workflow (#12006)
* Improve formatting in configurations docs (#11987)
* Add docs about Scheduler HA, how to use it and DB requirements (#11467)
* Add readme for core sensors, standardize capitalization schema (#11898)
* Updating 2.0 docs (#11842)
* Add providers, operators, and hooks readmes (#11829)
Misc/Internal
* Format all files (without excepions) by black (#12091)
* Enable Black - Python Auto Formmatter (#9550)
* Use PyUpgrade to use Python 3.6 features (#11447)
* Replace deprecated PythonOperator module with the new one (#12064)
* Remove unused JavaScript function (#12052)
* Code smell fixes for BackfillJob (#12005)
* All k8s object must comply with JSON Schema (#12003)
* Validate airflow chart values.yaml & values.schema.json (#11990)
* Remove unused "poke_exception_cache_ttl" param from SmartSensorOperator (#11972)
* Migrate from helm-unittest to python unittest (#11827)
* Migration commands shouldn't print a bunch of data. (#11961)
* Cache CLI parser objects (#11957)
* Add missing space to log message in task completion logs (#11934)
* Fix issue rendering k8s V1Pod (#11952)
* Make taskinstances pid and duration nullable (#11906)
Provider Packages PyPi Links:
https://pypi.org/project/apache-airflow-providers-amazon/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-apache-cassandra/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-apache-druid/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-apache-hdfs/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-apache-hive/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-apache-kylin/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-apache-livy/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-apache-pig/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-apache-pinot/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-apache-spark/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-apache-sqoop/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-celery/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-cloudant/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-databricks/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-datadog/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-dingding/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-discord/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-docker/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-elasticsearch/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-exasol/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-facebook/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-ftp/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-google/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-grpc/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-hashicorp/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-http/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-imap/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-jdbc/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-jenkins/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-jira/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-microsoft-azure/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-microsoft-mssql/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-microsoft-winrm/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-mongo/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-mysql/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-odbc/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-openfaas/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-opsgenie/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-oracle/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-pagerduty/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-papermill/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-plexus/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-postgres/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-presto/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-qubole/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-redis/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-salesforce/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-samba/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-segment/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-sftp/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-singularity/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-slack/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-snowflake/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-sqlite/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-ssh/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-vertica/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-yandex/1.0.0b1/
https://pypi.org/project/apache-airflow-providers-zendesk/1.0.0b1/
Re: Apache Airflow 2.0.0beta1 and providers 1.0.0beta1 available for testing.
Posted by Karolina Rosół <ka...@polidea.com>.
Yaaaaaay!
Karolina Rosół
Polidea <https://www.polidea.com/> | Head of Cloud & OSS
M: +48 606 630 236 <+48606630236>
E: karolina.rosol@polidea.com
[image: Polidea] <https://www.polidea.com/>
Check out our projects! <https://www.polidea.com/our-work>
[image: Github] <https://github.com/Polidea> [image: Facebook]
<https://www.facebook.com/Polidea.Software> [image: Twitter]
<https://twitter.com/polidea> [image: Linkedin]
<https://www.linkedin.com/company/polidea> [image: Instagram]
<https://instagram.com/polidea> [image: Behance]
<https://www.behance.net/polidea> [image: dribbble]
<https://dribbble.com/polideadesign>
On Tue, Nov 10, 2020 at 1:18 AM Jarek Potiuk <Ja...@polidea.com>
wrote:
> Woooho!
>
> On Tue, Nov 10, 2020 at 12:00 AM Ash Berlin-Taylor <as...@apache.org> wrote:
>
>> I'm proud to announce the availability of Apache Airlow 2.0.0.beta1 for
>> testing.
>>
>> This now marks us being "feature complete" for core Airflow -- from now
>> on any significant new features to core Airflow will have to wait until 2.1
>> -- but thanks to the split out of provider packages those can continue to
>> be developed.
>>
>> This "snapshot" is intended for members of the Airflow developer
>> community to test the build and get an early start on testing 2.0.0. For
>> clarity, this is not an official release of Apache Airflow either - that
>> doesn't happen until we make a release candidate and then vote on that, and
>> based on the expected timelines on the Airflow 2.0 planning page
>> <https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0+-+Planning>,
>> we expect that to happen the week of 7th December, 2020.
>>
>> Apache Airflow
>>
>> The beta snapshot is available at:
>>
>> https://dist.apache.org/repos/dist/dev/airflow/2.0.0b1/
>> <https://dist.apache.org/repos/dist/dev/airflow/2.0.0a1/>
>>
>> *apache-airflow-2.0.0b1-source.tar.gz* is a source release that comes
>> with INSTALL instructions.
>>
>> *apache-airflow-2.0.0b1-bin.tar.gz* is the binary Python "sdist" snapshot.
>>
>> *apache_airflow-2.0.0b1-py3-none-any.whl* is the binary Python wheel
>> snapshot.
>>
>> This snapshot *has* been pushed to PyPi.
>>
>> Major features since alpha2:
>>
>> * "Decorated Flow" renamed to "TaskFlow API" (we should be sticking with
>> this name now)
>> * Further significant performance improvements to CeleryExecutors task
>> launching speed (down from 2-3 seconds per task to 0.2seconds!)
>> * UI home page has been overhauled to make a better use of space.
>>
>> The full changelog for core airflow (excluding providers and a few other
>> internal things) since alpha2 is below.
>>
>> Provider Packages
>>
>> We have also released the 1.0.0b1 of the providers (not backports!) which
>> can be found at
>>
>> https://dist.apache.org/repos/dist/dev/airflow/providers/1.0.0b1/
>> We also made those versions available on PyPi for convenience ('pip
>> install --pre apache-airflow-providers-*'):
>>
>> https://pypi.org/search/?q=apache-airflow-providers
>>
>> The documentation and changelogs are available in the readme of the PyPI
>> packages -- see the end of this email for links to individual PyPi packages
>>
>>
>> Public keys for all releases are available at:
>> https://www.apache.org/dist/airflow/KEYS
>>
>> Airflow Changelog:
>>
>> *New Features*
>>
>> * Adds automated installation of dependent packages
>> * Render k8s yaml for tasks via the Airflow UI (#11815)
>> * Remove the ability to add hooks to airflow.hooks namespace (#12108)
>> * Add ability to specify pod_template_file in executor_config (#11784)
>> * Add SIGUSR2 handler to Scheduler to dump executor state (#12107)
>> * Remove the ability to import operators and sensors from plugins (#12072)
>> * Change Decorated Flow to TaskFlow API (#11895)
>> * Add Kubernetes cleanup-pods CLI command for Helm Chart (#11802)
>> * Perform "mini scheduling run" after task has finished (#11589)
>> * Allow airflow.providers to be installed in multiple python folders
>> (#10806)
>> * Add Flower Authentication to Helm Chart (#11836)
>> * Disable XCom pickling by default (#11991)
>> * Fix: Responsive layout of DAGs (Home) view (#11958)
>> * Speed up task execution in Celery by pre-loading "expensive" modules
>> (#11956)
>> * Fix oversized width of DAGs table with hide/reveal of "links" (#11866)
>> * Use resource permissions for Airflow view access (#11362)
>> * Make Dag Serialization a hard requirement (#11335)
>>
>> *Improvments*
>>
>> * Extend the same keyword args callable support in PythonOperator to some
>> other sensors/operators (#11922)
>> * Filter dags by owner (#11121)
>> * Remove BaseDag and BaseDagBag classes (#12195)
>> * Don't include provider datafiles in the apache-airflow sdist (#12196)
>> * Call scheduler "book-keeping" operations less frequently. (#12139)
>> * Fix broken 'Blocked Highlight' feature in UI (#12183)
>> * Add authentication to AWS with Google credentials (#12079)
>> * Proper title for XCom List View page (#12169)
>> * Make doc_md field nullable and raise json for non-existing dag in dag
>> detail endpoint (#12142)
>> * Retry Publishing Task to Celery Broker (#12140)
>> * Update to new helm stable repo (#12137)
>> * Override FAB table views where table width extends beyond parent
>> containers (#12048)
>> * Update doc images to reflect latest UI (#11984)
>> * Allow fractional seconds for timeout values (#11966)
>> * Log instead of raise an Error for unregistered OperatorLinks (#11959)
>> * Upgrade FAB to 3.1.1 (#11884)
>>
>> *Bug Fixes*
>>
>> * Include data files in (backport) provider packages (#12200)
>> * Sync FAB Permissions for all base views (#12162)
>> * fix spacing between table and pagination (#12160)
>> * Fix default values for Helm Chart (#12153)
>> * airflow info fixed for python 3.8+ (#12132)
>> * Correct failure message in sql_sensor.py. (#12057)
>> * Avoid unnecessary IF checks when generate Duration & Landing Time views
>> (#12075)
>> * Retry Dagbag.sync_to_db to avoid Deadlocks (#12046)
>> * Refine request check in api_connextion Pool endpoints (#12019)
>> * fix helm scheduler deployment / scheduler logs (#11685)
>> * fix helm chart worker deployment without kerberos (#11681)
>> * Unpin werkzeug & set default cookie_samesite to Lax (#11873)
>>
>> *Docs*
>>
>> * Fixed path of the test_core.py file in docs (#12191)
>> * Move metrics configuration to new section - metrics (#12165)
>> * Added 1.10.x section to Upgrading to 2.0 doc (#12173)
>> * Allow Connection Edit View to handle entries with NULL 'extra' (#12149)
>> * Move docs for max_db_retries option to core (#12167)
>> * Changed tutorial file to reflect name change to TaskFlow API (#12099)
>> * Fix typo in docker-context-files/README.md (#12078)
>> * Delete an environment-dependent value from CLI documentation (#12055)
>> * Adds updating note to ariflowignore fix (#12043)
>> * Fix incorrect .airflowignore behavior with multiple nested directories
>> (#11994)
>> * Replace Docs GIF with updated UI screenshots (#12044)
>> * Ignore the basepath when ignoring files via .airflowignore (#11993)
>> * Adds documentation about the optimized PR workflow (#12006)
>> * Improve formatting in configurations docs (#11987)
>> * Add docs about Scheduler HA, how to use it and DB requirements (#11467)
>> * Add readme for core sensors, standardize capitalization schema (#11898)
>> * Updating 2.0 docs (#11842)
>> * Add providers, operators, and hooks readmes (#11829)
>>
>> *Misc/Internal*
>>
>> * Format all files (without excepions) by black (#12091)
>> * Enable Black - Python Auto Formmatter (#9550)
>> * Use PyUpgrade to use Python 3.6 features (#11447)
>> * Replace deprecated PythonOperator module with the new one (#12064)
>> * Remove unused JavaScript function (#12052)
>> * Code smell fixes for BackfillJob (#12005)
>> * All k8s object must comply with JSON Schema (#12003)
>> * Validate airflow chart values.yaml & values.schema.json (#11990)
>> * Remove unused "poke_exception_cache_ttl" param from SmartSensorOperator
>> (#11972)
>> * Migrate from helm-unittest to python unittest (#11827)
>> * Migration commands shouldn't print a bunch of data. (#11961)
>> * Cache CLI parser objects (#11957)
>> * Add missing space to log message in task completion logs (#11934)
>> * Fix issue rendering k8s V1Pod (#11952)
>> * Make taskinstances pid and duration nullable (#11906)
>>
>> Provider Packages PyPi Links:
>>
>> https://pypi.org/project/apache-airflow-providers-amazon/1.0.0b1/
>>
>> https://pypi.org/project/apache-airflow-providers-apache-cassandra/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-apache-druid/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-apache-hdfs/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-apache-hive/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-apache-kylin/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-apache-livy/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-apache-pig/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-apache-pinot/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-apache-spark/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-apache-sqoop/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-celery/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-cloudant/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-databricks/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-datadog/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-dingding/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-discord/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-docker/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-elasticsearch/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-exasol/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-facebook/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-ftp/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-google/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-grpc/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-hashicorp/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-http/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-imap/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-jdbc/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-jenkins/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-jira/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-microsoft-azure/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-microsoft-mssql/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-microsoft-winrm/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-mongo/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-mysql/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-odbc/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-openfaas/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-opsgenie/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-oracle/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-pagerduty/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-papermill/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-plexus/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-postgres/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-presto/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-qubole/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-redis/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-salesforce/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-samba/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-segment/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-sftp/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-singularity/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-slack/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-snowflake/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-sqlite/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-ssh/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-vertica/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-yandex/1.0.0b1/
>> https://pypi.org/project/apache-airflow-providers-zendesk/1.0.0b1/
>>
>>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>
Re: Apache Airflow 2.0.0beta1 and providers 1.0.0beta1 available for testing.
Posted by Jarek Potiuk <Ja...@polidea.com>.
Woooho!
On Tue, Nov 10, 2020 at 12:00 AM Ash Berlin-Taylor <as...@apache.org> wrote:
> I'm proud to announce the availability of Apache Airlow 2.0.0.beta1 for
> testing.
>
> This now marks us being "feature complete" for core Airflow -- from now on
> any significant new features to core Airflow will have to wait until 2.1 --
> but thanks to the split out of provider packages those can continue to be
> developed.
>
> This "snapshot" is intended for members of the Airflow developer community
> to test the build and get an early start on testing 2.0.0. For clarity,
> this is not an official release of Apache Airflow either - that doesn't
> happen until we make a release candidate and then vote on that, and based
> on the expected timelines on the Airflow 2.0 planning page
> <https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0+-+Planning>,
> we expect that to happen the week of 7th December, 2020.
>
> Apache Airflow
>
> The beta snapshot is available at:
>
> https://dist.apache.org/repos/dist/dev/airflow/2.0.0b1/
> <https://dist.apache.org/repos/dist/dev/airflow/2.0.0a1/>
>
> *apache-airflow-2.0.0b1-source.tar.gz* is a source release that comes with
> INSTALL instructions.
>
> *apache-airflow-2.0.0b1-bin.tar.gz* is the binary Python "sdist" snapshot.
>
> *apache_airflow-2.0.0b1-py3-none-any.whl* is the binary Python wheel
> snapshot.
>
> This snapshot *has* been pushed to PyPi.
>
> Major features since alpha2:
>
> * "Decorated Flow" renamed to "TaskFlow API" (we should be sticking with
> this name now)
> * Further significant performance improvements to CeleryExecutors task
> launching speed (down from 2-3 seconds per task to 0.2seconds!)
> * UI home page has been overhauled to make a better use of space.
>
> The full changelog for core airflow (excluding providers and a few other
> internal things) since alpha2 is below.
>
> Provider Packages
>
> We have also released the 1.0.0b1 of the providers (not backports!) which
> can be found at
>
> https://dist.apache.org/repos/dist/dev/airflow/providers/1.0.0b1/
> We also made those versions available on PyPi for convenience ('pip
> install --pre apache-airflow-providers-*'):
>
> https://pypi.org/search/?q=apache-airflow-providers
>
> The documentation and changelogs are available in the readme of the PyPI
> packages -- see the end of this email for links to individual PyPi packages
>
>
> Public keys for all releases are available at:
> https://www.apache.org/dist/airflow/KEYS
>
> Airflow Changelog:
>
> *New Features*
>
> * Adds automated installation of dependent packages
> * Render k8s yaml for tasks via the Airflow UI (#11815)
> * Remove the ability to add hooks to airflow.hooks namespace (#12108)
> * Add ability to specify pod_template_file in executor_config (#11784)
> * Add SIGUSR2 handler to Scheduler to dump executor state (#12107)
> * Remove the ability to import operators and sensors from plugins (#12072)
> * Change Decorated Flow to TaskFlow API (#11895)
> * Add Kubernetes cleanup-pods CLI command for Helm Chart (#11802)
> * Perform "mini scheduling run" after task has finished (#11589)
> * Allow airflow.providers to be installed in multiple python folders
> (#10806)
> * Add Flower Authentication to Helm Chart (#11836)
> * Disable XCom pickling by default (#11991)
> * Fix: Responsive layout of DAGs (Home) view (#11958)
> * Speed up task execution in Celery by pre-loading "expensive" modules
> (#11956)
> * Fix oversized width of DAGs table with hide/reveal of "links" (#11866)
> * Use resource permissions for Airflow view access (#11362)
> * Make Dag Serialization a hard requirement (#11335)
>
> *Improvments*
>
> * Extend the same keyword args callable support in PythonOperator to some
> other sensors/operators (#11922)
> * Filter dags by owner (#11121)
> * Remove BaseDag and BaseDagBag classes (#12195)
> * Don't include provider datafiles in the apache-airflow sdist (#12196)
> * Call scheduler "book-keeping" operations less frequently. (#12139)
> * Fix broken 'Blocked Highlight' feature in UI (#12183)
> * Add authentication to AWS with Google credentials (#12079)
> * Proper title for XCom List View page (#12169)
> * Make doc_md field nullable and raise json for non-existing dag in dag
> detail endpoint (#12142)
> * Retry Publishing Task to Celery Broker (#12140)
> * Update to new helm stable repo (#12137)
> * Override FAB table views where table width extends beyond parent
> containers (#12048)
> * Update doc images to reflect latest UI (#11984)
> * Allow fractional seconds for timeout values (#11966)
> * Log instead of raise an Error for unregistered OperatorLinks (#11959)
> * Upgrade FAB to 3.1.1 (#11884)
>
> *Bug Fixes*
>
> * Include data files in (backport) provider packages (#12200)
> * Sync FAB Permissions for all base views (#12162)
> * fix spacing between table and pagination (#12160)
> * Fix default values for Helm Chart (#12153)
> * airflow info fixed for python 3.8+ (#12132)
> * Correct failure message in sql_sensor.py. (#12057)
> * Avoid unnecessary IF checks when generate Duration & Landing Time views
> (#12075)
> * Retry Dagbag.sync_to_db to avoid Deadlocks (#12046)
> * Refine request check in api_connextion Pool endpoints (#12019)
> * fix helm scheduler deployment / scheduler logs (#11685)
> * fix helm chart worker deployment without kerberos (#11681)
> * Unpin werkzeug & set default cookie_samesite to Lax (#11873)
>
> *Docs*
>
> * Fixed path of the test_core.py file in docs (#12191)
> * Move metrics configuration to new section - metrics (#12165)
> * Added 1.10.x section to Upgrading to 2.0 doc (#12173)
> * Allow Connection Edit View to handle entries with NULL 'extra' (#12149)
> * Move docs for max_db_retries option to core (#12167)
> * Changed tutorial file to reflect name change to TaskFlow API (#12099)
> * Fix typo in docker-context-files/README.md (#12078)
> * Delete an environment-dependent value from CLI documentation (#12055)
> * Adds updating note to ariflowignore fix (#12043)
> * Fix incorrect .airflowignore behavior with multiple nested directories
> (#11994)
> * Replace Docs GIF with updated UI screenshots (#12044)
> * Ignore the basepath when ignoring files via .airflowignore (#11993)
> * Adds documentation about the optimized PR workflow (#12006)
> * Improve formatting in configurations docs (#11987)
> * Add docs about Scheduler HA, how to use it and DB requirements (#11467)
> * Add readme for core sensors, standardize capitalization schema (#11898)
> * Updating 2.0 docs (#11842)
> * Add providers, operators, and hooks readmes (#11829)
>
> *Misc/Internal*
>
> * Format all files (without excepions) by black (#12091)
> * Enable Black - Python Auto Formmatter (#9550)
> * Use PyUpgrade to use Python 3.6 features (#11447)
> * Replace deprecated PythonOperator module with the new one (#12064)
> * Remove unused JavaScript function (#12052)
> * Code smell fixes for BackfillJob (#12005)
> * All k8s object must comply with JSON Schema (#12003)
> * Validate airflow chart values.yaml & values.schema.json (#11990)
> * Remove unused "poke_exception_cache_ttl" param from SmartSensorOperator
> (#11972)
> * Migrate from helm-unittest to python unittest (#11827)
> * Migration commands shouldn't print a bunch of data. (#11961)
> * Cache CLI parser objects (#11957)
> * Add missing space to log message in task completion logs (#11934)
> * Fix issue rendering k8s V1Pod (#11952)
> * Make taskinstances pid and duration nullable (#11906)
>
> Provider Packages PyPi Links:
>
> https://pypi.org/project/apache-airflow-providers-amazon/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-apache-cassandra/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-apache-druid/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-apache-hdfs/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-apache-hive/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-apache-kylin/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-apache-livy/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-apache-pig/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-apache-pinot/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-apache-spark/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-apache-sqoop/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-celery/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-cloudant/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-databricks/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-datadog/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-dingding/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-discord/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-docker/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-elasticsearch/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-exasol/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-facebook/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-ftp/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-google/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-grpc/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-hashicorp/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-http/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-imap/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-jdbc/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-jenkins/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-jira/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-microsoft-azure/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-microsoft-mssql/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-microsoft-winrm/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-mongo/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-mysql/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-odbc/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-openfaas/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-opsgenie/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-oracle/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-pagerduty/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-papermill/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-plexus/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-postgres/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-presto/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-qubole/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-redis/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-salesforce/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-samba/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-segment/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-sftp/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-singularity/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-slack/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-snowflake/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-sqlite/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-ssh/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-vertica/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-yandex/1.0.0b1/
> https://pypi.org/project/apache-airflow-providers-zendesk/1.0.0b1/
>
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>