You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by je...@apache.org on 2022/01/27 22:45:13 UTC
[airflow] branch v2-2-test updated (e898e0e -> 5ad965d)
This is an automated email from the ASF dual-hosted git repository.
jedcunningham pushed a change to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git.
discard e898e0e Improved instructions for custom image build with docker compose (#21052)
discard aae038c Add back legacy .piprc customization for pip (#21124)
discard 485e071 Update logging-tasks.rst (#21088)
discard 7f592d0 name mismatch (#21055)
discard 72ce671 Update v1.yaml (#21024)
discard b5da486 Return to the same place when triggering a DAG (#20955)
discard bda0a74 Add downgrade to some FAB migrations (#20874)
discard 3215d94 Add Roles from Azure OAUTH Response in security manager as it is currently not able map any AD roles to airflow ones (#20707)
discard 0337fcb Allow Viewing DagRuns and TIs if a user has DAG "read" perms (#20663)
discard 0bf4201 Add documentation and release policy on "latest" constraints (#21093)
discard bcc3f7f Logs in to Github Registry when preparing cache (#21069)
discard a85b649 Add a link to the DAG model in the Python API reference (#21060)
discard 17b7da5 Fix running airflow dags test <dag_id> <execution_dt> results in error when run twice (#21031)
discard 5eeefc0 Temporary limit Pandas version (#21045)
discard c9022f4 Update tutorial.rst (#21043)
discard b334800 Add image labels required by ArtifactHub (#21040)
discard 1b1bfc5 Fix grammar in ``dags.rst`` (#20988)
discard d5870f0 Doc: Added an enum param example (#20841)
discard c528166 Python3 requisite start local (#20777)
discard 2dbe1e9 Update metric name in documentation (#20764)
discard 7cd3fd6 Compare taskgroup and subdag (#20700)
discard 111e8c1 Update operators.rst (#20640)
discard 543a78b Improve documentation on ``Params`` (#20567)
discard 312577e Removes unnecessary --upgrade option from our examples (#20537)
discard e0b262c Deprecate smart sensors (#20151)
discard e2bb598 Fix grammar and typos in "Logging for Tasks" guide (#20146)
discard 8ecdcb9 Fix typo in MySQL Database creation code (Set up DB docs) (#20102)
discard 239b1dc Correct set-up-database.rst (#20090)
discard 346656f Add requirements.txt description (#20048)
discard 4608eaa Fix example code in Doc (#19824)
discard 7d37b0e Clean up ``default_args`` usage in docs (#19803)
discard ce21b8e Change the name of link to ASF downloads (#19441)
discard fbad277 Fix PostgresHook import in tutorial (#19374)
discard d46674a docs: reorder imports in tutorials 🎨 (#19035)
discard 2cc9ed0 Doc: Improve tutorial documentation and code (#19186)
discard 0b45b27 Add docker-compose explanation to conn localhost (#19076)
discard d183404 Update CSV ingest code for tutorial (#18960)
discard a718042 Adds Pendulum 1.x -> 2.x upgrade documentation (#18955)
discard ce341b6 Updating explicit arg example in TaskFlow API tutorial doc (#18907)
discard 3714aa3 Adds back documentation about context usage in Python/@task (#18868)
discard eb3ded7 Hide version selector for non-versioned packages (#21041)
discard f00583e Fix occasional external task sensor tests (#18853)
discard 7f6fc19 Restore stability and unquarantine all test_scheduler_job tests (#19860)
discard 0ea3608 Fix failing main. (#20094)
discard bcf18e5 Add roles to create_user test (#20773)
discard 9c6e2d6 Update Celery requirements
discard 34fe8dc Update md5 information about image after waiting (#21000)
discard 3a165e9 Update refreshing constraints instructions (#21001)
discard 995a841 Get rid of upload coverage warnings (#20994)
discard 77a82af Add extra sync when adding executable flag to installation scripts (#20987)
discard eff9929 Update base python image to be Python 3.7 by default (#20978)
discard cd05b1f Fix new buildkit builds on MacOS (#20963)
discard ff5ca5b Switch to 'buildkit' to build Airflow images (#20664)
discard c133e37 Fix errors thrown by some versions of Bash v4 (#20932)
discard e760582 Switch to non-vendored latest connexion library (#20910)
discard ec92a35 Uses CI images built in previous step to prepare PROD image (#20889)
discard 84b1819 Switch to new MySQL public key (#20912)
discard b3e16e2 add entry in release readme to update milestone in Issues (#20890)
discard 53a1fc7 Unpin ``argcomplete`` and ``colorlog`` (#20878)
discard 1f16854 Unpin ``cattrs`` (#20872)
discard d4195a1 Optimize dockerfiles for local rebuilds (#20238)
discard f56bc0c Cleaner output for Docker image building (#20747)
discard b9d175f Uses airflow user for build segment of docker image (#20744)
discard 3fea595 Modernize usage of PIP in Airflow images (#20726)
discard 8e2f07b Add color to pytest tests on CI (#20723)
discard d23a685 Fix naming convention for sdist provider packages (#20711)
discard f4b6d9f Be build -> built, and a stray space (#20703)
discard 9e9db3e Cleaner output of docker image building scripts (#20679)
discard 4c5661f Remove Python 2 from our images (#20680)
discard ec0b7dc Fix incorrect arguments (#20638)
discard 10dde0e Bump flask-appbuilder to >=3.3.4 (#20628)
discard 20501f1 upgrade celery 5.2.3 (#19703)
discard 0944c8a Fix Constraints failure in PRs (#20631)
discard 5a77fe7 Generate constraints in PRs when upgrading dependencies (#20624)
discard 7046d7d Fix generation of "Status provider" issue (#20621)
discard 34fc290 Add twine check for provider packages (#20619)
discard 41e9f81 Add known warning generated by snowflake new version (#20604)
discard bcaf486 Generate version documentation from single source of truth (#20594)
discard ef188ce Update the "releasing Airflow" docs (#20456)
discard d0a2066 Add possibility to ignore common deprecated message (#20444)
discard 07a33a0 Add exiting on error in prod image script (#20447)
discard 889580e Only list linked issues once in release issues (#20299)
discard ccb579e Fix MyPy errors in `scripts/in_container` (#20280)
discard b655f58 Fix MyPy errors in `dev/*` (#20261)
discard a24ab31 Update docs/tools for releasing core Airflow (#20211)
discard 0101026 Fix OOM error in tests when using public Github Runners. (#19809)
discard 5b85f6b Bring back Core and Other tests to be run in parallel (#19812)
discard 59d3219 Upload provider distribution artifacts during CI (#19807)
discard 39e4a27 Add retagging images accross repos (#19778)
discard e1bf9a8 Fix CI tests so they correctly fail in case of error! (#19678)
discard a86ddba Update Airflow release guide (#19663)
discard e253717 Fix dumping container logs on error (#19645)
discard 4a6c7e8 Fix failing CI phase with unhealthy container issue (#19633)
discard 536b7d5 Update docs about releasing providersk (#19549)
discard 97b78bc Add script to generate chart changelog annotations (#20555)
discard b2eee5e Add script to generate issue for status of testing of the rc (#19247)
discard a95187e Clarify that .asf.yml and codecov.yml should be changed in main (#19496)
discard 94f3e73 Update helm chart release docs (#19494)
discard 9a56fd5 Update description of release process for adding new major release (#19483)
discard 0a6d39f Clarify guidance re trust of keys in release docs (#19480)
discard 07ce313 Update known warnings for Python 3.7 (#19333)
discard 68ba68c Clarify rat test guidance in release check docs (#19296)
discard b693750 Fix release check script (#19238)
discard 586ea7a Dev: Clarify file naming in release verification doc (#19233)
discard 8b28949 Update "Release Airflow" doc (#19111)
discard d97c28b Bump croniter from `<1.1` to `<1.2` (#20489)
discard 5a3f792 Remove PyJWT upper bound from Dockerfile (#20503)
discard 8f5711e Increase time limit for Helm chart unit tests (#20525)
discard c82b84c Fix: pin pymongo < 4.0.0 (#20511)
discard 884ce34 Bump PyJWT from `<2` to `<3` (#20490)
discard 32400ae Add autoflake precommit to automatically remove unused code (#20466)
discard 1f05a22 Checks if the user running Breeze has permissions to run docker cmd (#20462)
discard 9dc2aae Add custom pip.conf to docker-context-files (#20445)
discard 9753e3c Add pre-commit that checks credentials are not persisted in CI (#20430)
discard 9f2985c Limit Snowflake connector to< 2.7.2 (#20395)
discard d3fd594 Change default python version in docker image docs (#20389)
discard 4ebe9b9 Fix race condition when flake checks run in parallel (#20294)
discard a7d5736 Speed up Helm Upgrade tests (#20289)
discard f1358fc YandexCloud provider: Support new Yandex SDK features: log_group_id, user-agent, maven packages (#20103)
discard 1fc1520 Remove Integration tests from MSSQL on Public Runners (#20231)
discard 4cf1f81 Increase limit of time for constraint job (#20230)
discard 18ab3ed Update minimum sphinx versions after upgrading sphinx-autoapi (#20170)
discard f59c150 Update Sphinx and Sphinx-AutoAPI (#20079)
discard 2fbd0c5 Upload provider artifacts before install/test step (#20137)
discard 7bf0031 Lift off upper bound for MarkupSafe (#20113)
discard ffe06e3 switch to follow_redirects on httpx.get call in CloudSQL provider (#20239)
discard 7dfa24b update upper bound for MarkupSafe (#19953)
discard 1f12bc8 Fix ``breeze kind-cluster shell`` (#20015)
discard 410231a Move to watchtower 2.0.1 (#19907)
discard a075bb7 Create dataproc serverless spark batches operator (#19248)
discard 3cdeb32 Fix failures with recent moto library 2.2.15 (#19693)
discard f694e32 Add hdfs requirement for hdfs provider (#19540)
discard 7e4aa2d Add dataproc metastore operators (#18945)
discard e90e2d6 Remove distutils usages for Python 3.10 (#19064)
new 2ad02ef Add dataproc metastore operators (#18945)
new 44f5dc5 Add hdfs requirement for hdfs provider (#19540)
new e2dcdfd Fix failures with recent moto library 2.2.15 (#19693)
new 67fc1d7 Create dataproc serverless spark batches operator (#19248)
new 95b9d48 Move to watchtower 2.0.1 (#19907)
new 85cce07 Fix ``breeze kind-cluster shell`` (#20015)
new 5f1236f update upper bound for MarkupSafe (#19953)
new f1c7f06 switch to follow_redirects on httpx.get call in CloudSQL provider (#20239)
new 7250d89 Lift off upper bound for MarkupSafe (#20113)
new 27fbbea Upload provider artifacts before install/test step (#20137)
new 96f88a6 Update Sphinx and Sphinx-AutoAPI (#20079)
new d761aff Update minimum sphinx versions after upgrading sphinx-autoapi (#20170)
new 689ded0 Increase limit of time for constraint job (#20230)
new 989e73c Remove Integration tests from MSSQL on Public Runners (#20231)
new 4c60fb2 YandexCloud provider: Support new Yandex SDK features: log_group_id, user-agent, maven packages (#20103)
new ac6b016 Speed up Helm Upgrade tests (#20289)
new 09602fc Fix race condition when flake checks run in parallel (#20294)
new 53d73dc Change default python version in docker image docs (#20389)
new 793aaa9 Limit Snowflake connector to< 2.7.2 (#20395)
new 2dbbe2b Add pre-commit that checks credentials are not persisted in CI (#20430)
new 48f39d1 Add custom pip.conf to docker-context-files (#20445)
new 0746bc8 Checks if the user running Breeze has permissions to run docker cmd (#20462)
new ef319b0 Add autoflake precommit to automatically remove unused code (#20466)
new 58a65e1 Bump PyJWT from `<2` to `<3` (#20490)
new 933716b Fix: pin pymongo < 4.0.0 (#20511)
new 0df0dcd Increase time limit for Helm chart unit tests (#20525)
new 95e71b7 Remove PyJWT upper bound from Dockerfile (#20503)
new bccf2f5 Bump croniter from `<1.1` to `<1.2` (#20489)
new e294620 Update "Release Airflow" doc (#19111)
new 59a6aca Dev: Clarify file naming in release verification doc (#19233)
new 23b2e7b Fix release check script (#19238)
new 35b5cd2 Clarify rat test guidance in release check docs (#19296)
new ae18469 Update known warnings for Python 3.7 (#19333)
new 734ab55 Clarify guidance re trust of keys in release docs (#19480)
new ae28789 Update description of release process for adding new major release (#19483)
new 8a66731 Update helm chart release docs (#19494)
new 3f41c95 Clarify that .asf.yml and codecov.yml should be changed in main (#19496)
new 04b380b Add script to generate issue for status of testing of the rc (#19247)
new b2c9085 Add script to generate chart changelog annotations (#20555)
new c92409c Update docs about releasing providersk (#19549)
new c7236c7 Fix failing CI phase with unhealthy container issue (#19633)
new 371ae8f Fix dumping container logs on error (#19645)
new a5ad8bb Update Airflow release guide (#19663)
new 40283e3 Fix CI tests so they correctly fail in case of error! (#19678)
new 84082d3 Add retagging images accross repos (#19778)
new 3af9ffd Upload provider distribution artifacts during CI (#19807)
new dfde1ba Bring back Core and Other tests to be run in parallel (#19812)
new b96093d Fix OOM error in tests when using public Github Runners. (#19809)
new 5917719 Update docs/tools for releasing core Airflow (#20211)
new 3162de5 Fix MyPy errors in `dev/*` (#20261)
new 3087dd2 Fix MyPy errors in `scripts/in_container` (#20280)
new b43fb3d Only list linked issues once in release issues (#20299)
new 92ef164 Add exiting on error in prod image script (#20447)
new ec08a5c Add possibility to ignore common deprecated message (#20444)
new ce55c30 Update the "releasing Airflow" docs (#20456)
new e02aa46 Generate version documentation from single source of truth (#20594)
new 8443c73 Add known warning generated by snowflake new version (#20604)
new 8a432bb Add twine check for provider packages (#20619)
new 3c3f246 Fix generation of "Status provider" issue (#20621)
new dfc4968 Generate constraints in PRs when upgrading dependencies (#20624)
new f19a89c Fix Constraints failure in PRs (#20631)
new e21fc84 upgrade celery 5.2.3 (#19703)
new bedf40c Bump flask-appbuilder to >=3.3.4 (#20628)
new f8bb2c2 Fix incorrect arguments (#20638)
new f05725e Cleaner output of docker image building scripts (#20679)
new a2fb0f4 Be build -> built, and a stray space (#20703)
new 2c7cea7 Fix naming convention for sdist provider packages (#20711)
new 907d107 Add color to pytest tests on CI (#20723)
new a4c57e5 Modernize usage of PIP in Airflow images (#20726)
new 172e536 Uses airflow user for build segment of docker image (#20744)
new 0cd9904 Cleaner output for Docker image building (#20747)
new 9b5e913 Optimize dockerfiles for local rebuilds (#20238)
new 33e3622 Unpin ``cattrs`` (#20872)
new eeeacc4 Unpin ``argcomplete`` and ``colorlog`` (#20878)
new 2abcd4f add entry in release readme to update milestone in Issues (#20890)
new ef0a980 Switch to new MySQL public key (#20912)
new bf7a755 Uses CI images built in previous step to prepare PROD image (#20889)
new c5082cb Switch to non-vendored latest connexion library (#20910)
new e36c54f Fix errors thrown by some versions of Bash v4 (#20932)
new 5ccbf30 Switch to 'buildkit' to build Airflow images (#20664)
new f466f1c Fix new buildkit builds on MacOS (#20963)
new fa83fa4 Update base python image to be Python 3.7 by default (#20978)
new 9d5c403 Add extra sync when adding executable flag to installation scripts (#20987)
new bf0dd93 Get rid of upload coverage warnings (#20994)
new 0f868a8 Update refreshing constraints instructions (#21001)
new 676e160 Update md5 information about image after waiting (#21000)
new d4660a8 Update Celery requirements
new 6f4d29c Add roles to create_user test (#20773)
new 3fa1535 Fix failing main. (#20094)
new ad00e8e Restore stability and unquarantine all test_scheduler_job tests (#19860)
new 83a4090 Fix occasional external task sensor tests (#18853)
new ab50aa7 Hide version selector for non-versioned packages (#21041)
new 9e34741 Adds back documentation about context usage in Python/@task (#18868)
new c3ead66 Updating explicit arg example in TaskFlow API tutorial doc (#18907)
new 97a7fe7 Adds Pendulum 1.x -> 2.x upgrade documentation (#18955)
new cc7f80d Update CSV ingest code for tutorial (#18960)
new 1463777 Add docker-compose explanation to conn localhost (#19076)
new 67b5f88 Doc: Improve tutorial documentation and code (#19186)
new 80d1b68 docs: reorder imports in tutorials 🎨 (#19035)
new 612e5c5 Fix PostgresHook import in tutorial (#19374)
new 8873abc Change the name of link to ASF downloads (#19441)
new 0072200 Clean up ``default_args`` usage in docs (#19803)
new 2ff09c3 Fix example code in Doc (#19824)
new 9a1c6b2 Add requirements.txt description (#20048)
new d8fe499 Correct set-up-database.rst (#20090)
new 7dfc7c1 Fix typo in MySQL Database creation code (Set up DB docs) (#20102)
new 52241bf Fix grammar and typos in "Logging for Tasks" guide (#20146)
new 25d7bbe Deprecate smart sensors (#20151)
new 50f4f9e Removes unnecessary --upgrade option from our examples (#20537)
new e43138f Improve documentation on ``Params`` (#20567)
new 915054f Update operators.rst (#20640)
new 88d1da8 Compare taskgroup and subdag (#20700)
new 08ddaea Update metric name in documentation (#20764)
new 8b199eb Python3 requisite start local (#20777)
new d0a9f9a Doc: Added an enum param example (#20841)
new 06828d6 Fix grammar in ``dags.rst`` (#20988)
new cf90511 Add image labels required by ArtifactHub (#21040)
new 72085cc Update tutorial.rst (#21043)
new a262d9c Temporary limit Pandas version (#21045)
new a889305 Fix running airflow dags test <dag_id> <execution_dt> results in error when run twice (#21031)
new b1862ca Add a link to the DAG model in the Python API reference (#21060)
new cc9a034 Logs in to Github Registry when preparing cache (#21069)
new f99f7df Add documentation and release policy on "latest" constraints (#21093)
new fab8b12 Allow Viewing DagRuns and TIs if a user has DAG "read" perms (#20663)
new a9c178e Add Roles from Azure OAUTH Response in security manager as it is currently not able map any AD roles to airflow ones (#20707)
new e671b92 Add downgrade to some FAB migrations (#20874)
new 512a10a Return to the same place when triggering a DAG (#20955)
new 40960c2 Update v1.yaml (#21024)
new 53d89b8 name mismatch (#21055)
new 46f7508 Update logging-tasks.rst (#21088)
new fb9a45e Add back legacy .piprc customization for pip (#21124)
new 000145b Improved instructions for custom image build with docker compose (#21052)
new 4dff21d Update `version_added` for `[email] from_email` (#21138)
new 753556f Do not set `TaskInstance.max_tries` in `refresh_from_task` (#21018)
new b7438f7 Fix 'airflow dags backfill --reset-dagruns' errors when run twice (#21062)
new 25c515e Type-annotate SkipMixin and BaseXCom (#20011)
new 91ec7b2 Helper for provide_session-decorated functions (#20104)
new 2a934b3 Fix session usage in ``/rendered-k8s`` view (#21006)
new 2785e19 Deprecate some functions in the experimental API (#19931)
new 5ad965d Avoid unintentional data loss when deleting DAGs (#20758)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (e898e0e)
\
N -- N -- N refs/heads/v2-2-test (5ad965d)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 140 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
Dockerfile | 4 +
Dockerfile.ci | 2 +
airflow/api/client/local_client.py | 29 +-
.../api/common/{experimental => }/delete_dag.py | 21 +-
airflow/api/common/experimental/delete_dag.py | 70 +----
airflow/api/common/experimental/get_code.py | 3 +
.../api/common/experimental/get_dag_run_state.py | 3 +
airflow/api/common/experimental/get_task.py | 3 +
.../api/common/experimental/get_task_instance.py | 3 +
airflow/api/common/experimental/pool.py | 6 +
airflow/api/common/experimental/trigger_dag.py | 115 +------
.../api/common/{experimental => }/trigger_dag.py | 5 +-
airflow/api_connexion/endpoints/dag_endpoint.py | 7 +-
airflow/cli/commands/dag_command.py | 4 +-
airflow/config_templates/config.yml | 2 +-
airflow/configuration.py | 19 +-
airflow/models/pool.py | 52 +++-
airflow/models/skipmixin.py | 15 +-
airflow/models/taskinstance.py | 4 +-
airflow/models/xcom.py | 343 ++++++++++++++-------
airflow/operators/sql.py | 16 +-
airflow/operators/trigger_dagrun.py | 2 +-
airflow/providers/jenkins/hooks/jenkins.py | 9 +-
airflow/providers/tableau/hooks/tableau.py | 23 +-
airflow/settings.py | 10 +-
airflow/utils/db.py | 15 +
airflow/utils/session.py | 11 +-
airflow/utils/strings.py | 14 +-
airflow/www/views.py | 18 +-
dev/provider_packages/remove_old_releases.py | 10 +-
docs/exts/sphinx_script_update.py | 9 +-
setup.cfg | 1 +
setup.py | 5 +-
tests/api/client/test_local_client.py | 31 +-
.../common/{experimental => }/test_delete_dag.py | 16 +-
.../common/{experimental => }/test_trigger_dag.py | 8 +-
tests/decorators/test_python_virtualenv.py | 41 +++
tests/models/test_pool.py | 71 +++++
tests/models/test_taskinstance.py | 6 +
tests/operators/test_python.py | 34 +-
40 files changed, 641 insertions(+), 419 deletions(-)
copy airflow/api/common/{experimental => }/delete_dag.py (82%)
copy airflow/api/common/{experimental => }/trigger_dag.py (95%)
rename tests/api/common/{experimental => }/test_delete_dag.py (91%)
rename tests/api/common/{experimental => }/test_trigger_dag.py (93%)