You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2022/01/22 15:04:26 UTC
[airflow] branch v2-2-test updated (713a807 -> 81c5e22)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 713a807 Docs: Clarify ``sentry_on`` value is not quoted with example (#20639)
new 97d90c1 breeze setup-autocomplete zshrc reload (#18893)
new 01edc7e Workaround docker-compose-v2 env passing (#18887)
new ef817be Fix comparision of docker versions (#18902)
new 49e582a Remove adding of "test-run" variables to dc_ci script (#18903)
new 40bf532 Decrease likelihood of memory issue in CI (#18852)
new ebf1ff7 Fix typos ``build.rst`` (#18935)
new 56aa22e Fix --github-image-id flag for Breeze (#18882) (#18946)
new 31eea57 Add decription on how you can customize image entrypoint (#18915)
new ecdadf5 CI: Increase parallel test timeout for Helm Chart tests (#18993)
new 7b1fb7e Skip updating constraints when only datetime changes (#19023)
new dd1a3fa Fix wrong commands in docs/breeze
new c8f492b Allow specifying extras when using breeze initialize_local_virtualenv (#19178)
new 80f4e5f Fix breeze docker version parsing (#19182)
new d5ee471 pyenv related docs added, warning message in breeze initialize-local-virtualenv command (#19100)
new 6f6e9c2 Skip triggerer in 'breeze start-airflow' if on 3.6 (#19305)
new 70c5869 Temporarily remove mypy checks to stop PRs from failing (#19345)
new e166a37 Enable mouse mode by default in start_airflow tmux session (#19325)
new cfa1bae Make scripts/in_container/check_environment.sh Google Shell Guide Compliant (#19350)
new 581e974 Fix --disable-mssql-client-installation error (#19295)
new 74cba25 Updates version of airflow in docker examples (#19455)
new 9f0271b Fix docker "after entrypoint" custom script example (#19495)
new 7f41a04 Optimizes running tests for public GitHub Runners. (#19512)
new 94b923d Disable test code coverage for PRs (#19523)
new 75dc1c0 Upload KinD logs on cancell too (#19554)
new 68f729c Stop polling when Webserver doesn't start up in Kube tests (#19598)
new e8a8566 Improve automation for docker image release (#19573)
new 97c0069 Move scripts for prod image preparation to dev (#19623)
new 82964ef Disable yarn-dev in start-airflow command (#19626)
new 4b09fac Fix failing CI phase with unhealthy container issue (#19633)
new 6561e7b Clean up dynamic `start_date` values from docs (#19607)
new 6aacc66 Misc. documentation typos and language improvements (#19599)
new 319d327 Fix dumping container logs on error (#19645)
new cd55810 Add more complete instruction for reproducing failed integration tests (#19646)
new 8a79d7e Fix CI tests so they correctly fail in case of error! (#19678)
new 3276864 Fix speed of yarn installation (#19697)
new 84c523d Speed up webserver start up in Kube tests (#19710)
new 01de1b7 Remove duplicate line call in CI (#19728)
new 8712710 Lower the recommended disk space requirements (#19775)
new 500946a Run Other tests sequentially for Public GitHub runners (#19766)
new da2f943 Add option to run PRs on public runners by maintainers. (#19772)
new c931b9c Sync committers in ci config for self-hosted runners (#19786)
new 069158e Add note to restart runners when updating committers (#19795)
new f9c444c Tests for Docker images in Python (#19737)
new 92b8ac3 Upload provider distribution artifacts during CI (#19807)
new 2e37124 Fixes failure of image building (#19813)
new 85cdc4d Bring back Core and Other tests to be run in parallel (#19812)
new ddf32f3 add kubernetes 1.21 support (#19557)
new 08844cf Fix PATH export in breeze tmux sessions (#19818)
new 208cf43 Use hyphen instead of underscore to match other artifacts (#19820)
new ebf2687 Rewrite image building tests to Python (#19819)
new b7bf671 Fix race condition when running mssql tests (#19863)
new 1e06225 Add tests for docker-compose quick start (#19874)
new a5488eb Remove the docker timeout workaround (#18872)
new 8a3abda Add ``semver`` to devel deps (#18818)
new b45c1c6 Add pandas requirements for providers that use pandas (#18997)
new a4a967f Upgrade the Dataproc package to 3.0.0 and migrate from v1beta2 to v1 api (#18879)
new bc8d1c6 Remove distutils usages for Python 3.10 (#19064)
new 8f5e5b3 Add dataproc metastore operators (#18945)
new 1be34fc Add hdfs requirement for hdfs provider (#19540)
new 70227ef Fix failures with recent moto library 2.2.15 (#19693)
new b176bef Create dataproc serverless spark batches operator (#19248)
new 5fa26e2 Move to watchtower 2.0.1 (#19907)
new 0c62f00 Fix ``breeze kind-cluster shell`` (#20015)
new f136bf0 Remove postgres 9.6 support (#19987)
new a7d3208 update upper bound for MarkupSafe (#19953)
new 8659399 switch to follow_redirects on httpx.get call in CloudSQL provider (#20239)
new 1450dfc Lift off upper bound for MarkupSafe (#20113)
new 3bce924 Upload provider artifacts before install/test step (#20137)
new af35b37 Update Sphinx and Sphinx-AutoAPI (#20079)
new 477927a Update minimum sphinx versions after upgrading sphinx-autoapi (#20170)
new 03a6d81 Increase limit of time for constraint job (#20230)
new bf66482 Remove Integration tests from MSSQL on Public Runners (#20231)
new 28038cc YandexCloud provider: Support new Yandex SDK features: log_group_id, user-agent, maven packages (#20103)
new fbc6319 Speed up Helm Upgrade tests (#20289)
new d2a5afa Fix race condition when flake checks run in parallel (#20294)
new 54c134a Change default python version in docker image docs (#20389)
new 9b929d5 Limit Snowflake connector to< 2.7.2 (#20395)
new 63dc2f8 Add pre-commit that checks credentials are not persisted in CI (#20430)
new 1276bc3 Add custom pip.conf to docker-context-files (#20445)
new 98e2f96 Checks if the user running Breeze has permissions to run docker cmd (#20462)
new 93c69d6 Add autoflake precommit to automatically remove unused code (#20466)
new f733316 Bump PyJWT from `<2` to `<3` (#20490)
new 43ae88b Fix: pin pymongo < 4.0.0 (#20511)
new d7201fd Increase time limit for Helm chart unit tests (#20525)
new 02b90f6 Remove PyJWT upper bound from Dockerfile (#20503)
new 9fbb214 Bump croniter from `<1.1` to `<1.2` (#20489)
new 3cfec5a Update "Release Airflow" doc (#19111)
new e759d5f Dev: Clarify file naming in release verification doc (#19233)
new 730c4aa Fix release check script (#19238)
new 4b0ff5f Clarify rat test guidance in release check docs (#19296)
new 4c7d7af Update known warnings for Python 3.7 (#19333)
new f818f3b Clarify guidance re trust of keys in release docs (#19480)
new 91e7848 Update description of release process for adding new major release (#19483)
new b26f254 Update helm chart release docs (#19494)
new 254fab4 Clarify that .asf.yml and codecov.yml should be changed in main (#19496)
new 38c0dfc Add script to generate issue for status of testing of the rc (#19247)
new 9253494 Add script to generate chart changelog annotations (#20555)
new 680df3c Update docs about releasing providersk (#19549)
new 6593bee Fix failing CI phase with unhealthy container issue (#19633)
new 43fef12 Fix dumping container logs on error (#19645)
new 91b1bb8 Update Airflow release guide (#19663)
new 939f6f3 Fix CI tests so they correctly fail in case of error! (#19678)
new 3d0b5c0 Add retagging images accross repos (#19778)
new 7d741df Upload provider distribution artifacts during CI (#19807)
new 7850c34 Bring back Core and Other tests to be run in parallel (#19812)
new d5a272d Fix OOM error in tests when using public Github Runners. (#19809)
new e853750 Update docs/tools for releasing core Airflow (#20211)
new 52c5a71 Fix MyPy errors in `dev/*` (#20261)
new 8e5ae7c Fix MyPy errors in `scripts/in_container` (#20280)
new 93c5092 Only list linked issues once in release issues (#20299)
new 467f636 Add exiting on error in prod image script (#20447)
new df5ed58 Add possibility to ignore common deprecated message (#20444)
new 69e1bea Update the "releasing Airflow" docs (#20456)
new daa2663 Generate version documentation from single source of truth (#20594)
new d4461d9 Add known warning generated by snowflake new version (#20604)
new d09643f Add twine check for provider packages (#20619)
new a48e61b Fix generation of "Status provider" issue (#20621)
new 069ae9c Generate constraints in PRs when upgrading dependencies (#20624)
new b53fdf6 Fix Constraints failure in PRs (#20631)
new 6406e3a upgrade celery 5.2.3 (#19703)
new f93b588 Bump flask-appbuilder to >=3.3.4 (#20628)
new 8b90e55 Fix incorrect arguments (#20638)
new 131d782 Remove Python 2 from our images (#20680)
new dc03e16 Cleaner output of docker image building scripts (#20679)
new 70b1f9c Be build -> built, and a stray space (#20703)
new e4400c3 Fix naming convention for sdist provider packages (#20711)
new ccc3a92 Add color to pytest tests on CI (#20723)
new 94ea009 Modernize usage of PIP in Airflow images (#20726)
new 45eedad Uses airflow user for build segment of docker image (#20744)
new 819d221 Cleaner output for Docker image building (#20747)
new 475298c Optimize dockerfiles for local rebuilds (#20238)
new 47b2259 Unpin ``cattrs`` (#20872)
new 81845ce Unpin ``argcomplete`` and ``colorlog`` (#20878)
new 64de384 add entry in release readme to update milestone in Issues (#20890)
new 284fdc2 Switch to new MySQL public key (#20912)
new ad392a3 Uses CI images built in previous step to prepare PROD image (#20889)
new fb0f634 Switch to non-vendored latest connexion library (#20910)
new abc213f Fix errors thrown by some versions of Bash v4 (#20932)
new b772374 Switch to 'buildkit' to build Airflow images (#20664)
new 57f25e2 Fix new buildkit builds on MacOS (#20963)
new ce9fc3b Update base python image to be Python 3.7 by default (#20978)
new 95de5e5 Add extra sync when adding executable flag to installation scripts (#20987)
new 5ffe160 Get rid of upload coverage warnings (#20994)
new 1fa244d Update refreshing constraints instructions (#21001)
new 38e7a6e Update md5 information about image after waiting (#21000)
new 81c5e22 Update Celery requirements
The 146 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/CODEOWNERS | 5 +-
.github/boring-cyborg.yml | 2 +-
.github/workflows/build-images.yml | 15 +-
.github/workflows/ci.yml | 182 +++-
.pre-commit-config.yaml | 24 +-
BREEZE.rst | 224 ++--
CI.rst | 18 +-
CONTRIBUTORS_QUICK_START.rst | 6 +-
Dockerfile | 354 ++++---
Dockerfile.ci | 171 ++--
IMAGES.rst | 176 +---
INSTALL | 4 +-
LOCAL_VIRTUALENV.rst | 7 +
PULL_REQUEST_WORKFLOW.rst | 16 +-
README.md | 35 +-
STATIC_CODE_CHECKS.rst | 6 +
TESTING.rst | 4 +-
UPDATING.md | 6 +-
airflow/_vendor/connexion/__init__.py | 48 -
airflow/_vendor/connexion/__main__.py | 3 -
airflow/_vendor/connexion/apis/__init__.py | 1 -
airflow/_vendor/connexion/apis/abstract.py | 446 --------
airflow/_vendor/connexion/apis/aiohttp_api.py | 394 --------
airflow/_vendor/connexion/apis/flask_api.py | 310 ------
airflow/_vendor/connexion/apis/flask_utils.py | 81 --
airflow/_vendor/connexion/apps/__init__.py | 1 -
airflow/_vendor/connexion/apps/abstract.py | 249 -----
airflow/_vendor/connexion/apps/aiohttp_app.py | 95 --
airflow/_vendor/connexion/apps/flask_app.py | 138 ---
airflow/_vendor/connexion/cli.py | 210 ----
airflow/_vendor/connexion/decorators/__init__.py | 0
.../connexion/decorators/coroutine_wrappers.py | 53 -
airflow/_vendor/connexion/decorators/decorator.py | 51 -
airflow/_vendor/connexion/decorators/metrics.py | 54 -
airflow/_vendor/connexion/decorators/parameter.py | 123 ---
airflow/_vendor/connexion/decorators/produces.py | 49 -
airflow/_vendor/connexion/decorators/response.py | 112 --
airflow/_vendor/connexion/decorators/security.py | 341 -------
.../_vendor/connexion/decorators/uri_parsing.py | 329 ------
airflow/_vendor/connexion/decorators/validation.py | 386 -------
airflow/_vendor/connexion/exceptions.py | 142 ---
airflow/_vendor/connexion/handlers.py | 85 --
airflow/_vendor/connexion/http_facts.py | 15 -
airflow/_vendor/connexion/json_schema.py | 114 ---
airflow/_vendor/connexion/jsonifier.py | 57 --
airflow/_vendor/connexion/lifecycle.py | 41 -
airflow/_vendor/connexion/mock.py | 47 -
airflow/_vendor/connexion/operations/__init__.py | 8 -
airflow/_vendor/connexion/operations/abstract.py | 445 --------
airflow/_vendor/connexion/operations/compat.py | 3 -
airflow/_vendor/connexion/operations/openapi.py | 380 -------
airflow/_vendor/connexion/operations/secure.py | 164 ---
airflow/_vendor/connexion/operations/swagger2.py | 310 ------
airflow/_vendor/connexion/options.py | 144 ---
airflow/_vendor/connexion/problem.py | 42 -
airflow/_vendor/connexion/resolver.py | 192 ----
airflow/_vendor/connexion/setup.cfg | 7 -
airflow/_vendor/connexion/setup.py | 119 ---
airflow/_vendor/connexion/spec.py | 262 -----
airflow/_vendor/connexion/utils.py | 250 -----
.../api_connexion/endpoints/connection_endpoint.py | 2 +-
airflow/api_connexion/endpoints/dag_endpoint.py | 2 +-
.../api_connexion/endpoints/dag_run_endpoint.py | 2 +-
.../endpoints/role_and_permission_endpoint.py | 2 +-
airflow/api_connexion/endpoints/user_endpoint.py | 2 +
airflow/api_connexion/exceptions.py | 2 +-
airflow/configuration.py | 19 +-
airflow/dag_processing/manager.py | 2 +-
airflow/exceptions.py | 4 +-
airflow/hooks/dbapi.py | 2 +-
airflow/kubernetes/pod.py | 2 +-
airflow/kubernetes/pod_launcher.py | 4 +-
airflow/kubernetes/pod_runtime_info_env.py | 4 +-
airflow/kubernetes/volume.py | 5 +-
airflow/kubernetes/volume_mount.py | 5 +-
.../2c6edca13270_resource_based_permissions.py | 1 -
...ad25_resource_based_permissions_for_default_.py | 1 -
airflow/models/dag.py | 3 +-
airflow/operators/generic_transfer.py | 2 +-
airflow/operators/sql.py | 16 +-
.../amazon/aws/log/cloudwatch_task_handler.py | 2 +-
airflow/providers/docker/operators/docker_swarm.py | 7 -
.../google/cloud/example_dags/example_dataproc.py | 49 +
.../example_dags/example_dataproc_metastore.py | 216 ++++
airflow/providers/google/cloud/hooks/cloud_sql.py | 10 +-
airflow/providers/google/cloud/hooks/dataproc.py | 221 +++-
.../google/cloud/hooks/dataproc_metastore.py | 676 +++++++++++++
airflow/providers/google/cloud/hooks/workflows.py | 4 +-
.../providers/google/cloud/operators/dataproc.py | 338 ++++++-
.../google/cloud/operators/dataproc_metastore.py | 1068 ++++++++++++++++++++
.../providers/google/cloud/operators/workflows.py | 4 +-
airflow/providers/google/cloud/sensors/dataproc.py | 2 +-
.../google/cloud/utils/credentials_provider.py | 10 +-
.../cloud/utils/mlengine_prediction_summary.py | 4 +
.../google/common/utils/id_token_credentials.py | 4 +
airflow/providers/google/provider.yaml | 11 +
airflow/providers/jenkins/hooks/jenkins.py | 9 +-
airflow/providers/postgres/hooks/postgres.py | 4 +-
airflow/providers/sqlite/hooks/sqlite.py | 2 +-
airflow/providers/tableau/hooks/tableau.py | 23 +-
.../example_dags/example_yandexcloud_dataproc.py | 12 +-
airflow/providers/yandex/hooks/yandex.py | 16 +-
.../yandex/operators/yandexcloud_dataproc.py | 40 +
airflow/sensors/base.py | 18 +-
airflow/smart_sensor_dags/smart_sensor_group.py | 10 +-
airflow/timetables/base.py | 1 -
airflow/utils/db.py | 2 -
airflow/utils/strings.py | 14 +-
airflow/www/ask_for_recompile_assets_if_needed.sh | 19 +-
airflow/www/decorators.py | 1 -
airflow/www/extensions/init_views.py | 5 +-
airflow/www/views.py | 8 -
breeze | 215 ++--
breeze-complete | 13 +-
chart/README.md | 2 +-
dev/ISSUE_TEMPLATE.md.jinja2 | 21 +
dev/README_RELEASE_AIRFLOW.md | 388 +++++--
dev/README_RELEASE_AIRFLOW_UPGRADE_CHECK.md | 37 +-
dev/README_RELEASE_HELM_CHART.md | 147 ++-
dev/README_RELEASE_PROVIDER_PACKAGES.md | 80 +-
dev/REFRESHING_CI_CACHE.md | 7 +-
dev/airflow-github | 9 +-
dev/airflow-license | 2 +-
dev/chart/build_changelog_annotations.py | 108 ++
dev/check_files.py | 159 ++-
dev/import_all_classes.py | 4 +-
.../ci/tools => dev}/prepare_prod_docker_images.sh | 35 +-
dev/prepare_release_issue.py | 298 ++++++
.../PROVIDER_ISSUE_TEMPLATE.md.jinja2 | 15 +-
dev/provider_packages/prepare_provider_packages.py | 193 +++-
dev/provider_packages/remove_old_releases.py | 10 +-
dev/refresh_images.sh | 7 +-
dev/retag_docker_images.py | 33 +-
dev/validate_version_added_fields_in_config.py | 8 +-
{tests/timetables => docker_tests}/__init__.py | 0
.../test_utils.py => docker_tests/command_utils.py | 37 +-
.../libs/helper.py => docker_tests/constants.py | 5 +-
docker_tests/docker_tests_utils.py | 100 ++
.../test_ci_image.py | 34 +-
docker_tests/test_docker_compose_quick_start.py | 161 +++
.../test_examples_of_prod_image_building.py | 64 ++
docker_tests/test_prod_image.py | 211 ++++
.../index.rst | 1 -
docs/apache-airflow-providers-google/index.rst | 2 +-
.../operators/cloud/dataproc.rst | 49 +
.../operators/cloud/dataproc_metastore.rst | 196 ++++
docs/apache-airflow/best-practices.rst | 7 +-
docs/apache-airflow/concepts/dags.rst | 8 +-
docs/apache-airflow/concepts/operators.rst | 3 +-
docs/apache-airflow/concepts/scheduler.rst | 4 +-
docs/apache-airflow/dag-run.rst | 10 +-
docs/apache-airflow/executor/kubernetes.rst | 6 +-
docs/apache-airflow/howto/set-up-database.rst | 2 +-
docs/apache-airflow/installation/prerequisites.rst | 4 +-
.../installation/supported-versions.rst | 28 +-
docs/apache-airflow/lineage.rst | 9 +-
docs/apache-airflow/tutorial.rst | 3 +-
docs/conf.py | 21 +
docs/docker-stack/build-arg-ref.rst | 14 +-
docs/docker-stack/build.rst | 32 +-
.../customizing/add-build-essential-custom.sh | 7 +-
.../docker-examples/customizing/custom-sources.sh | 7 +-
.../customizing/github-different-repository.sh | 1 +
.../docker-examples/customizing/github-main.sh | 1 +
.../{github-v2-1-test.sh => github-v2-2-test.sh} | 9 +-
.../customizing/pypi-dev-runtime-deps.sh | 7 +-
.../customizing/pypi-extras-and-deps.sh | 5 +-
.../customizing/pypi-selected-version.sh | 5 +-
.../extending/embedding-dags/test_dag.py | 10 +-
.../restricted/restricted_environments.sh | 13 +-
docs/docker-stack/entrypoint.rst | 10 +-
docs/docker-stack/index.rst | 4 +-
docs/docker-stack/recipes.rst | 2 +
docs/exts/docs_build/docs_builder.py | 5 +-
docs/exts/docs_build/run_patched_sphinx.py | 105 --
docs/exts/exampleinclude.py | 10 +-
docs/exts/sphinx_script_update.py | 9 +-
docs/helm-chart/index.rst | 2 +-
docs/helm-chart/manage-dags-files.rst | 4 +-
docs/helm-chart/quick-start.rst | 10 +-
docs/publish_docs.py | 2 +-
docs/spelling_wordlist.txt | 27 +-
licenses/LICENSE-connexion.txt | 9 -
scripts/ci/constraints/ci_branch_constraints.sh | 7 +-
scripts/ci/constraints/ci_commit_constraints.sh | 3 +-
scripts/ci/docker-compose/_docker.env | 4 +-
scripts/ci/docker-compose/_docker_compose.env | 49 -
scripts/ci/docker-compose/backend-mssql.yml | 8 +-
scripts/ci/docker-compose/base.yml | 56 +-
scripts/ci/docker-compose/integration-pinot.yml | 2 +-
scripts/ci/docker-compose/integration-statsd.yml | 2 +-
scripts/ci/docker-compose/local.yml | 1 +
scripts/ci/images/ci_prepare_ci_image_on_ci.sh | 5 -
scripts/ci/images/ci_prepare_prod_image_on_ci.sh | 4 -
scripts/ci/images/ci_push_ci_images.sh | 10 +-
scripts/ci/images/ci_push_production_images.sh | 11 +-
...h => ci_run_docker_compose_quick_start_test.sh} | 11 +-
scripts/ci/images/ci_run_docker_tests.py | 96 ++
scripts/ci/images/ci_run_prod_image_test.sh | 2 +-
.../ci_test_examples_of_prod_image_building.sh | 70 +-
.../images/ci_wait_for_and_verify_all_ci_images.sh | 2 +
.../ci_wait_for_and_verify_all_prod_images.sh | 2 +
.../ci/images/ci_wait_for_and_verify_ci_image.sh | 2 +
scripts/ci/installed_providers.txt | 22 +
scripts/ci/kubernetes/ci_run_helm_upgrade.sh | 6 +-
scripts/ci/kubernetes/ci_run_kubernetes_tests.sh | 4 +-
scripts/ci/libraries/_all_libs.sh | 2 -
scripts/ci/libraries/_build_images.sh | 449 +++-----
scripts/ci/libraries/_docker_engine_resources.sh | 2 +-
scripts/ci/libraries/_initialization.sh | 139 ++-
scripts/ci/libraries/_kind.sh | 3 +
scripts/ci/libraries/_local_mounts.sh | 1 +
scripts/ci/libraries/_md5sum.sh | 19 -
scripts/ci/libraries/_push_pull_remove_images.sh | 214 +---
scripts/ci/libraries/_runs.sh | 10 +-
scripts/ci/libraries/_spinner.sh | 55 -
scripts/ci/libraries/_start_end.sh | 22 -
scripts/ci/libraries/_testing.sh | 16 +-
scripts/ci/libraries/_verbosity.sh | 2 +-
scripts/ci/libraries/_verify_image.sh | 366 +------
.../pre_commit_checkout_no_credentials.py | 87 ++
scripts/ci/pre_commit/pre_commit_ci_build.sh | 30 +-
scripts/ci/pre_commit/pre_commit_flake8.sh | 2 +-
scripts/ci/pre_commit/pre_commit_mypy.sh | 5 +-
.../ci/pre_commit/pre_commit_update_versions.py | 25 +-
scripts/ci/pre_commit/supported_versions.py | 63 ++
.../ci_install_and_test_provider_packages.sh | 2 +-
.../ci/static_checks/in_container_bats_tests.sh | 4 +-
scripts/ci/static_checks/mypy.sh | 2 +-
scripts/ci/static_checks/ui_lint.sh | 2 +-
scripts/ci/static_checks/www_lint.sh | 2 +-
scripts/ci/testing/ci_run_airflow_testing.sh | 73 +-
.../ci_run_single_airflow_test_in_docker.sh | 27 +-
scripts/ci/tools/build_dockerhub.sh | 1 -
scripts/ci/tools/fix_ownership.sh | 6 +-
scripts/ci/tools/verify_docker_image.sh | 57 --
scripts/docker/common.sh | 34 +-
scripts/docker/compile_www_assets.sh | 46 +-
scripts/docker/install_additional_dependencies.sh | 30 +-
scripts/docker/install_airflow.sh | 32 +-
...install_airflow_dependencies_from_branch_tip.sh | 16 +-
.../docker/install_from_docker_context_files.sh | 26 +-
scripts/docker/install_mssql.sh | 15 +-
scripts/docker/install_mysql.sh | 16 +-
scripts/docker/install_pip_version.sh | 20 +-
...mpile_www_assets.sh => prepare_node_modules.sh} | 35 +-
scripts/in_container/_in_container_utils.sh | 18 +-
scripts/in_container/bin/run_tmux | 18 +-
scripts/in_container/check_environment.sh | 58 +-
scripts/in_container/check_junitxml_result.py | 56 +
scripts/in_container/entrypoint_ci.sh | 21 +-
scripts/in_container/prod/entrypoint_prod.sh | 2 +-
scripts/in_container/run_generate_constraints.sh | 2 +-
.../run_install_and_test_provider_packages.sh | 22 +
.../in_container/run_prepare_airflow_packages.sh | 2 +-
scripts/in_container/run_resource_check.sh | 4 +-
scripts/in_container/run_tmux_welcome.sh | 4 +-
setup.cfg | 35 +-
setup.py | 69 +-
tests/dag_processing/test_manager.py | 3 +
tests/decorators/test_python_virtualenv.py | 41 -
tests/operators/test_python.py | 34 +-
.../amazon/aws/hooks/test_cloud_formation.py | 19 +-
tests/providers/amazon/aws/hooks/test_logs.py | 4 +-
.../amazon/aws/log/test_cloudwatch_task_handler.py | 34 +-
.../docker/operators/test_docker_swarm.py | 48 -
.../providers/google/cloud/hooks/test_dataproc.py | 120 ++-
.../google/cloud/hooks/test_dataproc_metastore.py | 489 +++++++++
.../google/cloud/operators/test_dataproc.py | 126 +++
.../cloud/operators/test_dataproc_metastore.py | 396 ++++++++
.../test_dataproc_metastore_system.py} | 20 +-
.../google/cloud/operators/test_dataproc_system.py | 4 +
.../google/cloud/sensors/test_dataproc.py | 2 +-
.../yandex/operators/test_yandexcloud_dataproc.py | 11 +
tests/task/__init__.py | 2 -
.../task/task_runner/test_standard_task_runner.py | 2 +
tests/www/views/test_views_rendered.py | 2 +-
277 files changed, 8151 insertions(+), 9368 deletions(-)
delete mode 100755 airflow/_vendor/connexion/__init__.py
delete mode 100644 airflow/_vendor/connexion/__main__.py
delete mode 100644 airflow/_vendor/connexion/apis/__init__.py
delete mode 100644 airflow/_vendor/connexion/apis/abstract.py
delete mode 100644 airflow/_vendor/connexion/apis/aiohttp_api.py
delete mode 100644 airflow/_vendor/connexion/apis/flask_api.py
delete mode 100644 airflow/_vendor/connexion/apis/flask_utils.py
delete mode 100644 airflow/_vendor/connexion/apps/__init__.py
delete mode 100644 airflow/_vendor/connexion/apps/abstract.py
delete mode 100644 airflow/_vendor/connexion/apps/aiohttp_app.py
delete mode 100644 airflow/_vendor/connexion/apps/flask_app.py
delete mode 100644 airflow/_vendor/connexion/cli.py
delete mode 100644 airflow/_vendor/connexion/decorators/__init__.py
delete mode 100644 airflow/_vendor/connexion/decorators/coroutine_wrappers.py
delete mode 100644 airflow/_vendor/connexion/decorators/decorator.py
delete mode 100644 airflow/_vendor/connexion/decorators/metrics.py
delete mode 100644 airflow/_vendor/connexion/decorators/parameter.py
delete mode 100644 airflow/_vendor/connexion/decorators/produces.py
delete mode 100644 airflow/_vendor/connexion/decorators/response.py
delete mode 100644 airflow/_vendor/connexion/decorators/security.py
delete mode 100644 airflow/_vendor/connexion/decorators/uri_parsing.py
delete mode 100644 airflow/_vendor/connexion/decorators/validation.py
delete mode 100644 airflow/_vendor/connexion/exceptions.py
delete mode 100644 airflow/_vendor/connexion/handlers.py
delete mode 100644 airflow/_vendor/connexion/http_facts.py
delete mode 100644 airflow/_vendor/connexion/json_schema.py
delete mode 100644 airflow/_vendor/connexion/jsonifier.py
delete mode 100644 airflow/_vendor/connexion/lifecycle.py
delete mode 100644 airflow/_vendor/connexion/mock.py
delete mode 100644 airflow/_vendor/connexion/operations/__init__.py
delete mode 100644 airflow/_vendor/connexion/operations/abstract.py
delete mode 100644 airflow/_vendor/connexion/operations/compat.py
delete mode 100644 airflow/_vendor/connexion/operations/openapi.py
delete mode 100644 airflow/_vendor/connexion/operations/secure.py
delete mode 100644 airflow/_vendor/connexion/operations/swagger2.py
delete mode 100644 airflow/_vendor/connexion/options.py
delete mode 100644 airflow/_vendor/connexion/problem.py
delete mode 100644 airflow/_vendor/connexion/resolver.py
delete mode 100644 airflow/_vendor/connexion/setup.cfg
delete mode 100755 airflow/_vendor/connexion/setup.py
delete mode 100644 airflow/_vendor/connexion/spec.py
delete mode 100644 airflow/_vendor/connexion/utils.py
create mode 100644 airflow/providers/google/cloud/example_dags/example_dataproc_metastore.py
create mode 100644 airflow/providers/google/cloud/hooks/dataproc_metastore.py
create mode 100644 airflow/providers/google/cloud/operators/dataproc_metastore.py
create mode 100644 dev/ISSUE_TEMPLATE.md.jinja2
create mode 100755 dev/chart/build_changelog_annotations.py
rename {scripts/ci/tools => dev}/prepare_prod_docker_images.sh (57%)
create mode 100755 dev/prepare_release_issue.py
copy {tests/timetables => docker_tests}/__init__.py (100%)
copy tests/providers/alibaba/cloud/utils/test_utils.py => docker_tests/command_utils.py (53%)
copy airflow/example_dags/libs/helper.py => docker_tests/constants.py (91%)
create mode 100644 docker_tests/docker_tests_utils.py
copy airflow/ti_deps/deps/dagrun_exists_dep.py => docker_tests/test_ci_image.py (53%)
create mode 100644 docker_tests/test_docker_compose_quick_start.py
create mode 100644 docker_tests/test_examples_of_prod_image_building.py
create mode 100644 docker_tests/test_prod_image.py
create mode 100644 docs/apache-airflow-providers-google/operators/cloud/dataproc_metastore.rst
rename docs/docker-stack/docker-examples/customizing/{github-v2-1-test.sh => github-v2-2-test.sh} (84%)
delete mode 100755 docs/exts/docs_build/run_patched_sphinx.py
delete mode 100644 licenses/LICENSE-connexion.txt
delete mode 100644 scripts/ci/docker-compose/_docker_compose.env
copy scripts/ci/images/{ci_push_production_images.sh => ci_run_docker_compose_quick_start_test.sh} (72%)
create mode 100755 scripts/ci/images/ci_run_docker_tests.py
create mode 100644 scripts/ci/installed_providers.txt
delete mode 100644 scripts/ci/libraries/_spinner.sh
create mode 100755 scripts/ci/pre_commit/pre_commit_checkout_no_credentials.py
create mode 100755 scripts/ci/pre_commit/supported_versions.py
delete mode 100755 scripts/ci/tools/verify_docker_image.sh
copy scripts/docker/{compile_www_assets.sh => prepare_node_modules.sh} (63%)
create mode 100755 scripts/in_container/check_junitxml_result.py
create mode 100644 tests/providers/google/cloud/hooks/test_dataproc_metastore.py
create mode 100644 tests/providers/google/cloud/operators/test_dataproc_metastore.py
copy tests/providers/google/cloud/{transfers/test_gdrive_to_gcs_system.py => operators/test_dataproc_metastore_system.py} (71%)