You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ep...@apache.org on 2023/03/08 20:12:20 UTC
[airflow] branch v2-5-test updated (cfbfe7fd8e -> 9f8aa37f8a)
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a change to branch v2-5-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit cfbfe7fd8e Move help message to the google auth code (#29888)
omit 88f725e1ee Replace `unittests` in google provider tests by pure `pytest` (Part 1) (#27986)
omit 4744bc7f2d Updates to check_files to produce a more useful Dockerfile (#29469)
omit 83aa6d036a Avoid modifying PRs in Recheck old bug reports workflow (#29653)
omit 904dcd919f Add pre-commit hook to check session default value (#28007)
omit 8b1c44cce8 Add interactive testing in Breeze option in testing docs (#29340)
omit 7cd2ceccae Adjust some wording in the release process (#29278)
omit b0fa28cb84 docs: fix typo (#29256)
omit 1c93eb21f2 Refresh the CI documentation to reflect latest changes (#29217)
omit cac2f3f7c8 Fix MySQL integrations tests run (#29213)
omit 3bfcd8e391 pre-commit: add hook to sort `installed_providers.txt` (#29219)
omit 0211ea207e Also remove mypy cache when stopping breeze (#29184)
omit b87990dc9d Make sure DOCKER_BUILDKIT=1 variable is set for all builds (#29928)
omit d0966cfc2c Limit importlib-metadata backwport to < 5.0.0 (#29924)
omit b37ec74e02 Upgrade pip to version 23.0.1 (#29767)
omit ffdb121523 Propose to upgrade to newer dependencies in case build CI image fail (#29705)
omit 99b1cc6707 Do not crash when a version fails to parse (#29685)
omit 458c41fda6 Migrate breeze unit tests to pytest. (#29639)
omit cf89b09a62 Use "all" integrations to run tests when "all" is used (#29517)
omit 6a240c3b26 Bump stale bot version (#29491)
omit 3a3932440b Upgrade PIP to 23.0 version (#29421)
omit 4451aba667 Fix `breeze static_checks` type arg (#29292)
omit d2840e97cf Fix commit message for constraints (#29215)
omit d56606ea97 Fix asset compilation in start-airflow (#29185)
omit fbada04257 Fix `project-name` generation for Providers `test-type` in Breeze (#29030)
omit 04d1c17efc Fix hot loop with a burst of log messages (#28778) (#28780)
omit 626b1514d4 Exclude dependedbot PRs from status testing report (#28954)
omit 7945514559 Update boring-cyborg.yml with more PR labels (#28955)
omit 2b377377e4 Fix a few remaining references to flake8 (#28915)
omit a0045495a4 Removal of flake8 file has broken breeze bind mounts (#28909)
omit 6180b2d01c Fix pyzmq warning stopping provider package tests (#28901)
omit 81ca55c1e2 Fix bugs introduced when moving the code of issue generation to breeze (#28683)
omit 4e2af12f99 Make static checks generated file more stable accross the board (#29080)
omit 14fdfc07ac Switch to ruff for faster static checks (#28893)
omit 587b144569 Update black version automatically in pre-commit configuration (#28578)
omit 43b8ea65a2 Rerun flaky PinotDB integration test (#28562)
omit 56b0b76b3a Variables set in variables.env are automatically exported (#28633)
omit 6f25448868 Improve "other" test category selection (#28630)
omit 393bba236e Update codespell and fix typos (#28568)
omit 3784f2e25b add hostname argument to DockerOperator (#27822)
omit 7787fdb10f Fix discoverability of tests for ARM in Breeze (#28432)
omit d503914b60 Add deferrable mode to CloudBuildCreateBuildOperator (#27783)
omit 2ecbfd0118 Replace freezegun with time-machine (#28193)
omit 154ad9a02b Migrate amazon provider sensor tests from `unittests` to `pytest` (#28139)
omit caca35c546 Migrate remaining core sensors tests to `pytest` (#28204)
This update removed existing revisions from the reference, leaving the
reference pointing at a previous point in the repository history.
* -- * -- N refs/heads/v2-5-test (9f8aa37f8a)
\
O -- O -- O (cfbfe7fd8e)
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.codespellignorelines | 2 -
.dockerignore | 1 +
.flake8 | 8 +
.gitattributes | 1 +
.github/boring-cyborg.yml | 61 +----
.github/workflows/ci.yml | 23 +-
.github/workflows/recheck-old-bug-report.yml | 53 -----
.github/workflows/stale.yml | 21 +-
.pre-commit-config.yaml | 103 ++++----
.rat-excludes | 1 +
CI.rst | 128 +++-------
CI_DIAGRAMS.md | 85 ++-----
CONTRIBUTING.rst | 9 +-
CONTRIBUTORS_QUICK_START.rst | 4 +-
Dockerfile | 7 +-
Dockerfile.ci | 17 +-
IMAGES.rst | 2 +-
RELEASE_NOTES.rst | 2 +-
STATIC_CODE_CHECKS.rst | 36 +--
TESTING.rst | 75 +++---
...et_candidate_file_descriptor_ranges-to-us.patch | 76 ------
airflow/_vendor/daemon/module.mk | 27 ---
airflow/api/client/api_client.py | 2 +-
airflow/api/common/mark_tasks.py | 4 +-
airflow/cli/commands/connection_command.py | 2 +-
airflow/compat/functools.pyi | 1 -
airflow/decorators/__init__.pyi | 13 +-
airflow/example_dags/example_sensor_decorator.py | 1 -
airflow/example_dags/tutorial_taskflow_api.py | 1 -
airflow/hooks/dbapi.py | 6 +-
airflow/jobs/backfill_job.py | 8 +-
airflow/jobs/base_job.py | 3 +-
airflow/migrations/db_types.pyi | 1 -
airflow/models/taskinstance.py | 12 +-
airflow/models/xcom_arg.py | 1 -
airflow/providers/amazon/aws/hooks/emr.py | 10 +-
airflow/providers/amazon/aws/operators/sns.py | 2 +-
.../amazon/aws/transfers/dynamodb_to_s3.py | 4 +-
.../providers/cncf/kubernetes/utils/__init__.py | 2 -
airflow/providers/docker/operators/docker.py | 4 -
.../providers/google/cloud/hooks/cloud_build.py | 98 +-------
.../google/cloud/operators/cloud_build.py | 90 ++-----
.../providers/google/cloud/operators/dataproc.py | 2 +-
.../google/cloud/operators/kubernetes_engine.py | 2 +-
.../providers/google/cloud/triggers/cloud_build.py | 125 ----------
.../google/common/utils/id_token_credentials.py | 26 +-
airflow/providers/microsoft/azure/hooks/wasb.py | 2 +-
airflow/providers/odbc/hooks/odbc.py | 2 +-
airflow/providers/sftp/hooks/sftp.py | 2 +-
airflow/sentry.py | 6 +-
airflow/utils/code_utils.py | 2 +-
airflow/utils/context.pyi | 5 +-
airflow/utils/db_cleanup.py | 2 +-
airflow/utils/log/action_logger.py | 1 -
airflow/utils/process_utils.py | 4 +-
airflow/www/extensions/init_appbuilder.py | 6 +-
airflow/www/security.py | 4 +-
airflow/www/utils.py | 3 +-
airflow/www/views.py | 2 +-
dev/ISSUE_TEMPLATE.md.jinja2 | 7 +-
dev/README_RELEASE_HELM_CHART.md | 4 +-
dev/README_RELEASE_PROVIDER_PACKAGES.md | 14 +-
.../0002-implement-standalone-python-command.md | 2 +-
.../0008-fixing-group-permissions-before-build.md | 2 +-
.../airflow_breeze/commands/ci_image_commands.py | 47 ++--
.../airflow_breeze/commands/developer_commands.py | 12 +-
.../src/airflow_breeze/commands/main_command.py | 4 +-
.../commands/production_image_commands.py | 6 +-
.../commands/release_management_commands.py | 8 +-
.../airflow_breeze/commands/testing_commands.py | 8 +-
dev/breeze/src/airflow_breeze/global_constants.py | 11 +-
.../src/airflow_breeze/params/shell_params.py | 4 +-
dev/breeze/src/airflow_breeze/pre_commit_ids.py | 11 +-
.../pre_commit_ids_TEMPLATE.py.jinja2 | 1 -
.../airflow_breeze/utils/docker_command_utils.py | 1 +
.../utils/find_newer_dependencies.py | 5 +-
dev/breeze/src/airflow_breeze/utils/run_utils.py | 12 +-
.../src/airflow_breeze/utils/selective_checks.py | 10 +-
dev/breeze/tests/test_run_utils.py | 3 +-
dev/breeze/tests/test_selective_checks.py | 25 +-
dev/check_files.py | 7 +-
dev/deprecations/generate_deprecated_dicts.py | 217 +++++++++++++++++
dev/prepare_release_issue.py | 3 -
dev/provider_packages/prepare_provider_packages.py | 18 +-
dev/stats/get_important_pr_candidates.py | 2 +-
.../operators/cloud/cloud_build.rst | 50 ----
docs/apache-airflow/extra-packages-ref.rst | 2 +-
docs/apache-airflow/img/airflow_erd.sha256 | 2 +-
docs/build_docs.py | 6 +-
docs/docker-stack/build-arg-ref.rst | 2 +-
docs/exts/provider_init_hack.py | 4 +-
docs/spelling_wordlist.txt | 3 -
images/breeze/output-commands-hash.txt | 6 +-
...t_release-management_generate-issue-content.svg | 60 ++---
images/breeze/output_static-checks.svg | 78 +++---
images/breeze/output_stop.svg | 24 +-
provider_packages/.flake8 | 1 +
pyproject.toml | 108 +--------
scripts/ci/constraints/ci_commit_constraints.sh | 8 +-
scripts/ci/docker-compose/local.yml | 3 +
scripts/ci/installed_providers.txt | 1 +
.../ci/pre_commit/common_precommit_black_utils.py | 44 ----
scripts/ci/pre_commit/common_precommit_utils.py | 3 +-
.../pre_commit_check_pre_commit_hooks.py | 72 ++++--
.../ci/pre_commit/pre_commit_compile_www_assets.py | 2 +-
...migration_reference.py => pre_commit_flake8.py} | 13 +-
scripts/ci/pre_commit/pre_commit_insert_extras.py | 4 +-
.../ci/pre_commit/pre_commit_local_yml_mounts.py | 18 +-
scripts/ci/pre_commit/pre_commit_mypy.py | 13 +-
.../pre_commit_new_session_in_provide_session.py | 125 ----------
.../pre_commit_sort_installed_providers.py | 45 ----
.../pre_commit/pre_commit_update_black_version.py | 37 ---
scripts/docker/clean-logs.sh | 3 +-
scripts/docker/common.sh | 2 +-
scripts/docker/entrypoint_ci.sh | 13 +-
scripts/in_container/configure_environment.sh | 2 -
.../{run_mypy.sh => run_extract_tests.sh} | 8 +-
.../in_container/{run_mypy.sh => run_flake8.sh} | 5 +-
scripts/in_container/test_pytest_collection.py | 66 -----
scripts/in_container/verify_providers.py | 5 -
setup.cfg | 5 +-
setup.py | 12 +-
tests/api/client/test_local_client.py | 4 +-
tests/api_connexion/endpoints/test_dag_endpoint.py | 15 ++
.../endpoints/test_dag_run_endpoint.py | 4 +-
tests/api_connexion/schemas/test_dataset_schema.py | 4 +-
tests/conftest.py | 20 +-
tests/core/test_sentry.py | 4 +-
tests/dag_processing/test_manager.py | 6 +-
tests/executors/test_celery_executor.py | 6 +-
.../providers/apache/pinot/hooks/test_pinot.py | 2 -
tests/jobs/test_scheduler_job.py | 4 +-
tests/models/test_dag.py | 6 +-
tests/models/test_dagbag.py | 14 +-
tests/models/test_taskinstance.py | 18 +-
tests/models/test_timestamp.py | 6 +-
tests/operators/test_datetime.py | 16 +-
tests/operators/test_generic_transfer.py | 6 +-
tests/operators/test_latest_only_operator.py | 4 +-
tests/operators/test_weekday.py | 10 +-
tests/providers/amazon/aws/hooks/test_eks.py | 10 +-
tests/providers/amazon/aws/sensors/test_athena.py | 11 +-
tests/providers/amazon/aws/sensors/test_batch.py | 49 ++--
.../amazon/aws/sensors/test_cloud_formation.py | 8 +-
.../providers/amazon/aws/sensors/test_dms_task.py | 5 +-
tests/providers/amazon/aws/sensors/test_eks.py | 12 +-
.../providers/amazon/aws/sensors/test_emr_base.py | 4 +-
.../amazon/aws/sensors/test_emr_containers.py | 5 +-
.../amazon/aws/sensors/test_emr_job_flow.py | 10 +-
.../providers/amazon/aws/sensors/test_emr_step.py | 10 +-
tests/providers/amazon/aws/sensors/test_glacier.py | 7 +-
tests/providers/amazon/aws/sensors/test_glue.py | 9 +-
.../amazon/aws/sensors/test_glue_crawler.py | 15 +-
.../amazon/aws/sensors/test_quicksight.py | 51 ++--
tests/providers/amazon/aws/sensors/test_s3_key.py | 16 +-
.../amazon/aws/sensors/test_s3_keys_unchanged.py | 62 ++---
.../amazon/aws/sensors/test_sagemaker_base.py | 4 +-
.../amazon/aws/sensors/test_sagemaker_endpoint.py | 3 +-
.../amazon/aws/sensors/test_sagemaker_training.py | 3 +-
.../amazon/aws/sensors/test_sagemaker_transform.py | 3 +-
.../amazon/aws/sensors/test_sagemaker_tuning.py | 3 +-
tests/providers/amazon/aws/sensors/test_sqs.py | 5 +-
.../amazon/aws/sensors/test_step_function.py | 10 +-
.../amazon/aws/utils/test_eks_get_token.py | 4 +-
.../apache/hive/transfers/test_mssql_to_hive.py | 12 +-
.../apache/hive/transfers/test_mysql_to_hive.py | 8 +-
tests/providers/docker/operators/test_docker.py | 7 -
.../elasticsearch/log/test_es_task_handler.py | 15 +-
.../google/cloud/hooks/test_cloud_build.py | 42 +---
.../google/cloud/operators/test_cloud_build.py | 265 ++++-----------------
.../test_cloud_storage_transfer_service.py | 4 +-
.../cloud/transfers/test_bigquery_to_mssql.py | 9 +-
.../google/cloud/transfers/test_mssql_to_gcs.py | 8 +-
.../google/cloud/transfers/test_mysql_to_gcs.py | 11 +-
.../google/cloud/triggers/test_cloud_build.py | 238 ------------------
.../common/auth_backend/test_google_openid.py | 5 +-
.../google/common/hooks/test_base_google.py | 52 ++--
.../google/common/hooks/test_discovery_api.py | 9 +-
.../common/utils/test_id_token_credentials.py | 12 +-
.../google/firebase/hooks/test_firestore.py | 13 +-
.../google/firebase/operators/test_firestore.py | 3 +-
.../providers/google/leveldb/hooks/test_leveldb.py | 10 +-
.../google/leveldb/operators/test_leveldb.py | 14 +-
.../marketing_platform/hooks/test_analytics.py | 5 +-
.../hooks/test_campaign_manager.py | 6 +-
.../marketing_platform/hooks/test_display_video.py | 6 +-
.../marketing_platform/hooks/test_search_ads.py | 6 +-
.../marketing_platform/operators/test_analytics.py | 9 +-
.../operators/test_campaign_manager.py | 27 +--
.../operators/test_display_video.py | 31 ++-
.../operators/test_search_ads.py | 19 +-
.../sensors/test_campaign_manager.py | 4 +-
.../sensors/test_display_video.py | 6 +-
.../marketing_platform/sensors/test_search_ads.py | 4 +-
.../providers/google/suite/hooks/test_calendar.py | 10 +-
tests/providers/google/suite/hooks/test_drive.py | 15 +-
tests/providers/google/suite/hooks/test_sheets.py | 5 +-
tests/providers/google/suite/sensors/test_drive.py | 6 +-
.../google/suite/transfers/test_gcs_to_gdrive.py | 3 +-
.../google/suite/transfers/test_sql_to_sheets.py | 5 +-
tests/providers/google/test_go_module.py | 3 +-
tests/providers/microsoft/azure/hooks/test_asb.py | 8 +-
.../microsoft/azure/operators/test_asb.py | 6 +-
.../providers/microsoft/mssql/hooks/test_mssql.py | 6 +-
.../microsoft/mssql/operators/test_mssql.py | 10 +-
tests/providers/mysql/hooks/test_mysql.py | 11 +-
.../mysql/transfers/test_vertica_to_mysql.py | 8 +-
tests/providers/ssh/hooks/test_ssh.py | 4 +-
tests/sensors/test_base.py | 107 +++++----
tests/sensors/test_date_time.py | 13 +-
tests/sensors/test_external_task_sensor.py | 79 +++---
tests/sensors/test_time_sensor.py | 17 +-
.../cncf/kubernetes/example_spark_kubernetes.py | 3 +-
.../cloud/bigquery/example_bigquery_to_mssql.py | 8 +-
.../google/cloud/bigtable/example_bigtable.py | 6 +-
.../cloud/cloud_build/example_cloud_build.py | 18 +-
.../cloud/cloud_build/example_cloud_build_async.py | 220 -----------------
.../cloud_build/example_cloud_build_trigger.py | 12 +-
.../google/cloud/gcs/example_mssql_to_gcs.py | 9 +-
.../google/cloud/gcs/example_mysql_to_gcs.py | 9 +-
.../providers/google/leveldb/example_leveldb.py | 10 +-
.../microsoft/azure/example_azure_service_bus.py | 30 +--
.../providers/microsoft/mssql/example_mssql.py | 10 +-
tests/task/__init__.py | 4 +
tests/task/task_runner/conftest.py | 39 ---
tests/test_utils/get_all_tests.py | 4 +-
tests/ti_deps/deps/test_not_in_retry_period_dep.py | 6 +-
tests/ti_deps/deps/test_runnable_exec_date_dep.py | 6 +-
tests/timetables/test_interval_timetable.py | 8 +-
tests/timetables/test_trigger_timetable.py | 6 +-
tests/utils/log/test_file_processor_handler.py | 8 +-
tests/utils/test_serve_logs.py | 14 +-
tests/www/test_security.py | 10 +-
tests/www/views/test_views_grid.py | 23 +-
tests/www/views/test_views_tasks.py | 17 +-
235 files changed, 1499 insertions(+), 3127 deletions(-)
create mode 100644 .flake8
delete mode 100644 .github/workflows/recheck-old-bug-report.yml
delete mode 100644 airflow/_vendor/daemon/0001-Refactor-_get_candidate_file_descriptor_ranges-to-us.patch
delete mode 100644 airflow/_vendor/daemon/module.mk
delete mode 100644 airflow/providers/google/cloud/triggers/cloud_build.py
create mode 100644 dev/deprecations/generate_deprecated_dicts.py
create mode 120000 provider_packages/.flake8
delete mode 100644 scripts/ci/pre_commit/common_precommit_black_utils.py
copy scripts/ci/pre_commit/{pre_commit_migration_reference.py => pre_commit_flake8.py} (82%)
delete mode 100755 scripts/ci/pre_commit/pre_commit_new_session_in_provide_session.py
delete mode 100755 scripts/ci/pre_commit/pre_commit_sort_installed_providers.py
delete mode 100755 scripts/ci/pre_commit/pre_commit_update_black_version.py
copy scripts/in_container/{run_mypy.sh => run_extract_tests.sh} (82%)
copy scripts/in_container/{run_mypy.sh => run_flake8.sh} (88%)
delete mode 100755 scripts/in_container/test_pytest_collection.py
delete mode 100644 tests/providers/google/cloud/triggers/test_cloud_build.py
delete mode 100644 tests/system/providers/google/cloud/cloud_build/example_cloud_build_async.py
delete mode 100644 tests/task/task_runner/conftest.py