You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ep...@apache.org on 2024/04/03 07:15:41 UTC

(airflow) branch v2-9-stable updated (d48b910dc1 -> c8c1863289)

This is an automated email from the ASF dual-hosted git repository.

ephraimanierobi pushed a change to branch v2-9-stable
in repository https://gitbox.apache.org/repos/asf/airflow.git


    from d48b910dc1 Optimize preparing packages for PROD image builds (#38539)
     add 7fe1e2e105  Update log level to debug from warning about scheduled_duration metric (#38180)
     add e1933c8293 fixing the tz in next run id info (#38482)
     add b55aac8a88 Avoid accidentally upgrading airflow in compatibility check (#38545)
     add 7fc42b9eef Fixe test for PROD images to include >= conditions (#38574)
     add 7b177c67c7 Add pre-commit hook to sync template context vars (#38579)
     add ae17676098 Review and set variables in our workflows where appropriate (#38523)
     add cd9b0696ec Update description and use of default-python-version (#38546)
     add 6a80421aa1 Bring back release airflow command to be run in canary build (#38486)
     add f1b6d400c6 Don't error when (un)pausing (un)paused dag in the CLI (#38585)
     add 75aefd5ae9 Better fix for constraint generation dependency for PROD build (#38582)
     add 688b40e58d Fix secrets inheriting for docs build (#38596)
     add dcac0e183b Brings back mypy-checks (#38597)
     add 4cba4ef5e4 Rename to XComObjectStorageBackend (#38607)
     add a3ff1ce12f bugfix: properly serialize nested attrs classes (#38591)
     add 886cb55944 Add padding to bottom of log block. (#38610)
     add 373abd00c8 Fix `total_entries` count on the event logs endpoint (#38625)
     add 5e7793539c Rename `allowed_filter_attrs` to `allowed_sort_attrs` (#38626)
     add 3c214a5552 Mark more fields on API as dump-only (#38616)
     add 8ee1240467 Add 2.9.0b2 to issue template (#38586)
     add 17fa8d3d0c Upgrade gitpython as build dependency (#38644)
     add 4918174926 Prohibit to use unsupported DB backends in tests (#38530)
     add 5cc3dc85c3 Disable support of a legacy `LocalPath` in favor of stdlib `pathlib.Path` (#38624)
     add 4467cd6245 Add a migration script for encrypted trigger kwargs (#38358)
     add 9ed3535eae bump ruff to 0.3.5 (#38656)
     add 6746fea7ee More human friendly "show tables" output for db cleanup (#38654)
     add 0d32e45db6 Move cleanup-docker to a shell scripts from composite action (#38659)
     add 40379ed9a1 bump uv to 0.1.27 (#38657)
     add 90dc2cc8d4 Remove display of  param from the UI (#38660)
     add 4260f56050 Clean up unused triggers in a single query for all dialects except MySQL (#38663)
     add 9ec5c41e71 Improve stability of remove_task_decorator function (#38649)
     add 67c7392f84 Add exclude/include events filters to audit log (#38506)
     add af8c481dd6 Improve trigger assign_unassigned by merging alive_triggerer_ids and get_sorted_triggers queries (#38664)
     add a2e513c13a Add a task instance dependency for mapped dependencies (#37498)
     add 3f51039a06 Check-out the target commit when preparing PROD build artifacts (#38678)
     add 356897adad Remove image tag from Helm test name (#38621)
     add 367e8fed6c Use explicit and easier to use runs-on approach for CI workflows (#38601)
     add 471164bc92 Rename build arg for upgrade-to-newer-dependencies (#38637)
     add cf6357fafb Update known issue with Pendulum 3 in 2.8.1 release notes (#38675)
     add 8d12bb1e93 Update UI doc screenshots (#38680)
     add d46d7ecf5c Remove google-cloud-aiplatform when tests without pydantic are run (#38617)
     add c8c1863289 Update RELEASE_NOTES.rst

No new revisions were added by this update.

Summary of changes:
 .github/ISSUE_TEMPLATE/airflow_bug_report.yml      |   2 +-
 .github/actions/cleanup-docker/action.yml          |  26 --
 .github/workflows/additional-ci-image-checks.yml   |  75 +---
 .github/workflows/additional-prod-image-tests.yml  |  31 +-
 .github/workflows/basic-tests.yml                  |  83 +++-
 .github/workflows/build-images.yml                 |  47 +-
 .github/workflows/check-providers.yml              |  56 ++-
 .github/workflows/ci-image-build.yml               |  58 +--
 .github/workflows/ci.yml                           | 137 +++---
 .github/workflows/docs.yml                         | 126 ------
 .github/workflows/finalize-tests.yml               |  35 +-
 .github/workflows/generate-constraints.yml         | 132 ++++++
 .github/workflows/helm-tests.yml                   |  32 +-
 .github/workflows/integration-tests.yml            |  18 +-
 .github/workflows/k8s-tests.yml                    |  16 +-
 .github/workflows/prod-image-build.yml             | 117 +++--
 .github/workflows/prod-image-extra-checks.yml      |  17 +-
 .github/workflows/push-image-cache.yml             |  73 ++-
 .github/workflows/release_dockerhub_image.yml      |  14 +-
 .github/workflows/run-unit-tests.yml               |  25 +-
 .github/workflows/special-tests.yml                |  25 +-
 ...-generation.yml => static-checks-mypy-docs.yml} | 203 +++++----
 .pre-commit-config.yaml                            |   4 +-
 Dockerfile                                         |  13 +-
 Dockerfile.ci                                      |  15 +-
 RELEASE_NOTES.rst                                  |  24 +-
 .../api_connexion/endpoints/connection_endpoint.py |   4 +-
 .../api_connexion/endpoints/dag_run_endpoint.py    |   4 +-
 .../endpoints/dag_warning_endpoint.py              |   4 +-
 .../api_connexion/endpoints/event_log_endpoint.py  |  10 +-
 .../endpoints/import_error_endpoint.py             |   4 +-
 airflow/api_connexion/endpoints/pool_endpoint.py   |   4 +-
 .../api_connexion/endpoints/variable_endpoint.py   |   4 +-
 airflow/api_connexion/schemas/dag_schema.py        |  20 +-
 .../api_connexion/schemas/dag_warning_schema.py    |   6 +-
 airflow/api_connexion/schemas/error_schema.py      |   8 +-
 airflow/api_connexion/schemas/log_schema.py        |   4 +-
 airflow/api_connexion/schemas/plugin_schema.py     |  26 +-
 .../api_connexion/schemas/task_instance_schema.py  |  54 +--
 airflow/cli/commands/dag_command.py                |   3 +-
 ...py => 0140_2_9_0_update_trigger_kwargs_type.py} |  28 +-
 airflow/models/baseoperator.py                     |   2 +
 airflow/models/taskinstance.py                     |  15 +-
 airflow/models/trigger.py                          |  31 +-
 airflow/providers/common/io/xcom/backend.py        |   6 +-
 .../api_endpoints/role_and_permission_endpoint.py  |   4 +-
 .../auth_manager/api_endpoints/user_endpoint.py    |   4 +-
 airflow/providers/openlineage/plugins/listener.py  |   6 +-
 airflow/reproducible_build.yaml                    |   4 +-
 airflow/serialization/serde.py                     |   2 +-
 airflow/ti_deps/deps/mapped_task_upstream_dep.py   | 104 +++++
 airflow/utils/context.py                           |   7 +-
 airflow/utils/context.pyi                          |   4 +-
 airflow/utils/db.py                                |  41 +-
 airflow/utils/db_cleanup.py                        |  10 +-
 airflow/utils/decorators.py                        |  10 +-
 airflow/www/extensions/init_jinja_globals.py       |   2 +
 airflow/www/static/js/api/useEventLogs.tsx         |  12 +
 airflow/www/static/js/dag/details/AuditLog.tsx     |  83 +++-
 .../js/dag/details/taskInstance/Logs/LogBlock.tsx  |   1 -
 airflow/www/static/js/datasets/SearchBar.tsx       |  77 ++--
 airflow/www/templates/airflow/dag.html             |   7 +-
 airflow/www/views.py                               |   5 +
 dev/breeze/doc/ci/02_images.md                     |   2 +-
 dev/breeze/doc/ci/04_selective_checks.md           |   8 +-
 .../commands/minor_release_command.py              |  45 +-
 .../commands/release_candidate_command.py          |  81 ++--
 .../src/airflow_breeze/commands/release_command.py |  42 +-
 dev/breeze/src/airflow_breeze/global_constants.py  |   5 -
 .../src/airflow_breeze/params/build_ci_params.py   |   2 +-
 .../src/airflow_breeze/utils/selective_checks.py   |  18 +-
 dev/breeze/tests/test_selective_checks.py          |  14 +-
 docker_tests/test_prod_image.py                    |   4 +-
 .../xcom_backend.rst                               |   4 +-
 docs/apache-airflow/core-concepts/xcoms.rst        |   4 +-
 docs/apache-airflow/img/airflow_erd.sha256         |   2 +-
 docs/apache-airflow/img/airflow_erd.svg            |   8 +-
 docs/apache-airflow/img/audit_log.png              | Bin 0 -> 456601 bytes
 docs/apache-airflow/img/code.png                   | Bin 506092 -> 504063 bytes
 docs/apache-airflow/img/duration.png               | Bin 150876 -> 358076 bytes
 docs/apache-airflow/img/gantt.png                  | Bin 352148 -> 313750 bytes
 docs/apache-airflow/img/graph.png                  | Bin 429818 -> 417210 bytes
 docs/apache-airflow/img/grid.png                   | Bin 249561 -> 334128 bytes
 docs/apache-airflow/img/grid_task_details.png      | Bin 0 -> 227349 bytes
 docs/apache-airflow/img/landing_times.png          | Bin 84125 -> 303523 bytes
 docs/apache-airflow/img/run_types.png              | Bin 0 -> 5950 bytes
 .../img/task_manual_vs_scheduled.png               | Bin 5686 -> 0 bytes
 docs/apache-airflow/migrations-ref.rst             |   4 +-
 docs/apache-airflow/templates-ref.rst              |  17 +-
 docs/apache-airflow/ui.rst                         |  23 +-
 docs/docker-stack/build-arg-ref.rst                | 120 ++---
 hatch_build.py                                     |   2 +-
 pyproject.toml                                     |  21 +-
 .../run_anything.sh => ci/cleanup_docker.sh}       |   6 +-
 scripts/ci/docker-compose/devcontainer.env         |   2 +-
 .../pre_commit_template_context_key_sync.py        | 131 ++++++
 scripts/docker/entrypoint_ci.sh                    |   2 +-
 scripts/docker/install_additional_dependencies.sh  |   3 +-
 scripts/docker/install_airflow.sh                  |   6 +-
 .../in_container/install_airflow_and_providers.py  |   6 +
 .../in_container/run_prepare_airflow_packages.py   |  10 +-
 tests/always/test_project_structure.py             |   4 +
 .../endpoints/test_event_log_endpoint.py           |  18 +-
 tests/cli/commands/test_dag_command.py             |  14 +-
 tests/conftest.py                                  |  72 +--
 tests/dag_processing/test_processor.py             | 115 +++--
 .../providers/apache/drill}/__init__.py            |   0
 .../providers/apache/drill/operators}/__init__.py  |   0
 .../providers/apache/drill/operators/test_drill.py |   3 +-
 .../integration/providers/apache/hive}/__init__.py |   0
 .../providers/apache/hive/transfers}/__init__.py   |   0
 .../apache/hive/transfers/test_mssql_to_hive.py    |   3 +-
 .../cloud/transfers/test_bigquery_to_mssql.py      |   3 +-
 .../google/cloud/transfers/test_mssql_to_gcs.py    |   3 +-
 tests/providers/apache/drill/operators/__init__.py |  17 -
 tests/providers/common/io/xcom/test_backend.py     |  12 +-
 .../hooks/vertex_ai/test_batch_prediction_job.py   |   4 +
 .../vertex_ai/test_hyperparameter_tuning_job.py    |   4 +
 tests/serialization/test_dag_serialization.py      |   3 +
 tests/serialization/test_serde.py                  |  16 +
 .../ti_deps/deps/test_mapped_task_upstream_dep.py  | 496 +++++++++++++++++++++
 tests/utils/test_db_cleanup.py                     |  10 +-
 tests/utils/test_log_handlers.py                   |  39 +-
 tests/utils/test_python_virtualenv.py              |   5 +
 tests/www/views/test_views.py                      |   2 +
 125 files changed, 2340 insertions(+), 1119 deletions(-)
 delete mode 100644 .github/actions/cleanup-docker/action.yml
 delete mode 100644 .github/workflows/docs.yml
 create mode 100644 .github/workflows/generate-constraints.yml
 rename .github/workflows/{static-checks-mypy-and-constraints-generation.yml => static-checks-mypy-docs.yml} (56%)
 copy airflow/migrations/versions/{0139_2_9_0_add_display_name_for_dag_and_task_.py => 0140_2_9_0_update_trigger_kwargs_type.py} (60%)
 create mode 100644 airflow/ti_deps/deps/mapped_task_upstream_dep.py
 create mode 100644 docs/apache-airflow/img/audit_log.png
 create mode 100644 docs/apache-airflow/img/grid_task_details.png
 create mode 100644 docs/apache-airflow/img/run_types.png
 delete mode 100644 docs/apache-airflow/img/task_manual_vs_scheduled.png
 copy scripts/{in_container/run_anything.sh => ci/cleanup_docker.sh} (89%)
 create mode 100644 scripts/ci/pre_commit/pre_commit_template_context_key_sync.py
 copy {airflow/api_connexion => tests/integration/providers/apache/drill}/__init__.py (100%)
 copy {airflow/api_connexion => tests/integration/providers/apache/drill/operators}/__init__.py (100%)
 rename tests/{ => integration}/providers/apache/drill/operators/test_drill.py (94%)
 copy {airflow/api_connexion => tests/integration/providers/apache/hive}/__init__.py (100%)
 copy {airflow/api_connexion => tests/integration/providers/apache/hive/transfers}/__init__.py (100%)
 rename tests/{ => integration}/providers/apache/hive/transfers/test_mssql_to_hive.py (97%)
 rename tests/{ => integration}/providers/google/cloud/transfers/test_bigquery_to_mssql.py (93%)
 rename tests/{ => integration}/providers/google/cloud/transfers/test_mssql_to_gcs.py (98%)
 delete mode 100644 tests/providers/apache/drill/operators/__init__.py
 create mode 100644 tests/ti_deps/deps/test_mapped_task_upstream_dep.py