You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by di...@apache.org on 2024/11/05 20:00:07 UTC

(airflow) branch handle-stuck-in-queue updated (2113384b4e0 -> 6d639907113)

This is an automated email from the ASF dual-hosted git repository.

dimberman pushed a change to branch handle-stuck-in-queue
in repository https://gitbox.apache.org/repos/asf/airflow.git


    from 2113384b4e0 Update airflow/config_templates/config.yml
     add eefddcbd6ae next_execution_date explained in case of manual triggers (#43535)
     add c1bd9c5c014 Bedrock retired the Llama2 model, updated the test to use the closest replacement Llama3 model. (#43600)
     add 59ea748f740 Add min version to ipykernel,scrapbook, pywinrm (#43603)
     add a8921ae57a5 remove un used python_version param argument (#43609)
     add 17e51009a73 Check for empty list of integration tests before evaluation inside matrix that requires a value. (#43546)
     add 229c6a3e467 Explain how to use uv with airflow virtualenv and make it works (#43604)
     add ddc5670a8c6 mAKe Breeze installation and reinstallation support both uv and pipx (#43607)
     add 6fd7052f863 Move TaskInstance heartbeat directly on to TI row, not on Job row (#43599)
     add 45b0b2f15c5 Prepare docs for Oct 2nd wave of providers rc3 (#43613)
     add 0664782c19d Fix docstring for AthenaTrigger (#43616)
     add ff6038b8d15 updated git-sync to 4.3.0 (#41411)
     add 12950dd03b8 Enable back iterative development of latest providers with old airflows (#43617)
     add 20e6517ed33 AIP-84 Migrate DagWarning public endpoint to FastAPI (#42749)
     add 30147ffd7d8 AIP-84 Migrate backfill API to fast api (#43496)
     add 4950536d5eb Handle the scenario where a task is queued for longer than `task_queued_timeout`.
     add 1c34eea7e18 Merge branch 'handle-stuck-in-queue' of https://github.com/apache/airflow into handle-stuck-in-queue
     add c18f8f4edc6 k8s support
     add 6d639907113 Merge branch 'handle-stuck-in-queue' of https://github.com/apache/airflow into handle-stuck-in-queue

No new revisions were added by this update.

Summary of changes:
 .github/actions/install-pre-commit/action.yml      |  50 ++
 .github/workflows/basic-tests.yml                  |  13 +-
 .github/workflows/integration-tests.yml            |   1 +
 .github/workflows/static-checks-mypy-docs.yml      |  20 +-
 .gitignore                                         |   3 +
 Dockerfile.ci                                      |   7 +
 .../api_connexion/endpoints/backfill_endpoint.py   | 183 ----
 .../endpoints/dag_warning_endpoint.py              |   2 +
 airflow/api_connexion/openapi/v1.yaml              | 149 ----
 airflow/api_connexion/schemas/backfill_schema.py   | 100 ---
 airflow/api_fastapi/common/parameters.py           |  32 +
 .../api_fastapi/core_api/openapi/v1-generated.yaml | 944 +++++++++++++++------
 .../api_fastapi/core_api/routes/public/__init__.py |   5 +
 .../core_api/routes/public/backfills.py            | 187 ++++
 .../core_api/routes/public/dag_warning.py          |  72 ++
 .../serializers/{ui/dags.py => backfills.py}       |  39 +-
 .../serializers/{providers.py => dag_warning.py}   |  21 +-
 airflow/cli/cli_config.py                          |   2 -
 airflow/cli/commands/task_command.py               |   1 -
 airflow/executors/debug_executor.py                |   2 +-
 airflow/jobs/local_task_job_runner.py              |   3 +-
 airflow/jobs/scheduler_job_runner.py               |  18 +-
 ...5_3_0_0_add_last_heartbeat_at_directly_to_ti.py |  60 ++
 airflow/models/backfill.py                         |  35 +-
 airflow/models/taskinstance.py                     |  54 +-
 airflow/models/taskinstancehistory.py              |   1 -
 airflow/serialization/pydantic/taskinstance.py     |  16 +-
 airflow/settings.py                                |  10 +-
 airflow/task/standard_task_runner.py               |  23 +-
 airflow/ui/openapi-gen/queries/common.ts           | 222 +++--
 airflow/ui/openapi-gen/queries/prefetch.ts         | 230 +++--
 airflow/ui/openapi-gen/queries/queries.ts          | 550 +++++++++---
 airflow/ui/openapi-gen/queries/suspense.ts         | 286 +++++--
 airflow/ui/openapi-gen/requests/schemas.gen.ts     | 106 +++
 airflow/ui/openapi-gen/requests/services.gen.ts    | 471 +++++++---
 airflow/ui/openapi-gen/requests/types.gen.ts       | 480 +++++++++--
 airflow/utils/db.py                                |   2 +-
 airflow/utils/dot_renderer.py                      |   4 +-
 airflow/www/static/js/types/api-generated.ts       | 181 ----
 airflow/www/views.py                               |   4 +-
 chart/newsfragments/41411.significant.rst          |   3 +
 chart/values.schema.json                           |   2 +-
 chart/values.yaml                                  |   2 +-
 contributing-docs/03_contributors_quick_start.rst  |   2 +-
 contributing-docs/07_local_virtualenv.rst          | 300 +++----
 dev/breeze/doc/01_installation.rst                 |  25 +-
 .../airflow_breeze/commands/developer_commands.py  |   4 +-
 .../commands/release_management_commands.py        |  15 +-
 dev/breeze/src/airflow_breeze/global_constants.py  |  28 +-
 .../src/airflow_breeze/utils/kubernetes_utils.py   |   5 +-
 dev/breeze/src/airflow_breeze/utils/path_utils.py  |  13 +-
 .../src/airflow_breeze/utils/python_versions.py    |   7 +-
 dev/breeze/src/airflow_breeze/utils/reinstall.py   |  13 +-
 dev/breeze/src/airflow_breeze/utils/run_tests.py   |   8 +-
 dev/breeze/src/airflow_breeze/utils/run_utils.py   |  26 +-
 .../tests/test_pytest_args_for_test_types.py       |   2 -
 docs/apache-airflow-providers-amazon/commits.rst   | 108 ++-
 docs/apache-airflow-providers-fab/commits.rst      |  46 +-
 docs/apache-airflow/img/airflow_erd.sha256         |   2 +-
 docs/apache-airflow/img/airflow_erd.svg            | 310 ++++---
 docs/apache-airflow/migrations-ref.rst             |   4 +-
 docs/apache-airflow/templates-ref.rst              |   6 +-
 generated/provider_dependencies.json               |   8 +-
 hatch_build.py                                     |  10 +-
 .../src/airflow/providers/amazon/CHANGELOG.rst     |   2 +
 .../providers/amazon/aws/triggers/athena.py        |   3 +-
 .../apache/hive/transfers/mysql_to_hive.py         |  11 +-
 .../kubernetes/executors/kubernetes_executor.py    |   6 +
 .../executors/kubernetes_executor_types.py         |   1 +
 .../executors/kubernetes_executor_utils.py         |  24 +-
 providers/src/airflow/providers/fab/CHANGELOG.rst  |   8 +-
 .../google/cloud/transfers/mysql_to_gcs.py         |  11 +-
 .../providers/microsoft/azure/provider.yaml        |   2 +-
 .../src/airflow/providers/mysql/hooks/mysql.py     |  30 +-
 .../src/airflow/providers/mysql/provider.yaml      |   5 +-
 .../providers/mysql/transfers/vertica_to_mysql.py  |  10 +-
 .../src/airflow/providers/papermill/provider.yaml  |   4 +-
 providers/tests/mysql/operators/test_mysql.py      |  10 +-
 .../tests/system/amazon/aws/example_bedrock.py     |   2 +-
 providers/tests/system/papermill/conftest.py       |   2 +-
 .../docker-compose/providers-and-tests-sources.yml |   5 +-
 .../ci/pre_commit/check_ti_vs_tis_attributes.py    |   2 +
 scripts/ci/pre_commit/common_precommit_utils.py    |   7 +-
 scripts/docker/entrypoint_ci.sh                    |   7 +
 .../endpoints/test_backfill_endpoint.py            | 442 ----------
 .../test_mapped_task_instance_endpoint.py          |   1 -
 .../endpoints/test_task_instance_endpoint.py       |   1 -
 .../api_connexion/schemas/test_backfill_schema.py  |  55 --
 .../core_api/routes/public/test_backfills.py       | 309 +++++++
 .../core_api/routes/public/test_dag_warning.py     |  85 ++
 .../core_api/routes/public/test_task_instances.py  |   1 -
 tests/assets/test_manager.py                       |   1 +
 tests/cli/commands/test_task_command.py            |   2 +-
 tests/executors/test_debug_executor.py             |   2 +-
 tests/jobs/test_local_task_job.py                  |  18 +-
 tests/jobs/test_scheduler_job.py                   |  28 +-
 tests/models/test_backfill.py                      |  36 -
 tests/models/test_taskinstance.py                  |   2 +-
 tests/www/views/test_views_tasks.py                |  14 +-
 99 files changed, 4069 insertions(+), 2618 deletions(-)
 create mode 100644 .github/actions/install-pre-commit/action.yml
 delete mode 100644 airflow/api_connexion/endpoints/backfill_endpoint.py
 delete mode 100644 airflow/api_connexion/schemas/backfill_schema.py
 create mode 100644 airflow/api_fastapi/core_api/routes/public/backfills.py
 create mode 100644 airflow/api_fastapi/core_api/routes/public/dag_warning.py
 copy airflow/api_fastapi/core_api/serializers/{ui/dags.py => backfills.py} (50%)
 copy airflow/api_fastapi/core_api/serializers/{providers.py => dag_warning.py} (68%)
 create mode 100644 airflow/migrations/versions/0045_3_0_0_add_last_heartbeat_at_directly_to_ti.py
 create mode 100644 chart/newsfragments/41411.significant.rst
 delete mode 100644 tests/api_connexion/endpoints/test_backfill_endpoint.py
 delete mode 100644 tests/api_connexion/schemas/test_backfill_schema.py
 create mode 100644 tests/api_fastapi/core_api/routes/public/test_backfills.py
 create mode 100644 tests/api_fastapi/core_api/routes/public/test_dag_warning.py