You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ep...@apache.org on 2023/01/11 17:26:46 UTC
[airflow] branch v2-5-test updated (a6704d02ef -> f2a55bd2e5)
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a change to branch v2-5-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
from a6704d02ef Update 2.5.0 release date (#28081)
new 5012093007 Allow generation of connection URI to work when no conn type (#26765)
new 39791b6b2d fix: continue checking sla (#26968)
new 5a565c44e9 Additional info about Segmentation Fault in LocalTaskJob (#27381)
new f8db720253 Handle DAG disappearing mid-flight when dag verification happens (#27720)
new 7a03761c82 Note which versions of Python are supported (#27798)
new d174ef1de7 dagrun, next_dagruns_to_examine, add MySQL index hint (#27821)
new d34bd50683 Make sure we can get out of a faulty scheduler state (#27834)
new dcb9d4797d Don't log CLI actions if db not initialized (#27851)
new cc02bdc63b fix: current_state method on TaskInstance doesn't filter by map_index (#27898)
new 5302cc25af Order TIs by map_index (#27904)
new 9733de8d38 Fix deadlock when chaining multiple empty mapped tasks (#27964)
new 5bb6430463 Documentation for the LocalTaskJob return code counter (#27972)
new 9aa72e43b4 Ignore Blackification commit from Git Blame (#27981)
new 2016d4a604 Simplify dataset subgraph logic (#27987)
new 7646bf65d5 Prevent double loading of providers from local paths (#27988)
new be004cb2ca Add documentation for [core] mp_start_method config (#27993)
new e573389fcd docs: copyedit DAG (#27995)
new 3f3fd0f55a Handle bad zip files nicely when parsing DAGs. (#28011)
new c5a0a6b179 Use asserts instead of exceptions for executor not started (#28019)
new b322572c6e Add Andrey as committer (#28050)
new 51791a8cf4 Return list of tasks that will be changed (#28066)
new dacc8fc2d9 Resolve false warning about calling conf.get on moved item (#28075)
new 7c916b7ba7 Bump decode-uri-component from 0.2.0 to 0.2.2 in /airflow/www (#28080)
new 780a02838d Add airflow.api.auth.backend.session to backend sessions in compose (#28094)
new 4c15384e9e Improve handling of warnings in CI (#28096)
new b4efb3086f Don't emit FutureWarning when code not calling old key (#28109)
new 2bc0a0da43 Make BaseJob.most_recent_job favor "running" jobs (#28119)
new a7bcb4ba7e Improve run/task grid view actions (#28130)
new b7f6d0c02e Ensure the dagbag_size metric decreases when files are deleted (#28135)
new 10112966e2 Do not warn when airflow dags tests command is used (#28138)
new 524004e30a Ensure that warnings from `@dag` decorator are reported in dag file (#28153)
new 824963a06a fix next run datasets error (#28165)
new 2bfc0207ed Make invalid characters exception more readable (#28181)
new d69af8e7d9 Fix typo in Best Practice documentation (#28190)
new fa37bf7b82 Add custom pickling hooks to LazyXComAccess (#28191)
new e21a31576b Make live logs reading work for "other" k8s executors (#28213)
new 68a0fb7b4c Make arguments 'offset' and 'length' not required (#28234)
new 9220218114 Fix db clean warnings (#28243)
new 805dbc2304 Convert test_task_command to Pytest and unquarantine tests in it (#28247)
new 215011493b Trigger gevent monkeypatching via environment variable (#28283)
new 7f065092b2 Add SIGUSR2 handler for LocalTaskJob and workers to aid debugging (#28309)
new 4edd58845a Separate ER diagram dependencies to doc_gen extra (#28346)
new 3a8768eb45 Remove docs package from Publish_docs.py and switch to python in shebang (#28347)
new 2653349407 Correctly select a mapped task's "previous" task (#28379)
new 97f7de2041 Maintain manual scroll position in task logs (#28386)
new f6a39c1669 Show mapped task groups in graph (#28392)
new 2017150621 Dont show task/run durations when there is no start_date (#28395)
new 0e06d29afc A manual run can't look like a scheduled one (#28397)
new dc9b12b33e separate callModal from dag.js (#28410)
new 6efc2db205 Fix ti._try_number for deferred and up_for_reschedule tasks (#26993)
new 9daf9ac823 Add pre-commit hook to check session default value (#28007)
new 1987a3ad98 Add back join to zombie query that was dropped in #28198 (#28544)
new f2a55bd2e5 Add setters to MappedOperator on_*_callbacks (#28313)
The 53 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.git-blame-ignore-revs | 3 +-
.github/actions/post_tests/action.yml | 6 +
.gitignore | 2 +-
.pre-commit-config.yaml | 6 +
.rat-excludes | 3 +
CONTRIBUTING.rst | 21 +-
Dockerfile.ci | 3 +
INSTALL | 14 +-
STATIC_CODE_CHECKS.rst | 2 +
TESTING.rst | 3 +
airflow/__init__.py | 9 +
airflow/api/client/api_client.py | 2 +-
airflow/api/common/mark_tasks.py | 12 +-
airflow/cli/commands/task_command.py | 1 -
airflow/config_templates/config.yml | 15 +-
airflow/config_templates/default_airflow.cfg | 12 +-
airflow/configuration.py | 59 ++--
airflow/dag_processing/manager.py | 54 +++-
airflow/dag_processing/processor.py | 2 +-
airflow/executors/base_executor.py | 2 -
airflow/executors/dask_executor.py | 32 +-
airflow/executors/kubernetes_executor.py | 81 ++---
airflow/executors/local_executor.py | 39 +--
airflow/jobs/backfill_job.py | 8 +-
airflow/jobs/base_job.py | 18 +-
airflow/jobs/local_task_job.py | 49 +++
airflow/jobs/scheduler_job.py | 53 ++--
airflow/models/abstractoperator.py | 36 ++-
airflow/models/baseoperator.py | 21 +-
airflow/models/connection.py | 7 +-
airflow/models/dag.py | 26 +-
airflow/models/dagrun.py | 13 +-
airflow/models/mappedoperator.py | 16 +
airflow/models/taskinstance.py | 28 +-
airflow/models/xcom.py | 15 +
airflow/models/xcom_arg.py | 1 +
airflow/providers/microsoft/azure/hooks/wasb.py | 2 +-
airflow/providers_manager.py | 7 +
airflow/sentry.py | 6 +-
airflow/stats.py | 3 +-
airflow/ti_deps/deps/prev_dagrun_dep.py | 2 +-
airflow/utils/cli_action_loggers.py | 16 +-
airflow/utils/db_cleanup.py | 5 +-
airflow/utils/decorators.py | 22 ++
airflow/utils/log/file_task_handler.py | 23 +-
airflow/utils/log/log_reader.py | 2 +-
airflow/utils/state.py | 5 -
airflow/utils/task_group.py | 2 +
airflow/www/extensions/init_appbuilder.py | 6 +-
airflow/www/security.py | 4 +-
airflow/www/static/js/api/useClearRun.ts | 11 +-
airflow/www/static/js/api/useClearTask.ts | 12 +-
.../www/static/js/api/useDatasetDependencies.ts | 165 +++++------
airflow/www/static/js/api/useMarkFailedRun.ts | 10 +-
airflow/www/static/js/api/useMarkSuccessRun.ts | 10 +-
airflow/www/static/js/api/useQueueRun.ts | 10 +-
airflow/www/static/js/api/useTaskLog.ts | 20 +-
airflow/www/static/js/{dag.js => callModal.js} | 269 ++++++-----------
airflow/www/static/js/components/ConfirmDialog.tsx | 11 +-
airflow/www/static/js/dag.js | 327 +--------------------
airflow/www/static/js/dag/InstanceTooltip.test.tsx | 14 +
airflow/www/static/js/dag/InstanceTooltip.tsx | 24 +-
.../www/static/js/dag/details/dagRun/ClearRun.tsx | 6 +-
.../static/js/dag/details/dagRun/MarkFailedRun.tsx | 6 +-
.../js/dag/details/dagRun/MarkSuccessRun.tsx | 6 +-
.../www/static/js/dag/details/dagRun/QueueRun.tsx | 6 +-
airflow/www/static/js/dag/details/dagRun/index.tsx | 14 +-
.../static/js/dag/details/taskInstance/Details.tsx | 16 +-
.../js/dag/details/taskInstance/Logs/LogBlock.tsx | 90 ++++++
.../js/dag/details/taskInstance/Logs/index.tsx | 45 +--
.../dag/details/taskInstance/taskActions/Clear.tsx | 6 +-
.../taskInstance/taskActions/MarkFailed.tsx | 2 +-
.../taskInstance/taskActions/MarkSuccess.tsx | 2 +-
airflow/www/static/js/dags.js | 6 +-
airflow/www/static/js/gantt.js | 2 +-
airflow/www/static/js/graph.js | 59 ++--
airflow/www/utils.py | 15 +-
airflow/www/views.py | 36 +--
airflow/www/yarn.lock | 24 +-
dev/breeze/src/airflow_breeze/pre_commit_ids.py | 1 +
docs/apache-airflow/best-practices.rst | 2 +-
docs/apache-airflow/concepts/dags.rst | 4 +-
docs/apache-airflow/extra-packages-ref.rst | 4 +-
.../howto/docker-compose/docker-compose.yaml | 2 +-
docs/apache-airflow/logging-monitoring/metrics.rst | 88 +++---
docs/apache-airflow/project.rst | 1 +
docs/apache-airflow/start.rst | 3 +-
docs/publish_docs.py | 8 +-
images/breeze/output-commands-hash.txt | 2 +-
images/breeze/output_k8s.svg | 146 ++++-----
images/breeze/output_k8s_configure-cluster.svg | 152 +++++-----
images/breeze/output_k8s_create-cluster.svg | 156 +++++-----
images/breeze/output_k8s_delete-cluster.svg | 96 +++---
images/breeze/output_k8s_deploy-airflow.svg | 168 +++++------
images/breeze/output_k8s_k9s.svg | 92 +++---
images/breeze/output_k8s_logs.svg | 96 +++---
images/breeze/output_k8s_run-complete-tests.svg | 212 ++++++-------
images/breeze/output_k8s_shell.svg | 108 +++----
images/breeze/output_k8s_status.svg | 100 +++----
images/breeze/output_k8s_tests.svg | 164 +++++------
images/breeze/output_k8s_upload-k8s-image.svg | 144 ++++-----
images/breeze/output_setup.svg | 82 +++---
.../output_setup_regenerate-command-images.svg | 136 ++++-----
images/breeze/output_static-checks.svg | 240 +++++++--------
images/breeze/output_testing.svg | 74 ++---
images/breeze/output_testing_tests.svg | 212 ++++++-------
newsfragments/08212.misc.rst | 1 +
.../pre_commit_new_session_in_provide_session.py | 125 ++++++++
scripts/docker/entrypoint_ci.sh | 3 +
.../in_container/filter_out_warnings.py | 20 +-
scripts/in_container/run_ci_tests.sh | 8 +-
setup.cfg | 2 +-
setup.py | 7 +-
tests/always/test_connection.py | 8 +
tests/cli/commands/test_task_command.py | 54 ++--
tests/core/test_configuration.py | 62 ++++
tests/dag_processing/test_manager.py | 1 +
tests/dag_processing/test_processor.py | 43 +++
tests/jobs/test_base_job.py | 20 ++
tests/jobs/test_local_task_job.py | 37 ++-
tests/jobs/test_scheduler_job.py | 44 +++
tests/models/test_dag.py | 42 ++-
tests/models/test_dagrun.py | 127 +++++++-
tests/models/test_mappedoperator.py | 51 ++++
tests/models/test_taskinstance.py | 38 +++
tests/providers/slack/hooks/test_slack_webhook.py | 6 +-
tests/utils/test_db_cleanup.py | 17 ++
tests/utils/test_log_handlers.py | 36 +++
tests/utils/test_task_group.py | 3 +
tests/www/views/test_views_dagrun.py | 39 ++-
tests/www/views/test_views_grid.py | 4 +-
131 files changed, 3000 insertions(+), 2294 deletions(-)
copy airflow/www/static/js/{dag.js => callModal.js} (58%)
create mode 100644 airflow/www/static/js/dag/details/taskInstance/Logs/LogBlock.tsx
create mode 100644 newsfragments/08212.misc.rst
create mode 100755 scripts/ci/pre_commit/pre_commit_new_session_in_provide_session.py
copy airflow/api_connexion/schemas/version_schema.py => scripts/in_container/filter_out_warnings.py (73%)