You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2023/03/12 03:18:19 UTC
[airflow] branch v2-5-test updated (d4afb041f8 -> 6fede984d0)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch v2-5-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard d4afb041f8 Installing non-editable airflow for constraint generation (#30050)
add 5c0aabb290 Fix inconsitencies in checking edit permissions for a DAG (#20346)
add 7fe2ceaad6 Update doc for API clients release policy (#28521)
add b42ace20df Move provider issue generation to breeze (#28352)
add a0ec6fe9fc Improving the release process (#27829)
add 1456f11a99 listener plugin example added (#27905)
add 97912c84dd logging poke info when external dag is not none and task_id and task_ids are none (#28097)
add bc94f8177f Fix #28391 manual task trigger from UI fails for k8s executor (#28394)
add 652a6f48b3 introduce dag processor job (#28799)
add 7a03a733e3 Throttle streaming log reads (#28818)
add 3d5b89d553 Bump swagger-ui-dist from 3.52.0 to 4.1.3 in /airflow/www (#28824)
add 5b7b477ec2 More robust cleanup of executors in test_kubernetes_executor (#28281)
add 9e2cddfb6e Annotate KubernetesExecutor pods that we don't delete (#28844)
add adff51fa77 Email Config docs more explicit env var examples (#28845)
add fcd823eab8 clarify that the total_entries property isn't impact by pagination (#28867)
add 6efc15e046 KubenetesExecutor sends state even when successful (#28871)
add 43c93597fb Add dep context description for better log message (#28875)
add 6663802e74 Don't get ES log template from airflow local settings unless necessary (#28882)
add 46376e27e0 Be more selective when adopting pods with KubernetesExecutor (#28899)
add 763fc71805 Switch pull-request-target to base branch of pull request (#28921)
add 5ad119e409 Allow URI without authority and host blocks in `airflow connections add` (#28922)
add 874b1fd4d2 Refactor python operators/sensor tests (#28493)
add 60bb18c5be Update how PythonSensor returns values from python_callable (#28932)
add bba46bc181 improve quick start guide (#28949)
add 4624589f19 Move project and license docs down in menu to start with developer-focused docs (#28956)
add c0d8d52872 Bump epoch on python dependencies in Airlfow Image (#28960)
add 0c1272cefe Mark license block in doc as text (#28965)
add 123eb7ff9d Update go client gen command (#28967)
add f9b6324dc5 Resolve all variables in pickled XCom iterator (#28982)
add 857a80db85 Fix dag run trigger with a note. (#29228)
add 0065c6c007 Write action log to DB when DAG run is trigged via API (#28998)
add b6b2acabca fixing import error for dataset (#29007)
add 21ea2ce1f8 Remove erroneous TODO comment (#29015)
add 8de96dd2ae change context example to no longer us as variable (#29021)
add b6990132cd Only skip provider integration tests for non-main builds (#29022)
add 4cedd8076b Annotate and simplify code samples in DAGs doc (#29027)
add ed79c9988b Sanitize url_for arguments before they are passed (#29039)
add 5d63115644 Fix leak sensitive field via V1EnvVar on exception (#29016)
add 8b970a79a9 Clarify graph view dynamic task labels (#29042)
add 1450df11ae Fix kerberos authentication for the REST API. (#29054)
add cda95d1b73 Handling error on cluster policy itself (#29056)
add 250f39005d Fix pre-commit warning for exclude in inclusive-language check (#29057)
add 3b1152bbbb Several improvements to the Params doc (#29062)
add a719144d6f Capitalize dag to DAG (#29064)
add d269b78c7a Check for run_id url param when linking to graph/gantt views (#29066)
add e33a982984 emit dagrun failed duration when timeout (#29076)
add 4d13e2f970 Remove upper bound limitation for `pytest` (#29086)
add 2b161badd2 Fix warning in migrations about old config. (#29092)
add c2deb89be8 Explicit a few steps in the release process (#29098)
add 83bed35192 Update set-up-database.rst (#29104)
add 11df168d38 PR: Introduced fix to run tasks on Windows systems (#29107)
add 2ad138e4c7 Fix integration with `codecov` (#29145)
add f6598d3b52 Remove poke method to fall back to parent implementation (#29146)
add 7e956a440e General cleanup of things around KubernetesExecutor (#29148)
add 9efd09dd11 Update broken link for VS Code Quick Start guide (#29154)
add 9262990a3f Use `pod_name` when patching done KE pods (#29159)
add e5fbd65f37 fixed et.c type to etc. (#29163)
add 5819ffb9c1 Remove misleading docstring on dagrun_timeout (#29169)
add 6585ba9aa7 Bump ua-parser-js from 0.7.31 to 0.7.33 in /airflow/www (#29172)
add 37882afb04 Fix dev-mode start-airflow in Breeze (#29225)
add 52868a53b4 Fixing Task Duration view in case of manual DAG runs only (#22015) (#29195)
add e6347c771d Add trigger process to Airflow Docker docs (#29203)
add cd6b1f1040 redirect to the origin page with all the params (#29212)
add 24ebdc8bb8 Fix commit title for constraint commits (#29236)
add fbe83a68ca Update package version for go and python client (#29218)
add 6690f35075 Clarify description of worker control config (#29247)
add f25db5e649 Fixed grammatical issue (#29253)
add 0a9af855a0 Use `schedule` param rather than `timetable` in Timetables docs (#29255)
add 412adbf9e2 Initialize and validate conf early and intentionally (#29257)
add 59138dd5fd Change awkward wording about extenibility (#29261)
add 5dfd0320a1 Remove `notes` param from TriggerDagRunOperator docstring (#29298)
add 5fa3b6738d Fixed link to airflow variable in docs (#29338)
add 7ba7dffcb1 remove /legacy_tree endpoint (#29359)
add fa4028b722 Missing timezone in next_dagrun in parameterized docs (#29362)
add 6992dd4fc2 Increase max height for grid view elements (#29367)
add e2cce23e93 Remove stderr redirection for get-workflow-info (#29381)
add ce5b166ebe Mark Airflow directory in container as safe for git commands (#29386)
add 5fe5d7b278 Persist DAG and task doc values in TaskFlow API if explicitly set (#29399)
add 71790553de Ensure Serialized DAG is deleted (#29407)
add ed429f01d0 Improve health checks in example docker-compose and clarify usage (#29408)
add a41e1bac3b Fix Rest API update user output (#29409)
add 71ae0cd8fd Edgemodifier refactoring w/ labels in TaskGroup edge case (#29410)
add 6bdac035d6 Bring back fixed tenacity (#29440)
add 1b05812614 Remove ssh.rst from private-key check (#29412)
add b0632bbef8 datasets, next_run_datasets, remove unnecessary timestamp filter (#29441)
add d996532b35 Fix nested fields rendering in mapped operators (#29451)
add ae907508db Make `prev_logical_date` variable offset-aware (#29454)
add 9a7dd90390 Use newer setuptools v67.2.0 (#29465)
add 7928e89767 Use non-deprecated method when unpause DAG in the UI (#29475)
add c439634c7e Change permissions of config/password files created by airflow (#29495)
add 965e76d9ed Do not show version/node in UI traceback for unauthenticated user (#29501)
add 5bcb4dfeda Add Landing Times entry to UI docs (#29511)
add fa106151f6 Make options output of breeze command consistent and complete (#29097)
add 536665ffa1 Remove section check from get-value command (#29541)
add 101a9b3bb1 detect is 'docker-compose' existing (#29544)
add 2f88d4c31b Don't display empty rendered attrs in Task Instance Details page (#29545)
add df3bf60c6c Change expose_hostname default to false (#29547)
add 630ac5ffa7 Bump undici from 5.9.1 to 5.19.1 (#29583)
add a0a5ea26b5 Specific use-case: adding packages via requirements.txt in compose (#29598)
add eae19ced54 fix clear dag run openapi spec responses by adding additional return type (#29600)
add 8e280b68da convert moment with timezone to UTC instead of raising an exception (#29606)
add 0a0860892d Fix Scheduler crash when clear a previous run of a normal task that is now a mapped task (#29645)
add 2207eee838 FIx formatting of Dataset inlet/outlet note in TaskFlow concepts (#29678)
add ef19711703 Move extra links position in grid view (#29703)
add e8b4e1d51e Correct version label for data interval variables in documentation (#29704)
add aed369c95f Update docs re: template_fields typing and subclasses (#29725)
add 0411f50310 Description of dag_processing.last_duration (#29740)
add 0d9cec55e6 DAG list sorting lost when switching page (#29756)
add fe0f3f89db Fix compile www assets dev (#29769)
add a71d1a6cf1 better description for limit in api (#29773)
add c3d07fa150 Removed continue for not in (#29791)
add 3a916c8e3e Update ref anchor for env var link in Connection how-to doc (#29816)
add 23b8cbd78d Add a check for not templateable fields (#29821)
add ef8799ae68 POST /dagRuns API should 404 if dag not active (#29860)
add 44300c8c5f Correct link to best_practice(s) section (#29876)
add f4e05bc7c9 replace tree view with grid view in docs (#29879)
add 45e6f986e7 Correct argument name of Workday timetable in timetable.rst (#29896)
add 8c2be610f2 Normalising the usage of generate_pages (#29898)
add 867c408607 Update min version of python-deamon to fix containerd file limits (#29916)
add cedf8c49b4 Require explicit flag to remove MyPy cache when running breeze stop (#29493)
add ace19930b6 Make skipping provider tests part of selective-checks (#29216)
add cda0daa4d2 Fix circular imports when airflow starts (#29494)
add bae4e26878 Fix import cycle starting from airflow/__main__.py (#29523)
add e244f9b74a Revert "Google have re-worked their docs site ane appear not publish objects.inv (#28834)" (#29543)
add 414cef2415 Fix on_failure_callback when task receives a SIGTERM (#29743)
add 9fd79441dd Use python location in cache key used by Breeze (#29947)
add 39457d7d55 Update docker image python version to 3.7 in entrypoint doc (#29964)
add 88a39b207c Replace blockquote + Note: with RST note in docs (#29709)
add 2a24a38124 Update Airflow version to 2.5.2
add 76e96219c9 Fix grid logs for large logs (#29390)
add 222d545379 Fix validation of date-time field in API and Parameter schemas (#29395)
add cd5fcbf9df Update pipeline.rst - Fix query in `merge_data()` task (#29158)
add 6c6d83699c Replace deprecated k8s registry references (#29938)
add 4e72ada55f Move help message to the google auth code (#29888)
add 57fd64d75d Migrate remaining core sensors tests to `pytest` (#28204)
add 6767493395 Limit importlib-metadata backwport to < 5.0.0 (#29924)
add f0fd4b71cb Add Changelog for 2.5.2
add 8d6efad05e Add release commands to release management command config (#29995)
add 620e6e7424 Return error when group is not defined in breeze configuration (#29997)
add 0915a108c6 Add release flag to Breeze generate-issue-content (#28952)
add e3bfd4414d Make options output of breeze command consistent and complete (#29097)
add 89b3f2404c Fix bugs introduced when moving the code of issue generation to breeze (#28683)
add 321d602d35 Fix CI
new 6fede984d0 Installing non-editable airflow for constraint generation (#30050)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (d4afb041f8)
\
N -- N -- N refs/heads/v2-5-test (6fede984d0)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.coveragerc | 27 +-
.github/actions/breeze/action.yml | 6 +-
.github/actions/codecov-action | 1 -
.../get-target-branch-build-scripts/action.yml | 43 +
.github/actions/post_tests/action.yml | 20 +-
.github/workflows/build-images.yml | 82 +-
.github/workflows/ci.yml | 90 +-
.github/workflows/codeql-analysis.yml | 2 +-
.github/workflows/release_dockerhub_image.yml | 2 +-
.gitignore | 3 +
.gitmodules | 3 -
.pre-commit-config.yaml | 5 +-
BREEZE.rst | 48 +-
CI.rst | 2 +
CONTRIBUTORS_QUICK_START_CODESPACES.rst | 2 +-
Dockerfile.ci | 20 +-
README.md | 22 +-
RELEASE_NOTES.rst | 107 +++
STATIC_CODE_CHECKS.rst | 8 +
airflow/__init__.py | 10 +
airflow/__main__.py | 13 +-
...et_candidate_file_descriptor_ranges-to-us.patch | 76 ++
airflow/_vendor/daemon/module.mk | 27 +
airflow/api/auth/backend/kerberos_auth.py | 4 +-
.../api_connexion/endpoints/dag_run_endpoint.py | 19 +-
airflow/api_connexion/openapi/v1.yaml | 12 +-
airflow/api_connexion/schemas/dag_run_schema.py | 2 +-
airflow/cli/commands/config_command.py | 3 -
airflow/cli/commands/connection_command.py | 5 +-
airflow/cli/commands/dag_processor_command.py | 18 +-
airflow/cli/commands/standalone_command.py | 3 +-
airflow/cli/commands/task_command.py | 2 +-
airflow/config_templates/airflow_local_settings.py | 6 -
airflow/config_templates/config.yml | 22 +-
airflow/config_templates/default_airflow.cfg | 22 +-
airflow/configuration.py | 15 +
airflow/decorators/base.py | 4 +-
airflow/decorators/sensor.py | 5 -
airflow/example_dags/example_params_ui_tutorial.py | 234 +++++
airflow/example_dags/plugins/event_listener.py | 156 +++
.../plugins/listener_plugin.py} | 12 +-
airflow/exceptions.py | 10 +
airflow/executors/kubernetes_executor.py | 97 +-
airflow/jobs/dag_processor_job.py | 69 ++
airflow/jobs/local_task_job.py | 6 +-
airflow/jobs/scheduler_job.py | 6 +-
airflow/kubernetes/pod_generator.py | 2 +-
.../git_sync_template.yaml | 2 +-
...2_0_0_add_scheduling_decision_to_dagrun_and_.py | 2 +-
airflow/models/abstractoperator.py | 4 +-
airflow/models/dag.py | 33 +-
airflow/models/dagbag.py | 32 +-
airflow/models/dagrun.py | 8 +
airflow/models/mappedoperator.py | 6 +-
airflow/models/param.py | 30 +
airflow/models/serialized_dag.py | 2 +-
airflow/models/taskinstance.py | 40 +-
airflow/models/taskmixin.py | 14 +-
airflow/models/xcom.py | 12 +-
airflow/operators/trigger_dagrun.py | 1 -
.../google/common/utils/id_token_credentials.py | 26 +-
airflow/sensors/external_task.py | 28 +-
airflow/sensors/python.py | 5 +-
airflow/serialization/serialized_objects.py | 6 +
airflow/settings.py | 5 +-
airflow/ti_deps/dep_context.py | 1 +
airflow/triggers/temporal.py | 4 +-
airflow/utils/db.py | 1 +
airflow/utils/edgemodifier.py | 145 +--
airflow/utils/log/log_reader.py | 17 +-
airflow/utils/log/secrets_masker.py | 17 +-
airflow/utils/state.py | 3 -
airflow/utils/task_group.py | 18 +-
airflow/utils/timezone.py | 16 +-
airflow/www/auth.py | 2 +-
airflow/www/extensions/init_jinja_globals.py | 2 +-
airflow/www/package.json | 252 ++---
airflow/www/security.py | 16 +-
airflow/www/static/js/api/useTaskLog.ts | 2 +-
airflow/www/static/js/dag/details/Dag.tsx | 3 -
airflow/www/static/js/dag/details/dagRun/index.tsx | 3 -
.../js/dag/details/taskInstance/Logs/LogBlock.tsx | 2 +-
.../dag/details/taskInstance/Logs/index.test.tsx | 35 -
.../js/dag/details/taskInstance/Logs/index.tsx | 25 +-
.../js/dag/details/taskInstance/Logs/utils.ts | 24 +-
.../static/js/dag/details/taskInstance/index.tsx | 6 +-
airflow/www/static/js/dag/grid/index.tsx | 2 +-
airflow/www/static/js/graph.js | 17 +-
airflow/www/static/js/main.js | 3 +-
airflow/www/static/js/types/api-generated.ts | 10 +-
airflow/www/static/js/utils/useOffsetHeight.tsx | 2 +-
airflow/www/templates/airflow/dag.html | 8 +-
airflow/www/templates/airflow/task.html | 6 +-
airflow/www/utils.py | 70 +-
airflow/www/views.py | 113 +--
airflow/www/yarn.lock | 34 +-
chart/values.schema.json | 4 +-
chart/values.yaml | 4 +-
clients/gen/go.sh | 3 +-
clients/gen/python.sh | 2 +-
codecov.yml | 15 +-
dev/README_RELEASE_AIRFLOW.md | 440 ++-------
dev/README_RELEASE_PROVIDER_PACKAGES.md | 4 +-
dev/breeze/README.md | 2 +-
dev/breeze/setup.cfg | 7 +
.../src/airflow_breeze/commands/ci_commands.py | 12 +-
.../airflow_breeze/commands/ci_commands_config.py | 2 +
.../airflow_breeze/commands/ci_image_commands.py | 3 +-
.../commands/ci_image_commands_config.py | 1 -
.../airflow_breeze/commands/developer_commands.py | 13 +-
.../commands/developer_commands_config.py | 1 +
.../commands/kubernetes_commands_config.py | 4 -
.../commands/minor_release_command.py | 191 ++++
.../commands/production_image_commands_config.py | 1 -
.../commands/release_candidate_command.py | 348 +++++++
.../src/airflow_breeze/commands/release_command.py | 291 ++++++
.../commands/release_management_commands.py | 210 +++-
.../commands/release_management_commands_config.py | 35 +-
.../src/airflow_breeze/commands/setup_commands.py | 117 +++
.../commands/setup_commands_config.py | 9 +
.../airflow_breeze/commands/testing_commands.py | 9 +
.../commands/testing_commands_config.py | 1 +
.../src/airflow_breeze/params/shell_params.py | 1 +
.../provider_issue_TEMPLATE.md.jinja2} | 14 +-
dev/breeze/src/airflow_breeze/utils/confirm.py | 17 +
dev/breeze/src/airflow_breeze/utils/console.py | 4 +
.../src/airflow_breeze/utils/custom_param_types.py | 4 +
.../airflow_breeze/utils/docker_command_utils.py | 2 +
.../src/airflow_breeze/utils/github_actions.py | 4 +-
dev/breeze/src/airflow_breeze/utils/path_utils.py | 1 +
dev/breeze/src/airflow_breeze/utils/run_utils.py | 12 +-
.../src/airflow_breeze/utils/selective_checks.py | 40 +-
dev/breeze/tests/test_commands.py | 6 +
dev/breeze/tests/test_selective_checks.py | 7 +
dev/provider_packages/prepare_provider_packages.py | 196 +---
.../connections.rst | 2 +-
.../cluster-policies.rst | 4 +-
.../logging-monitoring/errors.rst | 2 +-
.../logging-monitoring/metrics.rst | 4 +-
.../modules_management.rst | 6 +-
.../production-deployment.rst | 4 +-
.../administration-and-deployment/scheduler.rst | 4 +-
.../security/access-control.rst | 4 +-
.../administration-and-deployment/security/api.rst | 3 +
.../security/kerberos.rst | 4 +-
.../authoring-and-scheduling/timetable.rst | 10 +-
docs/apache-airflow/best-practices.rst | 6 +-
docs/apache-airflow/cli-and-env-variables-ref.rst | 2 +-
docs/apache-airflow/core-concepts/dag-run.rst | 2 +-
docs/apache-airflow/core-concepts/dags.rst | 122 ++-
.../core-concepts/executor/debug.rst | 2 +-
.../core-concepts/executor/kubernetes.rst | 10 +-
docs/apache-airflow/core-concepts/operators.rst | 3 +-
docs/apache-airflow/core-concepts/params.rst | 64 +-
docs/apache-airflow/core-concepts/taskflow.rst | 9 +-
docs/apache-airflow/core-concepts/xcoms.rst | 6 +-
docs/apache-airflow/deprecated-rest-api-ref.rst | 4 +-
docs/apache-airflow/faq.rst | 14 +-
docs/apache-airflow/howto/add-dag-tags.rst | 2 +-
docs/apache-airflow/howto/connection.rst | 7 +-
docs/apache-airflow/howto/custom-operator.rst | 76 +-
.../apache-airflow/howto/docker-compose/airflow.sh | 11 +-
.../howto/docker-compose/docker-compose.yaml | 40 +-
docs/apache-airflow/howto/docker-compose/index.rst | 35 +
.../howto/dynamic-dag-generation.rst | 10 +-
docs/apache-airflow/howto/email-config.rst | 22 +-
docs/apache-airflow/howto/index.rst | 1 +
docs/apache-airflow/howto/listener-plugin.rst | 95 ++
docs/apache-airflow/howto/set-up-database.rst | 2 +-
docs/apache-airflow/howto/timetable.rst | 20 +-
docs/apache-airflow/img/airflow_erd.sha256 | 2 +-
docs/apache-airflow/img/landing_times.png | Bin 0 -> 84125 bytes
docs/apache-airflow/index.rst | 4 +-
.../installation/supported-versions.rst | 2 +-
docs/apache-airflow/license.rst | 2 +-
docs/apache-airflow/start.rst | 6 +-
docs/apache-airflow/templates-ref.rst | 12 +-
docs/apache-airflow/tutorial/fundamentals.rst | 2 +-
docs/apache-airflow/tutorial/pipeline.rst | 5 +-
docs/apache-airflow/tutorial/taskflow.rst | 2 +-
docs/apache-airflow/ui.rst | 18 +-
docs/conf.py | 35 +
docs/docker-stack/README.md | 10 +-
.../extending/add-apt-packages/Dockerfile | 2 +-
.../add-build-essential-extend/Dockerfile | 2 +-
.../extending/add-providers/Dockerfile | 2 +-
.../extending/add-pypi-packages/Dockerfile | 2 +-
.../extending/add-requirement-packages/Dockerfile | 2 +-
.../extending/custom-providers/Dockerfile | 2 +-
.../extending/embedding-dags/Dockerfile | 2 +-
.../extending/writable-directory/Dockerfile | 2 +-
docs/docker-stack/entrypoint.rst | 14 +-
docs/exts/docs_build/third_party_inventories.py | 24 +
docs/spelling_wordlist.txt | 9 +-
images/breeze/output-commands-hash.txt | 21 +-
images/breeze/output_build-docs.svg | 115 +--
images/breeze/output_ci-image.svg | 74 +-
images/breeze/output_ci-image_build.svg | 296 +++---
images/breeze/output_ci-image_pull.svg | 62 +-
images/breeze/output_ci-image_verify.svg | 34 +-
images/breeze/output_ci.svg | 90 +-
.../breeze/output_ci_find-newer-dependencies.svg | 56 +-
images/breeze/output_ci_fix-ownership.svg | 22 +-
images/breeze/output_ci_free-space.svg | 20 +-
images/breeze/output_ci_get-workflow-info.svg | 20 +-
images/breeze/output_ci_resource-check.svg | 16 +-
images/breeze/output_ci_selective-check.svg | 38 +-
images/breeze/output_cleanup.svg | 26 +-
images/breeze/output_compile-www-assets.svg | 24 +-
images/breeze/output_exec.svg | 16 +-
images/breeze/output_k8s.svg | 58 +-
images/breeze/output_k8s_build-k8s-image.svg | 50 +-
images/breeze/output_k8s_configure-cluster.svg | 58 +-
images/breeze/output_k8s_create-cluster.svg | 62 +-
images/breeze/output_k8s_delete-cluster.svg | 32 +-
images/breeze/output_k8s_deploy-airflow.svg | 68 +-
images/breeze/output_k8s_k9s.svg | 30 +-
images/breeze/output_k8s_logs.svg | 34 +-
images/breeze/output_k8s_run-complete-tests.svg | 88 +-
images/breeze/output_k8s_setup-env.svg | 22 +-
images/breeze/output_k8s_shell.svg | 38 +-
images/breeze/output_k8s_status.svg | 34 +-
images/breeze/output_k8s_tests.svg | 66 +-
images/breeze/output_k8s_upload-k8s-image.svg | 56 +-
images/breeze/output_prod-image.svg | 74 +-
images/breeze/output_prod-image_build.svg | 192 ++--
images/breeze/output_prod-image_pull.svg | 60 +-
images/breeze/output_prod-image_verify.svg | 34 +-
images/breeze/output_release-management.svg | 94 +-
...put_release-management_generate-constraints.svg | 62 +-
...management_generate-issue-content-providers.svg | 200 ++++
..._release-management_prepare-airflow-package.svg | 28 +-
...e-management_prepare-provider-documentation.svg | 65 +-
...elease-management_prepare-provider-packages.svg | 30 +-
...tput_release-management_release-prod-images.svg | 46 +-
...release-management_verify-provider-packages.svg | 140 +--
images/breeze/output_setup.svg | 22 +-
images/breeze/output_setup_autocomplete.svg | 24 +-
.../output_setup_check-all-params-in-groups.svg | 172 ++++
images/breeze/output_setup_config.svg | 40 +-
.../output_setup_regenerate-command-images.svg | 44 +-
images/breeze/output_setup_self-upgrade.svg | 18 +-
images/breeze/output_setup_version.svg | 16 +-
images/breeze/output_stop.svg | 32 +-
images/breeze/output_testing.svg | 24 +-
.../breeze/output_testing_docker-compose-tests.svg | 32 +-
images/breeze/output_testing_helm-tests.svg | 34 +-
images/breeze/output_testing_integration-tests.svg | 72 +-
images/breeze/output_testing_tests.svg | 90 +-
pyproject.toml | 2 +-
scripts/ci/constraints/ci_commit_constraints.sh | 2 +-
scripts/ci/docker-compose/_docker.env | 1 +
scripts/ci/docker-compose/base.yml | 1 +
scripts/ci/docker-compose/devcontainer.env | 1 +
.../pre_commit_compile_www_assets_dev.py | 8 +-
scripts/ci/pre_commit/pre_commit_mypy.py | 3 +-
.../ci/pre_commit/pre_commit_supported_versions.py | 2 +-
scripts/docker/entrypoint_ci.sh | 18 +-
scripts/in_container/run_ci_tests.sh | 1 -
.../in_container/run_prepare_airflow_packages.sh | 7 +
.../run_prepare_provider_documentation.sh | 5 -
setup.cfg | 12 +-
setup.py | 18 +-
.../endpoints/test_dag_run_endpoint.py | 94 +-
.../test_mapped_task_instance_endpoint.py | 7 +-
.../endpoints/test_task_instance_endpoint.py | 29 +-
tests/cli/commands/test_config_command.py | 12 +-
tests/cli/commands/test_connection_command.py | 67 +-
tests/cli/commands/test_dag_command.py | 2 +-
tests/cli/commands/test_dag_processor_command.py | 13 +-
tests/cli/commands/test_db_command.py | 4 +-
tests/conftest.py | 10 +-
tests/decorators/test_python.py | 168 ++--
tests/decorators/test_python_virtualenv.py | 13 -
tests/executors/test_kubernetes_executor.py | 416 ++++----
tests/models/test_dag.py | 159 +++-
tests/models/test_dagrun.py | 67 +-
tests/models/test_mappedoperator.py | 10 +-
tests/models/test_param.py | 70 +-
tests/models/test_serialized_dag.py | 2 +-
tests/models/test_taskinstance.py | 92 +-
tests/operators/test_python.py | 1003 +++++++-------------
.../common/utils/test_id_token_credentials.py | 7 +-
tests/sensors/test_date_time.py | 13 +-
tests/sensors/test_external_task_sensor.py | 127 ++-
tests/sensors/test_python.py | 135 +--
tests/sensors/test_time_sensor.py | 6 +-
tests/serialization/test_dag_serialization.py | 18 +-
tests/task/__init__.py | 4 -
.../task/task_runner/conftest.py | 28 +-
tests/test_utils/mock_operators.py | 41 +-
tests/triggers/test_temporal.py | 14 +-
tests/utils/test_edgemodifier.py | 501 ++++++++++
tests/www/test_security.py | 71 +-
tests/www/test_utils.py | 50 +-
tests/www/views/test_views_home.py | 36 +
tests/www/views/test_views_tasks.py | 13 +
297 files changed, 7674 insertions(+), 4260 deletions(-)
delete mode 160000 .github/actions/codecov-action
create mode 100644 .github/actions/get-target-branch-build-scripts/action.yml
create mode 100644 airflow/_vendor/daemon/0001-Refactor-_get_candidate_file_descriptor_ranges-to-us.patch
create mode 100644 airflow/_vendor/daemon/module.mk
create mode 100644 airflow/example_dags/example_params_ui_tutorial.py
create mode 100644 airflow/example_dags/plugins/event_listener.py
copy airflow/{providers/amazon/aws/utils/rds.py => example_dags/plugins/listener_plugin.py} (78%)
create mode 100644 airflow/jobs/dag_processor_job.py
create mode 100644 dev/breeze/src/airflow_breeze/commands/minor_release_command.py
create mode 100644 dev/breeze/src/airflow_breeze/commands/release_candidate_command.py
create mode 100644 dev/breeze/src/airflow_breeze/commands/release_command.py
rename dev/{provider_packages/PROVIDER_ISSUE_TEMPLATE.md.jinja2 => breeze/src/airflow_breeze/provider_issue_TEMPLATE.md.jinja2} (57%)
create mode 100644 docs/apache-airflow/howto/listener-plugin.rst
create mode 100644 docs/apache-airflow/img/landing_times.png
create mode 100644 images/breeze/output_release-management_generate-issue-content-providers.svg
create mode 100644 images/breeze/output_setup_check-all-params-in-groups.svg
copy airflow/www/extensions/init_robots.py => tests/task/task_runner/conftest.py (58%)
[airflow] 01/01: Installing non-editable airflow for constraint generation (#30050)
Posted by po...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch v2-5-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 6fede984d00aa23d8815d3b79ac002d4c144714e
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Sun Mar 12 03:20:47 2023 +0100
Installing non-editable airflow for constraint generation (#30050)
When we are generating constraints including released version of
providers or "no providers" (which effectively means preinstalled
version of providers only), we should use non-editable installation
of airflow.
Normally in CI, we have editable version of airflow and we install
providers from sources, but this leads to skipping installation
of the preinstalled providers (it is connected to the fact that we
were earlier installing as editable and "sources only" and those
packages are marked as required, which confuses newer version of
pip. It prints that the preinstalled packages are installed, but
it does not specify the version - apparently thinking the editable
version of the install already contains them.
Since our behaviour is pretty non-standard, this is not really a
bug in pip, but it is a bug in our approach when we are adding
providers for constraint generation. Rather than using editable
installation when we add providers, we should use "regular"
installation. This seems to remove the confusion and the preinstalled
packages are also installed.
(cherry picked from commit 05242e95bbfbaf153e4ae971fc0d0a5314d5bdb8)
---
scripts/in_container/_in_container_utils.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/in_container/_in_container_utils.sh b/scripts/in_container/_in_container_utils.sh
index 3240b2c90d..e0e8594954 100644
--- a/scripts/in_container/_in_container_utils.sh
+++ b/scripts/in_container/_in_container_utils.sh
@@ -261,7 +261,7 @@ function install_local_airflow_with_eager_upgrade() {
# we add eager requirements to make sure to take into account limitations that will allow us to
# install all providers
# shellcheck disable=SC2086
- pip install -e ".${extras}" ${EAGER_UPGRADE_ADDITIONAL_REQUIREMENTS} \
+ pip install ".${extras}" ${EAGER_UPGRADE_ADDITIONAL_REQUIREMENTS} \
--upgrade --upgrade-strategy eager
}
@@ -290,7 +290,7 @@ function install_all_providers_from_pypi_with_eager_upgrade() {
# Installing it with Airflow makes sure that the version of package that matches current
# Airflow requirements will be used.
# shellcheck disable=SC2086
- pip install -e ".[${NO_PROVIDERS_EXTRAS}]" "${packages_to_install[@]}" ${EAGER_UPGRADE_ADDITIONAL_REQUIREMENTS} \
+ pip install ".[${NO_PROVIDERS_EXTRAS}]" "${packages_to_install[@]}" ${EAGER_UPGRADE_ADDITIONAL_REQUIREMENTS} \
--upgrade --upgrade-strategy eager
}