You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by pi...@apache.org on 2023/03/09 00:07:54 UTC
[airflow] branch v2-5-test updated (f996075928 -> caef382367)
This is an automated email from the ASF dual-hosted git repository.
pierrejeambrun pushed a change to branch v2-5-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit f996075928 Add Changelog for 2.5.2
omit 36eea087cf Fix CI
new f167e41962 Fix CI
new caef382367 Add Changelog for 2.5.2
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (f996075928)
\
N -- N -- N refs/heads/v2-5-test (caef382367)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/spelling_wordlist.txt | 1 +
tests/models/test_dag.py | 123 +++++++++++++++------------------------------
2 files changed, 42 insertions(+), 82 deletions(-)
[airflow] 01/02: Fix CI
Posted by pi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
pierrejeambrun pushed a commit to branch v2-5-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit f167e419628296537020d9f02d179f830a76e80a
Author: pierrejeambrun <pi...@gmail.com>
AuthorDate: Thu Mar 9 00:19:05 2023 +0100
Fix CI
---
.pre-commit-config.yaml | 2 +-
docs/spelling_wordlist.txt | 1 +
tests/models/test_dag.py | 123 +++++++++++++++------------------------------
3 files changed, 43 insertions(+), 83 deletions(-)
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index dae2207a9a..bbd2c46db3 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -147,7 +147,7 @@ repos:
files: >
\.cfg$|\.conf$|\.ini$|\.ldif$|\.properties$|\.readthedocs$|\.service$|\.tf$|Dockerfile.*$
- repo: https://github.com/PyCQA/isort
- rev: 5.11.2
+ rev: 5.12.0
hooks:
- id: isort
name: Run isort to sort imports in Python files
diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt
index 3c3b855133..67cc6cfc65 100644
--- a/docs/spelling_wordlist.txt
+++ b/docs/spelling_wordlist.txt
@@ -283,6 +283,7 @@ conn
connectTimeoutMS
connexion
containerConfiguration
+containerd
ContainerGroup
containerinstance
ContainerPort
diff --git a/tests/models/test_dag.py b/tests/models/test_dag.py
index 59721f4347..4940d0d562 100644
--- a/tests/models/test_dag.py
+++ b/tests/models/test_dag.py
@@ -471,7 +471,6 @@ class TestDag:
session.close()
def test_get_task_instances_before(self):
-
BASE_DATE = timezone.datetime(2022, 7, 20, 20)
test_dag_id = "test_get_task_instances_before"
@@ -503,90 +502,53 @@ class TestDag:
REF_DATE = BASE_DATE
- assert set([dr.run_id for dr in [dr1]]) == set(
- [
- ti.run_id
- for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=1, session=session)
- ]
- )
- assert set([dr.run_id for dr in [dr1, dr2, dr3]]) == set(
- [
- ti.run_id
- for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=3, session=session)
- ]
- )
- assert set([dr.run_id for dr in [dr1, dr2, dr3, dr4, dr5]]) == set(
- [
- ti.run_id
- for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=5, session=session)
- ]
- )
- assert set([dr.run_id for dr in [dr1, dr2, dr3, dr4, dr5, dr6, dr7]]) == set(
- [
- ti.run_id
- for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=7, session=session)
- ]
- )
- assert set([dr.run_id for dr in [dr1, dr2, dr3, dr4, dr5, dr6, dr7, dr8]]) == set(
- [
- ti.run_id
- for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=9, session=session)
- ]
- )
- assert set([dr.run_id for dr in [dr1, dr2, dr3, dr4, dr5, dr6, dr7, dr8]]) == set(
- [
- ti.run_id
- for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=10, session=session)
- ]
- ) # stays constrained to available ones
+ assert {dr.run_id for dr in [dr1]} == {
+ ti.run_id for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=1, session=session)
+ }
+ assert {dr.run_id for dr in [dr1, dr2, dr3]} == {
+ ti.run_id for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=3, session=session)
+ }
+ assert {dr.run_id for dr in [dr1, dr2, dr3, dr4, dr5]} == {
+ ti.run_id for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=5, session=session)
+ }
+ assert {dr.run_id for dr in [dr1, dr2, dr3, dr4, dr5, dr6, dr7]} == {
+ ti.run_id for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=7, session=session)
+ }
+ assert {dr.run_id for dr in [dr1, dr2, dr3, dr4, dr5, dr6, dr7, dr8]} == {
+ ti.run_id for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=9, session=session)
+ }
+ assert {dr.run_id for dr in [dr1, dr2, dr3, dr4, dr5, dr6, dr7, dr8]} == {
+ ti.run_id
+ for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=10, session=session)
+ } # stays constrained to available ones
REF_DATE = BASE_DATE + timedelta(hours=-3.5)
- assert set([dr.run_id for dr in [dr4]]) == set(
- [
- ti.run_id
- for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=1, session=session)
- ]
- )
- assert set([dr.run_id for dr in [dr4, dr5, dr6]]) == set(
- [
- ti.run_id
- for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=3, session=session)
- ]
- )
- assert set([dr.run_id for dr in [dr4, dr5, dr6, dr7, dr8]]) == set(
- [
- ti.run_id
- for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=5, session=session)
- ]
- )
- assert set([dr.run_id for dr in [dr4, dr5, dr6, dr7, dr8]]) == set(
- [
- ti.run_id
- for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=6, session=session)
- ]
- ) # stays constrained to available ones
+ assert {dr.run_id for dr in [dr4]} == {
+ ti.run_id for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=1, session=session)
+ }
+ assert {dr.run_id for dr in [dr4, dr5, dr6]} == {
+ ti.run_id for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=3, session=session)
+ }
+ assert {dr.run_id for dr in [dr4, dr5, dr6, dr7, dr8]} == {
+ ti.run_id for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=5, session=session)
+ }
+ assert {dr.run_id for dr in [dr4, dr5, dr6, dr7, dr8]} == {
+ ti.run_id for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=6, session=session)
+ } # stays constrained to available ones
REF_DATE = BASE_DATE + timedelta(hours=-8)
- assert set([dr.run_id for dr in [dr8]]) == set(
- [
- ti.run_id
- for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=0, session=session)
- ]
- )
- assert set([dr.run_id for dr in [dr8]]) == set(
- [
- ti.run_id
- for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=1, session=session)
- ]
- )
- assert set([dr.run_id for dr in [dr8]]) == set(
- [
- ti.run_id
- for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=10, session=session)
- ]
- )
+ assert {dr.run_id for dr in [dr8]} == {
+ ti.run_id for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=0, session=session)
+ }
+ assert {dr.run_id for dr in [dr8]} == {
+ ti.run_id for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=1, session=session)
+ }
+ assert {dr.run_id for dr in [dr8]} == {
+ ti.run_id
+ for ti in test_dag.get_task_instances_before(base_date=REF_DATE, num=10, session=session)
+ }
session.close()
@@ -634,7 +596,6 @@ class TestDag:
assert isinstance(jinja_env, expected_env)
def test_resolve_template_files_value(self):
-
with NamedTemporaryFile(suffix=".template") as f:
f.write(b"{{ ds }}")
f.flush()
@@ -652,7 +613,6 @@ class TestDag:
assert task.test_field == "{{ ds }}"
def test_resolve_template_files_list(self):
-
with NamedTemporaryFile(suffix=".template") as f:
f.write(b"{{ ds }}")
f.flush()
@@ -1265,7 +1225,6 @@ class TestDag:
session.close()
def test_existing_dag_default_view(self):
-
with create_session() as session:
session.add(DagModel(dag_id="dag_default_view_old", default_view=None))
session.commit()
[airflow] 02/02: Add Changelog for 2.5.2
Posted by pi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
pierrejeambrun pushed a commit to branch v2-5-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit caef382367c5ee23c295b6755e23cac49ff51759
Author: pierrejeambrun <pi...@gmail.com>
AuthorDate: Wed Mar 8 23:22:09 2023 +0100
Add Changelog for 2.5.2
---
RELEASE_NOTES.rst | 107 ++++++++++++++++++++++++++++++++++++
newsfragments/29395.significant.rst | 8 ---
newsfragments/29547.significant.rst | 3 -
3 files changed, 107 insertions(+), 11 deletions(-)
diff --git a/RELEASE_NOTES.rst b/RELEASE_NOTES.rst
index ff777a75ed..7993e0899d 100644
--- a/RELEASE_NOTES.rst
+++ b/RELEASE_NOTES.rst
@@ -21,6 +21,113 @@
.. towncrier release notes start
+Airflow 2.5.2 (2023-03-13)
+--------------------------
+
+Significant Changes
+^^^^^^^^^^^^^^^^^^^
+
+The date-time fields passed as API parameters or Params should be RFC3339-compliant (#29395)
+""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+
+In case of API calls, it was possible that "+" passed as part of the date-time fields were not URL-encoded, and
+such date-time fields could pass validation. Such date-time parameters should now be URL-encoded (as ``%2B``).
+
+In case of parameters, we still allow IS8601-compliant date-time (so for example it is possible that
+' ' was used instead of ``T`` separating date from time and no timezone was specified) but we raise
+deprecation warning.
+
+Default for ``[webserver] expose_hostname`` changed to ``False`` (#29547)
+"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+
+The default for ``[webserver] expose_hostname`` has been set to ``False``, instead of ``True``. This means administrators must opt-in to expose webserver hostnames to end users.
+
+Bug Fixes
+^^^^^^^^^
+- Fix validation of date-time field in API and Parameter schemas (#29395)
+- Fix grid logs for large logs (#29390)
+- Fix on_failure_callback when task receives a SIGTERM (#29743)
+- Update min version of python-daemon to fix containerd file limits (#29916)
+- POST /dagRuns API should 404 if dag not active (#29860)
+- DAG list sorting lost when switching page (#29756)
+- Fix Scheduler crash when clear a previous run of a normal task that is now a mapped task (#29645)
+- Convert moment with timezone to UTC instead of raising an exception (#29606)
+- Fix clear dag run openapi spec responses by adding additional return type (#29600)
+- Don't display empty rendered attrs in Task Instance Details page (#29545)
+- Remove section check from get-value command (#29541)
+- Do not show version/node in UI traceback for unauthenticated user (#29501)
+- Make ``prev_logical_date`` variable offset-aware (#29454)
+- Fix nested fields rendering in mapped operators (#29451)
+- Datasets, next_run_datasets, remove unnecessary timestamp filter (#29441)
+- Edgemodifier refactoring w/ labels in TaskGroup edge case (#29410)
+- Fix Rest API update user output (#29409)
+- Ensure Serialized DAG is deleted (#29407)
+- Persist DAG and task doc values in TaskFlow API if explicitly set (#29399)
+- Redirect to the origin page with all the params (#29212)
+- Fixing Task Duration view in case of manual DAG runs only (#22015) (#29195)
+- Remove poke method to fall back to parent implementation (#29146)
+- PR: Introduced fix to run tasks on Windows systems (#29107)
+- Fix warning in migrations about old config. (#29092)
+- Emit dagrun failed duration when timeout (#29076)
+- Handling error on cluster policy itself (#29056)
+- Fix kerberos authentication for the REST API. (#29054)
+- Fix leak sensitive field via V1EnvVar on exception (#29016)
+- Sanitize url_for arguments before they are passed (#29039)
+- Fix dag run trigger with a note. (#29228)
+- Write action log to DB when DAG run is triggered via API (#28998)
+- Resolve all variables in pickled XCom iterator (#28982)
+- Allow URI without authority and host blocks in ``airflow connections add`` (#28922)
+- Be more selective when adopting pods with KubernetesExecutor (#28899)
+- KubenetesExecutor sends state even when successful (#28871)
+- Annotate KubernetesExecutor pods that we don't delete (#28844)
+- Throttle streaming log reads (#28818)
+- Introduce dag processor job (#28799)
+- Fix #28391 manual task trigger from UI fails for k8s executor (#28394)
+- Logging poke info when external dag is not none and task_id and task_ids are none (#28097)
+- Fix inconsitencies in checking edit permissions for a DAG (#20346)
+
+Misc/Internal
+^^^^^^^^^^^^^
+- Add a check for not templateable fields (#29821)
+- Removed continue for not in (#29791)
+- Move extra links position in grid view (#29703)
+- Bump undici from 5.9.1 to 5.19.1 (#29583)
+- Change expose_hostname default to false (#29547)
+- Change permissions of config/password files created by airflow (#29495)
+- Use newer setuptools v67.2.0 (#29465)
+- Increase max height for grid view elements (#29367)
+- Clarify description of worker control config (#29247)
+- Bump ua-parser-js from 0.7.31 to 0.7.33 in /airflow/www (#29172)
+- Remove upper bound limitation for ``pytest`` (#29086)
+- Check for run_id url param when linking to graph/gantt views (#29066)
+- Clarify graph view dynamic task labels (#29042)
+- Fixing import error for dataset (#29007)
+- Update how PythonSensor returns values from python_callable (#28932)
+- Add dep context description for better log message (#28875)
+- Bump swagger-ui-dist from 3.52.0 to 4.1.3 in /airflow/www (#28824)
+
+Doc Only Changes
+^^^^^^^^^^^^^^^^
+- Update pipeline.rst - Fix query in ``merge_data()`` task (#29158)
+- Correct argument name of Workday timetable in timetable.rst (#29896)
+- Update ref anchor for env var link in Connection how-to doc (#29816)
+- Better description for limit in api (#29773)
+- Description of dag_processing.last_duration (#29740)
+- Update docs re: template_fields typing and subclasses (#29725)
+- FIx formatting of Dataset inlet/outlet note in TaskFlow concepts (#29678)
+- Specific use-case: adding packages via requirements.txt in compose (#29598)
+- Detect is 'docker-compose' existing (#29544)
+- Add Landing Times entry to UI docs (#29511)
+- Improve health checks in example docker-compose and clarify usage (#29408)
+- Remove ``notes`` param from TriggerDagRunOperator docstring (#29298)
+- Use ``schedule`` param rather than ``timetable`` in Timetables docs (#29255)
+- Add trigger process to Airflow Docker docs (#29203)
+- Update set-up-database.rst (#29104)
+- Several improvements to the Params doc (#29062)
+- Email Config docs more explicit env var examples (#28845)
+- Listener plugin example added (#27905)
+
+
Airflow 2.5.1 (2023-01-20)
--------------------------
diff --git a/newsfragments/29395.significant.rst b/newsfragments/29395.significant.rst
deleted file mode 100644
index 14f094397d..0000000000
--- a/newsfragments/29395.significant.rst
+++ /dev/null
@@ -1,8 +0,0 @@
-The date-time fields passed as API parameters or Params should be RFC3339-compliant.
-
-In case of API calls, it was possible that "+" passed as part of the date-time fields were not URL-encoded, and
-such date-time fields could pass validation. Such date-time parameters should now be URL-encoded (as ``%2B``).
-
-In case of parameters, we still allow IS8601-compliant date-time (so for example it is possible that
-' ' was used instead of ``T`` separating date from time and no timezone was specified) but we raise
-deprecation warning.
diff --git a/newsfragments/29547.significant.rst b/newsfragments/29547.significant.rst
deleted file mode 100644
index 2a9f4c45c3..0000000000
--- a/newsfragments/29547.significant.rst
+++ /dev/null
@@ -1,3 +0,0 @@
-Default for ``[webserver] expose_hostname`` changed to ``False``
-
-The default for ``[webserver] expose_hostname`` has been set to ``False``, instead of ``True``. This means administrators must opt-in to expose webserver hostnames to end users.