You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ra...@apache.org on 2026/07/03 08:49:41 UTC

(airflow) branch v3-3-test updated (32b1ffa982d -> 767cee3c466)

This is an automated email from the ASF dual-hosted git repository.

vatsrahul1001 pushed a change to branch v3-3-test
in repository https://gitbox.apache.org/repos/asf/airflow.git


 discard 32b1ffa982d [v3-3-test] Clarify when last_automated_data_interval is None in timetable docs (#68461) (#69227)
 discard 60081077727 [v3-3-test] Fix deadline callback data persistence (#69073) (#69259)
    omit 8e77f9e7d12 [v3-3-test] Fix retry policy overrides not persisted to task instance history (#69235) (#69241)
    omit a039c5f368f [v3-3-test] [AIP-94] Mark config CLI commands as migrated to airflowctl (#68958) (#69258)
    omit fc2feaaa287 [v3-3-test] Mark providers commands for airflowctl(#68525) (#69232)
    omit cb49e750491 [v3-3-test] Add new Committers to CODEOWNERS (#69215) (#69219)
    omit 4ec3ca401fa Update SQLAlchemy documentation links from 1.4 to 2.0 (#69200) (#69203)
    omit 261fc1c9661 [v3-3-test] Fix drifting data intervals for monthly/yearly schedules with catchup disabled (#69143) (#69189)
     add fc7faf2bfd7 update release notes for 3.3.0rc1
     new 3ba9ddbcdba [v3-3-test] Fix drifting data intervals for monthly/yearly schedules with catchup disabled (#69143) (#69189)
     new 76893fb14ac Update SQLAlchemy documentation links from 1.4 to 2.0 (#69200) (#69203)
     new 84590c18331 [v3-3-test] Add new Committers to CODEOWNERS (#69215) (#69219)
     new 7ebb7dc0c2b [v3-3-test] Mark providers commands for airflowctl(#68525) (#69232)
     new e9eda507cc8 [v3-3-test] [AIP-94] Mark config CLI commands as migrated to airflowctl (#68958) (#69258)
     new e3418895b40 [v3-3-test] Fix retry policy overrides not persisted to task instance history (#69235) (#69241)
     new 7fe134e787d [v3-3-test] Fix deadline callback data persistence (#69073) (#69259)
     new 767cee3c466 [v3-3-test] Clarify when last_automated_data_interval is None in timetable docs (#68461) (#69227)

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (32b1ffa982d)
            \
             N -- N -- N   refs/heads/v3-3-test (767cee3c466)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 8 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 RELEASE_NOTES.rst          | 135 ++++++++++++++++++++++++---------------------
 docs/spelling_wordlist.txt |   1 +
 reproducible_build.yaml    |   4 +-
 3 files changed, 74 insertions(+), 66 deletions(-)


(airflow) 01/08: [v3-3-test] Fix drifting data intervals for monthly/yearly schedules with catchup disabled (#69143) (#69189)

Posted by ra...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

vatsrahul1001 pushed a commit to branch v3-3-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit 3ba9ddbcdba1b01ab20936d4bed9fa79c2326516
Author: github-actions[bot] <41...@users.noreply.github.com>
AuthorDate: Wed Jul 1 14:30:05 2026 +0800

    [v3-3-test] Fix drifting data intervals for monthly/yearly schedules with catchup disabled (#69143) (#69189)
    
    Co-authored-by: Shahar Epstein <60...@users.noreply.github.com>
---
 airflow-core/newsfragments/69143.bugfix.rst        |  1 +
 airflow-core/src/airflow/timetables/interval.py    | 26 +++++--
 .../unit/timetables/test_interval_timetable.py     | 84 ++++++++++++++++++++++
 3 files changed, 107 insertions(+), 4 deletions(-)

diff --git a/airflow-core/newsfragments/69143.bugfix.rst b/airflow-core/newsfragments/69143.bugfix.rst
new file mode 100644
index 00000000000..f6a2b7880d9
--- /dev/null
+++ b/airflow-core/newsfragments/69143.bugfix.rst
@@ -0,0 +1 @@
+Fix drifting, mis-aligned data intervals for monthly and yearly (``relativedelta``) schedules when ``catchup`` is disabled. The first interval is now anchored to the Dag's ``start_date`` instead of a fixed 30-day/365-day grid, matching the boundaries produced with ``catchup=True``.
diff --git a/airflow-core/src/airflow/timetables/interval.py b/airflow-core/src/airflow/timetables/interval.py
index 5064ee9be50..30a7f5033fa 100644
--- a/airflow-core/src/airflow/timetables/interval.py
+++ b/airflow-core/src/airflow/timetables/interval.py
@@ -224,8 +224,26 @@ class DeltaDataIntervalTimetable(DeltaMixin, _DataIntervalTimetable):
             + delta.seconds
         )
 
-    def _round(self, dt: DateTime) -> DateTime:
-        """Round the given time to the nearest interval."""
+    def _round(self, dt: DateTime, anchor: DateTime) -> DateTime:
+        """
+        Floor ``dt`` to the latest schedule boundary at or before it.
+
+        Months/years have no fixed second count, so the epoch-grid rounding
+        used for fixed deltas would drift. For them we anchor on ``anchor``
+        (the start_date) and advance one period at a time, so boundaries match
+        the catchup=True grid -- relativedelta day-clamping is path-dependent
+        (e.g. Jan 31 -> Feb 28 -> Mar 28), so a multiplied jump would land
+        elsewhere. Fixed deltas keep the historical epoch rounding and ignore
+        ``anchor``.
+
+        ``anchor`` must be at or before ``dt``; otherwise the forward stepping
+        cannot reach ``dt`` and the result is meaningless.
+        """
+        if isinstance(self._delta, relativedelta) and (self._delta.months or self._delta.years):
+            boundary = anchor
+            while self._get_next(boundary) <= dt:
+                boundary = self._get_next(boundary)
+            return boundary
         if isinstance(self._delta, datetime.timedelta):
             delta_in_seconds = self._delta.total_seconds()
         else:
@@ -243,8 +261,8 @@ class DeltaDataIntervalTimetable(DeltaMixin, _DataIntervalTimetable):
 
         This is slightly different from the cron version at terminal values.
         """
-        round_current_time = self._round(coerce_datetime(utcnow()))
-        new_start = self._get_prev(round_current_time)
+        now = coerce_datetime(utcnow())
+        new_start = self._get_prev(self._round(now, earliest or now))
         if earliest is None:
             return new_start
         return max(new_start, earliest)
diff --git a/airflow-core/tests/unit/timetables/test_interval_timetable.py b/airflow-core/tests/unit/timetables/test_interval_timetable.py
index d8ad62131a8..dbe27c6784f 100644
--- a/airflow-core/tests/unit/timetables/test_interval_timetable.py
+++ b/airflow-core/tests/unit/timetables/test_interval_timetable.py
@@ -43,6 +43,8 @@ OLD_INTERVAL = DataInterval(start=YESTERDAY, end=CURRENT_TIME)
 HOURLY_CRON_TIMETABLE = CronDataIntervalTimetable("@hourly", utc)
 HOURLY_TIMEDELTA_TIMETABLE = DeltaDataIntervalTimetable(datetime.timedelta(hours=1))
 HOURLY_RELATIVEDELTA_TIMETABLE = DeltaDataIntervalTimetable(dateutil.relativedelta.relativedelta(hours=1))
+MONTHLY_RELATIVEDELTA_TIMETABLE = DeltaDataIntervalTimetable(dateutil.relativedelta.relativedelta(months=1))
+YEARLY_RELATIVEDELTA_TIMETABLE = DeltaDataIntervalTimetable(dateutil.relativedelta.relativedelta(years=1))
 
 CRON_TIMETABLE = CronDataIntervalTimetable("30 16 * * *", utc)
 DELTA_FROM_MIDNIGHT = datetime.timedelta(minutes=30, hours=16)
@@ -139,6 +141,88 @@ def test_no_catchup_next_info_starts_at_current_time(
     assert next_info == DagRunInfo.interval(start=expected_start, end=CURRENT_TIME)
 
 
+@pytest.mark.parametrize(
+    "last_automated_data_interval",
+    [
+        pytest.param(None, id="first-run"),
+        pytest.param(
+            DataInterval(
+                pendulum.DateTime(2020, 1, 1, tzinfo=utc),
+                pendulum.DateTime(2020, 2, 1, tzinfo=utc),
+            ),
+            id="subsequent",
+        ),
+    ],
+)
+@pytest.mark.parametrize(
+    ("timetable", "start_date", "expected_start", "expected_end"),
+    [
+        pytest.param(
+            MONTHLY_RELATIVEDELTA_TIMETABLE,
+            pendulum.DateTime(2025, 1, 15, tzinfo=utc),
+            pendulum.DateTime(2026, 5, 15, tzinfo=utc),
+            pendulum.DateTime(2026, 6, 15, tzinfo=utc),
+            id="monthly",
+        ),
+        pytest.param(
+            YEARLY_RELATIVEDELTA_TIMETABLE,
+            pendulum.DateTime(2020, 3, 10, tzinfo=utc),
+            pendulum.DateTime(2025, 3, 10, tzinfo=utc),
+            pendulum.DateTime(2026, 3, 10, tzinfo=utc),
+            id="yearly",
+        ),
+    ],
+)
+@time_machine.travel(pendulum.DateTime(2026, 6, 29, tzinfo=utc))
+def test_no_catchup_calendar_delta_aligns_to_start_date(
+    timetable: Timetable,
+    start_date: pendulum.DateTime,
+    expected_start: pendulum.DateTime,
+    expected_end: pendulum.DateTime,
+    last_automated_data_interval: DataInterval | None,
+) -> None:
+    """``catchup=False`` with a relativedelta in months/years must stay aligned
+    to ``start_date`` and not drift onto a fixed 30-day/365-day epoch grid."""
+    next_info = timetable.next_dagrun_info(
+        last_automated_data_interval=last_automated_data_interval,
+        restriction=TimeRestriction(earliest=start_date, latest=None, catchup=False),
+    )
+    assert next_info == DagRunInfo.interval(start=expected_start, end=expected_end)
+
+
+@time_machine.travel(pendulum.DateTime(2026, 6, 29, 12, tzinfo=utc))
+def test_no_catchup_calendar_delta_without_start_date_ends_now() -> None:
+    """With no ``start_date`` to anchor on, the interval simply ends at now."""
+    next_info = MONTHLY_RELATIVEDELTA_TIMETABLE.next_dagrun_info(
+        last_automated_data_interval=None,
+        restriction=TimeRestriction(earliest=None, latest=None, catchup=False),
+    )
+    assert next_info == DagRunInfo.interval(
+        start=pendulum.DateTime(2026, 5, 29, 12, tzinfo=utc),
+        end=pendulum.DateTime(2026, 6, 29, 12, tzinfo=utc),
+    )
+
+
+@time_machine.travel(pendulum.DateTime(2026, 6, 29, tzinfo=utc))
+def test_no_catchup_calendar_delta_uses_one_period_at_a_time_clamping() -> None:
+    """Boundaries advance one relativedelta period at a time, so day-clamping is
+    path-dependent: Jan 31 + 1 month clamps to Feb 28, and because each step starts
+    from the previous boundary (not from Jan 31), the 28 then sticks --
+    Feb 28 -> Mar 28 -> ... -> May 28 -> Jun 28. A single multiplied jump off
+    ``start_date`` would instead re-clamp from the 31st and land elsewhere
+    (Jan 31 + 5 months -> Jun 30)."""
+    next_info = MONTHLY_RELATIVEDELTA_TIMETABLE.next_dagrun_info(
+        last_automated_data_interval=None,
+        restriction=TimeRestriction(
+            earliest=pendulum.DateTime(2025, 1, 31, tzinfo=utc), latest=None, catchup=False
+        ),
+    )
+    assert next_info == DagRunInfo.interval(
+        start=pendulum.DateTime(2026, 5, 28, tzinfo=utc),
+        end=pendulum.DateTime(2026, 6, 28, tzinfo=utc),
+    )
+
+
 @pytest.mark.parametrize(
     "timetable",
     [


(airflow) 02/08: Update SQLAlchemy documentation links from 1.4 to 2.0 (#69200) (#69203)

Posted by ra...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

vatsrahul1001 pushed a commit to branch v3-3-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit 76893fb14ace68892d3b4d33723c04eecf009b32
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Wed Jul 1 14:18:33 2026 +0200

    Update SQLAlchemy documentation links from 1.4 to 2.0 (#69200) (#69203)
    
    The SQLAlchemy 1.4 docs are frozen at an old version; the project has
    moved to 2.0. Point all references at the current docs so readers land
    on maintained content instead of a stale release.
    
    (cherry picked from commit a8770ba566845076a3d5d6adc723776cb0798747)
    
    Co-authored-by: Ashley Huang <12...@users.noreply.github.com>
---
 airflow-core/docs/howto/set-up-database.rst                  | 12 ++++++------
 airflow-core/src/airflow/assets/manager.py                   |  2 +-
 airflow-core/src/airflow/models/base.py                      |  2 +-
 .../sqlite/src/airflow/providers/sqlite/hooks/sqlite.py      |  4 ++--
 4 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/airflow-core/docs/howto/set-up-database.rst b/airflow-core/docs/howto/set-up-database.rst
index b9c8f1aae01..9036a8454f1 100644
--- a/airflow-core/docs/howto/set-up-database.rst
+++ b/airflow-core/docs/howto/set-up-database.rst
@@ -20,7 +20,7 @@
 Set up a Database Backend
 =========================
 
-Airflow was built to interact with its metadata using `SqlAlchemy <https://docs.sqlalchemy.org/en/14/>`__.
+Airflow was built to interact with its metadata using `SqlAlchemy <https://docs.sqlalchemy.org/en/20/>`__.
 
 The document below describes the database engine configurations, the necessary changes to their configuration to be used with Airflow, as well as changes to the Airflow configurations to connect to these databases.
 
@@ -66,7 +66,7 @@ the example below.
     $ airflow config get-value database sql_alchemy_conn
     sqlite:////tmp/airflow/airflow.db
 
-The exact format description is described in the SQLAlchemy documentation, see `Database Urls <https://docs.sqlalchemy.org/en/14/core/engines.html>`__. We will also show you some examples below.
+The exact format description is described in the SQLAlchemy documentation, see `Database Urls <https://docs.sqlalchemy.org/en/20/core/engines.html>`__. We will also show you some examples below.
 
 Setting up a SQLite Database
 ----------------------------
@@ -198,7 +198,7 @@ in the Postgres documentation to learn more.
    If you cannot change the prefix of your URL immediately, Airflow continues to work with SQLAlchemy
    1.3 and you can downgrade SQLAlchemy, but we recommend to update the prefix.
 
-   Details in the `SQLAlchemy Changelog <https://docs.sqlalchemy.org/en/14/changelog/changelog_14.html#change-3687655465c25a39b968b4f5f6e9170b>`_.
+   Details in the `SQLAlchemy Changelog <https://docs.sqlalchemy.org/en/20/changelog/changelog_14.html#change-3687655465c25a39b968b4f5f6e9170b>`_.
 
 We recommend using the ``psycopg2`` driver and specifying it in your SqlAlchemy connection string.
 
@@ -218,7 +218,7 @@ If you use a current Postgres user with custom search_path, search_path can be c
 
    ALTER USER airflow_user SET search_path = public;
 
-For more information regarding setup of the PostgreSQL connection, see `PostgreSQL dialect <https://docs.sqlalchemy.org/en/14/dialects/postgresql.html>`__ in SQLAlchemy documentation.
+For more information regarding setup of the PostgreSQL connection, see `PostgreSQL dialect <https://docs.sqlalchemy.org/en/20/dialects/postgresql.html>`__ in SQLAlchemy documentation.
 
 .. note::
 
@@ -307,7 +307,7 @@ We recommend using the ``mysqlclient`` driver and specifying it in your SqlAlche
     The integration of MySQL backend has only been validated using the ``mysqlclient`` driver
     during Apache Airflow's continuous integration (CI) process.
 
-If you want to use other drivers visit the `MySQL Dialect <https://docs.sqlalchemy.org/en/14/dialects/mysql.html>`__  in SQLAlchemy documentation for more information regarding download
+If you want to use other drivers visit the `MySQL Dialect <https://docs.sqlalchemy.org/en/20/dialects/mysql.html>`__  in SQLAlchemy documentation for more information regarding download
 and setup of the SqlAlchemy connection.
 
 In addition, you also should pay particular attention to MySQL's encoding. Although the ``utf8mb4`` character set is more and more popular for MySQL (actually, ``utf8mb4`` becomes default character set in MySQL8.0), using the ``utf8mb4`` encoding requires additional setting in Airflow 2+ (See more details in `#7570 <https://github.com/apache/airflow/pull/7570>`__.). If you use ``utf8mb4`` as character set, you should also set ``sql_engine_collation_for_ids=utf8mb3_bin``.
@@ -454,7 +454,7 @@ engine configuration).
 - Suitable for non-production environments like staging systems.
 
 You can do it with ``echo=True`` as sqlalchemy engine configuration as explained in the
-`SQLAlchemy logging documentation <https://docs.sqlalchemy.org/en/14/core/engines.html#configuring-logging>`_.
+`SQLAlchemy logging documentation <https://docs.sqlalchemy.org/en/20/core/engines.html#configuring-logging>`_.
 
 Use :ref:`config:database__sql_alchemy_engine_args` configuration parameter to set echo arg to True.
 
diff --git a/airflow-core/src/airflow/assets/manager.py b/airflow-core/src/airflow/assets/manager.py
index d3399cc6e00..c3491574e91 100644
--- a/airflow-core/src/airflow/assets/manager.py
+++ b/airflow-core/src/airflow/assets/manager.py
@@ -795,7 +795,7 @@ class AssetManager(LoggingMixin):
         def _queue_dagrun_if_needed(dag: DagModel) -> str | None:
             item = AssetDagRunQueue(target_dag_id=dag.dag_id, asset_id=asset_id)
             # Don't error whole transaction when a single RunQueue item conflicts.
-            # https://docs.sqlalchemy.org/en/14/orm/session_transaction.html#using-savepoint
+            # https://docs.sqlalchemy.org/en/20/orm/session_transaction.html#using-savepoint
             try:
                 with session.begin_nested():
                     session.merge(item)
diff --git a/airflow-core/src/airflow/models/base.py b/airflow-core/src/airflow/models/base.py
index 19bdae9d693..1c7af7b275a 100644
--- a/airflow-core/src/airflow/models/base.py
+++ b/airflow-core/src/airflow/models/base.py
@@ -28,7 +28,7 @@ SQL_ALCHEMY_SCHEMA = conf.get("database", "SQL_ALCHEMY_SCHEMA")
 
 # For more information about what the tokens in the naming convention
 # below mean, see:
-# https://docs.sqlalchemy.org/en/14/core/metadata.html#sqlalchemy.schema.MetaData.params.naming_convention
+# https://docs.sqlalchemy.org/en/20/core/metadata.html#sqlalchemy.schema.MetaData.params.naming_convention
 naming_convention = {
     "ix": "idx_%(column_0_N_label)s",
     "uq": "%(table_name)s_%(column_0_N_name)s_uq",
diff --git a/providers/sqlite/src/airflow/providers/sqlite/hooks/sqlite.py b/providers/sqlite/src/airflow/providers/sqlite/hooks/sqlite.py
index 4647a4b086e..99030b2067e 100644
--- a/providers/sqlite/src/airflow/providers/sqlite/hooks/sqlite.py
+++ b/providers/sqlite/src/airflow/providers/sqlite/hooks/sqlite.py
@@ -39,7 +39,7 @@ class SqliteHook(DbApiHook):
         """Return SQLite connection object."""
         sqlalchemy_uri = self.get_uri()
         # The sqlite3 connection does not use the sqlite scheme.
-        # See https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#uri-connections for details.
+        # See https://docs.sqlalchemy.org/en/20/dialects/sqlite.html#uri-connections for details.
         sqlite_uri = sqlalchemy_uri.replace("sqlite:///", "file:")
         conn = sqlite3.connect(sqlite_uri, uri=True)
         return conn
@@ -51,6 +51,6 @@ class SqliteHook(DbApiHook):
         # For sqlite, there is no schema in the connection URI. So we need to drop the trailing slash.
         airflow_sqlite_uri = airflow_uri.replace("/?", "?")
         # The sqlite connection has one more slash for path specification.
-        # See https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#connect-strings for details.
+        # See https://docs.sqlalchemy.org/en/20/dialects/sqlite.html#connect-strings for details.
         sqlalchemy_uri = airflow_sqlite_uri.replace("sqlite://", "sqlite:///")
         return sqlalchemy_uri


(airflow) 04/08: [v3-3-test] Mark providers commands for airflowctl(#68525) (#69232)

Posted by ra...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

vatsrahul1001 pushed a commit to branch v3-3-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit 7ebb7dc0c2b69cbe4edc5233fbc8e28aba17153c
Author: Henry Chen <he...@apache.org>
AuthorDate: Thu Jul 2 16:53:59 2026 +0800

    [v3-3-test] Mark providers commands for airflowctl(#68525) (#69232)
    
    The airflowctl migration direction now keeps existing airflow CLI behavior unchanged and only records commands that should no longer receive new development.
    
    (cherry picked from commit 17aa5415c7c36555b7fc645b9b27c21e2965c6d6)
---
 airflow-core/src/airflow/cli/commands/provider_command.py         | 3 +++
 airflow-core/tests/unit/cli/commands/test_command_deprecations.py | 4 +++-
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/airflow-core/src/airflow/cli/commands/provider_command.py b/airflow-core/src/airflow/cli/commands/provider_command.py
index 645618fd852..fa24b8bb8b8 100644
--- a/airflow-core/src/airflow/cli/commands/provider_command.py
+++ b/airflow-core/src/airflow/cli/commands/provider_command.py
@@ -22,6 +22,7 @@ import re
 import sys
 
 from airflow.cli.simple_table import AirflowConsole
+from airflow.cli.utils import deprecated_for_airflowctl
 from airflow.providers_manager import ProvidersManager
 from airflow.utils.cli import suppress_logs_and_warning
 from airflow.utils.providers_configuration_loader import providers_configuration_loaded
@@ -33,6 +34,7 @@ def _remove_rst_syntax(value: str) -> str:
     return re.sub("[`_<>]", "", value.strip(" \n."))
 
 
+@deprecated_for_airflowctl("airflowctl providers get")
 @suppress_logs_and_warning
 @providers_configuration_loaded
 def provider_get(args):
@@ -55,6 +57,7 @@ def provider_get(args):
         raise SystemExit(f"No such provider installed: {args.provider_name}")
 
 
+@deprecated_for_airflowctl("airflowctl providers list")
 @suppress_logs_and_warning
 @providers_configuration_loaded
 def providers_list(args):
diff --git a/airflow-core/tests/unit/cli/commands/test_command_deprecations.py b/airflow-core/tests/unit/cli/commands/test_command_deprecations.py
index 9300219fe5a..3f082c48b47 100644
--- a/airflow-core/tests/unit/cli/commands/test_command_deprecations.py
+++ b/airflow-core/tests/unit/cli/commands/test_command_deprecations.py
@@ -30,7 +30,7 @@ from __future__ import annotations
 
 import pytest
 
-from airflow.cli.commands import asset_command, dag_command, pool_command
+from airflow.cli.commands import asset_command, dag_command, pool_command, provider_command
 
 # (command callable, expected airflowctl replacement recorded by the decorator)
 MIGRATED_CLI_COMMANDS = [
@@ -43,6 +43,8 @@ MIGRATED_CLI_COMMANDS = [
     (pool_command.pool_import, "airflowctl pools import"),
     (pool_command.pool_export, "airflowctl pools export"),
     (asset_command.asset_materialize, "airflowctl assets materialize"),
+    (provider_command.provider_get, "airflowctl providers get"),
+    (provider_command.providers_list, "airflowctl providers list"),
 ]
 
 


(airflow) 06/08: [v3-3-test] Fix retry policy overrides not persisted to task instance history (#69235) (#69241)

Posted by ra...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

vatsrahul1001 pushed a commit to branch v3-3-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit e3418895b4027fb928974a13dd7a73ff8f2ed976
Author: github-actions[bot] <41...@users.noreply.github.com>
AuthorDate: Fri Jul 3 12:08:31 2026 +0530

    [v3-3-test] Fix retry policy overrides not persisted to task instance history (#69235) (#69241)
    
    AIP-105's per-try audit trail (retry_delay_override, retry_reason) is meant to
    live durably in task_instance_history — the columns on the live task_instance
    row are transient and cleared when the task next enters RUNNING. The Execution
    API retry handler wrote the overrides only into the live-row UPDATE and archived
    the task instance to history before those values were applied, so the history
    columns were always NULL. That silently dropped the audit trail the feature
    advertises, while retry timing (which reads the live row) stayed correct.
    
    Setting the overrides on the task instance before prepare_db_for_next_try() lets
    record_ti() snapshot them into task_instance_history, restoring the per-try
    record.
    (cherry picked from commit 3bfa5ff9a373aae6669f7cc7a6782da71e714186)
    
    Co-authored-by: Rahul Vats <43...@users.noreply.github.com>
---
 .../execution_api/routes/task_instances.py         |  7 ++++
 .../versions/head/test_task_instances.py           | 48 ++++++++++++++++++++++
 2 files changed, 55 insertions(+)

diff --git a/airflow-core/src/airflow/api_fastapi/execution_api/routes/task_instances.py b/airflow-core/src/airflow/api_fastapi/execution_api/routes/task_instances.py
index 0f862e16258..3b142a4e6b1 100644
--- a/airflow-core/src/airflow/api_fastapi/execution_api/routes/task_instances.py
+++ b/airflow-core/src/airflow/api_fastapi/execution_api/routes/task_instances.py
@@ -644,6 +644,13 @@ def _create_ti_state_update_query_and_update_state(
                 _handle_fail_fast_for_dag(ti=ti, dag_id=dag_id, session=session, dag_bag=dag_bag)
         elif isinstance(ti_patch_payload, TIRetryStatePayload):
             if ti is not None:
+                # Set the overrides on the TI *before* archiving so record_ti()
+                # snapshots them into task_instance_history (it copies attrs off
+                # the ti object). Otherwise the per-try audit trail is always NULL.
+                ti.retry_delay_override = ti_patch_payload.retry_delay_seconds
+                ti.retry_reason = (
+                    ti_patch_payload.retry_reason[:500] if ti_patch_payload.retry_reason else None
+                )
                 ti.prepare_db_for_next_try(session=session)
             # Store retry policy overrides so next_retry_datetime() can read them.
             # These are cleared when the task enters RUNNING (ti_run).
diff --git a/airflow-core/tests/unit/api_fastapi/execution_api/versions/head/test_task_instances.py b/airflow-core/tests/unit/api_fastapi/execution_api/versions/head/test_task_instances.py
index 67e0a4ee4ae..4920fc001f9 100644
--- a/airflow-core/tests/unit/api_fastapi/execution_api/versions/head/test_task_instances.py
+++ b/airflow-core/tests/unit/api_fastapi/execution_api/versions/head/test_task_instances.py
@@ -1888,6 +1888,44 @@ class TestTIUpdateState:
         assert ti.retry_delay_override == 42.5
         assert ti.retry_reason == "Rate limit: backing off"
 
+    def test_ti_update_state_retry_policy_overrides_persisted_in_history(
+        self, client, session, create_task_instance
+    ):
+        """Retry policy override + reason must be archived to task_instance_history.
+
+        record_ti() snapshots columns off the TI object, so the overrides must be set on
+        the TI before prepare_db_for_next_try() archives it. When they were written only
+        to the live-row UPDATE, the per-try audit trail in task_instance_history was
+        always NULL even though the live row was correct.
+        """
+        ti = create_task_instance(
+            task_id="test_retry_policy_override_history",
+            state=State.RUNNING,
+        )
+        session.commit()
+
+        response = client.patch(
+            f"/execution/task-instances/{ti.id}/state",
+            json={
+                "state": State.UP_FOR_RETRY,
+                "end_date": DEFAULT_END_DATE.isoformat(),
+                "retry_delay_seconds": 42.5,
+                "retry_reason": "Rate limit: backing off",
+            },
+        )
+
+        assert response.status_code == 204
+
+        tih = session.scalars(
+            select(TaskInstanceHistory).where(
+                TaskInstanceHistory.dag_id == ti.dag_id,
+                TaskInstanceHistory.task_id == ti.task_id,
+                TaskInstanceHistory.run_id == ti.run_id,
+            )
+        ).one()
+        assert tih.retry_delay_override == 42.5
+        assert tih.retry_reason == "Rate limit: backing off"
+
     def test_ti_update_state_retry_without_policy_overrides(self, client, session, create_task_instance):
         """Without retry policy fields, the columns remain NULL."""
         ti = create_task_instance(
@@ -1913,6 +1951,16 @@ class TestTIUpdateState:
         assert ti.retry_delay_override is None
         assert ti.retry_reason is None
 
+        tih = session.scalars(
+            select(TaskInstanceHistory).where(
+                TaskInstanceHistory.dag_id == ti.dag_id,
+                TaskInstanceHistory.task_id == ti.task_id,
+                TaskInstanceHistory.run_id == ti.run_id,
+            )
+        ).one()
+        assert tih.retry_delay_override is None
+        assert tih.retry_reason is None
+
     def test_ti_run_clears_retry_policy_overrides(self, client, session, create_task_instance):
         """When a task enters RUNNING, retry policy overrides from the previous attempt are cleared."""
         ti = create_task_instance(


(airflow) 07/08: [v3-3-test] Fix deadline callback data persistence (#69073) (#69259)

Posted by ra...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

vatsrahul1001 pushed a commit to branch v3-3-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit 7fe134e787d8fc3e115c1b211940422eb5fc67c1
Author: github-actions[bot] <41...@users.noreply.github.com>
AuthorDate: Fri Jul 3 14:04:14 2026 +0530

    [v3-3-test] Fix deadline callback data persistence (#69073) (#69259)
    
    * Fix deadline callback data persistence
    
    * Clarify deadline callback data updates
    (cherry picked from commit 57cff7c409c158e1c1d0458db11e4c0d00f79f4c)
    
    Co-authored-by: Shivam Rastogi <64...@users.noreply.github.com>
---
 airflow-core/src/airflow/models/deadline.py     | 29 ++++++-----
 airflow-core/tests/unit/models/test_deadline.py | 65 ++++++++++++++++++++++++-
 2 files changed, 78 insertions(+), 16 deletions(-)

diff --git a/airflow-core/src/airflow/models/deadline.py b/airflow-core/src/airflow/models/deadline.py
index f596c663014..67438ce428a 100644
--- a/airflow-core/src/airflow/models/deadline.py
+++ b/airflow-core/src/airflow/models/deadline.py
@@ -242,30 +242,29 @@ class Deadline(Base):
 
             return {
                 "dag_run": DAGRunResponse.model_validate(dagrun).model_dump(mode="json"),
-                "deadline": {"id": self.id, "deadline_time": self.deadline_time},
+                "deadline": {"id": str(self.id), "deadline_time": self.deadline_time},
             }
 
+        def callback_data_with_context():
+            data = self.callback.data.copy()
+            kwargs = dict(data.get("kwargs") or {})
+            kwargs["context"] = get_simple_context()
+            data["kwargs"] = kwargs
+            return data
+
         if isinstance(self.callback, TriggererCallback):
-            # Update the callback with context before queuing
-            if "kwargs" not in self.callback.data:
-                self.callback.data["kwargs"] = {}
-            self.callback.data["kwargs"] = (self.callback.data.get("kwargs") or {}) | {
-                "context": get_simple_context()
-            }
+            self.callback.data = callback_data_with_context()
 
             self.callback.queue(session=session)
             session.add(self.callback)
             session.flush()
 
         elif isinstance(self.callback, ExecutorCallback):
-            if "kwargs" not in self.callback.data:
-                self.callback.data["kwargs"] = {}
-            self.callback.data["kwargs"] = (self.callback.data.get("kwargs") or {}) | {
-                "context": get_simple_context()
-            }
-            self.callback.data["deadline_id"] = str(self.id)
-            self.callback.data["dag_run_id"] = str(self.dagrun.id)
-            self.callback.data["dag_id"] = self.dagrun.dag_id
+            data = callback_data_with_context()
+            data["deadline_id"] = str(self.id)
+            data["dag_run_id"] = str(self.dagrun.id)
+            data["dag_id"] = self.dagrun.dag_id
+            self.callback.data = data
 
             self.callback.state = CallbackState.PENDING
             session.add(self.callback)
diff --git a/airflow-core/tests/unit/models/test_deadline.py b/airflow-core/tests/unit/models/test_deadline.py
index 4fc1aada1cc..99f3d9b127b 100644
--- a/airflow-core/tests/unit/models/test_deadline.py
+++ b/airflow-core/tests/unit/models/test_deadline.py
@@ -252,10 +252,73 @@ class TestDeadline:
         context = callback_kwargs.pop("context")
         assert callback_kwargs == TEST_CALLBACK_KWARGS
 
-        assert context["deadline"]["id"] == deadline_orm.id
+        assert context["deadline"]["id"] == str(deadline_orm.id)
         assert context["deadline"]["deadline_time"].timestamp() == deadline_orm.deadline_time.timestamp()
         assert context["dag_run"] == DAGRunResponse.model_validate(dagrun).model_dump(mode="json")
 
+    @pytest.mark.db_test
+    def test_handle_miss_persists_triggerer_callback_context(self, dagrun, session):
+        deadline_orm = Deadline(
+            deadline_time=DEFAULT_DATE,
+            callback=AsyncCallback(TEST_CALLBACK_PATH, TEST_CALLBACK_KWARGS),
+            dagrun_id=dagrun.id,
+            dag_id=dagrun.dag_id,
+            deadline_alert_id=None,
+        )
+        session.add(deadline_orm)
+        session.flush()
+
+        callback_id = deadline_orm.callback.id
+        deadline_id = deadline_orm.id
+        deadline_time = deadline_orm.deadline_time
+        expected_dag_run = DAGRunResponse.model_validate(dagrun).model_dump(mode="json")
+
+        deadline_orm.handle_miss(session)
+        session.commit()
+        session.expunge_all()
+
+        callback = session.scalar(select(Deadline).where(Deadline.id == deadline_id)).callback
+        assert callback.id == callback_id
+
+        callback_kwargs = callback.data["kwargs"]
+        context = callback_kwargs["context"]
+        assert {
+            key: value for key, value in callback_kwargs.items() if key != "context"
+        } == TEST_CALLBACK_KWARGS
+        assert context["deadline"]["id"] == str(deadline_id)
+        assert context["deadline"]["deadline_time"].timestamp() == deadline_time.timestamp()
+        assert context["dag_run"] == expected_dag_run
+
+        callback.trigger = None
+        session.commit()
+
+    @pytest.mark.db_test
+    def test_handle_miss_persists_executor_callback_routing_data(self, dagrun, session):
+        deadline_orm = Deadline(
+            deadline_time=DEFAULT_DATE,
+            callback=SyncCallback(TEST_CALLBACK_PATH, TEST_CALLBACK_KWARGS),
+            dagrun_id=dagrun.id,
+            dag_id=dagrun.dag_id,
+            deadline_alert_id=None,
+        )
+        session.add(deadline_orm)
+        session.flush()
+
+        callback_id = deadline_orm.callback.id
+        deadline_id = deadline_orm.id
+        dagrun_id = dagrun.id
+        dag_id = dagrun.dag_id
+
+        deadline_orm.handle_miss(session)
+        session.commit()
+        session.expunge_all()
+
+        callback = session.scalar(select(Deadline).where(Deadline.id == deadline_id)).callback
+        assert callback.id == callback_id
+        assert callback.data["dag_run_id"] == str(dagrun_id)
+        assert callback.data["dag_id"] == dag_id
+        assert callback.data["deadline_id"] == str(deadline_id)
+
 
 @pytest.mark.db_test
 class TestCalculatedDeadlineDatabaseCalls:


(airflow) 08/08: [v3-3-test] Clarify when last_automated_data_interval is None in timetable docs (#68461) (#69227)

Posted by ra...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

vatsrahul1001 pushed a commit to branch v3-3-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit 767cee3c46631cad3a861ca504ee86ab384ff0b4
Author: github-actions[bot] <41...@users.noreply.github.com>
AuthorDate: Fri Jul 3 14:04:48 2026 +0530

    [v3-3-test] Clarify when last_automated_data_interval is None in timetable docs (#68461) (#69227)
    
    (cherry picked from commit b448e6ec58276f07fff50c170cba429aed492446)
    
    Co-authored-by: Vedant Agarwal <43...@users.noreply.github.com>
---
 airflow-core/docs/howto/timetable.rst       | 9 +++++++++
 airflow-core/src/airflow/timetables/base.py | 3 +++
 2 files changed, 12 insertions(+)

diff --git a/airflow-core/docs/howto/timetable.rst b/airflow-core/docs/howto/timetable.rst
index b5c1aeadb40..90308ca42e5 100644
--- a/airflow-core/docs/howto/timetable.rst
+++ b/airflow-core/docs/howto/timetable.rst
@@ -144,6 +144,15 @@ how the Dag and its tasks specify the schedule, and contains three attributes:
     (the *start* of the data interval), not when the run will be scheduled
     (usually after the end of the data interval).
 
+.. note::
+
+    ``last_automated_data_interval`` is only ever ``None`` when the Dag is
+    first picked up by the Dag processor: the first run is calculated at
+    parsing time, before any Dag run exists, and the result is stored on the
+    Dag. During scheduling, ``next_dagrun_info`` is always called with the
+    previous run's data interval already populated, so the ``None`` case does
+    not appear in scheduler logs when a Dag is unpaused for the first time.
+
 If there was a run scheduled previously, we should now schedule for the next
 non-holiday weekday by looping through subsequent days to find one that is not
 a Saturday, Sunday, or US holiday. If there was not a previous scheduled run,
diff --git a/airflow-core/src/airflow/timetables/base.py b/airflow-core/src/airflow/timetables/base.py
index 84d8aa2e46f..365f980b932 100644
--- a/airflow-core/src/airflow/timetables/base.py
+++ b/airflow-core/src/airflow/timetables/base.py
@@ -444,6 +444,9 @@ class Timetable(Protocol):
 
         :param last_automated_data_interval: The data interval of the associated
             DAG's last scheduled or backfilled run (manual runs not considered).
+            This is only ``None`` when the Dag is being scheduled for the first
+            time, which happens when the Dag processor first parses the Dag --
+            before any Dag run exists.
         :param restriction: Restriction to apply when scheduling the DAG run.
             See documentation of :class:`TimeRestriction` for details.
 


(airflow) 05/08: [v3-3-test] [AIP-94] Mark config CLI commands as migrated to airflowctl (#68958) (#69258)

Posted by ra...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

vatsrahul1001 pushed a commit to branch v3-3-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit e9eda507cc871cee55c1c3fe462bf678e7b06ebf
Author: Henry Chen <he...@apache.org>
AuthorDate: Fri Jul 3 00:26:01 2026 +0800

    [v3-3-test] [AIP-94] Mark config CLI commands as migrated to airflowctl (#68958) (#69258)
    
    Under AIP-94 the remote config CLI commands are frozen so new
    development goes to airflowctl. Following the marker-only direction
    from #68726 , record their airflowctl counterparts for maintainers
    without emitting any user-facing deprecation warning.
    
    (cherry picked from commit 6571dce244edbfc7560474cfc97e9b21e514427b)
    
    Co-authored-by: PoAn Yang <pa...@apache.org>
---
 airflow-core/src/airflow/cli/commands/config_command.py        |  3 +++
 .../tests/unit/cli/commands/test_command_deprecations.py       | 10 +++++++++-
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/airflow-core/src/airflow/cli/commands/config_command.py b/airflow-core/src/airflow/cli/commands/config_command.py
index 21f1dcfc4a9..548a2a21e05 100644
--- a/airflow-core/src/airflow/cli/commands/config_command.py
+++ b/airflow-core/src/airflow/cli/commands/config_command.py
@@ -28,6 +28,7 @@ import pygments
 from pygments.lexers.configs import IniLexer
 
 from airflow.cli.simple_table import AirflowConsole
+from airflow.cli.utils import deprecated_for_airflowctl
 from airflow.configuration import AIRFLOW_CONFIG, ConfigModifications, conf
 from airflow.exceptions import AirflowConfigException
 from airflow.utils.cli import should_use_colors
@@ -35,6 +36,7 @@ from airflow.utils.code_utils import get_terminal_formatter
 from airflow.utils.providers_configuration_loader import providers_configuration_loaded
 
 
+@deprecated_for_airflowctl("airflowctl config list")
 @providers_configuration_loaded
 def show_config(args):
     """Show current application configuration."""
@@ -63,6 +65,7 @@ def show_config(args):
     print(code)
 
 
+@deprecated_for_airflowctl("airflowctl config get")
 @providers_configuration_loaded
 def get_value(args):
     """Get one value from configuration."""
diff --git a/airflow-core/tests/unit/cli/commands/test_command_deprecations.py b/airflow-core/tests/unit/cli/commands/test_command_deprecations.py
index 3f082c48b47..bf9c181779c 100644
--- a/airflow-core/tests/unit/cli/commands/test_command_deprecations.py
+++ b/airflow-core/tests/unit/cli/commands/test_command_deprecations.py
@@ -30,7 +30,13 @@ from __future__ import annotations
 
 import pytest
 
-from airflow.cli.commands import asset_command, dag_command, pool_command, provider_command
+from airflow.cli.commands import (
+    asset_command,
+    config_command,
+    dag_command,
+    pool_command,
+    provider_command,
+)
 
 # (command callable, expected airflowctl replacement recorded by the decorator)
 MIGRATED_CLI_COMMANDS = [
@@ -45,6 +51,8 @@ MIGRATED_CLI_COMMANDS = [
     (asset_command.asset_materialize, "airflowctl assets materialize"),
     (provider_command.provider_get, "airflowctl providers get"),
     (provider_command.providers_list, "airflowctl providers list"),
+    (config_command.get_value, "airflowctl config get"),
+    (config_command.show_config, "airflowctl config list"),
 ]
 
 


(airflow) 03/08: [v3-3-test] Add new Committers to CODEOWNERS (#69215) (#69219)

Posted by ra...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

vatsrahul1001 pushed a commit to branch v3-3-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit 84590c18331365f97a7d8364fe83a4aebd153aae
Author: github-actions[bot] <41...@users.noreply.github.com>
AuthorDate: Wed Jul 1 21:34:35 2026 +0200

    [v3-3-test] Add new Committers to CODEOWNERS (#69215) (#69219)
    
    * Add Miretpl to Chart owners
    
    * Add henry3260 to CODEOWNERS
    (cherry picked from commit adb541ca6637b1dd8b7b9b175af916967e01da39)
    
    Co-authored-by: Przemysław Mirowski <17...@users.noreply.github.com>
---
 .github/CODEOWNERS | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
index 6b0836ce2b9..19a4483ab59 100644
--- a/.github/CODEOWNERS
+++ b/.github/CODEOWNERS
@@ -10,21 +10,21 @@
 /airflow-core/src/airflow/dag_processing @jedcunningham @ephraimbuddy
 
 # Helm Chart
-/chart/ @jedcunningham @hussein-awala @jscheffl @bugraoz93
+/chart/ @jedcunningham @hussein-awala @jscheffl @bugraoz93 @Miretpl
 
 # Docs
 /docs/*.py @potiuk @ashb @gopidesupavan @amoghrajesh @jscheffl @bugraoz93 @jason810496
 
 # API
 /airflow-core/src/airflow/api/ @ephraimbuddy @pierrejeambrun @rawwar @jason810496
-/airflow-core/src/airflow/api_fastapi/ @ephraimbuddy @pierrejeambrun @rawwar @jason810496 @bugraoz93 @shubhamraj-git @choo121600
+/airflow-core/src/airflow/api_fastapi/ @ephraimbuddy @pierrejeambrun @rawwar @jason810496 @bugraoz93 @shubhamraj-git @choo121600 @henry3260
 /airflow-core/src/airflow/api_fastapi/execution_api/ @ashb @kaxil @amoghrajesh
 
 # Airflow CTL
-/airflow-ctl/ @bugraoz93 @potiuk @dheerajturaga
+/airflow-ctl/ @bugraoz93 @potiuk @dheerajturaga @henry3260
 
 # Airflow CLI
-/airflow-core/src/airflow/cli/ @bugraoz93 @potiuk @dheerajturaga
+/airflow-core/src/airflow/cli/ @bugraoz93 @potiuk @dheerajturaga @henry3260
 
 # Auth manager
 /airflow-core/src/airflow/api_fastapi/auth/ @vincbeck
@@ -57,7 +57,7 @@ airflow-core/src/airflow/ui/public/i18n/locales/ru/ @jscheffl # + @puzzle-rpa-te
 airflow-core/src/airflow/ui/public/i18n/locales/th/ @potiuk # + @zkan @blackbass64 @lifnaja @Aphinan-Th @chonla @Srabasti
 airflow-core/src/airflow/ui/public/i18n/locales/tr/ @bugraoz93 # +@hasancatalgol
 airflow-core/src/airflow/ui/public/i18n/locales/zh-CN/ @potiuk # + @Fortytwoo @gyli
-airflow-core/src/airflow/ui/public/i18n/locales/zh-TW/ @Lee-W @jason810496 @guan404ming # + @RoyLee1224
+airflow-core/src/airflow/ui/public/i18n/locales/zh-TW/ @Lee-W @jason810496 @guan404ming @henry3260 # + @RoyLee1224
 
 # Security/Permissions
 /airflow-core/src/airflow/security/permissions.py @vincbeck
@@ -138,7 +138,7 @@ Dockerfile.ci @potiuk @ashb @gopidesupavan @amoghrajesh @jscheffl @bugraoz93 @ja
 /task-sdk-integration-tests/ @potiuk @ashb @gopidesupavan @amoghrajesh @bugraoz93 @kaxil @jason810496
 
 # airflowctl integration tests
-/airflow-ctl-tests/ @potiuk @ashb @gopidesupavan @amoghrajesh @bugraoz93 @jason810496
+/airflow-ctl-tests/ @potiuk @ashb @gopidesupavan @amoghrajesh @bugraoz93 @jason810496 @henry3260
 
 # Issue triage process
 # ISSUE_TRIAGE_PROCESS.rst