(airflow) branch backport-2cf2a1af91-v3-3-test created (now 426140317dd)

[ra...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

vatsrahul1001 pushed a change to branch backport-2cf2a1af91-v3-3-test
in repository https://gitbox.apache.org/repos/asf/airflow.git


      at 426140317dd Explicitly state 'durable execution' in airflow docs (#71863)

This branch includes the following new commits:

     new 426140317dd Explicitly state 'durable execution' in airflow docs (#71863)

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.

(airflow) 01/01: Explicitly state 'durable execution' in airflow docs (#71863)

[ra...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

vatsrahul1001 pushed a commit to branch backport-2cf2a1af91-v3-3-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit 426140317ddcda0043064cae4a423dca6ff43dc8
Author: Amogh Desai <am...@gmail.com>
AuthorDate: Wed Aug 26 10:23:33 2026 +0530

    Explicitly state 'durable execution' in airflow docs (#71863)
    
    (cherry picked from commit 2cf2a1af91cd86c19ed724f87af014dbd60fdb05)
---
 .../docs/core-concepts/resumable-tasks.rst         | 23 ++++++++++++++++++++--
 .../docs/core-concepts/task-state-store.rst        |  3 +++
 2 files changed, 24 insertions(+), 2 deletions(-)

diff --git a/airflow-core/docs/core-concepts/resumable-tasks.rst b/airflow-core/docs/core-concepts/resumable-tasks.rst
index ecb2bcaf4c0..45c993bda42 100644
--- a/airflow-core/docs/core-concepts/resumable-tasks.rst
+++ b/airflow-core/docs/core-concepts/resumable-tasks.rst
@@ -32,8 +32,27 @@ entire polling duration, and if the worker process is restarted or the host is p
 retries from scratch, losing all the progress made. Depending on the operator, that means the external
 job is submitted again, creating a duplicate run in context of the external system.
 
-Airflow recommends three approaches for handling long-running external work. Understanding the trade-offs
-between them helps you choose the right one for your situation.
+.. _concepts-durable-execution:
+
+Durable execution
+-----------------
+
+Surviving that failure mode is what **durable execution** means: a task outlives the loss of the
+process running it and can continue / re-attach from where it stopped, rather than repeating work
+already done or submitting a duplicate job to an external system.
+
+The :doc:`task state store <task-state-store>` is the mechanism Airflow provides to achieve it. It
+is the only per-task-instance storage that outlives a worker crash and is still readable by the
+next attempt, which is what lets a retry recover a checkpoint or an external job identifier written
+by the attempt before it. Durable execution is the outcome; the task state store is how you get it.
+
+Operator documentation across the provider ecosystem uses the same term for the operator-level
+feature built on this mechanism, normally exposed as a ``durable`` parameter. Spark, Kubernetes,
+Databricks, Snowflake, BigQuery, Redshift and Glue operators each have a "Durable execution"
+section describing what they persist and how they reconnect on retry.
+
+Airflow recommends three approaches for achieving this with long-running external work.
+Understanding the trade-offs between them helps you choose the right one for your situation.
 
 .. _concepts-resumable-tasks-deferrable:
 
diff --git a/airflow-core/docs/core-concepts/task-state-store.rst b/airflow-core/docs/core-concepts/task-state-store.rst
index 7702fb90f68..6d64ac3fb36 100644
--- a/airflow-core/docs/core-concepts/task-state-store.rst
+++ b/airflow-core/docs/core-concepts/task-state-store.rst
@@ -30,6 +30,9 @@ Task State Store
 
 Task store is a persistent key/value store scoped to a single task instance (``dag_id`` + ``run_id`` + ``task_id`` + ``map_index``). It survives worker crashes and task retries within the same Dag run, making it suitable for storing external job IDs, intra-task checkpoints, and progress metadata.
 
+Because it outlives a worker crash and stays readable by the next attempt, the task state store is the mechanism behind :ref:`durable execution <concepts-durable-execution>`, where a task continues
+from where it stopped instead of repeating work or submitting a duplicate external job. Provider operators that advertise a ``durable`` parameter are built on the API described here.
+
 Data persisted via task state store is accessed through the task context via ``context["task_state_store"]`` and exposes four methods: ``get``, ``set``, ``delete``, and ``clear``.