You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2026/10/14 09:17:16 UTC
(airflow) branch main updated: Graduate ClassifierRetryPolicy to stable in Common AI (#74707)
This is an automated email from the ASF dual-hosted git repository.
kaxil pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new a1c3755a23d Graduate ClassifierRetryPolicy to stable in Common AI (#74707)
a1c3755a23d is described below
commit a1c3755a23d6f30f98795486929543b1d0a347be
Author: Kaxil Naik <ka...@apache.org>
AuthorDate: Wed Oct 14 10:17:07 2026 +0100
Graduate ClassifierRetryPolicy to stable in Common AI (#74707)
Its fallback order (fallback_policy, then fallback_rules, then the task's
own retry settings) shipped and has not changed since, so the stated reason
for keeping it experimental no longer holds. Only the typesafe extra stays
experimental, because typesafe-sdk is pre-1.0. Also drop the stale
experimental labels the Why Common AI page still carried for features that
are already stable.
---
providers/common/ai/docs/decision_models.rst | 5 +++--
providers/common/ai/docs/retry_policies.rst | 5 -----
providers/common/ai/docs/stability.rst | 16 +++++++++++---
providers/common/ai/docs/why_common_ai.rst | 25 ++++++++++------------
.../airflow/providers/common/ai/policies/retry.py | 10 ---------
5 files changed, 27 insertions(+), 34 deletions(-)
diff --git a/providers/common/ai/docs/decision_models.rst b/providers/common/ai/docs/decision_models.rst
index 337b306d1a6..8320806ffb5 100644
--- a/providers/common/ai/docs/decision_models.rst
+++ b/providers/common/ai/docs/decision_models.rst
@@ -20,8 +20,9 @@ Decision models
.. note::
- Experimental: this can change or be removed in a minor release of this provider.
- See :ref:`howto/stability`.
+ Experimental: the ``typesafe`` extra can change or be removed in a minor release of this
+ provider, because the ``typesafe-sdk`` it installs is before its 1.0 release. See
+ :ref:`howto/stability`.
Every other model in this provider writes text. A decision model does not: you give it
some text and a typed question, and it answers with a value from a set you named in
diff --git a/providers/common/ai/docs/retry_policies.rst b/providers/common/ai/docs/retry_policies.rst
index 686eb5c7e0f..5b19a1f2966 100644
--- a/providers/common/ai/docs/retry_policies.rst
+++ b/providers/common/ai/docs/retry_policies.rst
@@ -162,11 +162,6 @@ an attempt is a separate mechanism on the connection; see
ClassifierRetryPolicy
---------------------
-.. note::
-
- Experimental: this can change or be removed in a minor release of this provider.
- See :ref:`howto/stability`.
-
``ClassifierRetryPolicy`` takes the retry decision away from the model. Its
``categories`` maps a category name to an
:class:`~airflow.providers.common.ai.policies.retry.ErrorCategory`: what
diff --git a/providers/common/ai/docs/stability.rst b/providers/common/ai/docs/stability.rst
index abf65413ff2..70ce591b8cb 100644
--- a/providers/common/ai/docs/stability.rst
+++ b/providers/common/ai/docs/stability.rst
@@ -173,6 +173,17 @@ Pydantic AI toolsets, but the Pydantic AI class they inherit from can change.
``redact_exception=False`` or a custom ``redactor`` replaces the masking. If the
model call fails, it falls back to ``fallback_rules`` and then to the task's own
retry settings; it never fails the task itself.
+ * - :class:`~airflow.providers.common.ai.policies.retry.ClassifierRetryPolicy` and
+ :class:`~airflow.providers.common.ai.policies.retry.ErrorCategory`
+ - The model can only answer with one of the names in ``categories``, and that name's
+ ``ErrorCategory`` decides whether the task retries and after what ``delay``.
+ Without ``categories`` the policy uses
+ :data:`~airflow.providers.common.ai.policies.retry.DEFAULT_CATEGORIES`, whose seven
+ names and their retry or fail split stay the same. When the model call fails, or
+ ``min_confidence`` is set and the answer is under it or carries no confidence, the
+ policy uses the RETRY or FAIL of ``fallback_policy``, then ``fallback_rules``, then
+ the task's own retry settings. Secrets are masked in the exception text as for
+ ``LLMRetryPolicy``.
* - Output review: ``require_approval``, ``enable_hitl_review`` and the review plugin
- With ``require_approval=True``, the task defers after generating output and returns
it only once a person approves it on the Required Actions page. A rejection, or a
@@ -191,9 +202,8 @@ Everything this provider ships that is not in the table above is experimental.
* - Feature
- Why it is experimental
- * - :class:`~airflow.providers.common.ai.policies.retry.ClassifierRetryPolicy`
- (:doc:`decision_models`)
- - What it does when the classifier is unsure or unavailable is changing.
+ * - The ``typesafe`` extra (:doc:`decision_models`)
+ - It installs ``typesafe-sdk``, which is before its 1.0 release.
* - :class:`~airflow.providers.common.ai.policies.decision.DecisionPolicy`, and
``min_confidence`` on
:class:`~airflow.providers.common.ai.policies.decision.BranchOption`
diff --git a/providers/common/ai/docs/why_common_ai.rst b/providers/common/ai/docs/why_common_ai.rst
index 992b54945fd..cfd3e7fa9f8 100644
--- a/providers/common/ai/docs/why_common_ai.rst
+++ b/providers/common/ai/docs/why_common_ai.rst
@@ -85,10 +85,9 @@ and a bucket of report files. What differs:
- **The tools.** The hand-written ``query`` runs any SQL the model writes, including
``DELETE``, on any table the connection can reach, and returns every row it gets.
``SQLToolset`` allows only the ``orders`` table, rejects statements that write, and caps
- the rows and bytes it returns. ``ObjectStorageToolset`` (experimental) keeps reads under
- the prefix you gave it. Both run Airflow's secrets masker over results, which hides
- values Airflow knows are secret, such as connection passwords, not personal data in your
- tables.
+ the rows and bytes it returns. ``ObjectStorageToolset`` keeps reads under the prefix you
+ gave it. Both run Airflow's secrets masker over results, which hides values Airflow knows
+ are secret, such as connection passwords, not personal data in your tables.
- **The model.** The plain task names a vendor in code and reads its key from the
environment. ``AgentOperator`` takes both from a connection, so moving to another vendor
or adding a fallback is a connection change. See :doc:`model_providers`.
@@ -137,7 +136,7 @@ What a plain task leaves to you
* - A person has to approve a tool call before it runs, such as a refund or a write
to a production table.
- A tool marked with Pydantic AI's approval API pauses ``AgentOperator`` before the
- call, without holding a worker slot. Airflow 3.3 and later. Experimental. Not with
+ call, without holding a worker slot. Airflow 3.3 and later. Not with
``durable=True`` or ``enable_hitl_review=True``; with either, a tool marked for
approval fails the task.
- :doc:`tool_approval`
@@ -150,10 +149,10 @@ What a plain task leaves to you
- :doc:`hitl_review`
* - The agent needs your databases, object storage or an API you already have a
hook for.
- - ``SQLToolset``, ``HookToolset`` and ``ObjectStorageToolset`` (experimental) build
- tools from an Airflow connection, with table allow-lists, read-only SQL and size
- limits. Results pass through Airflow's secrets masker before the model sees
- them. They work in an agent you run yourself too.
+ - ``SQLToolset``, ``HookToolset`` and ``ObjectStorageToolset`` build tools from an
+ Airflow connection, with table allow-lists, read-only SQL and size limits. Results
+ pass through Airflow's secrets masker before the model sees them. They work in an
+ agent you run yourself too.
- :doc:`toolsets/index`
* - A model vendor is down.
- ``fallback_conn_ids`` fails over to other connections in order.
@@ -162,11 +161,10 @@ What a plain task leaves to you
- The batch operator sends them through the OpenAI or Anthropic batch API at
roughly half the per-token cost. A retry re-attaches to the batch already
submitted if the prompts, model, output schema and connection are unchanged.
- With ``deferrable=True`` the wait does not hold a worker slot. Experimental.
+ With ``deferrable=True`` the wait does not hold a worker slot.
- :doc:`operators/llm_batch`
The provider runs on Airflow 2.11 and later; a row that needs a newer Airflow says so.
-Experimental features can change in a minor release; see :doc:`stability`.
Decide whether a failed task retries
------------------------------------
@@ -187,9 +185,8 @@ When the task fails, the model reads the exception and decides whether to retry
long to wait, so it can decide to wait out a rate limit and to fail a revoked credential
at once. The exception text goes through the secrets masker before the model sees it,
and ``fallback_rules`` decide when the model cannot be reached. ``ClassifierRetryPolicy``
-(experimental) has a cheaper model pick from categories you define, and can hand the
-cases it is unsure of to an ``LLMRetryPolicy``. Both need Airflow 3.3 or later. See
-:doc:`retry_policies`.
+has a cheaper model pick from categories you define, and can hand the cases it is unsure
+of to an ``LLMRetryPolicy``. Both need Airflow 3.3 or later. See :doc:`retry_policies`.
When a plain task is enough
---------------------------
diff --git a/providers/common/ai/src/airflow/providers/common/ai/policies/retry.py b/providers/common/ai/src/airflow/providers/common/ai/policies/retry.py
index cccdfbe4593..bb72791d0e6 100644
--- a/providers/common/ai/src/airflow/providers/common/ai/policies/retry.py
+++ b/providers/common/ai/src/airflow/providers/common/ai/policies/retry.py
@@ -129,11 +129,6 @@ class ErrorCategory:
"""
One kind of failure a :class:`ClassifierRetryPolicy` may name, and what it does when it does.
- .. note::
-
- Experimental: this can change or be removed in a minor release of this provider.
- See :ref:`howto/stability`.
-
The value of the policy's ``categories`` mapping, keyed by the category name the model
answers with.
@@ -459,11 +454,6 @@ class ClassifierRetryPolicy(_ModelRetryPolicy):
"""
Retry policy where the model names the kind of failure and the author's table decides.
- .. note::
-
- Experimental: this can change or be removed in a minor release of this provider.
- See :ref:`howto/stability`.
-
The model's only job is to pick one of ``categories``; it reads each one's description
from the output schema. Whether that category is retried, after how long, and how sure
the model has to be all come from the :class:`ErrorCategory` in the worker process.