You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2026/10/14 09:01:44 UTC
(airflow) branch commonai-classifier-stable created (now 81e4beaca77)
This is an automated email from the ASF dual-hosted git repository.
kaxil pushed a change to branch commonai-classifier-stable
in repository https://gitbox.apache.org/repos/asf/airflow.git
at 81e4beaca77 Graduate ClassifierRetryPolicy to stable in Common AI
This branch includes the following new commits:
new 81e4beaca77 Graduate ClassifierRetryPolicy to stable in Common AI
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(airflow) 01/01: Graduate ClassifierRetryPolicy to stable in Common AI
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxil pushed a commit to branch commonai-classifier-stable
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 81e4beaca77db4e8e5412c392b55cb927dc4d1ec
Author: Kaxil Naik <ka...@apache.org>
AuthorDate: Wed Oct 14 09:59:50 2026 +0100
Graduate ClassifierRetryPolicy to stable in Common AI
Its fallback order (fallback_policy, then fallback_rules, then the task's
own retry settings) shipped and has not changed since, so the stated reason
for keeping it experimental no longer holds. Only the typesafe extra stays
experimental, because typesafe-sdk is pre-1.0. Also drop the stale
experimental labels the Why Common AI page still carried for features that
are already stable.
---
providers/common/ai/docs/decision_models.rst | 5 +++--
providers/common/ai/docs/retry_policies.rst | 5 -----
providers/common/ai/docs/stability.rst | 16 +++++++++++---
providers/common/ai/docs/why_common_ai.rst | 25 ++++++++++------------
.../airflow/providers/common/ai/policies/retry.py | 10 ---------
5 files changed, 27 insertions(+), 34 deletions(-)
diff --git a/providers/common/ai/docs/decision_models.rst b/providers/common/ai/docs/decision_models.rst
index 337b306d1a6..8320806ffb5 100644
--- a/providers/common/ai/docs/decision_models.rst
+++ b/providers/common/ai/docs/decision_models.rst
@@ -20,8 +20,9 @@ Decision models
.. note::
- Experimental: this can change or be removed in a minor release of this provider.
- See :ref:`howto/stability`.
+ Experimental: the ``typesafe`` extra can change or be removed in a minor release of this
+ provider, because the ``typesafe-sdk`` it installs is before its 1.0 release. See
+ :ref:`howto/stability`.
Every other model in this provider writes text. A decision model does not: you give it
some text and a typed question, and it answers with a value from a set you named in
diff --git a/providers/common/ai/docs/retry_policies.rst b/providers/common/ai/docs/retry_policies.rst
index 686eb5c7e0f..5b19a1f2966 100644
--- a/providers/common/ai/docs/retry_policies.rst
+++ b/providers/common/ai/docs/retry_policies.rst
@@ -162,11 +162,6 @@ an attempt is a separate mechanism on the connection; see
ClassifierRetryPolicy
---------------------
-.. note::
-
- Experimental: this can change or be removed in a minor release of this provider.
- See :ref:`howto/stability`.
-
``ClassifierRetryPolicy`` takes the retry decision away from the model. Its
``categories`` maps a category name to an
:class:`~airflow.providers.common.ai.policies.retry.ErrorCategory`: what
diff --git a/providers/common/ai/docs/stability.rst b/providers/common/ai/docs/stability.rst
index abf65413ff2..70ce591b8cb 100644
--- a/providers/common/ai/docs/stability.rst
+++ b/providers/common/ai/docs/stability.rst
@@ -173,6 +173,17 @@ Pydantic AI toolsets, but the Pydantic AI class they inherit from can change.
``redact_exception=False`` or a custom ``redactor`` replaces the masking. If the
model call fails, it falls back to ``fallback_rules`` and then to the task's own
retry settings; it never fails the task itself.
+ * - :class:`~airflow.providers.common.ai.policies.retry.ClassifierRetryPolicy` and
+ :class:`~airflow.providers.common.ai.policies.retry.ErrorCategory`
+ - The model can only answer with one of the names in ``categories``, and that name's
+ ``ErrorCategory`` decides whether the task retries and after what ``delay``.
+ Without ``categories`` the policy uses
+ :data:`~airflow.providers.common.ai.policies.retry.DEFAULT_CATEGORIES`, whose seven
+ names and their retry or fail split stay the same. When the model call fails, or
+ ``min_confidence`` is set and the answer is under it or carries no confidence, the
+ policy uses the RETRY or FAIL of ``fallback_policy``, then ``fallback_rules``, then
+ the task's own retry settings. Secrets are masked in the exception text as for
+ ``LLMRetryPolicy``.
* - Output review: ``require_approval``, ``enable_hitl_review`` and the review plugin
- With ``require_approval=True``, the task defers after generating output and returns
it only once a person approves it on the Required Actions page. A rejection, or a
@@ -191,9 +202,8 @@ Everything this provider ships that is not in the table above is experimental.
* - Feature
- Why it is experimental
- * - :class:`~airflow.providers.common.ai.policies.retry.ClassifierRetryPolicy`
- (:doc:`decision_models`)
- - What it does when the classifier is unsure or unavailable is changing.
+ * - The ``typesafe`` extra (:doc:`decision_models`)
+ - It installs ``typesafe-sdk``, which is before its 1.0 release.
* - :class:`~airflow.providers.common.ai.policies.decision.DecisionPolicy`, and
``min_confidence`` on
:class:`~airflow.providers.common.ai.policies.decision.BranchOption`
diff --git a/providers/common/ai/docs/why_common_ai.rst b/providers/common/ai/docs/why_common_ai.rst
index 992b54945fd..cfd3e7fa9f8 100644
--- a/providers/common/ai/docs/why_common_ai.rst
+++ b/providers/common/ai/docs/why_common_ai.rst
@@ -85,10 +85,9 @@ and a bucket of report files. What differs:
- **The tools.** The hand-written ``query`` runs any SQL the model writes, including
``DELETE``, on any table the connection can reach, and returns every row it gets.
``SQLToolset`` allows only the ``orders`` table, rejects statements that write, and caps
- the rows and bytes it returns. ``ObjectStorageToolset`` (experimental) keeps reads under
- the prefix you gave it. Both run Airflow's secrets masker over results, which hides
- values Airflow knows are secret, such as connection passwords, not personal data in your
- tables.
+ the rows and bytes it returns. ``ObjectStorageToolset`` keeps reads under the prefix you
+ gave it. Both run Airflow's secrets masker over results, which hides values Airflow knows
+ are secret, such as connection passwords, not personal data in your tables.
- **The model.** The plain task names a vendor in code and reads its key from the
environment. ``AgentOperator`` takes both from a connection, so moving to another vendor
or adding a fallback is a connection change. See :doc:`model_providers`.
@@ -137,7 +136,7 @@ What a plain task leaves to you
* - A person has to approve a tool call before it runs, such as a refund or a write
to a production table.
- A tool marked with Pydantic AI's approval API pauses ``AgentOperator`` before the
- call, without holding a worker slot. Airflow 3.3 and later. Experimental. Not with
+ call, without holding a worker slot. Airflow 3.3 and later. Not with
``durable=True`` or ``enable_hitl_review=True``; with either, a tool marked for
approval fails the task.
- :doc:`tool_approval`
@@ -150,10 +149,10 @@ What a plain task leaves to you
- :doc:`hitl_review`
* - The agent needs your databases, object storage or an API you already have a
hook for.
- - ``SQLToolset``, ``HookToolset`` and ``ObjectStorageToolset`` (experimental) build
- tools from an Airflow connection, with table allow-lists, read-only SQL and size
- limits. Results pass through Airflow's secrets masker before the model sees
- them. They work in an agent you run yourself too.
+ - ``SQLToolset``, ``HookToolset`` and ``ObjectStorageToolset`` build tools from an
+ Airflow connection, with table allow-lists, read-only SQL and size limits. Results
+ pass through Airflow's secrets masker before the model sees them. They work in an
+ agent you run yourself too.
- :doc:`toolsets/index`
* - A model vendor is down.
- ``fallback_conn_ids`` fails over to other connections in order.
@@ -162,11 +161,10 @@ What a plain task leaves to you
- The batch operator sends them through the OpenAI or Anthropic batch API at
roughly half the per-token cost. A retry re-attaches to the batch already
submitted if the prompts, model, output schema and connection are unchanged.
- With ``deferrable=True`` the wait does not hold a worker slot. Experimental.
+ With ``deferrable=True`` the wait does not hold a worker slot.
- :doc:`operators/llm_batch`
The provider runs on Airflow 2.11 and later; a row that needs a newer Airflow says so.
-Experimental features can change in a minor release; see :doc:`stability`.
Decide whether a failed task retries
------------------------------------
@@ -187,9 +185,8 @@ When the task fails, the model reads the exception and decides whether to retry
long to wait, so it can decide to wait out a rate limit and to fail a revoked credential
at once. The exception text goes through the secrets masker before the model sees it,
and ``fallback_rules`` decide when the model cannot be reached. ``ClassifierRetryPolicy``
-(experimental) has a cheaper model pick from categories you define, and can hand the
-cases it is unsure of to an ``LLMRetryPolicy``. Both need Airflow 3.3 or later. See
-:doc:`retry_policies`.
+has a cheaper model pick from categories you define, and can hand the cases it is unsure
+of to an ``LLMRetryPolicy``. Both need Airflow 3.3 or later. See :doc:`retry_policies`.
When a plain task is enough
---------------------------
diff --git a/providers/common/ai/src/airflow/providers/common/ai/policies/retry.py b/providers/common/ai/src/airflow/providers/common/ai/policies/retry.py
index cccdfbe4593..bb72791d0e6 100644
--- a/providers/common/ai/src/airflow/providers/common/ai/policies/retry.py
+++ b/providers/common/ai/src/airflow/providers/common/ai/policies/retry.py
@@ -129,11 +129,6 @@ class ErrorCategory:
"""
One kind of failure a :class:`ClassifierRetryPolicy` may name, and what it does when it does.
- .. note::
-
- Experimental: this can change or be removed in a minor release of this provider.
- See :ref:`howto/stability`.
-
The value of the policy's ``categories`` mapping, keyed by the category name the model
answers with.
@@ -459,11 +454,6 @@ class ClassifierRetryPolicy(_ModelRetryPolicy):
"""
Retry policy where the model names the kind of failure and the author's table decides.
- .. note::
-
- Experimental: this can change or be removed in a minor release of this provider.
- See :ref:`howto/stability`.
-
The model's only job is to pick one of ``categories``; it reads each one's description
from the output schema. Whether that category is retried, after how long, and how sure
the model has to be all come from the :class:`ErrorCategory` in the worker process.