You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "kaxil (via GitHub)" <gi...@apache.org> on 2026/09/18 22:55:29 UTC
[PR] Add support for TypeSafe Jev classifier models [airflow]
kaxil opened a new pull request, #73363:
URL: https://github.com/apache/airflow/pull/73363
A classifier model answers typed questions and refuses anything else. You give it text and a question whose answers you named in advance, and it returns one of them plus a confidence; ask it for a string and the request is rejected before it leaves your process. pydantic-ai 2.45.0 added `TypeSafeModel` for TypeSafe's Jev in pydantic/pydantic-ai#8450, which is the first model of that shape it supports.
It needs no provider code. `TypeSafeProvider.__init__` accepts exactly the `api_key` and `base_url` that `PydanticAIHook._get_provider_kwargs` already passes, and `infer_model("typesafe:jev-1.13.0")` resolves through the same path as every other model, so setting `model_id` or the connection's Model field is the whole integration. The diff is one `[project.optional-dependencies]` line, a guide, an example Dag, and the dependency plumbing that lets the extra resolve. Nothing under `src/` changes except the new example, which is why there are no new tests.
Over half the diff is the guide. "It works" is not much use to a Dag author facing a model that cannot write text, so the guide covers when a classifier model is the right choice, where it fits among the operators already here, and where it does not.
```python
LLMBranchOperator(
task_id="route_failure",
prompt=failure_text,
llm_conn_id="jev_default",
model_id="typesafe:jev-1.13.0",
system_prompt="Pick the remediation that addresses the cause, not the symptom.",
)
```
## Design rationale
**Opt-in extra rather than a base dependency.** The extra pulls `typesafe-sdk`, which is pre-1.0. Keeping it opt-in means its churn cannot break a plain provider install, which is the same reasoning already applied to `code-mode` and `shields`.
**The three temporary `exclude-newer` pins are the part worth a second look.** `pydantic-ai-slim` 2.45.0 was published inside the repo's rolling four-day `exclude-newer` window, so without them the `typesafe` extra cannot resolve at all. `pydantic-graph` is pinned in lockstep because `pydantic-ai-slim` requires the exact matching version, and `typesafe-sdk` because the extra's `>=0.6.0` floor is only satisfied by releases inside the same window. They follow the `pydantic-ai-skills` and `google-cloud-aiplatform` entries already in both blocks, including the "Remove once the rolling window advances past …" comment, and they become deletable on their own once the window passes 2026-09-18. Verified that the repo's own override is what does the work: `uv pip install --dry-run 'pydantic-ai-slim[typesafe]>=2.45.0'` from the workspace root resolves with no flags, adding `typesafe-sdk==0.7.0` as the only new package. `uv.lock` and `generated/provider_dependencies.json` are untouched beca
use neither tracks provider optional extras.
**The guide documents four limits rather than claiming a clean fit**, each verified against the live API rather than inferred:
- A bounded `float` field carries **no** confidence, because there the probability is the answer. This is the trap most likely to bite, since the natural gating pattern (`confidence or 0.0`) silently reads as zero confidence on a model that was certain.
- Which tool a call needs is itself a pick, so a classifier model can make it, but it cannot write a tool's arguments. A zero-argument tool it calls itself; an argument-taking one raises `ToolCallProposed`, a `ModelAPIError` rather than a refusal, which `FallbackModel` hands to a text model behind it.
- `LLMBranchOperator` needs two or more downstream tasks. The operator permits one, and a one-option pick is refused.
- `LLMRetryPolicy` does not fit: three of `ErrorClassification`'s four fields are shapes Jev refuses.
**Placed with the model-provider extras, not on its own.** The group's stated membership rule is that each extra name mirrors the identically named `pydantic-ai-slim` group, which `typesafe` does, and it is selected the same way as the others, through `extra["model"]` on a generic `pydanticai` connection. The bullet says it differs in kind and links the guide, which seemed better than inventing a fifth extras category for one member.
## Gotchas
**Confidence is not reachable from the operators, and that limits what branching on this model can do.** It lives in `provider_details` on the model response, which no operator exposes on XCom by default. `AgentOperator` carries it inside the `message_history` transcript when that is enabled, and otherwise it is dropped. A branch is decided inside `LLMBranchOperator` before any downstream task can look at it, so when two options both fit the evidence the branch turns on a margin nothing surfaces. The guide says this plainly and shows the hook-direct pattern for gating, and the example Dag uses a separable branch for the operator and keeps the confidence gate in a task.
`jev-latest` moves when TypeSafe ship a release, so the guide and the example pin `typesafe:jev-1.13.0`. A confidence threshold tuned against one version is not guaranteed to mean the same thing after the next.
## Follow-ups
Upstream's own confidence gate is a response handler on `FallbackModel`:
```python
FallbackModel("typesafe:jev-latest", "openai:…", fallback_on=[ModelAPIError, unsure])
```
That is currently unreachable from every operator in this provider, because `create_agent` passes the model positionally, so `agent_params={"model": FallbackModel(...)}` raises `TypeError: got multiple values for argument 'model'`. A `model=` passthrough would close the gap above without this provider inventing any routing policy of its own. Left out of this PR to keep it to the extra and the docs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add support for TypeSafe Jev classifier models [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil merged PR #73363:
URL: https://github.com/apache/airflow/pull/73363
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org