You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "pierrejeambrun (via GitHub)" <gi...@apache.org> on 2026/10/05 16:27:39 UTC
[PR] Expose the resolved LLM model name on a dedicated XCom key [airflow]
pierrejeambrun opened a new pull request, #74272:
URL: https://github.com/apache/airflow/pull/74272
The model that actually answered a run was only reachable buried inside the `decision` XCom record (for `LLMOperator`) or the task logs. This publishes it under a dedicated, stable `model_name` XCom key from both the LLM and the agent operator, so downstream tasks — and the UI — can read which model ran without parsing the decision record.
- `LLMOperator` (and `LLMBranchOperator`, by inheritance): push the resolved `response.model_name` under `model_name`.
- `AgentOperator`: push it in `_emit_run_metadata`, alongside the existing `run_id` / `usage` run-metadata XComs.
The resolved name is the model that actually responded (e.g. `claude-sonnet-4-5-20250929`), which can differ from the configured `model_id` when the model comes from the connection's `extra.model` or a fallback.
First of a two-PR series — the follow-up adds a UI panel that surfaces this as a first-class "Model" badge on the task-instance view.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Opus 4.8)
Generated-by: Claude Code (Opus 4.8) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Expose the resolved LLM model name on a dedicated XCom key [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #74272:
URL: https://github.com/apache/airflow/pull/74272#discussion_r4196740368
##########
providers/common/ai/src/airflow/providers/common/ai/utils/logging.py:
##########
@@ -94,6 +94,11 @@ def _log_cache_and_cost(logger: Logger | logging.Logger, usage: RunUsage) -> Non
logger.info("LLM run cost: $%s (USD, best-effort)", format(usage.cost, "f"))
+# XCom key the LLM and agent operators publish the run's resolved model name under, so downstream
+# tasks and the UI can read which model actually answered without parsing the decision record.
+MODEL_NAME_XCOM_KEY = "model_name"
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Expose the resolved LLM model name on a dedicated XCom key [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on PR #74272:
URL: https://github.com/apache/airflow/pull/74272#issuecomment-6037974481
Soften wording (to mention no auto refresh yet)
<img width="857" height="462" alt="Screenshot 2026-10-07 at 14 31 54" src="https://github.com/user-attachments/assets/579bd18d-bcf8-438b-b8f5-c9dc6a1db2dd" />
Rename to AI Model (plugin and tab)
<img width="1185" height="576" alt="Screenshot 2026-10-07 at 14 32 09" src="https://github.com/user-attachments/assets/1c3c6cce-f200-4475-8327-afe311b79611" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Expose the resolved LLM model name on a dedicated XCom key [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #74272:
URL: https://github.com/apache/airflow/pull/74272#discussion_r4218598343
##########
providers/common/ai/src/airflow/providers/common/ai/utils/logging.py:
##########
@@ -94,6 +94,11 @@ def _log_cache_and_cost(logger: Logger | logging.Logger, usage: RunUsage) -> Non
logger.info("LLM run cost: $%s (USD, best-effort)", format(usage.cost, "f"))
+# XCom key the LLM and agent operators publish the run's resolved model name under, so downstream
+# tasks and the UI can read which model actually answered without parsing the decision record.
+MODEL_NAME_XCOM_KEY = "__AIRFLOW__COMMON_AI_MODEL_NAME__"
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Expose the resolved LLM model name on a dedicated XCom key [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #74272:
URL: https://github.com/apache/airflow/pull/74272#discussion_r4195220389
##########
providers/common/ai/src/airflow/providers/common/ai/utils/logging.py:
##########
@@ -94,6 +94,11 @@ def _log_cache_and_cost(logger: Logger | logging.Logger, usage: RunUsage) -> Non
logger.info("LLM run cost: $%s (USD, best-effort)", format(usage.cost, "f"))
+# XCom key the LLM and agent operators publish the run's resolved model name under, so downstream
+# tasks and the UI can read which model actually answered without parsing the decision record.
+MODEL_NAME_XCOM_KEY = "model_name"
Review Comment:
I thought about this yes, we can do that. I didn't because "decision" is just "decision", but we can to it with models to avoid collisions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Expose the resolved LLM model name on a dedicated XCom key [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #74272:
URL: https://github.com/apache/airflow/pull/74272#discussion_r4195220389
##########
providers/common/ai/src/airflow/providers/common/ai/utils/logging.py:
##########
@@ -94,6 +94,11 @@ def _log_cache_and_cost(logger: Logger | logging.Logger, usage: RunUsage) -> Non
logger.info("LLM run cost: $%s (USD, best-effort)", format(usage.cost, "f"))
+# XCom key the LLM and agent operators publish the run's resolved model name under, so downstream
+# tasks and the UI can read which model actually answered without parsing the decision record.
+MODEL_NAME_XCOM_KEY = "model_name"
Review Comment:
I thought about this yes, we can do that. I didn't because "decision" is just "decision", but we can to it with models key to avoid collisions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Expose the resolved LLM model name on a dedicated XCom key [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #74272:
URL: https://github.com/apache/airflow/pull/74272#discussion_r4194849275
##########
providers/common/ai/src/airflow/providers/common/ai/utils/logging.py:
##########
@@ -94,6 +94,11 @@ def _log_cache_and_cost(logger: Logger | logging.Logger, usage: RunUsage) -> Non
logger.info("LLM run cost: $%s (USD, best-effort)", format(usage.cost, "f"))
+# XCom key the LLM and agent operators publish the run's resolved model name under, so downstream
+# tasks and the UI can read which model actually answered without parsing the decision record.
+MODEL_NAME_XCOM_KEY = "model_name"
Review Comment:
Should we make this ``__AIRFLOW__COMMON_AI_MODEL_NAME__`` or something similar -- so they are 0 chances of users adding such XCom too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Expose the resolved LLM model name on a dedicated XCom key [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #74272:
URL: https://github.com/apache/airflow/pull/74272#discussion_r4211601932
##########
providers/common/ai/src/airflow/providers/common/ai/operators/llm.py:
##########
@@ -308,6 +308,8 @@ def execute(self, context: Context) -> Any:
output = result.output
model_confidence = ModelConfidence.from_result(result)
+ if model_confidence.model is not None:
+ self._push_xcom(context, MODEL_NAME_XCOM_KEY, model_confidence.model)
Review Comment:
`LLMBranchOperator` doesn't get this by inheritance. Its `execute` (llm_branch.py:206) replaces this one without calling `super().execute()`, builds its own `ModelConfidence.from_result(result)` and only pushes `decision`, so `@task.llm_branch` never publishes the key even though the description and the comment on the constant say it does. Could the push move into a small helper that both `execute`s call, with a test in test_llm_branch.py? A case with `model_name=None` asserting the key is absent would be good too, since every fixture sets a model today.
`LLMSQLQueryOperator`, `LLMSchemaCompareOperator` and `LLMFileAnalysisOperator` also override `execute`. Fine to leave those out (as #74355 does), but then the description should say `LLMOperator` only.
##########
providers/common/ai/src/airflow/providers/common/ai/utils/logging.py:
##########
@@ -94,6 +94,11 @@ def _log_cache_and_cost(logger: Logger | logging.Logger, usage: RunUsage) -> Non
logger.info("LLM run cost: $%s (USD, best-effort)", format(usage.cost, "f"))
+# XCom key the LLM and agent operators publish the run's resolved model name under, so downstream
+# tasks and the UI can read which model actually answered without parsing the decision record.
+MODEL_NAME_XCOM_KEY = "__AIRFLOW__COMMON_AI_MODEL_NAME__"
Review Comment:
Can this go into observability.rst next to `run_id` and `usage`? That's where users find which keys exist and which operators push them, and its Scope bullet still says those XComs come only from `AgentOperator`. Worth saying there that with `enable_hitl_review` this is the initial run's model, same as the other two, since `regenerate_with_feedback` doesn't re-push it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Expose the resolved LLM model name on a dedicated XCom key [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun merged PR #74272:
URL: https://github.com/apache/airflow/pull/74272
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Expose the resolved LLM model name on a dedicated XCom key [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #74272:
URL: https://github.com/apache/airflow/pull/74272#discussion_r4219017491
##########
providers/common/ai/src/airflow/providers/common/ai/operators/llm.py:
##########
@@ -308,6 +308,8 @@ def execute(self, context: Context) -> Any:
output = result.output
model_confidence = ModelConfidence.from_result(result)
+ if model_confidence.model is not None:
+ self._push_xcom(context, MODEL_NAME_XCOM_KEY, model_confidence.model)
Review Comment:
Updated, only focusing on AgentOperator and LLMOperator for now. Subsequence PRs for all 4 remaining operators later. (LLMSQLQueryOperator, LLMSchemaCompareOperator and LLMFileAnalysisOperator, LLMBranchOperator)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org