You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jobs@airflow.apache.org by GitBox <gi...@apache.org> on 2026/07/14 18:17:39 UTC
[GH] (airflow/common-ai-trace-plugin): Workflow run "Tests (AMD)" failed!
The GitHub Actions job "Tests (AMD)" on airflow.git/common-ai-trace-plugin has failed.
Run started by GitHub user kaxil (triggered by kaxil).
Head commit for run:
53e7a0ec70b46263160f42e896362ab21ea0b99a / Kaxil Naik <ka...@gmail.com>
Add AI Trace observability panel to the common.ai provider
Adds an AI Trace AirflowPlugin: a FastAPI sub-app mounted on the API
server plus a React panel on the UI, so a GenAI agent run (AgentOperator
/ @task.agent / @task.llm) can be inspected in context -- conversation,
model, tokens, cost, latency, and the full observation tree with a
duration waterfall -- from the task instance and from a deployment-wide
list, without leaving Airflow.
The panel reads traces from two sources:
- A configured tracing backend (Langfuse today): the trace is resolved
from the task instance's own OTel span context and read back through
the backend's API. No new data is written anywhere.
- A backend-free ObjectStorage trace store: when [common.ai]
trace_store_path is set, agent spans are written as standard OTLP JSON
lines under {dag_id}/{run_id}/{task_id}/{map_index}/{try_number}.jsonl
and the panel reads them directly. This needs no Langfuse, no
collector, and no core tracing -- just a path -- and targets local dev
and quick testing. Files are standard OTLP JSON, so a collector's
otlpjsonfilereceiver can replay them into a real backend later.
Costs are estimated at read time via genai-prices (already a transitive
dependency of pydantic-ai-slim) from the model, provider, and token
counts on each span, so old files are priced retroactively and unknown
or self-hosted models simply show no cost. The OTLP JSON encoder is
vendored because the upstream opentelemetry-exporter-otlp-json-file
release is currently uninstallable from PyPI (its opentelemetry-proto-json
dependency was never published); it follows the OTLP JSON-Protobuf
encoding so files stay collector-replayable.
Security: every endpoint is RBAC-gated. The task-instance panel checks
TASK_INSTANCE access to the DAG; the bare-id lookups resolve the trace
(or observation) to its owning task instance and require access to that
DAG, so a user authorized for one DAG cannot read another's captured
content. Task-instance coordinates are validated before they reach an
ObjectStoragePath join (it does not normalize ".."), and trace ids are
validated as 32-hex before use in a reverse lookup or file scan.
Includes unit tests for the OTLP JSON encoder (wire-format round trip
against real SDK spans), the store reader and normalizer, read-time cost
estimation, and the endpoints in both modes (RBAC, path-traversal
rejection, trace-id validation, per-DAG authorization, and 404 paths).
Known limitations: the trace-store layout is a proof of concept, not a
stable on-disk contract; no retention is applied; object stores flush a
task's spans at process exit rather than live; cost is an estimate, not
billing data.
Report URL: https://github.com/apache/airflow/actions/runs/29352463050
With regards,
GitHub Actions via GitBox
---------------------------------------------------------------------
To unsubscribe, e-mail: jobs-unsubscribe@airflow.apache.org
For additional commands, e-mail: jobs-help@airflow.apache.org