You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "jason810496 (via GitHub)" <gi...@apache.org> on 2026/07/15 03:03:27 UTC
[PR] Make Elasticsearch log handler respect the per-run log ID template [airflow]
jason810496 opened a new pull request, #69896:
URL: https://github.com/apache/airflow/pull/69896
- related: #69520 (discussion on dropping vs. fixing the LogTemplate flow)
- **Stacked on #69688**, only the last commit (`providers/elasticsearch`) is new; the base commits are the core plumbing that delivers the pinned template to workers and will disappear from the diff once #69688 merges.
## Why
The `LogTemplate` model pins the log ID template to each Dag run so task logs written before a config change stay readable, but the Airflow 3 paths of the Elasticsearch handler render `log_id` from the current conf value on both the write (worker) and read (API server) sides — so changing `[elasticsearch] log_id_template` orphans every previously written log document.
## What
- Write side: `RemoteLogIO.upload` opts into the `ti_context` keyword (covering both the `write_stdout`/filebeat and direct-write branches) and renders the log ID from the Dag-run-pinned template delivered via `TIRunContext.log_id_template` (#69688).
- Read side (`_read`, `get_external_log_url`, `RemoteLogIO.read`): restore the per-run `LogTemplate` lookup with a conf fallback when the DB is unreachable or the row is missing; worker-side `RuntimeTI`s skip the DB lookup entirely.
- `_render_log_id` now always supplies the full placeholder set (`logical_date`, `execution_date`, `data_interval_start/end`) because templates pinned by older rows use them and `str.format` raises `KeyError` on missing names.
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes, with help of Claude Code (Fable 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make Elasticsearch log handler respect the per-run log ID template [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #69896:
URL: https://github.com/apache/airflow/pull/69896#issuecomment-5465752377
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make Elasticsearch log handler respect the per-run log ID template [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #69896: Make Elasticsearch log handler respect the per-run log ID template
URL: https://github.com/apache/airflow/pull/69896
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org