You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/03/06 14:27:10 UTC
[PR] [v3-1-test] perf: use load_only() in get_dag_runs eager loading to reduce data fetched per task instance (#62482) [airflow]
github-actions[bot] opened a new pull request, #62996:
URL: https://github.com/apache/airflow/pull/62996
* perf: use load_only() in eager_load_dag_run_for_validation to reduce data fetched
The get_dag_runs API endpoint was slow on large deployments because
eager_load_dag_run_for_validation() used selectinload on task_instances and
task_instances_histories without restricting which columns were fetched.
This caused SQLAlchemy to load all heavyweight columns (executor_config with
pickled data, hostname, rendered fields, etc.) for every task instance across
every DAG run in the result page — even though only dag_version_id is needed
to traverse the association proxy to DagVersion.
Add load_only(TaskInstance.dag_version_id) and
load_only(TaskInstanceHistory.dag_version_id) to the selectinload chains so
the SELECT for task instances fetches only the identity columns and the FK
needed to resolve the dag_version relationship, significantly reducing the
volume of data transferred from the database on busy deployments.
Fixes #62025
* Fix static checks
---------
(cherry picked from commit 13af96b80868ef91ca623d35afcd76003bfbda90)
Co-authored-by: Lakshmi Sravya <38...@users.noreply.github.com>
Co-authored-by: pierrejeambrun <pi...@gmail.com>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [v3-1-test] perf: use load_only() in get_dag_runs eager loading to reduce data fetched per task instance (#62482) [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun merged PR #62996:
URL: https://github.com/apache/airflow/pull/62996
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org