You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "choo121600 (via GitHub)" <gi...@apache.org> on 2026/06/14 05:42:47 UTC
[I] Add an aggregate Dag schedule view: typical daily run times across all Dags [airflow]
choo121600 opened a new issue, #68532:
URL: https://github.com/apache/airflow/issues/68532
### Description
Add a built-in UI view that visualizes the **typical (mean / median) start and end time of each Dag over a 24-hour timeline**, aggregated from recent successful Dag runs.
This is **not** the existing Gantt view. The current Gantt is *per Dag run* — it shows when each task started/ended within a single run.
The proposed view operates at a different level: it aggregates across **all Dags** to answer "across my whole deployment, when during the day does each Dag usually run?"
<img width="2168" height="1402" alt="Image" src="https://github.com/user-attachments/assets/6db18d2f-5561-4b8b-996b-6bda55f83c7d" />
### Use case/motivation
Operators running many Dags currently have no at-a-glance way to see how scheduled load is distributed across the day. To understand it today you have to open each Dag's runs one by one.
This view makes the whole schedule visible at once, which helps to:
- spot **overlapping heavy windows** where many Dags pile up at the same hour,
- find **quiet windows** for maintenance / deploys / resource-intensive jobs,
- notice **drift** when a Dag's typical run time shifts over recent runs,
- plan capacity and SLAs across the fleet.
### Related issues
_No response_
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add an aggregate Dag schedule view: typical daily run times across all Dags [airflow]
Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #68532:
URL: https://github.com/apache/airflow/issues/68532#issuecomment-4733316954
Not quite sure how this is useful:
> typical (mean / median) start and end time of each Dag over a 24-hour timeline
If I run a DAG hourly, that would make the average noon. That's pretty useless IMO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add an aggregate Dag schedule view: typical daily run times across all Dags [airflow]
Posted by "venkatamandavilli-code (via GitHub)" <gi...@apache.org>.
venkatamandavilli-code commented on issue #68532:
URL: https://github.com/apache/airflow/issues/68532#issuecomment-4723452865
Thanks for raising this enhancement.
In large enterprise environments with hundreds or even thousands of DAGs, it can be difficult for platform teams to understand workload distribution throughout the day. An aggregate schedule view would provide valuable operational visibility by helping teams identify peak execution windows, scheduling bottlenecks, and opportunities to better distribute workloads across available resources.
From an operations and governance perspective, this could also support capacity planning, infrastructure optimization, and proactive monitoring of critical business workflows. A visual representation of typical DAG execution patterns across the environment would make it easier to identify schedule concentration and reduce the risk of resource contention during peak periods.
I believe this would be especially useful for organizations running Airflow as a shared platform across multiple business teams.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add an aggregate Dag schedule view: typical daily run times across all Dags [airflow]
Posted by "choo121600 (via GitHub)" <gi...@apache.org>.
choo121600 commented on issue #68532:
URL: https://github.com/apache/airflow/issues/68532#issuecomment-4760918105
Thanks @jroachgolf84 — that's a fair point. You're right that a mean/median start time becomes meaningless for high-frequency schedules (an hourly Dag averaging to "noon" doesn't tell us anything useful), so I'd drop the averaging idea.
Let me reframe around the two problems I'm actually trying to solve:
1. **Scheduled Dags**: a 24-hour view of when Dags are expected to run and how long they typically take. For cron/timetable Dags, start times are deterministic and can be derived directly from the timetable. Durations come from recent run history.
2. **Asset/data-aware triggered Dags**: these have no cron schedule, so historical runs are the only way to understand when they typically fire. This is the case I'm most interested in, and as far as I can tell it isn't covered by the calendar view, Gantt view, or schedule metadata today.
Questions for the community:
* Is there already a way to get a fleet-level, time-of-day view of when Dags run (especially asset-triggered ones) that I'm missing?
* Does anyone else feel this need, particularly for asset-triggered Dags?
If the demand isn't broad enough for a core UI feature, I'm happy to keep this as a plugin. I mainly wanted to understand whether others are running into the same visibility gap around asset-triggered Dag timing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add an aggregate Dag schedule view: typical daily run times across all Dags [airflow]
Posted by "venkatamandavilli-code (via GitHub)" <gi...@apache.org>.
venkatamandavilli-code commented on issue #68532:
URL: https://github.com/apache/airflow/issues/68532#issuecomment-4723518955
Thanks for raising this request.
From an operational perspective, visual indicators can be very helpful when investigating workflow issues or reviewing historical executions. Notes often contain important context such as incident details, business explanations, remediation actions, or handoff information that may not be immediately visible through task status alone.
Having a clear visual indicator at the DAG run or task instance level would improve discoverability and reduce the time required for troubleshooting, especially in environments where multiple teams collaborate on the same Airflow platform.
This could be particularly useful for production support teams that rely on operational notes to understand why specific actions were taken during previous runs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add an aggregate Dag schedule view: typical daily run times across all Dags [airflow]
Posted by "choo121600 (via GitHub)" <gi...@apache.org>.
choo121600 closed issue #68532: Add an aggregate Dag schedule view: typical daily run times across all Dags
URL: https://github.com/apache/airflow/issues/68532
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org