You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "matrach (via GitHub)" <gi...@apache.org> on 2025/01/24 01:10:30 UTC

Re: [I] Scheduler performance with large number of mapped task instances [airflow]

matrach commented on issue #45991:
URL: https://github.com/apache/airflow/issues/45991#issuecomment-2611334138

   I've mentioned the mini-scheduler above, but if I understand correctly, it resolved the scheduling state for ALL task instances, but then ignored the ones not downstream...?
   
   It appears that the scheduler for mapped tasks takes at least quadratic time in their quantity, as [here](https://github.com/apache/airflow/blob/main/airflow/ti_deps/deps/trigger_rule_dep.py#L255) it loops over all finished task instances, and does so for all schedulable task instances.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org