[GitHub] [airflow] yuqian90 commented on issue #14205: Scheduler "deadlocks" itself when max_active_runs_per_dag is reached by up_for_retry tasks

[GitBox <gi...@apache.org>]

yuqian90 commented on issue #14205:
URL: https://github.com/apache/airflow/issues/14205#issuecomment-832457091


   The symptom I'm seeing is that whenever any single DAG hits the `max_active_runs_per_dag` limit, the scheduler log will be full of lines like this. At this point, the scheduler basically stops scheduling any task for any dags. The fact that one dag hitting this limit causes all other dags to be halted is pretty bad. Our workaround at this moment is to make `max_active_runs_per_dag` a big number that is unlikely going to be hit.
   
   ```
   DAG ... already has .. active runs, not queuing any tasks
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org