[GitHub] [airflow] halilduygulu edited a comment on issue #19622: Tasks get stuck in "scheduled" state and starved when dags with huge amount of tasks is scheduled

[GitBox <gi...@apache.org>]

halilduygulu edited a comment on issue #19622:
URL: https://github.com/apache/airflow/issues/19622#issuecomment-975789786


   Hi I am having the same case I think. airflow version 2.2.2 , just upgraded from 2.0.2
   A huge dag running with concurrency = 10, has 280 tasks and there are other dags which their last tasks are in scheduled state forever now. I increased max_tis_per_query = 300 , it was 100, but did not see any improvement. Weird thing is if I ran another dag it runs fine until last task, then stuck, now I got 5 dags waiting in last step with plenty workers idle.
   
   my config
   ```
   parallelism = 32
   max_active_tasks_per_dag = 14
   max_active_runs_per_dag = 10
   max_tis_per_query = 300
   ```
   
   what can I do for a workaround now?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org