[GitHub] [airflow] MichailKaramanos commented on issue #34624: Tasks stuck in queued state when using Triggerer process along with max_active_tis_per_dag

["MichailKaramanos (via GitHub)" <gi...@apache.org>]

MichailKaramanos commented on issue #34624:
URL: https://github.com/apache/airflow/issues/34624#issuecomment-1737232295

   Hi @hussein-awala,
   First, thanks for the reply. 
   I just to give you some feedback around this: In production we patched the to test the proposed solution, and we ended up removing the `max_active_tis_per_dag` limits and set the DAG `max_active_runs=1`! We only left the triggerer process to deal with the deferrable tasks.
   There is something bigger around this problem. I mean, the tasks are still stuck now bouncing between QUEUED -> SCHEDULED without letting the DAG progression further on!
   
   Here are the logs:
   ```
   [2023-09-27T11:40:06.975+0000] {base_executor.py:280} ERROR - could not queue task TaskInstanceKey(dag_id='...', task_id='...', run_id='scheduled__2023-09-27T01:00:00+00:00', try_number=1, map_index=-1) (still running after 12 attempts)
   [2023-09-27T11:40:06.975+0000] {base_executor.py:280} ERROR - could not queue task TaskInstanceKey(dag_id='...', task_id='...', run_id='scheduled__2023-09-27T01:00:00+00:00', try_number=1, map_index=-1) (still running after 12 attempts)
   [2023-09-27T11:40:06.975+0000] {base_executor.py:280} ERROR - could not queue task TaskInstanceKey(dag_id='...', task_id='...', run_id='scheduled__2023-09-27T01:00:00+00:00', try_number=1, map_index=-1) (still running after 12 attempts)
   
   ``` 
   Weird is that the deferrable tasks terminated with success and remain in queued state (same before the patch by the way)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org