You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/05/26 18:47:31 UTC

[GitHub] [airflow] MatrixManAtYrService edited a comment on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

MatrixManAtYrService edited a comment on issue #13542:
URL: https://github.com/apache/airflow/issues/13542#issuecomment-849031715


   While trying to recreate this, I wrote a [stress test](https://github.com/MatrixManAtYrService/airflow-git-sync/blob/master/scheduler_stress.py) which I ran overnight on my local microk8s cluster (release:2.0.1+beb8af5ac6c438c29e2c186145115fb1334a3735 configured like [this](https://github.com/MatrixManAtYrService/airflow-git-sync/blob/master/zsh.stdin)).
   
   I was hoping that it would get fully stuck by the time I woke.  Instead there were only two stuck tasks: 
    
   ![stucktasks](https://user-images.githubusercontent.com/5834582/119712787-34db0f80-be1e-11eb-9a41-875dd88c0566.gif)
   
   Deleting the scheduler pod and letting kubernetes recreate it did cause the stuck tasks to complete.  My current guess is that it won't get stuck in a restarting-scheduler-doesnt-help kind of way until the number of stuck-queued tasks matches `dag_concurrency`.  I'll update here if I'm able to confirm that guess.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org