[GitHub] [airflow] tothandor commented on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

[GitBox <gi...@apache.org>]

tothandor commented on issue #13542:
URL: https://github.com/apache/airflow/issues/13542#issuecomment-818568914


   I also have these symptoms with Airflow 2.0.1, though I am using a Celery executor.
   I have inspected the workers with `strace` and it seems, that they're unable to read from some pipe:
   ```bash
   # strace -p 259969
   strace: Process 259969 attached
   read(8,
   ```
   
   If I check the file descriptors, it's gone missing:
   ```
   # ls -l /proc/259969/fd/8                                                                                                                                                                                         
   lr-x------. 1 airflow airflow 64 Apr 13 10:31 /proc/259969/fd/8 -> 'pipe:[864564522]'
   ```
   
   And the parent process is unable to write to a socket:
   ```bash
   # strace -p 252882                                                                                                                                                                                              
   strace: Process 252882 attached
   write(6, "\0\0\1t\200\4\225i\1\0\0\0\0\0\0\214\34airflow.utils.d"..., 376
   ```
   And the socket as well is missing.
   ```bash
   # ls -l /proc/252882/fd/6                                                                                                                                                                                         
   lrwx------. 1 airflow airflow 64 Apr 13 10:30 /proc/252882/fd/6 -> 'socket:[864501296]'
   ```
   After killing the stucked workers things start to work again for a while.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org