[GitHub] [airflow] mongakshay removed a comment on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

[GitBox <gi...@apache.org>]

mongakshay removed a comment on issue #13542:
URL: https://github.com/apache/airflow/issues/13542#issuecomment-892630614


   I am also noticing this issue with my Airflow v2.1.2 instance, where the task is forever in queued state, and in the scheduler log I see 
   ```
   airflow-scheduler1_1  | [2021-08-04 12:45:23,286] {base_executor.py:85} ERROR - could not queue task TaskInstanceKey(dag_id='example_complex', task_id='create_tag_template_result2', execution_date=datetime.datetime(2021, 8, 3, 22, 39, 34, 577031, tzinfo=Timezone('UTC')), try_number=1)
   ```
   
   When I sent the SIGUSR2 signal to scheduler I received:
   
   ```
   airflow-scheduler1_1  | SIGUSR2 received, printing debug
   airflow-scheduler1_1  | --------------------------------------------------------------------------------
   airflow-scheduler1_1  | [2021-08-04 13:07:37,230] {base_executor.py:305} INFO - executor.queued (0)
   airflow-scheduler1_1  | 	
   airflow-scheduler1_1  | [2021-08-04 13:07:37,231] {base_executor.py:307} INFO - executor.running (1)
   airflow-scheduler1_1  | 	TaskInstanceKey(dag_id='example_complex', task_id='create_tag_template_result2', execution_date=datetime.datetime(2021, 8, 3, 22, 39, 34, 577031, tzinfo=Timezone('UTC')), try_number=1)
   airflow-scheduler1_1  | [2021-08-04 13:07:37,233] {base_executor.py:311} INFO - executor.event_buffer (0)
   airflow-scheduler1_1  | 	
   airflow-scheduler1_1  | [2021-08-04 13:07:37,234] {celery_executor.py:372} INFO - executor.tasks (1)
   airflow-scheduler1_1  | 	(TaskInstanceKey(dag_id='example_complex', task_id='create_tag_template_result2', execution_date=datetime.datetime(2021, 8, 3, 22, 39, 34, 577031, tzinfo=Timezone('UTC')), try_number=1), <AsyncResult: 1ed20983-dbd3-47c6-8404-345fa0fb75ff>)
   airflow-scheduler1_1  | [2021-08-04 13:07:37,235] {celery_executor.py:377} INFO - executor.adopted_task_timeouts (0)
   ```
   
   So basically it is trying to queue a running task ? and failing the `if` check 
   ```
           if task_instance.key not in self.queued_tasks and task_instance.key not in self.running:
               self.log.info("Adding to queue: %s", command)
               self.queued_tasks[task_instance.key] = (command, priority, queue, task_instance)
           else:
               self.log.error("could not queue task %s", task_instance.key)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org