You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "ziyangRen (via GitHub)" <gi...@apache.org> on 2025/07/02 02:11:11 UTC

Re: [I] The new task will be scheduled many times during the first dag_run. [airflow]

ziyangRen commented on issue #52276:
URL: https://github.com/apache/airflow/issues/52276#issuecomment-3026109887

   @sjyangkevin I have some new findings to add here. We used the celery executor, and there are three celery nodes: 94, 95, and 96. When the "remove" phenomenon occurred, we checked the MySQL binlog log and discovered the following update statements:
   `airflow_2025[airflow_2025] @  [10.48.8.95] │ UPDATE task_instance SET state='removed', updated_at='2025-07-01 09:00:34.904710' WHERE task_instance.dag_id = 'analysis_v1' AND task_instance.task_id = 'analysis_v1.cs_hehe_complain_rate_cs' AND task_instance.run_id = 'scheduled__2025-06-29T16:00:00+00:00' AND task_instance.map_index = -1`
   
   I noticed that this statement **was executed by 95**. However, the queue we configured for **this task has 94 nodes**. I think this is very helpful for troubleshooting the issue.
   
   ![Image](https://github.com/user-attachments/assets/8f5c4694-c5d3-4e4f-8155-77c8d3ba9a6b)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org