Re: [I] StaleDataError when running DAG with dynamic task mapping [airflow]

["mro-dmoura (via GitHub)" <gi...@apache.org>]

mro-dmoura commented on issue #42006:
URL: https://github.com/apache/airflow/issues/42006#issuecomment-2413884711

   I'm experiencing the same issue. Using Celery Executor in EKS with PostgreSQL on RDS as metadata database. The dynamic generated tasks affect other tasks and DAGs, causing random task killed without logs:
   [2024-10-14, 05:14:48 CDT] {scheduler_job_runner.py:843} ERROR - Executor reports task instance <TaskInstance: taskname [queued]> finished (failed) although the task says it's queued. (Info: None) Was the task killed externally?
   
   And in the scheduler logs, the StaleDataError:
   sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org