[GitHub] [airflow] ozw1z5rd commented on issue #8521: MySQL deadlock when using DAG serialization

[GitBox <gi...@apache.org>]

ozw1z5rd commented on issue #8521:
URL: https://github.com/apache/airflow/issues/8521#issuecomment-666575284


   Hi, I have the same issue. 
   
   I was looking in `models/renderedtifields.py` file and I noticed that 
   
   `def delete_old_records( ` 
   
   contains a line that loads the number or rendered fields to keep:
   
   `num_to_keep=conf.getint("core", "max_num_rendered_ti_fields_per_task", fallback=0)` 
   
   and if this value is <= 0 the function will return doing nothing.  
   
   ```
    if num_to_keep <= 0:
        return
   ```
   
   Since the dead lock is about the insert and the delete in that table, setting max_num_rendered_ti_fields_per_task = 0 inside the [core] config ... perhaps can fix the issue.
   
   Of course it does not work. 
   
   Using `SHOW ENGINE INNODB STATUS ` I see queries like this: 
   
   ```
   DELETE FROM rendered_task_instance_fields 
   WHERE rendered_task_instance_fields.dag_id = 'PARTITIONADD' 
   AND rendered_task_instance_fields.task_id = 'partition_add' 
   AND (rendered_task_instance_fields.dag_id, rendered_task_instance_fields.task_id, rendered_task_instance_fields.execution_date) NOT IN (
          SELECT subq1.dag_id, subq1.task_id, subq1.execution_date
         FROM (
                SELECT rendered_task_instance_fields.dag_id AS dag_id, rendered_task_instance_fields.task_id AS task_id,         
                              rendered_task_instance_fields.execution_date AS execution_date
                 FROM rendered_task_instance_fields
                WHERE rendered_task_instance_fields.dag_id = 'PARTITIONADD' 
                AND rendered_task_instance_fields.task_id = 'partition_add' 
               ORDER BY rendered_task_instance_fields.execution_date DESC
               LIMIT 30
          ) 
   AS subq1
   )
   ```
   -----> Please note LIMIT 30
   
   I found this code inside` models/taskinstance.py`
   
   ```
    if STORE_SERIALIZED_DAGS:
        RTIF.write(RTIF(ti=self, render_templates=False), session=session)
        RTIF.delete_old_records(self.task_id, self.dag_id, session=session)
   ```
   and it's the unique place where delete_old_records is called, so ... it is weird, is it not? 
   from which point of the universe comes that "30"? 
   
   I'll investigate better tomorrow...
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org