[GitHub] [airflow] sangreddy32 commented on pull request #14031: Disable row level locking for Mariadb and MySQL <8

[GitBox <gi...@apache.org>]

sangreddy32 commented on PR #14031:
URL: https://github.com/apache/airflow/pull/14031#issuecomment-1179187360

   We have a similar issue in the Airflow '2.3.1' and MySQL 'Ver 14.14 Distrib 5.7.37'. Our scheduler shuts down whenever we get Deadlocks as below. Any solution or temporary fix greatly appreciated. 
   
   
   
   
   
    rowcount = cursor.executemany(statement, parameters)
     File "/efs/airflow-venv/cdw-prod/airflow/lib/python3.9/site-packages/MySQLdb/cursors.py", line 239, in executemany
       self.rowcount = sum(self.execute(query, arg) for arg in args)
     File "/efs/airflow-venv/cdw-prod/airflow/lib/python3.9/site-packages/MySQLdb/cursors.py", line 239, in <genexpr>
       self.rowcount = sum(self.execute(query, arg) for arg in args)
     File "/efs/airflow-venv/cdw-prod/airflow/lib/python3.9/site-packages/MySQLdb/cursors.py", line 206, in execute
       res = self._query(query)
     File "/efs/airflow-venv/cdw-prod/airflow/lib/python3.9/site-packages/MySQLdb/cursors.py", line 319, in _query
       db.query(q)
     File "/efs/airflow-venv/cdw-prod/airflow/lib/python3.9/site-packages/MySQLdb/connections.py", line 254, in query
       _mysql.connection.query(self, query)
   sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction')
   [SQL: UPDATE dag_run SET last_scheduling_decision=%s WHERE dag_run.id = %s]
   [parameters: ((datetime.datetime(2022, 7, 8, 11, 0, 23, 20492), 4611), (datetime.datetime(2022, 7, 8, 11, 0, 23, 92501), 4612), (datetime.datetime(2022, 7, 8, 11, 0, 23, 180288), 4613), (datetime.datetime(2022, 7, 8, 11, 0, 23, 261947), 4616), (datetime.datetime(2022, 7, 8, 11, 0, 23, 360666), 4619), (datetime.datetime(2022, 7, 8, 11, 0, 23, 484052), 4620), (datetime.datetime(2022, 7, 8, 11, 0, 23, 605315), 4621), (datetime.datetime(2022, 7, 8, 11, 0, 23, 696649), 4622)  ... displaying 10 of 17 total bound parameter sets ...  (datetime.datetime(2022, 7, 8, 11, 0, 22, 901407), 4632), (datetime.datetime(2022, 7, 8, 11, 0, 22, 964056), 4633))]
   (Background on this error at: http://sqlalche.me/e/14/e3q8)
   [^[[0m] {^[[34mlocal_executor.py:^[[0m390} INFO^[[0m - Shutting down LocalExecutor; waiting for running tasks to finish.  Signal again if you don't want to wait.^[[0m
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org