[jira] [Created] (AIRFLOW-4333) DAGs wont run due to mysql lock? UPDATE dag SET last_scheduler_run=%s WHERE dag.dag_id = %s

["t oo (JIRA)" <ji...@apache.org>]

t oo created AIRFLOW-4333:
-----------------------------

             Summary: DAGs wont run due to mysql lock? UPDATE dag SET last_scheduler_run=%s WHERE dag.dag_id = %s
                 Key: AIRFLOW-4333
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-4333
             Project: Apache Airflow
          Issue Type: Bug
          Components: db
    Affects Versions: 1.10.3
            Reporter: t oo


Process DagFileProcessor2805-Process:
Traceback (most recent call last):
 File "/usr/lib64/python2.7/multiprocessing/process.py", line 267, in _bootstrap
 self.run()
 File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
 self._target(*self._args, **self._kwargs)
 File "/home/ec2-user/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 402, in helper
 pickle_dags)
 File "/home/ec2-user/venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line 73, in wrapper
 return func(*args, **kwargs)
 File "/home/ec2-user/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 1731, in process_file
 dag.sync_to_db()
 File "/home/ec2-user/venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line 73, in wrapper
 return func(*args, **kwargs)
 File "/home/ec2-user/venv/local/lib/python2.7/site-packages/airflow/models/__init__.py", line 4123, in sync_to_db
 session.commit()
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1023, in commit
 self.transaction.commit()
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 487, in commit
 self._prepare_impl()
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 466, in _prepare_impl
 self.session.flush()
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 2446, in flush
 self._flush(objects)
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 2584, in _flush
 transaction.rollback(_capture_exception=True)
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 67, in __exit__
 compat.reraise(exc_type, exc_value, exc_tb)
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 2544, in _flush
 flush_context.execute()
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 416, in execute
 rec.execute(self)
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 583, in execute
 uow,
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 236, in save_obj
 update,
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 976, in _emit_update_statements
 statement, multiparams
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 980, in execute
 return meth(self, multiparams, params)
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 273, in _execute_on_connection
 return connection._execute_clauseelement(self, multiparams, params)
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1099, in _execute_clauseelement
 distilled_params,
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1240, in _execute_context
 e, statement, parameters, cursor, context
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1458, in _handle_dbapi_exception
 util.raise_from_cause(sqlalchemy_exception, exc_info)
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 296, in raise_from_cause
 reraise(type(exception), exception, tb=exc_tb, cause=cause)
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context
 cursor, statement, parameters, context
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute
 cursor.execute(statement, parameters)
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 206, in execute
 res = self._query(query)
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 312, in _query
 db.query(q)
 File "/home/ec2-user/venv/local/lib64/python2.7/site-packages/MySQLdb/connections.py", line 224, in query
 _mysql.connection.query(self, query)
OperationalError: (MySQLdb._exceptions.OperationalError) (1205, 'Lock wait timeout exceeded; try restarting transaction') [SQL: u'UPDATE dag SET last_scheduler_run=%s WHERE dag.dag_id = %s'] [parameters: (datetime.datetime(2019, 4, 16, 16, 10, 56, 190949, tzinfo=<Timezone [UTC]>), 'netpos')] (Background on this error at: http://sqlalche.me/e/e3q8)
[2019-04-16 16:11:52,712] \{sqlalchemy.py:81} WARNING - DB connection invalidated. Reconnecting...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)