[jira] [Created] (AIRFLOW-656) Lock wait timeout exceeded on XCOM table

["Chris Riccomini (JIRA)" <ji...@apache.org>]

Chris Riccomini created AIRFLOW-656:
---------------------------------------

             Summary: Lock wait timeout exceeded on XCOM table
                 Key: AIRFLOW-656
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-656
             Project: Apache Airflow
          Issue Type: Bug
          Components: xcom
    Affects Versions: Airflow 1.7.1.2
            Reporter: Chris Riccomini
            Assignee: Chris Riccomini


We are seeing task failures with this periodically:

(_mysql_exceptions.OperationalError) (1205, 'Lock wait timeout
exceeded; try restarting transaction') [SQL: u'DELETE FROM xcom WHERE
xcom.`key` = %s AND xcom.execution_date = %s AND xcom.task_id = %s AND
xcom.dag_id = %s'] [parameters: (u'return_value',
datetime.datetime(2016, 11, 28, 15, 47),
'gcs2bq_fraud.fraud_rules_executions',
'db_monolith_fraud_fraud_rules_executions_partition_15m')]

According to [~maxime.beauchemin@apache.org]:

{quote}
I just looked quickly and it looks like no index is defined on the XCom
model beyond the PK. We should add a composite btree index on dag_id,
task_id and execution_date. I don't think MySQL will let you include `key`
in there as it's too long (512). The 3 other fields are plenty selective so
that will work just as well.

Creating the index straight in the db should fix the problem right away in
your environment, but it'd be nice to add it to the model definition and
generate the db migration script for it.
{quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)