[jira] [Created] (AIRFLOW-5473) SLA calculation doesn't work when you have manual triggers

["Thiago Rigo (Jira)" <ji...@apache.org>]

Thiago Rigo created AIRFLOW-5473:
------------------------------------

             Summary: SLA calculation doesn't work when you have manual triggers
                 Key: AIRFLOW-5473
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5473
             Project: Apache Airflow
          Issue Type: Bug
          Components: scheduler
    Affects Versions: 1.10.4
            Reporter: Thiago Rigo


SLA calculation doesn't work properly when you have manual triggers of the DAG.

I believe the solution should be to filter for scheduled executions only when doing the calculation. Code snippet from jobs.py from manage_slas method.

 
{code:python}
sq = (
            session
            .query(
                TI.task_id,
                func.max(TI.execution_date).label('max_ti'))
            .with_hint(TI, 'USE INDEX (PRIMARY)', dialect_name='mysql')
            .filter(TI.dag_id == dag.dag_id)
            .filter(TI.state == State.SUCCESS)
            .filter(TI.task_id.in_(dag.task_ids))
            .group_by(TI.task_id).subquery('sq')
        )

        max_tis = session.query(TI).filter(
            TI.dag_id == dag.dag_id,
            TI.task_id == sq.c.task_id,
            TI.execution_date == sq.c.max_ti,
        ).all()

        ts = datetime.utcnow()
        SlaMiss = models.SlaMiss
        for ti in max_tis:
            task = dag.get_task(ti.task_id)
            dttm = ti.execution_date
            if task.sla:
                dttm = dag.following_schedule(dttm)
                while dttm < datetime.utcnow():
                    following_schedule = dag.following_schedule(dttm)
                    if following_schedule + task.sla < datetime.utcnow():
                        session.merge(models.SlaMiss(
                            task_id=ti.task_id,
                            dag_id=ti.dag_id,
                            execution_date=dttm,
                            timestamp=ts))
                    dttm = dag.following_schedule(dttm)
{code}

In case you have manual execution, the max_ti will already be D-1, so when you call `dag.following_schedule(dttm)` twice it will be a date in the future, hence SLA miss will never be stored.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)