[jira] [Created] (AIRFLOW-5188) Dag Bag not in-sync with DB

["Oliver Ricken (JIRA)" <ji...@apache.org>]

Oliver Ricken created AIRFLOW-5188:
--------------------------------------

             Summary: Dag Bag not in-sync with DB
                 Key: AIRFLOW-5188
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5188
             Project: Apache Airflow
          Issue Type: Bug
          Components: DAG, database, scheduler, webserver
    Affects Versions: 1.10.4
         Environment: CentOS 7, Maria-DB, python 3.6.7, Airflow 1.10.4
            Reporter: Oliver Ricken


Dear all,
we have upgraded our Airflow installation from 1.10.2 to 1.10.4 without any immidiate hick-ups. However, now, we experience problems we did not observe before the upgrade.

When changing the START_DATE of a DAG, we also change the DAG_ID to avoid any misbehaviour of the scheduler due to changed stats of the DAG. This results in new DAGs appearing in the Dag Bag and old ones disappearing – at least it did before the upgrade.


Now, the picture is different. I changed the START_DATE and DAG_ID of a dag, the Dag Bag lists it correctly (e.g. by manually issuing "airflow list_dags") – only the latest version is displayed, the old one is gone.
However, the Web-UI still shows the old dag, along with the new dag. Removing it via "airflow delete_dag" does not work, prompting "Dag is still in Dag Bag, remove dag file first.". However, accessing dag details via the Web-UI fails, prompting "dag is missing", indicating that is NOT in the Dag Bag anymore (which seems to be correct).

To complete the picture: both dags are present in the DB.
Restarting the webserver did not help either. Restarting the scheduler has not yet been an option due to running dags. This would be my next escalation step.

Can anyone comment on similar observations particularly with respect to having upgraded to 1.10.4?

Cheers


Oliver



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)