[GitHub] [airflow] nik-davis commented on issue #13504: Scheduler is unable to find serialized DAG in the serialized_dag table

[GitBox <gi...@apache.org>]

nik-davis commented on issue #13504:
URL: https://github.com/apache/airflow/issues/13504#issuecomment-762806828


   Would just like to add our temporary solution that is helping us get around this issue, and seems to be working quite nicely. We've added a python script to run before starting the scheduler which will serialize any missing DAGs, so if it fails on this error it will be fixed the next time it starts up.
   
   Here's serialize_missing_dags.py:
   
   ```
   from airflow.models import DagBag
   from airflow.models.serialized_dag import SerializedDagModel
   
   dag_bag = DagBag()
   
   # Check DB for missing serialized DAGs, and add them if missing
   for dag_id in dag_bag.dag_ids:
       if not SerializedDagModel.get(dag_id):
           dag = dag_bag.get_dag(dag_id)
           SerializedDagModel.write_dag(dag)
   ```
   Which we call before starting the scheduler: `python serialize_missing_dags.py && exec airflow scheduler`
   
   I hope this helps!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org