[GitHub] [airflow] georborodin opened a new issue #17442: Deleted DAG raises SerializedDagNotFound exception when accessing webserver

[GitBox <gi...@apache.org>]

georborodin opened a new issue #17442:
URL: https://github.com/apache/airflow/issues/17442


   **Apache Airflow version**: 2.1.2
   
   
   **Environment**:
   
   - **Docker image**: official apache/airflow:2.1.2 extended with pip installed packages
   - **Executor**: CeleryExecutor
   - **Database**: PostgreSQL (engine version: 12.5)
   
   **What happened**:
   
   I've encountered `SerializedDagNotFound: DAG 'my_dag_id' not found in serialized_dag table` after deleting DAG via web UI. Exception is raised each time when UI is accessed. The exception itself does not impact UI, but still an event is sent to Sentry each time I interact with it.
   
   **What you expected to happen**:
   
   After DAG deletion I've expected that all records of it apart from logs would be deleted, but it's DAG run was still showing up in Webserver UI (even though, I couldn't find any records of DAG run in the metadb, apart from dag_tag table still containing records related to deleted DAG (Which may be a reason for a new issue), but manual deletion of those records had no impact. 
   
   After I've deleted DAG Run via Web UI, the exception is no longer raised.
   
   **How to reproduce it**:
   
   To this moment, I've only encountered this with one DAG, which was used for debugging purposes. It consists of multiple PythonOperators and it uses the same boilerplate code I use for dynamic DAG generation (return DAG object from `create_dag` function, add it to globals), but in it I've replaced dynamic generation logic with just `dag = create_dag(*my_args, **my_kwargs)`. I think this may have been caused by DAG run still running, when DAG was deleted, but cannot support this theory with actual information. 
   
   <details>
     <summary>Traceback</summary> 
   
   ```python
   SerializedDagNotFound: DAG 'my_dag_id' not found in serialized_dag table
     File "flask/app.py", line 2447, in wsgi_app
       response = self.full_dispatch_request()
     File "flask/app.py", line 1952, in full_dispatch_request
       rv = self.handle_user_exception(e)
     File "flask/app.py", line 1821, in handle_user_exception
       reraise(exc_type, exc_value, tb)
     File "flask/_compat.py", line 39, in reraise
       raise value
     File "flask/app.py", line 1950, in full_dispatch_request
       rv = self.dispatch_request()
     File "flask/app.py", line 1936, in dispatch_request
       return self.view_functions[rule.endpoint](**req.view_args)
     File "airflow/www/auth.py", line 34, in decorated
       return func(*args, **kwargs)
     File "airflow/utils/session.py", line 70, in wrapper
       return func(*args, session=session, **kwargs)
     File "airflow/www/views.py", line 1679, in blocked
       dag = current_app.dag_bag.get_dag(dag_id)
     File "airflow/utils/session.py", line 70, in wrapper
       return func(*args, session=session, **kwargs)
     File "airflow/models/dagbag.py", line 186, in get_dag
       self._add_dag_from_db(dag_id=dag_id, session=session)
     File "airflow/models/dagbag.py", line 258, in _add_dag_from_db
       raise SerializedDagNotFound(f"DAG '{dag_id}' not found in serialized_dag table")
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil closed issue #17442: Deleted DAG raises SerializedDagNotFound exception when accessing webserver

[GitBox <gi...@apache.org>]

kaxil closed issue #17442:
URL: https://github.com/apache/airflow/issues/17442


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org