[GitHub] [airflow] mik-laj edited a comment on pull request #8718: Set store_serialized_dags from config in UI

[GitBox <gi...@apache.org>]

mik-laj edited a comment on pull request #8718:
URL: https://github.com/apache/airflow/pull/8718#issuecomment-624179448


   Can you check if DAG cleaning works if it contains ExternalTaskMarker and recursive = True? In my opinion, this can still cause problems in some cases. Here is a potential path that can cause problems.
   1. https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/www/views.py#L1138
   or 
   https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/www/views.py#L1156
   2. https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/www/views.py#L1138
   3. https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/www/views.py#L1072
   4. https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/models/dag.py#L956
   5. https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/models/dag.py#L1055
   6. https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/models/dag.py#L1083-L1084
   
   Can you try running this DAG with serialization enabled and disabled?
   https://github.com/apache/airflow/blob/8d6f1aa4b5bb8809ffc55dc0c62e6d0e89f331e5/airflow/example_dags/example_external_task_marker_dag.py
   If you will test on different versions of Airflow, can you also check how long the whole DAG Run will be done? I suspect there will be a big difference between Airflow 1.10.10 and older versions, but I haven't checked it yet.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org