[GitHub] [airflow] uranusjr commented on a change in pull request #17121: [WIP] Deactivating DAGs which have been removed from files

[GitBox <gi...@apache.org>]

uranusjr commented on a change in pull request #17121:
URL: https://github.com/apache/airflow/pull/17121#discussion_r700963529



##########
File path: airflow/dag_processing/processor.py
##########
@@ -645,3 +648,12 @@ def process_file(
             self.log.exception("Error logging import errors!")
 
         return len(dagbag.dags), len(dagbag.import_errors)
+
+    def _deactivate_missing_dags(self, session: Session, dagbag: DagBag, file_path: str) -> None:
+        deactivated = (
+            session.query(DagModel)
+            .filter(DagModel.fileloc == file_path, DagModel.is_active, ~DagModel.dag_id.in_(dagbag.dag_ids))
+            .update({DagModel.is_active: False}, synchronize_session="fetch")
+        )
+        if deactivated:
+            self.log.info("Deactivated %i DAGs which are no longer present in %s", deactivated, file_path)

Review comment:
       I wonder if it’d be worthwhile to log the DAG IDs here. (Is it easy though? I know in SQL you can do `UPDATE ... RETURNING` but I don’t know what dbs can handle that nor how it’s done in SQLAlchemy.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org