[GitHub] [airflow] SamWheating commented on a change in pull request #17121: Deactivating DAGs which have been removed from files

[GitBox <gi...@apache.org>]

SamWheating commented on a change in pull request #17121:
URL: https://github.com/apache/airflow/pull/17121#discussion_r704563433



##########
File path: airflow/dag_processing/processor.py
##########
@@ -645,3 +648,12 @@ def process_file(
             self.log.exception("Error logging import errors!")
 
         return len(dagbag.dags), len(dagbag.import_errors)
+
+    def _deactivate_missing_dags(self, session: Session, dagbag: DagBag, file_path: str) -> None:
+        deactivated = (
+            session.query(DagModel)
+            .filter(DagModel.fileloc == file_path, DagModel.is_active, ~DagModel.dag_id.in_(dagbag.dag_ids))
+            .update({DagModel.is_active: False}, synchronize_session="fetch")
+        )
+        if deactivated:
+            self.log.info("Deactivated %i DAGs which are no longer present in %s", deactivated, file_path)

Review comment:
       > Or we can only show the IDs on Postgres and MSSQL.
   
   Do you think its worth doing that? I feel like its a bit of an anti-pattern to have code like this around:
   ```python
   if conf.get('core', 'sql_alchemy_conn').startswith(('mssql', 'postgresql')):
   	# run one query
   else:
   	# run a different query
   ```
   
   So I'd be inclined to just keep it as-is. Thoughts?
   	




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org