[GitHub] [airflow] mik-laj commented on a change in pull request #13199: Create dag dependencies view

[GitBox <gi...@apache.org>]

mik-laj commented on a change in pull request #13199:
URL: https://github.com/apache/airflow/pull/13199#discussion_r546442769



##########
File path: airflow/www/views.py
##########
@@ -3772,3 +3772,102 @@ def autocomplete(self, session=None):
         payload = [row[0] for row in dag_ids_query.union(owners_query).limit(10).all()]
 
         return wwwutils.json_response(payload)
+
+
+class DagDependenciesView(AirflowBaseView):
+    """View to show dependencies between DAGs"""
+
+    refresh_interval = conf.getint(
+        "dag_dependencies_plugin",
+        "refresh_interval",
+        fallback=conf.getint("scheduler", "dag_dir_list_interval"),
+    )
+    last_refresh = datetime.utcnow() - timedelta(seconds=refresh_interval)
+    nodes = []
+    edges = []
+
+    @expose('/dag-dependencies')
+    @auth.has_access(
+        [
+            (permissions.ACTION_CAN_READ, permissions.RESOURCE_DAG),
+        ]
+    )
+    @gzipped
+    @action_logging
+    def list(self):
+        """Display DAG dependencies"""
+        title = "DAG Dependencies"
+
+        if datetime.utcnow() > self.last_refresh + timedelta(seconds=self.refresh_interval):
+            self._calculate_graph()
+            self.last_refresh = datetime.utcnow()
+
+        return self.render_template(
+            "airflow/dag_dependencies.html",
+            title=title,
+            nodes=self.nodes,
+            edges=self.edges,
+            last_refresh=self.last_refresh.strftime("%Y-%m-%d %H:%M:%S"),
+            arrange=conf.get("webserver", "dag_orientation"),
+            width=request.args.get("width", "100%"),
+            height=request.args.get("height", "800"),
+        )
+
+    def _calculate_graph(self):
+
+        current_app.dag_bag.collect_dags_from_db()

Review comment:
       I fear this call the most. Some DAGs can contain a thousand tasks, and we can have several such DAGs. It will be very slow to load them all into memory.  Do you think we can optimize it?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org