You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/05/09 15:48:00 UTC
[jira] [Commented] (AIRFLOW-2393) UI tree view struggles with large
dags (60 tasks)x25 dag histories
[ https://issues.apache.org/jira/browse/AIRFLOW-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468984#comment-16468984 ]
ASF subversion and git services commented on AIRFLOW-2393:
----------------------------------------------------------
Commit 2a55ffe0cd2c04fde2eecdbeff2fa727067069e1 in incubator-airflow's branch refs/heads/master from Tao feng
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=2a55ffe ]
[AIRFLOW-2086][AIRFLOW-2393] Customize default dagrun number in tree view
Closes #3279 from feng-tao/reduce-tree-view
This introduces a new configuration variable to set the default
number of dag runs displayed in the tree view. For large DAGs, this
could cause timeouts in the webserver.
> UI tree view struggles with large dags (60 tasks)x25 dag histories
> ------------------------------------------------------------------
>
> Key: AIRFLOW-2393
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2393
> Project: Apache Airflow
> Issue Type: Improvement
> Components: ui
> Affects Versions: 1.9.0
> Reporter: Badger
> Priority: Major
>
> Hi,
> We are noticing the tree view is taking a long time to render as our DAG has become more complex. We will need to start breaking our dag apart in order to continue to use the user interface.
> The basic problem is that a reasonably complex DAG (60 operators) x the standard 25 dag run histories on the tree view causes a 350MB json response (compressed to 8MB) to be downloaded, this then needs the browser to render it.
> On quick observation this appears to be because, the response appears to contain all meta-data for each task.
> Is this something others think is a problem. We occasionally have to refresh due to memory errors and have already increased the RAM allocated to the box.
> A suggestion might be to load specific instance history when a user hovers over the task, rather than exporting all of the history on page load. I'd look at contributing a PR but haven't had chance to take a look at this area of the code base.
> Thanks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)