[jira] [Assigned] (AIRFLOW-1311) Improve Webserver Load Time For Large DAGs

["Edgar Rodriguez (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Edgar Rodriguez reassigned AIRFLOW-1311:
----------------------------------------

    Assignee: Kevin Yang  (was: Edgar Rodriguez)

> Improve Webserver Load Time For Large DAGs
> ------------------------------------------
>
>                 Key: AIRFLOW-1311
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1311
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: webserver
>            Reporter: Dan Davydov
>            Assignee: Kevin Yang
>            Priority: Major
>
> Large DAGs can take an extremely long time to load in the Airflow UI (minutes/timeout).
> The fixes are as follows:
> 1. Lazy load DAGs (load up to a certain # of tasks by default, prioritizing tasks by their depth, and allow users to expand sections for these DAGs, ideally prefetch deeper tasks once the initial set of tasks has rendered )
> 2. Identify bottlenecks/performance issues in both the frontend/backend for rendering DAGs on the webserver and fix them. Airflow should be more performant for displaying DAGs that are somewhat large, e.g. DAGs that have up to 500 nodes and 2000 edges (dependencies from one task to another) should render within a couple of seconds.
> 3. Make DAG loading asynchronous in the UI (once the top-level tasks have loaded display them immediately). We might not want to do this as users might try to click something only to have the UI change from underneath them
> [~saguziel]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)