[jira] [Commented] (AIRFLOW-6871) Tree view unusable for large DAGs

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073200#comment-17073200 ] 

ASF subversion and git services commented on AIRFLOW-6871:
----------------------------------------------------------

Commit b3e8286fcc4b6f45c0655ba8fd2c1b35fa2fa5a2 in airflow's branch refs/heads/v1-10-test from Kaxil Naik
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=b3e8286 ]

[AIRFLOW-6871] optimize tree view for large DAGs (#7492)

This change reduces page size by more than 10X and
reduces page load time by 3-5X. As a result, the
tree view can now load large DAGs that were causing
5XX error without the patch.

List of optimizations applied to the view handler:
* only seralize used task instance attributes to json instead of the
  whole ORM object
* encode task instance attributes as array instead of dict
* encode datetime in unix timestamp instead of iso formmat string
* push task instance attribute construction into client side JS
* remove redundant task instance attributes
* simplify reduce_nodes() logic, remove unnecessary if statements
* seralize JSON as string to be used with JSON.parse on the client side
  to speed up browser JS parse time
* remove spaces in seralized JSON string to reduce payload size


> Tree view unusable for large DAGs
> ---------------------------------
>
>                 Key: AIRFLOW-6871
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6871
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: webserver
>    Affects Versions: 2.0.0
>            Reporter: QP Hou
>            Assignee: QP Hou
>            Priority: Major
>
> By default airflow loads 25 runs in the tree view. For our main DAG, it's just not usable because all we get is a 5xx error when hitting that page. Manually overriding number of runs to 15 makes the page to load again, but it's very slow and takes more than 1 minute to render.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)