[jira] [Commented] (AIRFLOW-2776) Tree view JSON is unnecessarily large

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16747037#comment-16747037 ] 

ASF subversion and git services commented on AIRFLOW-2776:
----------------------------------------------------------

Commit 7019a8d684909469b9dda8c793c5c8b6ebfbc4cc in airflow's branch refs/heads/v1-10-test from Abdul Nimeri
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=7019a8d ]

[AIRFLOW-2776] Compress tree view JSON

The tree view generates JSON that can be massive
for bigger DAGs,
up to 10s of MBs. The JSON is currently
prettified, which both
takes up more CPU time during serialization, and
slows down
everything else that uses it. Considering the JSON
is only
meant to be used programmatically, this is an easy
win

Closes #3620 from abdul-stripe/smaller-tree-view-
json


> Tree view JSON is unnecessarily large
> -------------------------------------
>
>                 Key: AIRFLOW-2776
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2776
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Abdul Nimeri
>            Assignee: Abdul Nimeri
>            Priority: Minor
>             Fix For: 2.0.0
>
>
> The tree view generates JSON that can be massive for bigger DAGs –– some of our tree views at stripe have 10s of MBs of JSON.
> The [generated JSON is prettified|https://github.com/apache/incubator-airflow/blob/52c745da71a6da798f7322956967b5e818b56e48/airflow/www/views.py#L1480], which both takes up more CPU time during serialization, as well as slowing everything else that uses it. We patched this on stripe's fork by removing all whitespace and had a ton of speedup for bigger tree views. Considering the JSON is only meant to be used programmatically, this is probably an easy win.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)