You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2018/09/02 17:57:01 UTC

[jira] [Commented] (AIRFLOW-2615) Webserver parent not using cached app

    [ https://issues.apache.org/jira/browse/AIRFLOW-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16601288#comment-16601288 ] 

Apache Spark commented on AIRFLOW-2615:
---------------------------------------

User 'yrqls21' has created a pull request for this issue:
https://github.com/apache/incubator-airflow/pull/3506

> Webserver parent not using cached app
> -------------------------------------
>
>                 Key: AIRFLOW-2615
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2615
>             Project: Apache Airflow
>          Issue Type: Bug
>    Affects Versions: 1.10.0
>            Reporter: Kevin Yang
>            Assignee: Kevin Yang
>            Priority: Major
>             Fix For: 2.0.0
>
>
> From what I can tell, the app cached [here|https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L790] attempt to cache the app for later use-likely to be for the expensive DagBag() creation. Before I dive into the webserver parsing everything in one process problem, I was hoping this cached app would save me sometime. However it seems to me that every subprocess spun up by gunicorn is trying to create the DagBag() right after they've been created--make sense to me since we didn't share the cached app to the subprocess( doubt we can). If what I observed is true, why do we cache the app at all in the parent process?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)