You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Josh Carp (JIRA)" <ji...@apache.org> on 2018/12/28 04:32:00 UTC
[jira] [Commented] (AIRFLOW-3562) Remove DagBag dependency inside
the webserver
[ https://issues.apache.org/jira/browse/AIRFLOW-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730032#comment-16730032 ]
Josh Carp commented on AIRFLOW-3562:
------------------------------------
I think most uses of the `DagBag` in the webserver will be easy to replace with the database, but what about rendering code?
> Remove DagBag dependency inside the webserver
> ---------------------------------------------
>
> Key: AIRFLOW-3562
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3562
> Project: Apache Airflow
> Issue Type: Improvement
> Components: webserver
> Affects Versions: 1.10.0, 1.10.1
> Reporter: Peter van 't Hof
> Assignee: Peter van 't Hof
> Priority: Critical
> Fix For: 2.0.0
>
>
> Currently the webserver does depend on the dag directory and reparsing the dag files. Because this happens with gunicorn each process has there own DagBag. The means the webserver is statefull and depending which process is assigned a request the information can be changed. If all information will come from the database this will not happen. The only process that should add dag/dagruns is the scheduler.
> The webserver should have enough on only the database. Currently not all information is already in the database. This could be fixable, already started this in AIRFLOW-3561.
> I do notice this might give some conflicts with the brace change that is coming. Any suggestion how to approach this?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)