You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Jeremiah Lowin (JIRA)" <ji...@apache.org> on 2017/03/17 22:04:42 UTC

[jira] [Created] (AIRFLOW-1005) Speed up Airflow startup time

Jeremiah Lowin created AIRFLOW-1005:
---------------------------------------

             Summary: Speed up Airflow startup time
                 Key: AIRFLOW-1005
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1005
             Project: Apache Airflow
          Issue Type: Improvement
    Affects Versions: 1.8.0
            Reporter: Jeremiah Lowin
            Assignee: Jeremiah Lowin
            Priority: Minor


Airflow takes approximately 1 second to import. It turns out that more than half the time is spend doing two things: importing Cryptography to create Fernet keys and importing Alembic. 

The first Cryptography import is in configuration.py and is only necessary if Airflow is generating a new airflow.cfg file (but currently gets run every time). Therefore it can be easily deferred.

The second is in models.py to check if encryption is turned on. This can also be deferred until encryption checks are actually needed.

Alembic is always imported even though it is only needed for running initdb() and upgradedb(). It can be lazily imported inside those functions.

These simple changes reduce Airflow's startup time by half on my machine.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)