You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Nam Ngo (JIRA)" <ji...@apache.org> on 2016/05/18 12:47:13 UTC

[jira] [Created] (AIRFLOW-129) Allow CELERYD_PREFETCH_MULTIPLIER to be configurable

Nam Ngo created AIRFLOW-129:
-------------------------------

             Summary: Allow CELERYD_PREFETCH_MULTIPLIER to be configurable
                 Key: AIRFLOW-129
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-129
             Project: Apache Airflow
          Issue Type: Improvement
          Components: celery
    Affects Versions: Airflow 1.7.0
            Reporter: Nam Ngo


Airflow needs to allow everyone to customise their prefetch limit. Some might have short running task and don't want the overhead of celery latency.

More on that here:
http://docs.celeryproject.org/en/latest/userguide/optimizing.html#optimizing-prefetch-limit



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)