You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Ash Berlin-Taylor (JIRA)" <ji...@apache.org> on 2018/03/05 23:03:00 UTC

[jira] [Resolved] (AIRFLOW-129) Allow CELERYD_PREFETCH_MULTIPLIER to be configurable

     [ https://issues.apache.org/jira/browse/AIRFLOW-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ash Berlin-Taylor resolved AIRFLOW-129.
---------------------------------------
       Resolution: Fixed
    Fix Version/s: Airflow 1.9.0

Not the nicest interface for configuring, but it is now possible to do without patching Airflow.

> Allow CELERYD_PREFETCH_MULTIPLIER to be configurable
> ----------------------------------------------------
>
>                 Key: AIRFLOW-129
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-129
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: celery
>    Affects Versions: Airflow 1.7.0
>            Reporter: Nam Ngo
>            Priority: Major
>             Fix For: Airflow 1.9.0
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Airflow needs to allow everyone to customise their prefetch limit. Some might have short running task and don't want the overhead of celery latency.
> More on that here:
> http://docs.celeryproject.org/en/latest/userguide/optimizing.html#optimizing-prefetch-limit



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)