[GitHub] [airflow] nicnguyen3103 commented on issue #11225: celery executer with SQS and predefined_queues

[GitBox <gi...@apache.org>]

nicnguyen3103 commented on issue #11225:
URL: https://github.com/apache/airflow/issues/11225#issuecomment-997587302


   For anyone who is still struggling with the import path of celery config here is how to import it: 
   
   1. Create a celery_config_extend.py and put it in AIRFLOW_HOME folder. By default, airflow will have this path added in the sys.path https://airflow.apache.org/docs/apache-airflow/stable/modules_management.html.
   2. Setup the script by like @charliegriefer or other comments above. If you want to keep the original config and extends some extra config you can write the config like this, it will keep the original DEFAULT_CELERY_CONFIG and add/replace the key:value when applicable
   ```
   from airflow.config_templates.default_celery import DEFAULT_CELERY_CONFIG
   
   CELERY_CONFIG = dict(
       DEFAULT_CELERY_CONFIG,
       **{
           'key1': value1, # add new key value pair or replace the existing one if key is in DEFAULT_CELERY_CONFIG
           ....
       }
   )
   ```
   3. Set the environment variable `AIRFLOW__CELERY__CELERY_CONFIG_OPTIONS=airflow.celery_default_config.CELERY_CONFIG` to import your new config
   
   Airflow now should pick up your new celery config and apply it to the worker
    
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org