[jira] [Created] (AIRFLOW-7060) EXECUTOR variable don't work in airflow.cfg in breeze shell

["JeonChangWon (Jira)" <ji...@apache.org>]

JeonChangWon created AIRFLOW-7060:
-------------------------------------

             Summary: EXECUTOR variable don't work in airflow.cfg in breeze shell
                 Key: AIRFLOW-7060
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-7060
             Project: Apache Airflow
          Issue Type: Bug
          Components: celery, cli
    Affects Versions: 2.0.0
            Reporter: JeonChangWon
            Assignee: JeonChangWon


Because the default breeze environment set `AIRFLOW__CORE__EXECUTOR` to `SequentialExecutor`, I can't change it to 'CeleryExecutor' through  airflow.cfg. 

Problem 1:
When I execute the command `airflow celery` in the breeze default environment, an error occurred. Because the `celery` command is added dynamically when the executor is set to CeleryExecutor by airflow.cfg.
{code:java}
if conf.get("core", "EXECUTOR") == ExecutorLoader.CELERY_EXECUTOR or BUILD_DOCS: ...{code}
 

So, I changed airflow.cfg file from SequentialExecutor to CeleryExecutor.

but above error still exist because of the environment variable `AIRFLOW__CORE__EXECUTOR` which is set to "SequentialExecutor" by 'backend-sqlite.yml' file

 

Problem 2:

In addition,   the below error can occur after I changed

`AIRFLOW__CORE__EXECUTOR` to "CeleryExecutor" because "Celery Executor" don't use sqlite.
{code:java}
Traceback (most recent call last): File "/usr/local/bin/airflow", line 7, in <module> exec(compile(f.read(), __file__, 'exec')) File "/opt/airflow/airflow/bin/airflow", line 26, in <module> from airflow.bin.cli import CLIFactory File "/opt/airflow/airflow/__init__.py", line 34, in <module> from airflow import settings File "/opt/airflow/airflow/settings.py", line 34, in <module> from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf # NOQA F401 File "/opt/airflow/airflow/configuration.py", line 602, in <module> conf.read(AIRFLOW_CONFIG) File "/opt/airflow/airflow/configuration.py", line 309, in read self._validate() File "/opt/airflow/airflow/configuration.py", line 185, in _validate self.get('core', 'executor'))) airflow.exceptions.AirflowConfigException: error: cannot use sqlite with the CeleryExecutor{code}
 

Solution 2:

 

I execute breeze command with an option like 
{code:java}
./breeze -i redis -b postgres{code}
And change airflow.cfg SequentialExecutor to CeleryExecutor.
I finally could use the celery command and the celery worker works well.

 

 

I will send PR for Problem 1



--
This message was sent by Atlassian Jira
(v8.3.4#803005)