You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/10/13 07:01:29 UTC
[GitHub] [airflow] TreyYi opened a new issue, #27025: I used the default docker-compose.yaml to setup CeleryExecutor, but SequentialExecutor is set in airflow.cfg
TreyYi opened a new issue, #27025:
URL: https://github.com/apache/airflow/issues/27025
### Apache Airflow version
2.4.1
### What happened
Hi all,
I used the default docker-compose.yaml to setup CeleryExecutor, but SequentialExecutor is set in airflow.cfg.
```
version: '3'
x-airflow-common:
&airflow-common
# In order to add custom dependencies or upgrade provider packages you can use your extended image.
# Comment the image line, place your Dockerfile in the directory where you placed the docker-compose.yaml
# and uncomment the "build" line below, Then run `docker-compose build` to build the images.
image: ${AIRFLOW_IMAGE_NAME:-apache/airflow:2.4.1}
# build: .
environment:
&airflow-common-env
AIRFLOW__CORE__EXECUTOR: CeleryExecutor
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:airflow@postgres/airflow
# For backward compatibility, with Airflow <2.3
AIRFLOW__CORE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:airflow@postgres/airflow
AIRFLOW__CELERY__RESULT_BACKEND: db+postgresql://airflow:airflow@postgres/airflow
AIRFLOW__CELERY__BROKER_URL: redis://:@redis:6379/0
AIRFLOW__CORE__FERNET_KEY: ''
AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION: 'true'
AIRFLOW__CORE__LOAD_EXAMPLES: 'false'
AIRFLOW__API__AUTH_BACKENDS: 'airflow.api.auth.backend.basic_auth'
_PIP_ADDITIONAL_REQUIREMENTS: ${_PIP_ADDITIONAL_REQUIREMENTS:-}
volumes:
- ./:/opt/airflow
user: "${AIRFLOW_UID:-50000}:0"
depends_on:
&airflow-common-depends-on
redis:
condition: service_healthy
postgres:
condition: service_healthy
```
I see that `AIRFLOW__CORE__EXECUTOR` is set to `CeleryExecutor`.
However, after running docker-compose up and check the airflow.cfg file in the container, executor is set to `SequentialExecutor`.
Did I miss something to configure in advance?
### What you think should happen instead
_No response_
### How to reproduce
_No response_
### Operating System
Amazon Linux 2
### Versions of Apache Airflow Providers
_No response_
### Deployment
Docker-Compose
### Deployment details
airflow version 2.4.1
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #27025: I used the default docker-compose.yaml to setup CeleryExecutor, but SequentialExecutor is set in airflow.cfg
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #27025:
URL: https://github.com/apache/airflow/issues/27025#issuecomment-1291434015
Precisely. veriables do not change files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #27025: I used the default docker-compose.yaml to setup CeleryExecutor, but SequentialExecutor is set in airflow.cfg
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #27025: I used the default docker-compose.yaml to setup CeleryExecutor, but SequentialExecutor is set in airflow.cfg
URL: https://github.com/apache/airflow/issues/27025
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hussein-awala commented on issue #27025: I used the default docker-compose.yaml to setup CeleryExecutor, but SequentialExecutor is set in airflow.cfg
Posted by GitBox <gi...@apache.org>.
hussein-awala commented on issue #27025:
URL: https://github.com/apache/airflow/issues/27025#issuecomment-1287523151
I don't think this is a bug, as the 'airflow.cfg' file contains the default configs and not the active configs.
Airflow [checks](https://github.com/apache/airflow/blob/main/airflow/configuration.py#L564-L588) if the conf exists in the environment variables first, if not, it tries to load it from file `airflow.cfg`.
You can create a shell on the scheduler container, and try to get the `core executor` conf:
```bash
airflow config get-value core executor
# and to get all the active conf
airflow config list
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org