You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/03/15 09:33:02 UTC
[GitHub] [airflow] oursy opened a new issue #14796: Apple M1 Docker-compose up airflow-init produces airflow command error
oursy opened a new issue #14796:
URL: https://github.com/apache/airflow/issues/14796
<!--
Welcome to Apache Airflow! For a smooth issue process, try to answer the following questions.
Don't worry if they're not all applicable; just try to include what you can :-)
If you need to include code snippets or logs, please put them in fenced code
blocks. If they're super-long, please use the details tag like
<details><summary>super-long log</summary> lots of stuff </details>
Please delete these comment blocks before submitting the issue.
-->
<!--
IMPORTANT!!!
PLEASE CHECK "SIMILAR TO X EXISTING ISSUES" OPTION IF VISIBLE
NEXT TO "SUBMIT NEW ISSUE" BUTTON!!!
PLEASE CHECK IF THIS ISSUE HAS BEEN REPORTED PREVIOUSLY USING SEARCH!!!
Please complete the next sections or the issue will be closed.
These questions are the first thing we need to know to understand the context.
-->
**Apache Airflow version**: 2.0.1
**Environment**:
- **OS** (e.g. from /etc/os-release): macOS 11.2.2 M1
- **Kernel** (e.g. `uname -a`): Darwin Darwin Kernel Version 20.3.0
- **Install tools**: docker-compose
**What happened**:
Following the steps in the Quick start > Running Airflow in Docker page (https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html#initializing-environment):
1. Pulled down the pre-made docker-compose.yaml file (https://airflow.apache.org/docs/apache-airflow/2.0.1/docker-compose.yaml)
2. Ran docker-compose up airflow-init within the same directory as the docker-compose file
3. This command failed with: airflow command error: argument GROUP_OR_COMMAND: airflow upgradedb command, has been removed, please use airflow db upgrade, see help above.
<!-- (please include exact error messages if you can) -->
**What you expected to happen**:
The latest stable official airflow docker image to be pulled, and airflow-init to complete successfully.
<!-- What do you think went wrong? -->
**How to reproduce it**:
Follow the instructions per the quick start docs (https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html) to pull down the docker-compose.yaml file, then attempt to run docker-compose up airflow-init.
1. `mkdir ./dags ./logs ./plugins
echo -e "AIRFLOW_UID=$(id -u)\nAIRFLOW_GID=0" > .env`
2. `docker-compose up airflow-init`
<!---
As minimally and precisely as possible. Keep in mind we do not have access to your cluster or dags.
If you are using kubernetes, please attempt to recreate the issue using minikube or kind.
## Install minikube/kind
- Minikube https://minikube.sigs.k8s.io/docs/start/
- Kind https://kind.sigs.k8s.io/docs/user/quick-start/
If this is a UI bug, please provide a screenshot of the bug or a link to a youtube video of the bug in action
You can include images using the .md style of
To record a screencast, mac users can use QuickTime and then create an unlisted youtube video with the resulting .mov file.
--->
**Anything else we need to know**:
<!--
How often does this problem occur? Once? Every time etc?
Any relevant logs to include? Put them here in side a detail tag:
<details><summary>x.log</summary> lots of stuff </details>
-->
Below is the log information
```log
$ docker-compose up airflow-init
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Creating network "airflow_default" with the default driver
Creating volume "airflow_postgres-db-volume" with default driver
Pulling postgres (postgres:13)...
13: Pulling from library/postgres
706ee5d0a6b5: Pull complete
f0d0c8ce025c: Pull complete
d65206854bd5: Pull complete
e02ab81971e4: Pull complete
3e4d9f964bbd: Pull complete
22d34800f722: Pull complete
b580fcf09f2e: Pull complete
927098918ec9: Pull complete
135acccac104: Pull complete
e83b2f02fcec: Pull complete
5a4f5dd074b7: Pull complete
a7c009ac8442: Pull complete
f27c1fa930aa: Pull complete
97e124b88a98: Pull complete
Digest: sha256:c83014a2b46834ef6d17f64c8e4a70089901a8c0dee158f1ca5ccae032ea32e5
Status: Downloaded newer image for postgres:13
Pulling redis (redis:latest)...
latest: Pulling from library/redis
706ee5d0a6b5: Already exists
ca1c5a94560f: Pull complete
36348e11b8f8: Pull complete
b3e4d340a8f5: Pull complete
556088962a68: Pull complete
461f118c9c37: Pull complete
Digest: sha256:e97d506be34a39fa69f45eea846080d6985c2c9ee338c0d408c7ea4347f014a5
Status: Downloaded newer image for redis:latest
Pulling airflow-init (apache/airflow:2.0.1)...
2.0.1: Pulling from apache/airflow
45b42c59be33: Pull complete
f875e16ab19c: Pull complete
a4fb13de3215: Pull complete
f3dd9aaf319e: Pull complete
451a8f7ee895: Pull complete
e2d0adf49357: Pull complete
52856765f525: Pull complete
fd428150aaa6: Pull complete
b4fa7f6b9775: Pull complete
b34f0a9234cb: Pull complete
537882d8c1ac: Pull complete
dffdc965d748: Pull complete
a97e5a67cb46: Pull complete
c401f5284516: Pull complete
af49f5db1642: Pull complete
c61449868181: Pull complete
0b574505c97e: Pull complete
a65349ef32b2: Pull complete
Digest: sha256:2bf4fe559e9e699487a50d4e587358bb0b3c90b59c8fefeccc8a4fb2e83f3d09
Status: Downloaded newer image for apache/airflow:2.0.1
Creating airflow_postgres_1 ... done
Creating airflow_redis_1 ... done
Creating airflow_airflow-init_1 ... done
Attaching to airflow_airflow-init_1
airflow-init_1 | BACKEND=postgresql+psycopg2
airflow-init_1 | DB_HOST=postgres
airflow-init_1 | DB_PORT=5432
airflow-init_1 |
airflow-init_1 | [2021-03-15 09:24:55,179] {cli_action_loggers.py:105} WARNING - Failed to log action with (psycopg2.errors.UndefinedTable) relation "log" does not exist
airflow-init_1 | LINE 1: INSERT INTO log (dttm, dag_id, task_id, event, execution_dat...
airflow-init_1 | ^
airflow-init_1 |
airflow-init_1 | [SQL: INSERT INTO log (dttm, dag_id, task_id, event, execution_date, owner, extra) VALUES (%(dttm)s, %(dag_id)s, %(task_id)s, %(event)s, %(execution_date)s, %(owner)s, %(extra)s) RETURNING log.id]
airflow-init_1 | [parameters: {'dttm': datetime.datetime(2021, 3, 15, 9, 24, 55, 145656, tzinfo=Timezone('UTC')), 'dag_id': None, 'task_id': None, 'event': 'cli_upgradedb', 'execution_date': None, 'owner': 'default', 'extra': '{"host_name": "1c70efb17222", "full_command": "[\'/home/airflow/.local/bin/airflow\', \'db\', \'upgrade\']"}'}]
airflow-init_1 | (Background on this error at: http://sqlalche.me/e/13/f405)
airflow-init_1 | DB: postgresql+psycopg2://airflow:***@postgres/airflow
airflow-init_1 | [2021-03-15 09:24:56,100] {db.py:674} INFO - Creating tables
airflow-init_1 | INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
airflow-init_1 | INFO [alembic.runtime.migration] Will assume transactional DDL.
airflow-init_1 | WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | qemu: uncaught target signal 4 (Illegal instruction) - core dumped
airflow-init_1 | /entrypoint: line 192: 26 Illegal instruction airflow db upgrade
airflow-init_1 |
airflow-init_1 | airflow command error: argument GROUP_OR_COMMAND: `airflow upgradedb` command, has been removed, please use `airflow db upgrade`, see help above.
airflow-init_1 | usage: airflow [-h] GROUP_OR_COMMAND ...
airflow-init_1 |
airflow-init_1 | positional arguments:
airflow-init_1 | GROUP_OR_COMMAND
airflow-init_1 |
airflow-init_1 | Groups:
airflow-init_1 | celery Celery components
airflow-init_1 | config View configuration
airflow-init_1 | connections Manage connections
airflow-init_1 | dags Manage DAGs
airflow-init_1 | db Database operations
airflow-init_1 | kubernetes Tools to help run the KubernetesExecutor
airflow-init_1 | pools Manage pools
airflow-init_1 | providers Display providers
airflow-init_1 | roles Manage roles
airflow-init_1 | tasks Manage tasks
airflow-init_1 | users Manage users
airflow-init_1 | variables Manage variables
airflow-init_1 |
airflow-init_1 | Commands:
airflow-init_1 | cheat-sheet Display cheat sheet
airflow-init_1 | info Show information about current Airflow and environment
airflow-init_1 | kerberos Start a kerberos ticket renewer
airflow-init_1 | plugins Dump information about loaded plugins
airflow-init_1 | rotate-fernet-key
airflow-init_1 | Rotate encrypted connection credentials and variables
airflow-init_1 | scheduler Start a scheduler instance
airflow-init_1 | sync-perm Update permissions for existing roles and DAGs
airflow-init_1 | version Show the version
airflow-init_1 | webserver Start a Airflow webserver instance
airflow-init_1 |
airflow-init_1 | optional arguments:
airflow-init_1 | -h, --help show this help message and exit
airflow-init_1 | [2021-03-15 09:25:25,537] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | qemu: uncaught target signal 4 (Illegal instruction) - core dumped
airflow-init_1 | /entrypoint: line 129: 53 Illegal instruction airflow users create --username "${_AIRFLOW_WWW_USER_USERNAME="admin"}" --firstname "${_AIRFLOW_WWW_USER_FIRSTNAME="Airflow"}" --lastname "${_AIRFLOW_WWW_USER_LASTNME="Admin"}" --email "${_AIRFLOW_WWW_USER_EMAIL="airflowadmin@example.com"}" --role "${_AIRFLOW_WWW_USER_ROLE="Admin"}" --password "${local_password}"
airflow-init_1 |
airflow-init_1 | airflow command error: argument GROUP_OR_COMMAND: `airflow create_user` command, has been removed, please use `airflow users create`, see help above.
airflow-init_1 | usage: airflow [-h] GROUP_OR_COMMAND ...
airflow-init_1 |
airflow-init_1 | positional arguments:
airflow-init_1 | GROUP_OR_COMMAND
airflow-init_1 |
airflow-init_1 | Groups:
airflow-init_1 | celery Celery components
airflow-init_1 | config View configuration
airflow-init_1 | connections Manage connections
airflow-init_1 | dags Manage DAGs
airflow-init_1 | db Database operations
airflow-init_1 | kubernetes Tools to help run the KubernetesExecutor
airflow-init_1 | pools Manage pools
airflow-init_1 | providers Display providers
airflow-init_1 | roles Manage roles
airflow-init_1 | tasks Manage tasks
airflow-init_1 | users Manage users
airflow-init_1 | variables Manage variables
airflow-init_1 |
airflow-init_1 | Commands:
airflow-init_1 | cheat-sheet Display cheat sheet
airflow-init_1 | info Show information about current Airflow and environment
airflow-init_1 | kerberos Start a kerberos ticket renewer
airflow-init_1 | plugins Dump information about loaded plugins
airflow-init_1 | rotate-fernet-key
airflow-init_1 | Rotate encrypted connection credentials and variables
airflow-init_1 | scheduler Start a scheduler instance
airflow-init_1 | sync-perm Update permissions for existing roles and DAGs
airflow-init_1 | version Show the version
airflow-init_1 | webserver Start a Airflow webserver instance
airflow-init_1 |
airflow-init_1 | optional arguments:
airflow-init_1 | -h, --help show this help message and exit
airflow-init_1 | 2.0.1
airflow_airflow-init_1 exited with code 0
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #14796: Apple M1 Docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #14796:
URL: https://github.com/apache/airflow/issues/14796
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #14796: Apple M1 Docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #14796:
URL: https://github.com/apache/airflow/issues/14796#issuecomment-799267116
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #14796: Apple M1 Docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #14796:
URL: https://github.com/apache/airflow/issues/14796#issuecomment-799340809
> mkdir ./dags ./logs ./plugins echo -e "AIRFLOW_UID=$(id -u)\nAIRFLOW_GID=0" > .env
This command should only be run on Linux.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14796: Apple M1 Docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #14796:
URL: https://github.com/apache/airflow/issues/14796#issuecomment-799377307
Airflow has no support for M1 - this M1 is an ARM device and we do not support ARM in docker images. There are some anecdotal evidences of people who were able to install Airflow on M1 device but it is neither tested nor supported.
If you would like to use Airflow with M1. I suggest to open a feature request for that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org