You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/22 18:44:30 UTC
[GitHub] [airflow] chrisrfu opened a new issue #14365: docker-compose up airflow-init produces airflow command error
chrisrfu opened a new issue #14365:
URL: https://github.com/apache/airflow/issues/14365
**Apache Airflow version**: 2.0.1
**Environment**:
- OS (e.g. from /etc/os-release): macOS 10.15.7
- Kernel (e.g. uname -a): Darwin Kernel Version 19.6.0
- Install tools: docker-compose
**What happened**:
Following the steps in the _Quick start > Running Airflow in Docker_ page (https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html#initializing-environment):
1. Pulled down the pre-made docker-compose.yaml file (https://airflow.apache.org/docs/apache-airflow/2.0.1/docker-compose.yaml)
2. Ran `docker-compose up airflow-init` within the same directory as the docker-compose file
3. This command failed with: airflow command error: argument GROUP_OR_COMMAND: `airflow upgradedb` command, has been removed, please use `airflow db upgrade`, see help above.
**What you expected to happen**:
The latest stable official airflow docker image to be pulled, and airflow-init to complete successfully.
**How to reproduce it**:
For the exact traceback block below, I stopped and removed all of my docker images:
1. `docker stop (docker ps -q)`
2. `docker rm (docker ps -a -q)`
3. `docker images -a -q | xargs docker rmi -f`
4. Follow the instructions per the quick start docs (https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html) to pull down the docker-compose.yaml file, then attempt to run `docker-compose up airflow-init`.
**Anything else we need to know**:
This is very similar issue to #14143
Below is the traceback block
> ~/C/P/docker-apache-airflow docker-compose up airflow-init conda-env
Pulling postgres (postgres:13)...
13: Pulling from library/postgres
45b42c59be33: Already exists
40adec129f1a: Pull complete
b4c431d00c78: Pull complete
2696974e2815: Pull complete
564b77596399: Pull complete
5044045cf6f2: Pull complete
d736e67e6ac3: Pull complete
390c1c9a5ae4: Pull complete
c0e62f172284: Pull complete
ebcdc659c5bf: Pull complete
29be22cb3acc: Pull complete
f63c47038e66: Pull complete
77a0c198cde5: Pull complete
c8752d5b785c: Pull complete
Digest: sha256:5cfcfb0516ddd317109e2746a462b1dc398d152936640a2b69284629f48dc288
Status: Downloaded newer image for postgres:13
Pulling redis (redis:latest)...
latest: Pulling from library/redis
45b42c59be33: Already exists
5ce2e937bf62: Pull complete
2a031498ff58: Pull complete
78f77a50d3fe: Pull complete
60553ab7cb08: Pull complete
68240431d2fc: Pull complete
Digest: sha256:f29bcfb891678a0c6a0fc5da0b32ce1ac685af87c0f3aa9327e562da8d3f3b88
Status: Downloaded newer image for redis:latest
Pulling airflow-init (apache/airflow:2.0.1)...
2.0.1: Pulling from apache/airflow
45b42c59be33: Already exists
f875e16ab19c: Already exists
a4fb13de3215: Pull complete
f3dd9aaf319e: Pull complete
451a8f7ee895: Pull complete
e2d0adf49357: Pull complete
52856765f525: Pull complete
fd428150aaa6: Pull complete
b4fa7f6b9775: Pull complete
b34f0a9234cb: Pull complete
537882d8c1ac: Pull complete
dffdc965d748: Pull complete
a97e5a67cb46: Pull complete
c401f5284516: Pull complete
af49f5db1642: Pull complete
c61449868181: Pull complete
0b574505c97e: Pull complete
a65349ef32b2: Pull complete
Digest: sha256:2bf4fe559e9e699487a50d4e587358bb0b3c90b59c8fefeccc8a4fb2e83f3d09
Status: Downloaded newer image for apache/airflow:2.0.1
Creating docker-apache-airflow_postgres_1 ... done
Creating docker-apache-airflow_redis_1 ... done
Creating docker-apache-airflow_airflow-init_1 ... done
Attaching to docker-apache-airflow_airflow-init_1
airflow-init_1 | BACKEND=postgresql+psycopg2
airflow-init_1 | DB_HOST=postgres
airflow-init_1 | DB_PORT=5432
airflow-init_1 |
airflow-init_1 | DB: postgresql+psycopg2://airflow:***@postgres/airflow
airflow-init_1 | [2021-02-22 18:19:41,935] {db.py:674} INFO - Creating tables
airflow-init_1 | INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
airflow-init_1 | INFO [alembic.runtime.migration] Will assume transactional DDL.
airflow-init_1 | WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | Traceback (most recent call last):
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 162, in _catch_revision_errors
airflow-init_1 | yield
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 356, in _upgrade_revs
airflow-init_1 | revs = list(revs)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 904, in _iterate_revisions
airflow-init_1 | requested_lowers = self.get_revisions(lower)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 455, in get_revisions
airflow-init_1 | return sum([self.get_revisions(id_elem) for id_elem in id_], ())
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 455, in <listcomp>
airflow-init_1 | return sum([self.get_revisions(id_elem) for id_elem in id_], ())
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 460, in get_revisions
airflow-init_1 | for rev_id in resolved_id
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 460, in <genexpr>
airflow-init_1 | for rev_id in resolved_id
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 536, in _revision_for_ident
airflow-init_1 | resolved_id,
airflow-init_1 | alembic.script.revision.ResolutionError: No such revision or branch '449b4072c2da'
airflow-init_1 |
airflow-init_1 | The above exception was the direct cause of the following exception:
airflow-init_1 |
airflow-init_1 | Traceback (most recent call last):
airflow-init_1 | File "/home/airflow/.local/bin/airflow", line 8, in <module>
airflow-init_1 | sys.exit(main())
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/__main__.py", line 40, in main
airflow-init_1 | args.func(args)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/cli/cli_parser.py", line 48, in command
airflow-init_1 | return func(*args, **kwargs)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/cli.py", line 89, in wrapper
airflow-init_1 | return f(*args, **kwargs)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/cli/commands/db_command.py", line 48, in upgradedb
airflow-init_1 | db.upgradedb()
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/db.py", line 684, in upgradedb
airflow-init_1 | command.upgrade(config, 'heads')
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/command.py", line 294, in upgrade
airflow-init_1 | script.run_env()
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 481, in run_env
airflow-init_1 | util.load_python_file(self.dir, "env.py")
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
airflow-init_1 | module = load_module_py(module_id, path)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/util/compat.py", line 182, in load_module_py
airflow-init_1 | spec.loader.exec_module(module)
airflow-init_1 | File "<frozen importlib._bootstrap_external>", line 678, in exec_module
airflow-init_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/migrations/env.py", line 108, in <module>
airflow-init_1 | run_migrations_online()
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online
airflow-init_1 | context.run_migrations()
airflow-init_1 | File "<string>", line 8, in run_migrations
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
airflow-init_1 | self.get_context().run_migrations(**kw)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/runtime/migration.py", line 548, in run_migrations
airflow-init_1 | for step in self._migrations_fn(heads, self):
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/command.py", line 283, in upgrade
airflow-init_1 | return script._upgrade_revs(revision, rev)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 361, in _upgrade_revs
airflow-init_1 | for script in reversed(list(revs))
airflow-init_1 | File "/usr/local/lib/python3.6/contextlib.py", line 99, in __exit__
airflow-init_1 | self.gen.throw(type, value, traceback)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 194, in _catch_revision_errors
airflow-init_1 | compat.raise_(util.CommandError(resolution), from_=re)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/util/compat.py", line 294, in raise_
airflow-init_1 | raise exception
airflow-init_1 | alembic.util.exc.CommandError: Can't locate revision identified by '449b4072c2da'
airflow-init_1 | usage: airflow [-h] GROUP_OR_COMMAND ...
airflow-init_1 |
airflow-init_1 | positional arguments:
airflow-init_1 | GROUP_OR_COMMAND
airflow-init_1 |
airflow-init_1 | Groups:
airflow-init_1 | celery Celery components
airflow-init_1 | config View configuration
airflow-init_1 | connections Manage connections
airflow-init_1 | dags Manage DAGs
airflow-init_1 | db Database operations
airflow-init_1 | kubernetes Tools to help run the KubernetesExecutor
airflow-init_1 | pools Manage pools
airflow-init_1 | providers Display providers
airflow-init_1 | roles Manage roles
airflow-init_1 | tasks Manage tasks
airflow-init_1 | users Manage users
airflow-init_1 | variables Manage variables
airflow-init_1 |
airflow-init_1 | Commands:
airflow-init_1 | cheat-sheet Display cheat sheet
airflow-init_1 | info Show information about current Airflow and environment
airflow-init_1 | kerberos Start a kerberos ticket renewer
airflow-init_1 | plugins Dump information about loaded plugins
airflow-init_1 | rotate-fernet-key
airflow-init_1 | Rotate encrypted connection credentials and variables
airflow-init_1 | scheduler Start a scheduler instance
airflow-init_1 | sync-perm Update permissions for existing roles and DAGs
airflow-init_1 | version Show the version
airflow-init_1 | webserver Start a Airflow webserver instance
airflow-init_1 |
airflow-init_1 | optional arguments:
airflow-init_1 | -h, --help show this help message and exit
airflow-init_1 |
airflow-init_1 | airflow command error: argument GROUP_OR_COMMAND: `airflow upgradedb` command, has been removed, please use `airflow db upgrade`, see help above.
airflow-init_1 | [2021-02-22 18:19:50,692] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-22 18:19:51,231] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-22 18:19:52,417] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-22 18:19:52,514] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | airflow already exist in the db
airflow-init_1 | 2.0.1
docker-apache-airflow_airflow-init_1 exited with code 0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jhtimmins commented on issue #14365: docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
jhtimmins commented on issue #14365:
URL: https://github.com/apache/airflow/issues/14365#issuecomment-783790768
@chrisrfu It's hard to tell exactly what's going wrong here, but it appears that there's some type of mismatch between old/new data or old/new versions.
For example, `alembic.script.revision.ResolutionError: No such revision or branch '449b4072c2da'` seems to refer to an existing alembic revision number. It could also be that you need to upgrade docker.
How are you using `conda-env`? Is that an existing environment file?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #14365: docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #14365:
URL: https://github.com/apache/airflow/issues/14365#issuecomment-785062689
Looks like you have tried installing different version of Airflow, maybe you tried to install the image for Airflow Master and then rolled back to 2.0.1
Run
```
docker-compose down --volumes --rmi all
```
and then re-run your original commands.
I can tell you tried with Master because it is complaining about alembic version: `449b4072c2da` -- which is only available in Master.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chrisrfu closed issue #14365: docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
chrisrfu closed issue #14365:
URL: https://github.com/apache/airflow/issues/14365
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chrisrfu commented on issue #14365: docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
chrisrfu commented on issue #14365:
URL: https://github.com/apache/airflow/issues/14365#issuecomment-784794697
I am not on an M1 chip
MacBook Pro (16-inch, 2019)
2.3 GHz 8-Core Intel Core i9
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] marcbizal commented on issue #14365: docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
marcbizal commented on issue #14365:
URL: https://github.com/apache/airflow/issues/14365#issuecomment-784572647
I have an identical issue.
**Environment:**
- OS: macOS 11.2.1
- Kernel: Darwin Kernel Version 20.3.0
It's worth noting that this is on an M1 chip for me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chrisrfu commented on issue #14365: docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
chrisrfu commented on issue #14365:
URL: https://github.com/apache/airflow/issues/14365#issuecomment-785267821
It worked!
I increased the memory from 2GB to 4GB and kept CPU at 8, Swap at 1GB, and Disk image size at 59.6GB (my default Docker settings)
Thank you so much for your help!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chrisrfu commented on issue #14365: docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
chrisrfu commented on issue #14365:
URL: https://github.com/apache/airflow/issues/14365#issuecomment-783923512
docker is updated to the latest 3.1.0 version. my bad, `conda-env` is a typo
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chrisrfu commented on issue #14365: docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
chrisrfu commented on issue #14365:
URL: https://github.com/apache/airflow/issues/14365#issuecomment-785214213
There's no more
```
airflow-init_1 | airflow already exist in the db
```
which was solved by:
instead of `docker stop (docker ps -q)`, `docker rm (docker ps -a -q)`, and then `docker-compose down --volumes --rm all`
just going with `docker-compose down --volumes --rm all`and then re-running my original commands
> Creating network "docker-apache-airflow-201_default" with the default driver
Creating volume "docker-apache-airflow-201_postgres-db-volume" with default driver
Pulling postgres (postgres:13)...
13: Pulling from library/postgres
45b42c59be33: Already exists
40adec129f1a: Pull complete
b4c431d00c78: Pull complete
2696974e2815: Pull complete
564b77596399: Pull complete
5044045cf6f2: Pull complete
d736e67e6ac3: Pull complete
390c1c9a5ae4: Pull complete
c0e62f172284: Pull complete
ebcdc659c5bf: Pull complete
29be22cb3acc: Pull complete
f63c47038e66: Pull complete
77a0c198cde5: Pull complete
c8752d5b785c: Pull complete
Digest: sha256:5cfcfb0516ddd317109e2746a462b1dc398d152936640a2b69284629f48dc288
Status: Downloaded newer image for postgres:13
Pulling redis (redis:latest)...
latest: Pulling from library/redis
45b42c59be33: Already exists
5ce2e937bf62: Pull complete
2a031498ff58: Pull complete
7407cbab5258: Pull complete
c2d6c3b35be8: Pull complete
1997ce073dcd: Pull complete
Digest: sha256:26bd5f3d2bbfb21d0e4f03d642570e63c8776d2a9e5963e4057280fb3b4e3e29
Status: Downloaded newer image for redis:latest
Pulling airflow-init (apache/airflow:2.0.1)...
2.0.1: Pulling from apache/airflow
45b42c59be33: Already exists
f875e16ab19c: Already exists
a4fb13de3215: Pull complete
f3dd9aaf319e: Pull complete
451a8f7ee895: Pull complete
e2d0adf49357: Pull complete
52856765f525: Pull complete
fd428150aaa6: Pull complete
b4fa7f6b9775: Pull complete
b34f0a9234cb: Pull complete
537882d8c1ac: Pull complete
dffdc965d748: Pull complete
a97e5a67cb46: Pull complete
c401f5284516: Pull complete
af49f5db1642: Pull complete
c61449868181: Pull complete
0b574505c97e: Pull complete
a65349ef32b2: Pull complete
Digest: sha256:2bf4fe559e9e699487a50d4e587358bb0b3c90b59c8fefeccc8a4fb2e83f3d09
Status: Downloaded newer image for apache/airflow:2.0.1
Creating docker-apache-airflow-201_postgres_1 ... done
Creating docker-apache-airflow-201_redis_1 ... done
Creating docker-apache-airflow-201_airflow-init_1 ... done
Attaching to docker-apache-airflow-201_airflow-init_1
airflow-init_1 | BACKEND=postgresql+psycopg2
airflow-init_1 | DB_HOST=postgres
airflow-init_1 | DB_PORT=5432
airflow-init_1 |
airflow-init_1 | [2021-02-24 16:23:54,595] {cli_action_loggers.py:105} WARNING - Failed to log action with (psycopg2.errors.UndefinedTable) relation "log" does not exist
airflow-init_1 | LINE 1: INSERT INTO log (dttm, dag_id, task_id, event, execution_dat...
airflow-init_1 | ^
airflow-init_1 |
airflow-init_1 | [SQL: INSERT INTO log (dttm, dag_id, task_id, event, execution_date, owner, extra) VALUES (%(dttm)s, %(dag_id)s, %(task_id)s, %(event)s, %(execution_date)s, %(owner)s, %(extra)s) RETURNING log.id]
airflow-init_1 | [parameters: {'dttm': datetime.datetime(2021, 2, 24, 16, 23, 54, 583379, tzinfo=Timezone('UTC')), 'dag_id': None, 'task_id': None, 'event': 'cli_upgradedb', 'execution_date': None, 'owner': 'airflow', 'extra': '{"host_name": "93ea84d65b76", "full_command": "[\'/home/airflow/.local/bin/airflow\', \'db\', \'upgrade\']"}'}]
airflow-init_1 | (Background on this error at: http://sqlalche.me/e/13/f405)
airflow-init_1 | DB: postgresql+psycopg2://airflow:***@postgres/airflow
airflow-init_1 | [2021-02-24 16:23:54,814] {db.py:674} INFO - Creating tables
airflow-init_1 | INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
airflow-init_1 | INFO [alembic.runtime.migration] Will assume transactional DDL.
airflow-init_1 | WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade -> e3a246e0dc1, current schema
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade e3a246e0dc1 -> 1507a7289a2f, create is_encrypted
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 1507a7289a2f -> 13eb55f81627, maintain history for compatibility with earlier migrations
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 13eb55f81627 -> 338e90f54d61, More logging into task_instance
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 338e90f54d61 -> 52d714495f0, job_id indices
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 52d714495f0 -> 502898887f84, Adding extra to Log
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 502898887f84 -> 1b38cef5b76e, add dagrun
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 1b38cef5b76e -> 2e541a1dcfed, task_duration
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 2e541a1dcfed -> 40e67319e3a9, dagrun_config
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 40e67319e3a9 -> 561833c1c74b, add password column to user
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 561833c1c74b -> 4446e08588, dagrun start end
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 4446e08588 -> bbc73705a13e, Add notification_sent column to sla_miss
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade bbc73705a13e -> bba5a7cfc896, Add a column to track the encryption state of the 'Extra' field in connection
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade bba5a7cfc896 -> 1968acfc09e3, add is_encrypted column to variable table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 1968acfc09e3 -> 2e82aab8ef20, rename user table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 2e82aab8ef20 -> 211e584da130, add TI state index
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 211e584da130 -> 64de9cddf6c9, add task fails journal table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 64de9cddf6c9 -> f2ca10b85618, add dag_stats table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade f2ca10b85618 -> 4addfa1236f1, Add fractional seconds to mysql tables
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 4addfa1236f1 -> 8504051e801b, xcom dag task indices
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 8504051e801b -> 5e7d17757c7a, add pid field to TaskInstance
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 5e7d17757c7a -> 127d2bf2dfa7, Add dag_id/state index on dag_run table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 127d2bf2dfa7 -> cc1e65623dc7, add max tries column to task instance
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade cc1e65623dc7 -> bdaa763e6c56, Make xcom value column a large binary
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade bdaa763e6c56 -> 947454bf1dff, add ti job_id index
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 947454bf1dff -> d2ae31099d61, Increase text size for MySQL (not relevant for other DBs' text types)
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade d2ae31099d61 -> 0e2a74e0fc9f, Add time zone awareness
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade d2ae31099d61 -> 33ae817a1ff4, kubernetes_resource_checkpointing
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 33ae817a1ff4 -> 27c6a30d7c24, kubernetes_resource_checkpointing
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 27c6a30d7c24 -> 86770d1215c0, add kubernetes scheduler uniqueness
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 86770d1215c0, 0e2a74e0fc9f -> 05f30312d566, merge heads
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 05f30312d566 -> f23433877c24, fix mysql not null constraint
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade f23433877c24 -> 856955da8476, fix sqlite foreign key
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 856955da8476 -> 9635ae0956e7, index-faskfail
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 9635ae0956e7 -> dd25f486b8ea, add idx_log_dag
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade dd25f486b8ea -> bf00311e1990, add index to taskinstance
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 9635ae0956e7 -> 0a2a5b66e19d, add task_reschedule table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 0a2a5b66e19d, bf00311e1990 -> 03bc53e68815, merge_heads_2
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 03bc53e68815 -> 41f5f12752f8, add superuser field
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 41f5f12752f8 -> c8ffec048a3b, add fields to dag
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade c8ffec048a3b -> dd4ecb8fbee3, Add schedule interval to dag
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade dd4ecb8fbee3 -> 939bb1e647c8, task reschedule fk on cascade delete
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 939bb1e647c8 -> 6e96a59344a4, Make TaskInstance.pool not nullable
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 6e96a59344a4 -> d38e04c12aa2, add serialized_dag table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade d38e04c12aa2 -> b3b105409875, add root_dag_id to DAG
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 6e96a59344a4 -> 74effc47d867, change datetime to datetime2(6) on MSSQL tables
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 939bb1e647c8 -> 004c1210f153, increase queue name size limit
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade c8ffec048a3b -> a56c9515abdc, Remove dag_stat table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade a56c9515abdc, 004c1210f153, 74effc47d867, b3b105409875 -> 08364691d074, Merge the four heads back together
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 08364691d074 -> fe461863935f, increase_length_for_connection_password
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade fe461863935f -> 7939bcff74ba, Add DagTags table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 7939bcff74ba -> a4c2fd67d16b, add pool_slots field to task_instance
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade a4c2fd67d16b -> 852ae6c715af, Add RenderedTaskInstanceFields table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 852ae6c715af -> 952da73b5eff, add dag_code table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 952da73b5eff -> a66efa278eea, Add Precision to execution_date in RenderedTaskInstanceFields table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade a66efa278eea -> da3f683c3a5a, Add dag_hash Column to serialized_dag table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade da3f683c3a5a -> 92c57b58940d, Create FAB Tables
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 92c57b58940d -> 03afc6b6f902, Increase length of FAB ab_view_menu.name column
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 03afc6b6f902 -> cf5dc11e79ad, drop_user_and_chart
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade cf5dc11e79ad -> bbf4a7ad0465, Remove id column from xcom
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade bbf4a7ad0465 -> b25a55525161, Increase length of pool name
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade b25a55525161 -> 3c20cacc0044, Add DagRun run_type
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 3c20cacc0044 -> 8f966b9c467a, Set conn_type as non-nullable
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 8f966b9c467a -> 8d48763f6d53, add unique constraint to conn_id
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 8d48763f6d53 -> e38be357a868, Add sensor_instance table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade e38be357a868 -> b247b1e3d1ed, Add queued by Job ID to TI
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade b247b1e3d1ed -> e1a11ece99cc, Add external executor ID to TI
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade e1a11ece99cc -> bef4f3d11e8b, Drop KubeResourceVersion and KubeWorkerId
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade bef4f3d11e8b -> 98271e7606e2, Add scheduling_decision to DagRun and DAG
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 98271e7606e2 -> 52d53670a240, fix_mssql_exec_date_rendered_task_instance_fields_for_MSSQL
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 52d53670a240 -> 364159666cbd, Add creating_job_id to DagRun table
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 364159666cbd -> 45ba3f1493b9, add-k8s-yaml-to-rendered-templates
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 45ba3f1493b9 -> 849da589634d, Prefix DAG permissions.
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 849da589634d -> 2c6edca13270, Resource based permissions.
airflow-init_1 | [2021-02-24 16:23:59,385] {manager.py:727} WARNING - No user yet created, use flask fab command to do it.
airflow-init_1 | [2021-02-24 16:23:59,858] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-24 16:23:59,930] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 2c6edca13270 -> 61ec73d9401f, Add description field to connection
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 61ec73d9401f -> 64a7d6477aae, fix description field in connection to be text
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade 64a7d6477aae -> e959f08ac86c, Change field in DagCode to MEDIUMTEXT for MySql
airflow-init_1 | INFO [alembic.runtime.migration] Running upgrade e959f08ac86c -> 82b7c48c147f, Remove can_read permission on config resource for User and Viewer role
airflow-init_1 | [2021-02-24 16:24:02,276] {manager.py:727} WARNING - No user yet created, use flask fab command to do it.
airflow-init_1 | [2021-02-24 16:24:02,611] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-24 16:24:02,688] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | Upgrades done
airflow-init_1 | [2021-02-24 16:24:06,994] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-24 16:24:07,445] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-24 16:24:07,534] {manager.py:727} WARNING - No user yet created, use flask fab command to do it.
airflow-init_1 | [2021-02-24 16:24:08,633] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-24 16:24:08,709] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | Admin user airflow created
airflow-init_1 | 2.0.1
docker-apache-airflow-201_airflow-init_1 exited with code 0
However the Exceptions still exists and as I run `docker-compose up`, the webserver fails
Below is a small snippet of the many retries
> docker-apache-airflow-201_airflow-webserver_1 exited with code 1
airflow-webserver_1 |
airflow-webserver_1 | [2021-02-24 16:41:37,569] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-webserver_1 | [2021-02-24 16:41:38,063] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-webserver_1 | ____________ _____________
airflow-webserver_1 | ____ |__( )_________ __/__ /________ __
airflow-webserver_1 | ____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
airflow-webserver_1 | ___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
airflow-webserver_1 | _/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
airflow-webserver_1 | [2021-02-24 16:41:38,109] {dagbag.py:448} INFO - Filling up the DagBag from /dev/null
airflow-webserver_1 | [2021-02-24 16:41:39,077] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-webserver_1 | [2021-02-24 16:41:39,151] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-webserver_1 | [2021-02-24 16:41:41 +0000] [23] [INFO] Starting gunicorn 19.10.0
airflow-webserver_1 | [2021-02-24 16:41:41 +0000] [23] [INFO] Listening at: http://0.0.0.0:8080 (23)
airflow-webserver_1 | [2021-02-24 16:41:41 +0000] [23] [INFO] Using worker: sync
airflow-webserver_1 | [2021-02-24 16:41:41 +0000] [27] [INFO] Booting worker with pid: 27
airflow-webserver_1 | [2021-02-24 16:41:41 +0000] [28] [INFO] Booting worker with pid: 28
airflow-webserver_1 | [2021-02-24 16:41:41 +0000] [29] [INFO] Booting worker with pid: 29
airflow-webserver_1 | [2021-02-24 16:41:41 +0000] [30] [INFO] Booting worker with pid: 30
airflow-webserver_1 | [2021-02-24 16:41:45,912] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-webserver_1 | [2021-02-24 16:41:45,967] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-webserver_1 | Running the Gunicorn Server with:
airflow-webserver_1 | Workers: 4 sync
airflow-webserver_1 | Host: 0.0.0.0:8080
airflow-webserver_1 | Timeout: 120
airflow-webserver_1 | Logfiles: - -
airflow-webserver_1 | Access Logformat:
airflow-webserver_1 | =================================================================
airflow-webserver_1 | [2021-02-24 16:41:45,994] {webserver_command.py:255} ERROR - [0 / 0] Some workers seem to have died and gunicorn did not restart them as expected
airflow-webserver_1 | [2021-02-24 16:41:46,097] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-webserver_1 | [2021-02-24 16:41:46,097] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
docker-apache-airflow-201_airflow-webserver_1 exited with code 137
airflow-webserver_1 |
airflow-webserver_1 | [2021-02-24 16:41:51,031] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-webserver_1 | [2021-02-24 16:41:51,646] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-webserver_1 | ____________ _____________
airflow-webserver_1 | ____ |__( )_________ __/__ /________ __
airflow-webserver_1 | ____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
airflow-webserver_1 | ___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
airflow-webserver_1 | _/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
airflow-webserver_1 | [2021-02-24 16:41:51,703] {dagbag.py:448} INFO - Filling up the DagBag from /dev/null
airflow-webserver_1 | [2021-02-24 16:41:52,651] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-webserver_1 | [2021-02-24 16:41:52,729] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-webserver_1 | [2021-02-24 16:41:55 +0000] [23] [INFO] Starting gunicorn 19.10.0
airflow-webserver_1 | [2021-02-24 16:41:55 +0000] [23] [INFO] Listening at: http://0.0.0.0:8080 (23)
airflow-webserver_1 | [2021-02-24 16:41:55 +0000] [23] [INFO] Using worker: sync
airflow-webserver_1 | [2021-02-24 16:41:55 +0000] [27] [INFO] Booting worker with pid: 27
airflow-webserver_1 | [2021-02-24 16:41:55 +0000] [28] [INFO] Booting worker with pid: 28
airflow-webserver_1 | [2021-02-24 16:41:55 +0000] [29] [INFO] Booting worker with pid: 29
airflow-webserver_1 | [2021-02-24 16:41:55 +0000] [30] [INFO] Booting worker with pid: 30
airflow-webserver_1 | Running the Gunicorn Server with:
airflow-webserver_1 | Workers: 4 sync
airflow-webserver_1 | Host: 0.0.0.0:8080
airflow-webserver_1 | Timeout: 120
airflow-webserver_1 | Logfiles: - -
airflow-webserver_1 | Access Logformat:
airflow-webserver_1 | =================================================================
airflow-webserver_1 | [2021-02-24 16:42:00,595] {webserver_command.py:255} ERROR - [0 / 0] Some workers seem to have died and gunicorn did not restart them as expected
airflow-webserver_1 | [2021-02-24 16:42:01,329] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
docker-apache-airflow-201_airflow-webserver_1 exited with code 137
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jhtimmins commented on issue #14365: docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
jhtimmins commented on issue #14365:
URL: https://github.com/apache/airflow/issues/14365#issuecomment-784667888
@chrisrfu Are you on an M1 chip as well?
@kaxil did you recently address an issue where this came up?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #14365: docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #14365:
URL: https://github.com/apache/airflow/issues/14365#issuecomment-785245364
You can ignore the following line:
```
airflow-webserver_1 | [2021-02-24 16:42:01,329] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
```
The container exited because it did not have enough memory:
```
docker-apache-airflow-201_airflow-webserver_1 exited with code 137
```
Try increasing the memory: https://docs.docker.com/docker-for-mac/space/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #14365: docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #14365:
URL: https://github.com/apache/airflow/issues/14365#issuecomment-783588762
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chrisrfu commented on issue #14365: docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
chrisrfu commented on issue #14365:
URL: https://github.com/apache/airflow/issues/14365#issuecomment-785147085
i ran `docker-compose down --volumes --rmi all` and then `docker images -a -q | xargs docker rmi -f`
for good measure and re-ran my original commands and still no dice
> Pulling postgres (postgres:13)...
13: Pulling from library/postgres
45b42c59be33: Already exists
40adec129f1a: Pull complete
b4c431d00c78: Pull complete
2696974e2815: Pull complete
564b77596399: Pull complete
5044045cf6f2: Pull complete
d736e67e6ac3: Pull complete
390c1c9a5ae4: Pull complete
c0e62f172284: Pull complete
ebcdc659c5bf: Pull complete
29be22cb3acc: Pull complete
f63c47038e66: Pull complete
77a0c198cde5: Pull complete
c8752d5b785c: Pull complete
Digest: sha256:5cfcfb0516ddd317109e2746a462b1dc398d152936640a2b69284629f48dc288
Status: Downloaded newer image for postgres:13
Pulling redis (redis:latest)...
latest: Pulling from library/redis
45b42c59be33: Already exists
5ce2e937bf62: Pull complete
2a031498ff58: Pull complete
7407cbab5258: Pull complete
c2d6c3b35be8: Pull complete
1997ce073dcd: Pull complete
Digest: sha256:26bd5f3d2bbfb21d0e4f03d642570e63c8776d2a9e5963e4057280fb3b4e3e29
Status: Downloaded newer image for redis:latest
Pulling airflow-init (apache/airflow:2.0.1)...
2.0.1: Pulling from apache/airflow
45b42c59be33: Already exists
f875e16ab19c: Already exists
a4fb13de3215: Pull complete
f3dd9aaf319e: Pull complete
451a8f7ee895: Pull complete
e2d0adf49357: Pull complete
52856765f525: Pull complete
fd428150aaa6: Pull complete
b4fa7f6b9775: Pull complete
b34f0a9234cb: Pull complete
537882d8c1ac: Pull complete
dffdc965d748: Pull complete
a97e5a67cb46: Pull complete
c401f5284516: Pull complete
af49f5db1642: Pull complete
c61449868181: Pull complete
0b574505c97e: Pull complete
a65349ef32b2: Pull complete
Digest: sha256:2bf4fe559e9e699487a50d4e587358bb0b3c90b59c8fefeccc8a4fb2e83f3d09
Status: Downloaded newer image for apache/airflow:2.0.1
Creating docker-apache-airflow_postgres_1 ... done
Creating docker-apache-airflow_redis_1 ... done
Creating docker-apache-airflow_airflow-init_1 ... done
Attaching to docker-apache-airflow_airflow-init_1
airflow-init_1 | BACKEND=postgresql+psycopg2
airflow-init_1 | DB_HOST=postgres
airflow-init_1 | DB_PORT=5432
airflow-init_1 |
airflow-init_1 | DB: postgresql+psycopg2://airflow:***@postgres/airflow
airflow-init_1 | [2021-02-24 15:11:24,568] {db.py:674} INFO - Creating tables
airflow-init_1 | INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
airflow-init_1 | INFO [alembic.runtime.migration] Will assume transactional DDL.
airflow-init_1 | WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | Traceback (most recent call last):
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 162, in _catch_revision_errors
airflow-init_1 | yield
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 356, in _upgrade_revs
airflow-init_1 | revs = list(revs)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 904, in _iterate_revisions
airflow-init_1 | requested_lowers = self.get_revisions(lower)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 455, in get_revisions
airflow-init_1 | return sum([self.get_revisions(id_elem) for id_elem in id_], ())
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 455, in <listcomp>
airflow-init_1 | return sum([self.get_revisions(id_elem) for id_elem in id_], ())
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 460, in get_revisions
airflow-init_1 | for rev_id in resolved_id
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 460, in <genexpr>
airflow-init_1 | for rev_id in resolved_id
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 536, in _revision_for_ident
airflow-init_1 | resolved_id,
airflow-init_1 | alembic.script.revision.ResolutionError: No such revision or branch '449b4072c2da'
airflow-init_1 |
airflow-init_1 | The above exception was the direct cause of the following exception:
airflow-init_1 |
airflow-init_1 | Traceback (most recent call last):
airflow-init_1 | File "/home/airflow/.local/bin/airflow", line 8, in <module>
airflow-init_1 | sys.exit(main())
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/__main__.py", line 40, in main
airflow-init_1 | args.func(args)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/cli/cli_parser.py", line 48, in command
airflow-init_1 | return func(*args, **kwargs)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/cli.py", line 89, in wrapper
airflow-init_1 | return f(*args, **kwargs)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/cli/commands/db_command.py", line 48, in upgradedb
airflow-init_1 | db.upgradedb()
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/db.py", line 684, in upgradedb
airflow-init_1 | command.upgrade(config, 'heads')
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/command.py", line 294, in upgrade
airflow-init_1 | script.run_env()
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 481, in run_env
airflow-init_1 | util.load_python_file(self.dir, "env.py")
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
airflow-init_1 | module = load_module_py(module_id, path)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/util/compat.py", line 182, in load_module_py
airflow-init_1 | spec.loader.exec_module(module)
airflow-init_1 | File "<frozen importlib._bootstrap_external>", line 678, in exec_module
airflow-init_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/migrations/env.py", line 108, in <module>
airflow-init_1 | run_migrations_online()
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online
airflow-init_1 | context.run_migrations()
airflow-init_1 | File "<string>", line 8, in run_migrations
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
airflow-init_1 | self.get_context().run_migrations(**kw)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/runtime/migration.py", line 548, in run_migrations
airflow-init_1 | for step in self._migrations_fn(heads, self):
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/command.py", line 283, in upgrade
airflow-init_1 | return script._upgrade_revs(revision, rev)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 361, in _upgrade_revs
airflow-init_1 | for script in reversed(list(revs))
airflow-init_1 | File "/usr/local/lib/python3.6/contextlib.py", line 99, in __exit__
airflow-init_1 | self.gen.throw(type, value, traceback)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/script/base.py", line 194, in _catch_revision_errors
airflow-init_1 | compat.raise_(util.CommandError(resolution), from_=re)
airflow-init_1 | File "/home/airflow/.local/lib/python3.6/site-packages/alembic/util/compat.py", line 294, in raise_
airflow-init_1 | raise exception
airflow-init_1 | alembic.util.exc.CommandError: Can't locate revision identified by '449b4072c2da'
airflow-init_1 | usage: airflow [-h] GROUP_OR_COMMAND ...
airflow-init_1 |
airflow-init_1 | positional arguments:
airflow-init_1 | GROUP_OR_COMMAND
airflow-init_1 |
airflow-init_1 | Groups:
airflow-init_1 | celery Celery components
airflow-init_1 | config View configuration
airflow-init_1 | connections Manage connections
airflow-init_1 | dags Manage DAGs
airflow-init_1 | db Database operations
airflow-init_1 | kubernetes Tools to help run the KubernetesExecutor
airflow-init_1 | pools Manage pools
airflow-init_1 | providers Display providers
airflow-init_1 | roles Manage roles
airflow-init_1 | tasks Manage tasks
airflow-init_1 | users Manage users
airflow-init_1 | variables Manage variables
airflow-init_1 |
airflow-init_1 | Commands:
airflow-init_1 | cheat-sheet Display cheat sheet
airflow-init_1 | info Show information about current Airflow and environment
airflow-init_1 | kerberos Start a kerberos ticket renewer
airflow-init_1 | plugins Dump information about loaded plugins
airflow-init_1 | rotate-fernet-key
airflow-init_1 | Rotate encrypted connection credentials and variables
airflow-init_1 | scheduler Start a scheduler instance
airflow-init_1 | sync-perm Update permissions for existing roles and DAGs
airflow-init_1 | version Show the version
airflow-init_1 | webserver Start a Airflow webserver instance
airflow-init_1 |
airflow-init_1 | optional arguments:
airflow-init_1 | -h, --help show this help message and exit
airflow-init_1 |
airflow-init_1 | airflow command error: argument GROUP_OR_COMMAND: `airflow upgradedb` command, has been removed, please use `airflow db upgrade`, see help above.
airflow-init_1 | [2021-02-24 15:11:31,808] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-24 15:11:32,232] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-24 15:11:33,231] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | [2021-02-24 15:11:33,303] {providers_manager.py:299} WARNING - Exception when importing 'airflow.providers.microsoft.azure.hooks.wasb.WasbHook' from 'apache-airflow-providers-microsoft-azure' package: No module named 'azure.storage.blob'
airflow-init_1 | airflow already exist in the db
airflow-init_1 | 2.0.1
docker-apache-airflow_airflow-init_1 exited with code 0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #14365: docker-compose up airflow-init produces airflow command error
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #14365:
URL: https://github.com/apache/airflow/issues/14365#issuecomment-785173521
It already shows:
```
airflow-init_1 | airflow already exist in the db
```
which means the data is already stored in the DB being used.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org