You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/23 03:56:55 UTC
[GitHub] [airflow] JackyKwok42 opened a new issue #13268: Airflow fails to initdb with kubernetes 11.0.0
JackyKwok42 opened a new issue #13268:
URL: https://github.com/apache/airflow/issues/13268
**Apache Airflow version**:
1.10.12
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
1.18
**Environment**:
- **Cloud provider or hardware configuration**: AWS Fargate
- **OS** (e.g. from /etc/os-release):
- **Kernel** (e.g. `uname -a`):
- **Install tools**: pip 20.2.4, python 3.8
- **Others**:
**What happened**:
I encountered an issue at the airflow initdb stage:
```
INFO [alembic.runtime.migration] Running upgrade 127d2bf2dfa7 -> cc1e65623dc7, add max tries column to task instance
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 37, in <module>
args.func(args)
File "/usr/local/lib/python3.8/site-packages/airflow/utils/cli.py", line 76, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/airflow/bin/cli.py", line 1382, in upgradedb
db.upgradedb()
File "/usr/local/lib/python3.8/site-packages/airflow/utils/db.py", line 386, in upgradedb
command.upgrade(config, 'heads')
File "/usr/local/lib/python3.8/site-packages/alembic/command.py", line 298, in upgrade
script.run_env()
File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 489, in run_env
util.load_python_file(self.dir, "env.py")
File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
module = load_module_py(module_id, path)
File "/usr/local/lib/python3.8/site-packages/alembic/util/compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/local/lib/python3.8/site-packages/airflow/migrations/env.py", line 97, in <module>
run_migrations_online()
File "/usr/local/lib/python3.8/site-packages/airflow/migrations/env.py", line 91, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/usr/local/lib/python3.8/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
self.get_context().run_migrations(**kw)
File "/usr/local/lib/python3.8/site-packages/alembic/runtime/migration.py", line 520, in run_migrations
step.migration_fn(**kw)
File "/usr/local/lib/python3.8/site-packages/airflow/migrations/versions/cc1e65623dc7_add_max_tries_column_to_task_instance.py", line 70, in upgrade
dagbag = DagBag(settings.DAGS_FOLDER)
File "/usr/local/lib/python3.8/site-packages/airflow/models/dagbag.py", line 95, in __init__
executor = get_default_executor()
File "/usr/local/lib/python3.8/site-packages/airflow/executors/__init__.py", line 48, in get_default_executor
DEFAULT_EXECUTOR = _get_executor(executor_name)
File "/usr/local/lib/python3.8/site-packages/airflow/executors/__init__.py", line 85, in _get_executor
from airflow.executors.kubernetes_executor import KubernetesExecutor
File "/usr/local/lib/python3.8/site-packages/airflow/executors/kubernetes_executor.py", line 33, in <module>
from kubernetes import watch, client
ImportError: cannot import name 'watch' from 'kubernetes' (/usr/local/lib/python3.8/site-packages/airflow/kubernetes/__init__.py)
```
**What you expected to happen**:
I expected the Airflow DB to be initialised as per the instructions.
**How to reproduce it**:
On a fresh installation, follow the Quick Start guide until the apache initdb stage.
**Anything else we need to know**:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #13268: Airflow fails to initdb with kubernetes 11.0.0
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #13268:
URL: https://github.com/apache/airflow/issues/13268
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #13268: Airflow fails to initdb with kubernetes 11.0.0
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13268:
URL: https://github.com/apache/airflow/issues/13268#issuecomment-749950628
I am closing it for now, unless there is more information/the upgrade advice did not work.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] devender-yadav commented on issue #13268: Airflow fails to initdb with kubernetes 11.0.0
Posted by GitBox <gi...@apache.org>.
devender-yadav commented on issue #13268:
URL: https://github.com/apache/airflow/issues/13268#issuecomment-840472826
I am getting the same error `ImportError: cannot import name 'watch' from 'kubernetes'` with `kubernetes==11.0.0` and `apache-airflow==1.10.15`.
Kubernetes server version - 1.17
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #13268: Airflow fails to initdb with kubernetes 11.0.0
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #13268:
URL: https://github.com/apache/airflow/issues/13268#issuecomment-749915552
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #13268: Airflow fails to initdb with kubernetes 11.0.0
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13268:
URL: https://github.com/apache/airflow/issues/13268#issuecomment-749950288
It looks like you might have wrong version of kubernetes library. Can you please check with pip freeze what version you have and whether `pip check` does not show any conflicts`?
Also there were a lot of fixes to Kubernetes Executor between 1.10.12 and 1.10.14 (see the https://airflow.apache.org/docs/apache-airflow/stable/changelog.html) and I recommend you upgrade to 1.10.14 or even better. try 2.0.0 already. 2.0.0 is a major overhaul of Airlfow with better UI, much faster and with many new features. 1.10 series is not going to be continued. Only critical and security bugfixes will be fixed, so migrating to 2.0.0 is heartily recommended.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org