You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/11/08 13:37:13 UTC
[GitHub] [airflow] tnk-dev opened a new issue #19467: ModuleNotFoundError: No module named 'wtforms.compat'
tnk-dev opened a new issue #19467:
URL: https://github.com/apache/airflow/issues/19467
### Apache Airflow version
2.2.1 (latest released)
### Operating System
Docker python:3.8-slim image
### Versions of Apache Airflow Providers
```pip freeze
alembic==1.7.4
anyio==3.3.4
apache-airflow==2.2.1
apache-airflow-providers-amazon==1.4.0
apache-airflow-providers-apache-hive==2.0.3
apache-airflow-providers-ftp==2.0.1
apache-airflow-providers-http==2.0.1
apache-airflow-providers-imap==2.0.1
apache-airflow-providers-jdbc==2.0.1
apache-airflow-providers-postgres==2.3.0
apache-airflow-providers-sqlite==2.0.1
apache-airflow-providers-ssh==2.3.0
apispec==3.3.2
argcomplete==1.12.3
astroid==2.6.6
atomicwrites==1.4.0
attrs==20.3.0
awscli==1.19.97
Babel==2.9.1
bcrypt==3.2.0
bleach==4.1.0
blinker==1.4
boto3==1.17.97
botocore==1.20.97
cattrs==1.5.0
certifi==2021.10.8
cffi==1.15.0
charset-normalizer==2.0.7
click==7.1.2
clickclick==20.10.2
colorama==0.4.3
colorlog==5.0.1
commonmark==0.9.1
croniter==1.0.15
cryptography==35.0.0
defusedxml==0.7.1
dill==0.3.4
dnspython==2.1.0
docutils==0.15.2
email-validator==1.1.3
Flask==1.1.4
Flask-AppBuilder==3.3.4
Flask-Babel==2.0.0
Flask-Caching==1.10.1
Flask-JWT-Extended==3.25.1
Flask-Login==0.4.1
Flask-OpenID==1.3.0
Flask-SQLAlchemy==2.5.1
Flask-WTF==0.14.3
future==0.18.2
graphviz==0.18
gunicorn==20.1.0
h11==0.12.0
hmsclient==0.1.1
httpcore==0.13.7
httpx==0.20.0
idna==3.3
importlib-metadata==4.8.1
importlib-resources==5.4.0
inflection==0.5.1
iso8601==1.0.0
isodate==0.6.0
isort==5.10.0
itsdangerous==1.1.0
JayDeBeApi==1.2.3
Jinja2==2.11.3
jmespath==0.10.0
JPype1==1.3.0
jsonschema==3.2.0
lazy-object-proxy==1.6.0
lockfile==0.12.2
Mako==1.1.5
Markdown==3.3.4
MarkupSafe==1.1.1
marshmallow==3.14.0
marshmallow-enum==1.5.1
marshmallow-oneofschema==3.0.1
marshmallow-sqlalchemy==0.26.1
mccabe==0.6.1
more-itertools==8.10.0
mypy==0.812
mypy-extensions==0.4.3
numpy==1.21.4
openapi-schema-validator==0.1.5
openapi-spec-validator==0.3.1
packaging==21.2
pandas==1.3.4
paramiko==2.8.0
pendulum==2.1.2
pluggy==0.13.1
prison==0.2.1
psutil==5.8.0
psycopg2-binary==2.9.1
pure-sasl==0.6.2
py==1.11.0
pyasn1==0.4.8
pycparser==2.21
Pygments==2.10.0
PyHive==0.6.4
PyJWT==1.7.1
pylint==2.7.4
PyNaCl==1.4.0
pyparsing==2.4.7
pyrsistent==0.18.0
pysftp==0.2.9
pytest==5.2.2
pytest-responses==0.5.0
python-daemon==2.3.0
python-dateutil==2.8.2
python-nvd3==0.15.0
python-slugify==4.0.1
python3-openid==3.2.0
pytz==2021.3
pytzdata==2020.1
PyYAML==5.4.1
requests==2.26.0
requests-toolbelt==0.9.1
responses==0.15.0
rfc3986==1.5.0
rich==10.13.0
rsa==4.7.2
s3transfer==0.4.2
sasl==0.3.1
setproctitle==1.2.2
six==1.16.0
sniffio==1.2.0
SQLAlchemy==1.3.23
SQLAlchemy-JSONField==1.0.0
SQLAlchemy-Utils==0.37.9
sshtunnel==0.4.0
swagger-ui-bundle==0.0.9
tableau-api-lib==0.1.14
tabulate==0.8.9
tenacity==8.0.1
termcolor==1.1.0
text-unidecode==1.3
thrift==0.15.0
thrift-sasl==0.4.3
toml==0.10.2
typed-ast==1.4.3
typeguard==2.13.0
typing-extensions==3.10.0.2
unicodecsv==0.14.1
urllib3==1.26.7
watchtower==0.7.3
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==1.0.1
wrapt==1.12.1
WTForms==3.0.0
zipp==3.6.0
```
### Deployment
Virtualenv installation
### Deployment details
Hey Airflow,
while developing new dags etc today without changing any underlying infrastructure the following happened when running:
```bash
airflow db init
airflow db upgrade
```
Our Dockerfile
```yaml
FROM python:3.8-slim
# Airflow
ARG ENV_FILE
ARG PRIVATE_SSH_KEY_FILE
ENV PIP_VERSION=20.2.4
ENV LANGUAGE="en_US.UTF-8"
ENV LANG="en_US.UTF-8"
ENV LC_ALL="en_US.UTF-8"
ENV LC_CTYPE="en_US.UTF-8"
ENV LC_MESSAGES="en_US.UTF-8"
ENV AWS_DEFAULT_REGION="eu-central-1"
RUN apt-get update -y
RUN apt-get install -y locales
RUN sed -i 's/^# en_US.UTF-8 UTF-8$/en_US.UTF-8 UTF-8/g' /etc/locale.gen \
&& locale-gen \
&& update-locale LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8
ENV AIRFLOW_HOME=/usr/local/around-airflow
RUN mkdir -p ${AIRFLOW_HOME}
COPY requirements.txt ${AIRFLOW_HOME}/requirements.txt
RUN apt-get install -y build-essential ssh jq libffi-dev libssl-dev libsasl2-dev && \
pip3 install --upgrade pip==${PIP_VERSION}&& \
pip3 install --no-cache -r ${AIRFLOW_HOME}/requirements.txt && \
apt-get purge -y build-essential && \
apt-get autoremove -y --purge && \
apt-get clean
RUN useradd -d ${AIRFLOW_HOME} airflow
COPY scripts ${AIRFLOW_HOME}/scripts
COPY tests ${AIRFLOW_HOME}/tests
COPY generated_airflow.cfg ${AIRFLOW_HOME}/airflow.cfg
COPY dags ${AIRFLOW_HOME}/dags
COPY $ENV_FILE ${AIRFLOW_HOME}/.env
COPY mypy.ini .pylintrc ${AIRFLOW_HOME}/
RUN mkdir -p ${AIRFLOW_HOME}/.ssh
COPY $PRIVATE_SSH_KEY_FILE ${AIRFLOW_HOME}/.ssh/id_rsa
RUN chmod 400 ${AIRFLOW_HOME}/.ssh/id_rsa
COPY .airflowignore ${AIRFLOW_HOME}/.airflowignore
RUN chown -R airflow ${AIRFLOW_HOME}
EXPOSE 8080
USER airflow
WORKDIR ${AIRFLOW_HOME}
ENTRYPOINT ["/bin/bash", "-l", "-c"]
CMD ["bash scripts/entrypoint.sh"]
```
This is unexpected behaviour since we didn't change any pip package versions nor added any packages requirements.txt
### What happened
```
DB: sqlite:////usr/local/around-airflow/airflow.db
300[2021-11-08 12:27:48,868] {{db.py:674}} INFO - Creating tables
301INFO [alembic.runtime.migration] Context impl SQLiteImpl.
302INFO [alembic.runtime.migration] Will assume non-transactional DDL.
303Traceback (most recent call last):
304 File "/usr/local/bin/airflow", line 8, in <module>
305 sys.exit(main())
306 File "/usr/local/lib/python3.8/site-packages/airflow/__main__.py", line 40, in main
307 args.func(args)
308 File "/usr/local/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 48, in command
309 return func(*args, **kwargs)
310 File "/usr/local/lib/python3.8/site-packages/airflow/cli/commands/db_command.py", line 31, in initdb
311 db.initdb()
312 File "/usr/local/lib/python3.8/site-packages/airflow/utils/db.py", line 549, in initdb
313 upgradedb()
314 File "/usr/local/lib/python3.8/site-packages/airflow/utils/db.py", line 684, in upgradedb
315 command.upgrade(config, 'heads')
316 File "/usr/local/lib/python3.8/site-packages/alembic/command.py", line 320, in upgrade
317 script.run_env()
318 File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 563, in run_env
319 util.load_python_file(self.dir, "env.py")
320 File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file
321 module = load_module_py(module_id, path)
322 File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py
323 spec.loader.exec_module(module) # type: ignore
324 File "<frozen importlib._bootstrap_external>", line 783, in exec_module
325 File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
326 File "/usr/local/lib/python3.8/site-packages/airflow/migrations/env.py", line 108, in <module>
327 run_migrations_online()
328 File "/usr/local/lib/python3.8/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online
329 context.run_migrations()
330 File "<string>", line 8, in run_migrations
331 File "/usr/local/lib/python3.8/site-packages/alembic/runtime/environment.py", line 851, in run_migrations
332 self.get_context().run_migrations(**kw)
333 File "/usr/local/lib/python3.8/site-packages/alembic/runtime/migration.py", line 608, in run_migrations
334 for step in self._migrations_fn(heads, self):
335 File "/usr/local/lib/python3.8/site-packages/alembic/command.py", line 309, in upgrade
336 return script._upgrade_revs(revision, rev)
337 File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 439, in _upgrade_revs
338 for script in reversed(list(revs))
339 File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 793, in iterate_revisions
340 revisions, heads = fn(
341 File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 1393, in _collect_upgrade_revisions
342 targets: Collection["Revision"] = self._parse_upgrade_target(
343 File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 1193, in _parse_upgrade_target
344 return self.get_revisions(target)
345 File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 527, in get_revisions
346 resolved_id, branch_label = self._resolve_revision_number(
347 File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 747, in _resolve_revision_number
348 self._revision_map
349 File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 893, in __get__
350 obj.__dict__[self.__name__] = result = self.fget(obj)
351 File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 189, in _revision_map
352 for revision in self._generator():
353 File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 136, in _load_revisions
354 script = Script._from_filename(self, vers, file_)
355 File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 999, in _from_filename
356 module = util.load_python_file(dir_, filename)
357 File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file
358 module = load_module_py(module_id, path)
359 File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py
360 spec.loader.exec_module(module) # type: ignore
361 File "<frozen importlib._bootstrap_external>", line 783, in exec_module
362 File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
363 File "/usr/local/lib/python3.8/site-packages/airflow/migrations/versions/82b7c48c147f_remove_can_read_permission_on_config_.py", line 28, in <module>
364 from airflow.www.app import create_app
365 File "/usr/local/lib/python3.8/site-packages/airflow/www/app.py", line 23, in <module>
366 from flask_appbuilder import SQLA
367 File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/__init__.py", line 6, in <module>
368 from .base import AppBuilder # noqa: F401
369 File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/base.py", line 8, in <module>
370 from .api.manager import OpenApiManager
371 File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/api/manager.py", line 7, in <module>
372 from flask_appbuilder.baseviews import BaseView
373 File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/baseviews.py", line 21, in <module>
374 from .forms import GeneralModelConverter
375 File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/forms.py", line 16, in <module>
376 from .fields import EnumField, QuerySelectField, QuerySelectMultipleField
377 File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/fields.py", line 6, in <module>
378 from wtforms.compat import string_types, text_type
379ModuleNotFoundError: No module named 'wtforms.compat'
```
### What you expected to happen
I expected to build airflow as usual
### How to reproduce
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] tnk-dev commented on issue #19467: ModuleNotFoundError: No module named 'wtforms.compat'
Posted by GitBox <gi...@apache.org>.
tnk-dev commented on issue #19467:
URL: https://github.com/apache/airflow/issues/19467#issuecomment-963171923
```txt
wheel
cattrs
apache-airflow[crypto,postgres,hive,jdbc,ssh]==2.2.1 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.2.1/constraints-3.6.txt"
apache-airflow-providers-amazon==1.4.0
awscli==1.19.97
boto3==1.17.97
botocore==1.20.97
mypy==0.812
pylint==2.7.4
pytest==5.2.2
pytest-responses==0.5.0
sqlalchemy==1.3.23
tableau-api-lib==0.1.14
```
This is my requirements.txt
Same error!
Am I missing something else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] isaac-florence commented on issue #19467: ModuleNotFoundError: No module named 'wtforms.compat'
Posted by GitBox <gi...@apache.org>.
isaac-florence commented on issue #19467:
URL: https://github.com/apache/airflow/issues/19467#issuecomment-964992208
I have had exactly the same issue this morning, except I cannot move where the constraints file is used as `requirements.txt` is used by the source-to-image (s21) build logic. Nothing changed in our code but the K8s deployment failed this morning with this code change.
[requirements.txt](https://github.com/apache/airflow/files/7511877/requirements.txt)
Please can you advise?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #19467: ModuleNotFoundError: No module named 'wtforms.compat'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #19467:
URL: https://github.com/apache/airflow/issues/19467#issuecomment-963165188
You should install airflow using constraints https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] tnk-dev commented on issue #19467: ModuleNotFoundError: No module named 'wtforms.compat'
Posted by GitBox <gi...@apache.org>.
tnk-dev commented on issue #19467:
URL: https://github.com/apache/airflow/issues/19467#issuecomment-963192259
You're right this solved the issue. Thank you so much and have a nice day :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #19467: ModuleNotFoundError: No module named 'wtforms.compat'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #19467:
URL: https://github.com/apache/airflow/issues/19467#issuecomment-963177572
Also - If you have conflicts then you should only install your conficting dependencies separately after you installed airflow - this should be a separate pip install command.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] isaac-florence edited a comment on issue #19467: ModuleNotFoundError: No module named 'wtforms.compat'
Posted by GitBox <gi...@apache.org>.
isaac-florence edited a comment on issue #19467:
URL: https://github.com/apache/airflow/issues/19467#issuecomment-964992208
I have had exactly the same issue this morning, except I cannot move where the constraints file is used as `requirements.txt` is used by the source-to-image (s21) build logic. Nothing changed in our code but the K8s deployment failed this morning with this code change.
Please can you advise?
```
apache-airflow==2.1.1 -c https://raw.githubusercontent.com/apache/airflow/constraints-2.1.1/constraints-3.9.txt
apache-airflow-providers-cncf-kubernetes
apache-airflow-providers-postgres
Authlib
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #19467: ModuleNotFoundError: No module named 'wtforms.compat'
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #19467:
URL: https://github.com/apache/airflow/issues/19467
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #19467: ModuleNotFoundError: No module named 'wtforms.compat'
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #19467:
URL: https://github.com/apache/airflow/issues/19467#issuecomment-963157864
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] isaac-florence removed a comment on issue #19467: ModuleNotFoundError: No module named 'wtforms.compat'
Posted by GitBox <gi...@apache.org>.
isaac-florence removed a comment on issue #19467:
URL: https://github.com/apache/airflow/issues/19467#issuecomment-964992208
I have had exactly the same issue this morning, except I cannot move where the constraints file is used as `requirements.txt` is used by the source-to-image (s21) build logic. Nothing changed in our code but the K8s deployment failed this morning with this code change.
Please can you advise?
```
apache-airflow==2.1.1 -c https://raw.githubusercontent.com/apache/airflow/constraints-2.1.1/constraints-3.9.txt
apache-airflow-providers-cncf-kubernetes
apache-airflow-providers-postgres
Authlib
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] tnk-dev edited a comment on issue #19467: ModuleNotFoundError: No module named 'wtforms.compat'
Posted by GitBox <gi...@apache.org>.
tnk-dev edited a comment on issue #19467:
URL: https://github.com/apache/airflow/issues/19467#issuecomment-963171923
```txt
wheel
cattrs
apache-airflow[crypto,postgres,hive,jdbc,ssh]==2.2.1 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.2.1/constraints-3.8.txt"
apache-airflow-providers-amazon==1.4.0
awscli==1.19.97
boto3==1.17.97
botocore==1.20.97
mypy==0.812
pylint==2.7.4
pytest==5.2.2
pytest-responses==0.5.0
sqlalchemy==1.3.23
tableau-api-lib==0.1.14
```
This is my requirements.txt
Same error!
Am I missing something else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #19467: ModuleNotFoundError: No module named 'wtforms.compat'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #19467:
URL: https://github.com/apache/airflow/issues/19467#issuecomment-963176099
--constraints should be added to `pip install` command not to requirements.txt. Constraints added in requirements.txt make no sense.
```
pip3 install --no-cache -r ${AIRFLOW_HOME}/requirements.txt --constraint ....
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org