You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/11 02:22:59 UTC
[GitHub] [airflow] colinglaes opened a new issue #13603: ModuleNotFoundError: No module named 'airflow.providers.http'
colinglaes opened a new issue #13603:
URL: https://github.com/apache/airflow/issues/13603
**Apache Airflow version**: 2.0.0
**Environment**:
- **Cloud provider or hardware configuration**:
- **OS** (e.g. from /etc/os-release): 18.04.5 LTS (Bionic Beaver)
- **Kernel** (e.g. `uname -a`): Linux orion 5.4.0-60-generic #67~18.04.1-Ubuntu SMP Tue Jan 5 22:01:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
- **Install tools**: https://airflow.apache.org/docs/apache-airflow/stable/installation.html & https://airflow.apache.org/docs/apache-airflow/stable/upgrading-to-2.html
- **Python**: 3.6.9
**What happened**:
I got to upgrading to airflow 2 but when i received the following error message when running `airflow db upgrade`
```
[cglaes@orion : ~ : 01:55] $ airflow db upgrade
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324: DeprecationWarning: The log_format option in [core] has been moved to the log_format option in [logging] - the old setting has been used, but please update your config.
option = self._get_option_from_config_file(deprecated_key, deprecated_section, key, kwargs, section)
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324: DeprecationWarning: The simple_log_format option in [core] has been moved to the simple_log_format option in [logging] - the old setting has been used, but please update your config.
option = self._get_option_from_config_file(deprecated_key, deprecated_section, key, kwargs, section)
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324 DeprecationWarning: The logging_config_class option in [core] has been moved to the logging_config_class option in [logging] - the old setting has been used, but please update your config.
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324 DeprecationWarning: The logging_level option in [core] has been moved to the logging_level option in [logging] - the old setting has been used, but please update your config.
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324 DeprecationWarning: The fab_logging_level option in [core] has been moved to the fab_logging_level option in [logging] - the old setting has been used, but please update your config.
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324 DeprecationWarning: The colored_log_format option in [core] has been moved to the colored_log_format option in [logging] - the old setting has been used, but please update your config.
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324 DeprecationWarning: The colored_console_log option in [core] has been moved to the colored_console_log option in [logging] - the old setting has been used, but please update your config.
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324 DeprecationWarning: The colored_formatter_class option in [core] has been moved to the colored_formatter_class option in [logging] - the old setting has been used, but please update your config.
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324 DeprecationWarning: The base_log_folder option in [core] has been moved to the base_log_folder option in [logging] - the old setting has been used, but please update your config.
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324 DeprecationWarning: The dag_processor_manager_log_location option in [core] has been moved to the dag_processor_manager_log_location option in [logging] - the old setting has been used, but please update your config.
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324 DeprecationWarning: The log_filename_template option in [core] has been moved to the log_filename_template option in [logging] - the old setting has been used, but please update your config.
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324 DeprecationWarning: The log_processor_filename_template option in [core] has been moved to the log_processor_filename_template option in [logging] - the old setting has been used, but please update your config.
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324 DeprecationWarning: The remote_logging option in [core] has been moved to the remote_logging option in [logging] - the old setting has been used, but please update your config.
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324 DeprecationWarning: The task_log_reader option in [core] has been moved to the task_log_reader option in [logging] - the old setting has been used, but please update your config.
/home/cglaes/.local/lib/python3.6/site-packages/airflow/configuration.py:324 DeprecationWarning: The statsd_on option in [scheduler] has been moved to the statsd_on option in [metrics] - the old setting has been used, but please update your config.
DB: postgresql://localhost:5432/airflow
[2021-01-11 01:56:07,048] {db.py:678} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
Traceback (most recent call last):
File "/home/cglaes/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/__main__.py", line 40, in main
args.func(args)
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/utils/cli.py", line 89, in wrapper
return f(*args, **kwargs)
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/cli/commands/db_command.py", line 48, in upgradedb
db.upgradedb()
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/utils/db.py", line 688, in upgradedb
command.upgrade(config, 'heads')
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/command.py", line 298, in upgrade
script.run_env()
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/script/base.py", line 489, in run_env
util.load_python_file(self.dir, "env.py")
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
module = load_module_py(module_id, path)
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/util/compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/migrations/env.py", line 108, in <module>
run_migrations_online()
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
self.get_context().run_migrations(**kw)
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/runtime/migration.py", line 511, in run_migrations
for step in self._migrations_fn(heads, self):
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/command.py", line 287, in upgrade
return script._upgrade_revs(revision, rev)
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/script/base.py", line 364, in _upgrade_revs
revs = list(revs)
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 765, in _iterate_revisions
requested_lowers = self.get_revisions(lower)
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 319, in get_revisions
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 319, in <listcomp>
return sum([self.get_revisions(id_elem) for id_elem in id_], ())
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 321, in get_revisions
resolved_id, branch_label = self._resolve_revision_number(id_)
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 501, in _resolve_revision_number
self._revision_map
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/util/langhelpers.py", line 230, in __get__
obj.__dict__[self.__name__] = result = self.fget(obj)
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/script/revision.py", line 123, in _revision_map
for revision in self._generator():
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/script/base.py", line 112, in _load_revisions
script = Script._from_filename(self, vers, file_)
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/script/base.py", line 906, in _from_filename
module = util.load_python_file(dir_, filename)
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
module = load_module_py(module_id, path)
File "/home/cglaes/.local/lib/python3.6/site-packages/alembic/util/compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/migrations/versions/2c6edca13270_resource_based_permissions.py", line 27, in <module>
from airflow.www.app import create_app
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/www/app.py", line 38, in <module>
from airflow.www.extensions.init_views import (
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/www/extensions/init_views.py", line 29, in <module>
from airflow.www.views import lazy_add_provider_discovered_options_to_connection_form
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/www/views.py", line 2836, in <module>
class ConnectionFormWidget(FormWidget):
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/www/views.py", line 2839, in ConnectionFormWidget
field_behaviours = json.dumps(ProvidersManager().field_behaviours)
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/providers_manager.py", line 397, in field_behaviours
self.initialize_providers_manager()
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/providers_manager.py", line 129, in initialize_providers_manager
self._discover_all_providers_from_packages()
File "/home/cglaes/.local/lib/python3.6/site-packages/airflow/providers_manager.py", line 148, in _discover_all_providers_from_packages
provider_info = entry_point.load()()
File "/home/cglaes/.local/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 105, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'airflow.providers.http'
[cglaes@orion : ~ : 01:56] $
```
**What you expected to happen**:
Successful upgrade with out
**How to reproduce it**:
Follow tutorial: https://airflow.apache.org/docs/apache-airflow/stable/installation.html & https://airflow.apache.org/docs/apache-airflow/stable/upgrading-to-2.html
**Anything else we need to know**:
```
[cglaes@orion : ~ : 02:19] $ pip freeze
alembic==1.4.3
apache-airflow==2.0.0
apache-airflow-providers-ftp==1.0.0
apache-airflow-providers-http==1.0.0
apache-airflow-providers-imap==1.0.0
apache-airflow-providers-postgres==1.0.0
apache-airflow-providers-sqlite==1.0.0
apache-airflow-upgrade-check==1.1.0
apispec==3.3.2
apturl==0.5.2
argcomplete==1.12.2
asn1crypto==0.24.0
attrs==20.3.0
awscli==1.18.69
Babel==2.9.0
backcall==0.1.0
bitarray==1.2.1
bleach==2.1.2
botocore==1.16.19
bottle==0.12.13
Brlapi==0.6.6
cached-property==1.5.2
cachetools==4.2.0
cattrs==1.0.0
certifi==2020.11.8
cffi==1.14.4
chardet==3.0.4
click==6.7
clickclick==20.10.2
cloudpickle==1.3.0
colorama==0.4.4
colorlog==4.0.2
command-not-found==0.3
commonmark==0.9.1
configparser==3.5.3
connexion==2.7.0
croniter==0.3.36
cryptography==3.2.1
cupshelpers==1.0
cycler==0.10.0
dask==2.15.0
dataclasses==0.7
decorator==4.1.2
defer==1.0.6
defusedxml==0.6.0
dill==0.3.3
distributed==2.15.2
distro-info===0.18ubuntu0.18.04.1
dnspython==1.16.0
docker==4.2.0
docker-pycreds==0.2.1
docutils==0.15.2
email-validator==1.1.2
entrypoints==0.2.3.post1
flake8==3.7.9
Flask==1.1.2
Flask-Admin==1.5.4
Flask-AppBuilder==3.1.1
Flask-Babel==1.0.0
Flask-Caching==1.9.0
Flask-JWT-Extended==3.25.0
Flask-Login==0.4.1
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.4.4
flask-swagger==0.2.13
Flask-WTF==0.14.3
fsspec==0.7.3
funcsigs==1.0.2
future==0.18.2
fxcmpy==1.2.6
Glances==2.11.1
google-auth==1.24.0
graphviz==0.15
gunicorn==19.10.0
HeapDict==1.0.1
html5lib==0.999999999
httplib2==0.9.2
idna==2.8
importlib-metadata==1.7.0
importlib-resources==1.5.0
impyla==0.16.2
inflection==0.5.1
influxdb==4.1.1
iotop==0.6
ipykernel==4.8.2
ipython==7.13.0
ipython-genutils==0.2.0
ipywidgets==6.0.0
iso8601==0.1.13
itsdangerous==1.1.0
jedi==0.17.0
Jinja2==2.11.2
jmespath==0.9.3
json-merge-patch==0.2
json5==0.9.5
jsonschema==3.2.0
jupyter-client==5.2.2
jupyter-core==4.4.0
jupyterlab==2.2.4
jupyterlab-server==1.2.0
keyring==10.6.0
keyrings.alt==3.0
kubernetes==11.0.0
language-selector==0.1
launchpadlib==1.10.6
lazr.restfulclient==0.13.5
lazr.uri==1.0.3
lazy-object-proxy==1.4.3
locket==0.2.0
lockfile==0.12.2
louis==3.5.0
macaroonbakery==1.1.3
Mako==1.1.3
Markdown==2.6.11
MarkupSafe==1.1.1
marshmallow==3.10.0
marshmallow-enum==1.5.1
marshmallow-oneofschema==2.0.1
marshmallow-sqlalchemy==0.23.1
matplotlib==2.1.1
mccabe==0.6.1
mistune==0.8.3
msgpack==1.0.0
mypy-extensions==0.4.3
natsort==7.1.0
nbconvert==5.3.1
nbformat==4.4.0
netifaces==0.10.4
notebook==5.2.2
numpy==1.19.4
oauth==1.0.1
oauthlib==3.1.0
olefile==0.45.1
openapi-spec-validator==0.2.9
packaging==20.7
pandas==1.1.4
pandocfilters==1.4.2
parso==0.7.0
partd==1.1.0
pendulum==2.1.2
pep562==1.0
pexpect==4.2.1
pickleshare==0.7.5
Pillow==5.1.0
ply==3.11
prison==0.1.3
prompt-toolkit==3.0.5
protobuf==3.0.0
psutil==5.7.3
psycopg2==2.8.5
psycopg2-binary==2.8.6
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycairo==1.16.2
pycodestyle==2.5.0
pycparser==2.20
pycrypto==2.6.1
pycryptodomex==3.4.7
pycups==1.9.73
pyflakes==2.1.1
Pygments==2.7.2
pygobject==3.26.1
PyJWT==1.7.1
pymacaroons==0.13.0
PyNaCl==1.1.2
pyparsing==2.4.7
pyRFC3339==1.0
pyrsistent==0.17.3
pysmi==0.2.2
pysnmp==4.4.3
pystache==0.5.4
python-apt==1.6.5+ubuntu0.5
python-box==4.2.3
python-daemon==2.2.4
python-dateutil==2.8.1
python-debian==0.1.32
python-editor==1.0.4
python-nvd3==0.15.0
python-slugify==4.0.1
python3-openid==3.2.0
pytz==2020.4
pytzdata==2020.1
pyxdg==0.25
PyYAML==5.3.1
pyzmq==16.0.2
reportlab==3.4.0
requests==2.23.0
requests-oauthlib==1.3.0
requests-unixsocket==0.1.5
rich==9.2.0
roman==2.0.0
rsa==3.4.2
ruamel.yaml==0.16.10
ruamel.yaml.clib==0.2.0
s3transfer==0.3.3
screen-resolution-extra==0.0.0
SecretStorage==2.3.1
setproctitle==1.2
simplegeneric==0.8.1
simplejson==3.13.2
six==1.15.0
socketIO-client==0.7.2
sortedcontainers==2.1.0
SQLAlchemy==1.3.20
SQLAlchemy-JSONField==1.0.0
SQLAlchemy-Utils==0.36.8
ssh-import-id==5.7
supervisor==4.1.0
swagger-ui-bundle==0.0.8
system-service==0.3
systemd-python==234
tabulate==0.8.7
tblib==1.6.0
tenacity==6.2.0
termcolor==1.1.0
terminado==0.7
testpath==0.3.1
text-unidecode==1.3
thrift==0.13.0
thriftpy2==0.4.11
toml==0.10.0
toolz==0.10.0
tornado==6.0.4
tqdm==4.42.1
traitlets==4.3.2
typing==3.7.4.3
typing-extensions==3.7.4.3
tzlocal==1.5.1
ubuntu-drivers-common==0.0.0
ufw==0.36
unattended-upgrades==0.1
unicodecsv==0.14.1
urllib3==1.25.11
usb-creator==0.3.3
virtualenv==16.7.4
wadllib==1.3.2
wcwidth==0.1.9
webencodings==0.5
websocket-client==0.44.0
Werkzeug==1.0.1
WTForms==2.3.3
xkit==0.0.0
zict==2.0.0
zipp==3.4.0
zope.deprecation==4.4.0
zope.interface==4.3.2
[cglaes@orion : ~ : 02:22] $
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #13603: Airflow 2.0 - ModuleNotFoundError: No module named 'airflow.providers.http'
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #13603:
URL: https://github.com/apache/airflow/issues/13603
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #13603: ModuleNotFoundError: No module named 'airflow.providers.http'
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #13603:
URL: https://github.com/apache/airflow/issues/13603#issuecomment-757592500
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] colinglaes commented on issue #13603: Airflow 2.0 - ModuleNotFoundError: No module named 'airflow.providers.http'
Posted by GitBox <gi...@apache.org>.
colinglaes commented on issue #13603:
URL: https://github.com/apache/airflow/issues/13603#issuecomment-762526877
@potiuk I just went through the process again and have successfully got the scheduler and webserver running. I believe that the source of my issue was that i had some `apache_airflow_providers_*` installed under `/usr/local/lib/python3.6/dist-packages` and others under `/home/cglaes/.local/lib/python3.6/site-packages` from earlier botched attempts that i hadn't removed in my last installation attempt.
I did have some thoughts though as to how the documentation / error messages could be improved though
**https://airflow.apache.org/docs/apache-airflow/stable/installation.html**
- `pip upgrade --pip==20.2.4` is an invalid command
**Messages from `airflow upgrade_check`**
- "fernet_key in airflow.cfg must be explicitly set empty as fernet mechanism is enabledby default. This means that the apache-airflow[crypto] extra-packages are always installed.However, this requires that your operating system has libffi-dev installed.". I think it might be clearer to explicitly state that the config key should be set as `fernet_key = None` instead of saying it "must be explicitly set empty". Not everyone knows what empty means in this case (i could see many people setting it like `fernet_key = `)
- "1. The following invalid keys were found in your airflow.cfg` comes up and recommends removing the deprecated keys, additionally it states "Please generate a pod_template_file by running `airflow generate_pod_template`", there isn't an explicit statement that running this command is conditional on having a Kubernetes setup. Maybe these checks could be separated to improve clarity.
Let me know if it would be helpful for you guys if i paste the output from my terminal
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #13603: Airflow 2.0 - ModuleNotFoundError: No module named 'airflow.providers.http'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13603:
URL: https://github.com/apache/airflow/issues/13603#issuecomment-757923733
You probably likely have Airflow installed in `--editable` mode from sources rather than packages.
You should remove the 'apache-airflow-providers*' if you install airflow in --editable mode (Though I would argue this is not the best Idea to run airflow in --editable mode in production. (Though I am kind of surprised how many people do that).
You can achieve that by setting `INSTALL_PROVIDERS_FROM_SOURCES` variable to
This has been already addressed in #13439 in 2.0.1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #13603: Airflow 2.0 - ModuleNotFoundError: No module named 'airflow.providers.http'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13603:
URL: https://github.com/apache/airflow/issues/13603#issuecomment-758825007
@colinglaes - was my diagnosis correct? Did you manage to fix the problem?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] colinglaes commented on issue #13603: Airflow 2.0 - ModuleNotFoundError: No module named 'airflow.providers.http'
Posted by GitBox <gi...@apache.org>.
colinglaes commented on issue #13603:
URL: https://github.com/apache/airflow/issues/13603#issuecomment-762442453
@potiuk Thanks for following up, I didn't use that flag when installing. Let me purge my installation and go through the installation steps again, retain the command and output history from the terminal here to give you better context.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #13603: Airflow 2.0 - ModuleNotFoundError: No module named 'airflow.providers.http'
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #13603:
URL: https://github.com/apache/airflow/issues/13603#issuecomment-757923733
You probably have Airflow installed in `--editable` mode from sources rather than packages.
You should remove the 'apache-airflow-providers*' if you install airflow in --editable mode (Though I would argue this is not the best Idea to run airflow in --editable mode in production. (Though I am kind of surprised how many people do that).
You can achieve that by setting `INSTALL_PROVIDERS_FROM_SOURCES` variable to `true`
This has been already addressed in #13439 in 2.0.1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #13603: Airflow 2.0 - ModuleNotFoundError: No module named 'airflow.providers.http'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13603:
URL: https://github.com/apache/airflow/issues/13603#issuecomment-759558933
I am looking at similar problem so I might come back with it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #13603: Airflow 2.0 - ModuleNotFoundError: No module named 'airflow.providers.http'
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #13603:
URL: https://github.com/apache/airflow/issues/13603#issuecomment-757923733
You probably likely have Airflow installed in `--editable` mode from sources rather than packages.
You should remove the 'apache-airflow-providers*' if you install airflow in --editable mode (Though I would argue this is not the best Idea to run airflow in --editable mode in production. (Though I am kind of surprised how many people do that).
You can achieve that by setting `INSTALL_PROVIDERS_FROM_SOURCES` variable to `true`
This has been already addressed in #13439 in 2.0.1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #13603: Airflow 2.0 - ModuleNotFoundError: No module named 'airflow.providers.http'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13603:
URL: https://github.com/apache/airflow/issues/13603#issuecomment-762553532
> @potiuk I just went through the process again and have successfully got the scheduler and webserver running. I believe that the source of my issue was that i had some `apache_airflow_providers_*` installed under `/usr/local/lib/python3.6/dist-packages` and others under `/home/cglaes/.local/lib/python3.6/site-packages` from earlier botched attempts that i hadn't removed in my last installation attempt.
Cool.
> I did have some thoughts though as to how the documentation / error messages could be improved though
>
> **https://airflow.apache.org/docs/apache-airflow/stable/installation.html**
>
> * `pip upgrade --pip==20.2.4` is an invalid command
Yep. This is already fixed in the upcoming 2.0.1 version: https://github.com/apache/airflow/blob/master/docs/apache-airflow/installation.rst
> **Messages from `airflow upgrade_check`**
>
> * "fernet_key in airflow.cfg must be explicitly set empty as fernet mechanism is enabledby default. This means that the apache-airflow[crypto] extra-packages are always installed.However, this requires that your operating system has libffi-dev installed.". I think it might be clearer to explicitly state that the config key should be set as `fernet_key = None` instead of saying it "must be explicitly set empty". Not everyone knows what empty means in this case (i could see many people setting it like `fernet_key = `)
> * "1. The following invalid keys were found in your airflow.cfg`comes up and recommends removing the deprecated keys, additionally it states "Please generate a pod_template_file by running`airflow generate_pod_template`", there isn't an explicit statement that running this command is conditional on having a Kubernetes setup. Maybe these checks could be separated to improve clarity.
>
> Let me know if it would be helpful for you guys if i paste the output from my terminal
Would you be so kind to open Pull Request fixing those messages/statements ?
The Upgrade check is here: https://github.com/apache/airflow/tree/v1-10-stable/airflow/upgrade and it would be great if you could make a PR to v1-10-stable branch to fix those.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org