You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/04/20 10:44:32 UTC
[GitHub] [airflow] ClassyLion opened a new issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
ClassyLion opened a new issue #15451:
URL: https://github.com/apache/airflow/issues/15451
**Apache Airflow version**:
2.0.2
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
v1.18.18
**Environment**:
Cloud provider or hardware configuration: AWS
**What happened**:
Updated to Airflow 2.0.2 and a new warning appeared in webserver logs:
```
WARNING - Exception when importing 'airflow.providers.google.common.hooks.leveldb.LevelDBHook' from 'apache-airflow-providers-google' package: No module named 'airflow.providers.google.common.hooks.leveldb'
```
**What you expected to happen**:
No warning.
**How to reproduce it**:
Don't know the specific details. Have tried adding `pip install --upgrade apache-airflow-providers-google` but the error was still there.
**Anything else we need to know**:
I am not using LevelDB for anything in my code, as a result I don't understand from where this error is coming.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] oliakhovych edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
oliakhovych edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-1025547819
I have the same problem during completing command:
`docker-compose up`
Output:
`WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.google.leveldb.hooks.leveldb.LevelDBHook' from 'apache-airflow-providers-google' package: No module named 'plyvel'`
I tried this (didn't help me):
`pip install --upgrade apache-airflow-providers-google`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-824001450
For the future reference @KIRY4 - this is the root cause (added in FAB 3.2.0). For now i fixed constraints viaa #15470 but in a few days when FAB 3.2.3 gets released we will be able to us this: https://github.com/dpgaspar/Flask-AppBuilder/pull/1602#issuecomment-823962936
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] KIRY4 commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
KIRY4 commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-829278278
@potiuk fixed, now everything works perfect! thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ClassyLion commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
ClassyLion commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-823219979
Airflow 2.0.1 and 2.0.0 did not have this behaviour. I've been actively checking logs recently with the upgrade to Airflow 2, so I can say that this warning was not there.
I thought that this warning could be ignored, it's just coming up quite often in the logs, thus increasing their length.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-824003807
BTW. I recommend to use corresponding `docker build .` command to build the image rather than breeze.
See the latest documentation for this with nice examples: https://airflow.apache.org/docs/docker-stack/build.html#customizing-the-image
It's much simpler now to build the images and when you use docker build commands by default the released packages are used from PYPI to build the image rather than local source code. Unless you want to use latest master, version of Airflow/providers - this is the recommended way now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-824003807
BTW. I recommend to use corresponding `docker build .` command to build the image rather than breeze.
See the latest documentation for this with nice examples: https://airflow.apache.org/docs/docker-stack/build.html#customizing-the-image
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] KIRY4 edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
KIRY4 edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-823726044
I'm not sure if it related with mentioned error. But now I'm trying to login to freshly installed airflow 2.0.2 with LDAP integration and I can't login with my credentials. Log of web pod full of mentioned error/warning:
`WARNING - Exception when importing 'airflow.providers.google.common.hooks.leveldb.LevelDBHook' from 'apache-airflow-providers-google' package: No module named 'airflow.providers.google.common.hooks.leveldb'.`
```
airflow@airflow-web-749bb5988d-hv8xj:/opt/airflow$ airflow version
2.0.2
```
Everything works fine for me with 2.0.0 and 2.0.1 with my credentials from AD. My Airflow running on AKS if it matters.
Here is my webserver_config.py (maybe it also will be useful):
```
import os
from airflow import configuration as conf
from flask_appbuilder.security.manager import AUTH_LDAP
basedir = os.path.abspath(os.path.dirname(__file__))
SQLALCHEMY_DATABASE_URI = conf.get('core', 'SQL_ALCHEMY_CONN')
CSRF_ENABLED = True
AUTH_TYPE = AUTH_LDAP
AUTH_ROLE_ADMIN = 'Admin'
AUTH_USER_REGISTRATION = True
#AUTH_USER_REGISTRATION_ROLE = "Admin"
AUTH_USER_REGISTRATION_ROLE = "User"
AUTH_LDAP_SERVER = 'ldaps://***.com:636'
AUTH_LDAP_SEARCH = "DC=***,DC=com"
AUTH_LDAP_BIND_USER = '***'
AUTH_LDAP_BIND_PASSWORD = '***'
AUTH_LDAP_UID_FIELD = '***'
AUTH_LDAP_USE_TLS = False
AUTH_LDAP_ALLOW_SELF_SIGNED = False
AUTH_LDAP_TLS_CACERTFILE = '/opt/airflow/ldap/ldap_ca.crt'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-836703445
It is solved in the most recent version of the google provider (3.0.0). Currently when you install 2.0.2 you get the previous version of provider installed, but you can manually upgrade it - by extending the image and adding
`pip install --upgrade apache-airflow-providers-google`
It will be solved by out-of-the-box installation when we release `2.0.3`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-823245238
Thanks @ClassyLion.
It's a bug (harmless) in definition of the google provider 2.2.0 in fact:
In `provider.yaml`:
`airflow.providers.google.common.hooks.leveldb.LevelDBHook`
should be:
`airflow.providers.google.common.leveldb.LevelDBHook`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-827764391
@KIRY4 -> would it be possible that you check if upgrading FAB to 3.2.3 ( released yesterday) fixes the problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-1025630319
Upgrading dependencies in docker-compose requires you to update the image, not upgrade dependencie manually. You should read more how docker-compose works - see the note in https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html
> DO NOT expect the Docker Compose below will be enough to run production-ready Docker Compose Airflow installation using it. This is truly quick-start docker-compose for you to get Airflow up and running locally and get your hands dirty with Airflow. Configuring a Docker-Compose installation that is ready for production requires an intrinsic knowledge of Docker Compose, a lot of customization and possibly even writing the Docker Compose file that will suit your needs from the scratch. It’s probably OK if you want to run Docker Compose-based deployment, but short of becoming a Docker Compose expert, it’s highly unlikely you will get robust deployment with it.
> If you want to get an easy to configure Docker-based deployment that Airflow Community develops, supports and can provide support with deployment, you should consider using Kubernetes and deploying Airflow using Official Airflow Community Helm Chart.
If you are not ready to manage and build your images, don't use docker compose other than just for quick start. Use local quick start instead: https://airflow.apache.org/docs/apache-airflow/stable/start/local.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] oliakhovych commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
oliakhovych commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-1025547819
I have the same problem during completing command:
`docker-compose up`
Output:
`WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.google.leveldb.hooks.leveldb.LevelDBHook' from 'apache-airflow-providers-google' package: No module named 'plyvel'`
I tried this (didn`t help me):
`pip install --upgrade apache-airflow-providers-google`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] maryampashmi edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
maryampashmi edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-836684818
> airflow.providers.google.leveldb.hooks.LevelDBHook
This is not solved. I am getting the same error. Just tried to create a docker to install the module separately and this is the error I am getting
```
=> ERROR [2/2] RUN pip install --upgrade "pip==21.1.1" && pip install airflow.providers.google.leveldb.hooks.LevelDBHook && pip install 7.7s
------
> [2/2] RUN pip install --upgrade "pip==21.1.1" && pip install airflow.providers.google.leveldb.hooks.LevelDBHook && pip install apache-airflow-providers-oracle==1.1.0 && pip install apache-airflow-providers-mysql==1.1.0 && pip install azure-storage-blob:
#5 0.265 + pip install --upgrade pip==21.1.1
#5 0.990 Requirement already satisfied: pip==21.1.1 in /home/airflow/.local/lib/python3.6/site-packages (21.1.1)
#5 6.744 + pip install airflow.providers.google.leveldb.hooks.LevelDBHook
#5 7.610 ERROR: Could not find a version that satisfies the requirement airflow.providers.google.leveldb.hooks.LevelDBHook (from versions: none)
#5 7.610 ERROR: No matching distribution found for airflow.providers.google.leveldb.hooks.LevelDBHook
------
executor failed running [/bin/bash -o pipefail -e -u -x -c pip install --upgrade "pip==21.1.1" && pip install airflow.providers.google.leveldb.hooks.LevelDBHook && pip install apache-airflow-providers-oracle==1.1.0 && pip install apache-airflow-providers-mysql==1.1.0 && pip install azure-storage-blob]: exit code: 1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] KIRY4 commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
KIRY4 commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-823903390
@potiuk
I'm building bit customized image using Breeze:
```
AIRFLOW_VERSION=2.0.2
./breeze-airflow/breeze build-image \
--production-image --python 3.8 --install-airflow-version=$AIRFLOW_VERSION \
--additional-extras=doc,jdbc,odbc,oracle,snowflake --additional-python-deps="jaydebeapi azure-storage-blob" \
--image-tag myacrdev01.azurecr.io/airflow:$AIRFLOW_VERSION-base
```
Later deploying it using following Helm chart: https://github.com/airflow-helm/charts/tree/main/charts/airflow on AKS 1.18.14.
And yes, I'm getting following in logs from web pod:
`[2021-04-21 08:59:23,812] {manager.py:887} ERROR - LDAP search for '(sAMAccountName=***)' in scope 'DC=***,DC=com' returned multiple results`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] maryampashmi commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
maryampashmi commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-836684818
> airflow.providers.google.leveldb.hooks.LevelDBHook
This is didn't solved. I am getting the same error. Just tried to create a docker to install the module separately and this is the error I am getting
```
=> ERROR [2/2] RUN pip install --upgrade "pip==21.1.1" && pip install airflow.providers.google.leveldb.hooks.LevelDBHook && pip install 7.7s
------
> [2/2] RUN pip install --upgrade "pip==21.1.1" && pip install airflow.providers.google.leveldb.hooks.LevelDBHook && pip install apache-airflow-providers-oracle==1.1.0 && pip install apache-airflow-providers-mysql==1.1.0 && pip install azure-storage-blob:
#5 0.265 + pip install --upgrade pip==21.1.1
#5 0.990 Requirement already satisfied: pip==21.1.1 in /home/airflow/.local/lib/python3.6/site-packages (21.1.1)
#5 6.744 + pip install airflow.providers.google.leveldb.hooks.LevelDBHook
#5 7.610 ERROR: Could not find a version that satisfies the requirement airflow.providers.google.leveldb.hooks.LevelDBHook (from versions: none)
#5 7.610 ERROR: No matching distribution found for airflow.providers.google.leveldb.hooks.LevelDBHook
------
executor failed running [/bin/bash -o pipefail -e -u -x -c pip install --upgrade "pip==21.1.1" && pip install airflow.providers.google.leveldb.hooks.LevelDBHook && pip install apache-airflow-providers-oracle==1.1.0 && pip install apache-airflow-providers-mysql==1.1.0 && pip install azure-storage-blob]: exit code: 1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] KIRY4 edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
KIRY4 edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-823726044
I'm not sure if it related with mentioned error. But now I'm trying to login to freshly installed airflow 2.0.2 with LDAP integration and I can't login. Log of web pod full of mentioned error/warning:
`WARNING - Exception when importing 'airflow.providers.google.common.hooks.leveldb.LevelDBHook' from 'apache-airflow-providers-google' package: No module named 'airflow.providers.google.common.hooks.leveldb'.`
Everything works fine for me with 2.0.0 and 2.0.1 with my credentials from AD. My Airflow deployed on AKS if it matters.
Here is my webserver_config.py (maybe it also will be useful):
```
import os
from airflow import configuration as conf
from flask_appbuilder.security.manager import AUTH_LDAP
basedir = os.path.abspath(os.path.dirname(__file__))
SQLALCHEMY_DATABASE_URI = conf.get('core', 'SQL_ALCHEMY_CONN')
CSRF_ENABLED = True
AUTH_TYPE = AUTH_LDAP
AUTH_ROLE_ADMIN = 'Admin'
AUTH_USER_REGISTRATION = True
#AUTH_USER_REGISTRATION_ROLE = "Admin"
AUTH_USER_REGISTRATION_ROLE = "User"
AUTH_LDAP_SERVER = 'ldaps://***.com:636'
AUTH_LDAP_SEARCH = "DC=***,DC=com"
AUTH_LDAP_BIND_USER = '***'
AUTH_LDAP_BIND_PASSWORD = '***'
AUTH_LDAP_UID_FIELD = '***'
AUTH_LDAP_USE_TLS = False
AUTH_LDAP_ALLOW_SELF_SIGNED = False
AUTH_LDAP_TLS_CACERTFILE = '/opt/airflow/ldap/ldap_ca.crt'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] KIRY4 commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
KIRY4 commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-823726044
I'm not sure if it related with mentioned error. But now I'm trying to login to freshly installed airflow 2.0.2 with LDAP integration and I can't login. Log of web pod full of mentioned error/warning:
`WARNING - Exception when importing 'airflow.providers.google.common.hooks.leveldb.LevelDBHook' from 'apache-airflow-providers-google' package: No module named 'airflow.providers.google.common.hooks.leveldb'.`
Everything works fine for me with 2.0.0 and 2.0.1 with my credentials from AD. My Airflow deployed on AKS if it if it matters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] KIRY4 edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
KIRY4 edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-829278278
@potiuk fixed! now everything works perfect! thank you! I mean that I successfully login but warning still exist in logs:
WARNING - Exception when importing 'airflow.providers.google.common.hooks.leveldb.LevelDBHook' from 'apache-airflow-providers-google' package: No module named 'airflow.providers.google.common.hooks.leveldb'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] KIRY4 edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
KIRY4 edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-823726044
I'm not sure if it related with mentioned error. But now I'm trying to login to freshly installed airflow 2.0.2 with LDAP integration and I can't login with my credentials. Log of web pod full of mentioned error/warning:
`WARNING - Exception when importing 'airflow.providers.google.common.hooks.leveldb.LevelDBHook' from 'apache-airflow-providers-google' package: No module named 'airflow.providers.google.common.hooks.leveldb'.`
Everything works fine for me with 2.0.0 and 2.0.1 with my credentials from AD. My Airflow running on AKS if it matters.
Here is my webserver_config.py (maybe it also will be useful):
```
import os
from airflow import configuration as conf
from flask_appbuilder.security.manager import AUTH_LDAP
basedir = os.path.abspath(os.path.dirname(__file__))
SQLALCHEMY_DATABASE_URI = conf.get('core', 'SQL_ALCHEMY_CONN')
CSRF_ENABLED = True
AUTH_TYPE = AUTH_LDAP
AUTH_ROLE_ADMIN = 'Admin'
AUTH_USER_REGISTRATION = True
#AUTH_USER_REGISTRATION_ROLE = "Admin"
AUTH_USER_REGISTRATION_ROLE = "User"
AUTH_LDAP_SERVER = 'ldaps://***.com:636'
AUTH_LDAP_SEARCH = "DC=***,DC=com"
AUTH_LDAP_BIND_USER = '***'
AUTH_LDAP_BIND_PASSWORD = '***'
AUTH_LDAP_UID_FIELD = '***'
AUTH_LDAP_USE_TLS = False
AUTH_LDAP_ALLOW_SELF_SIGNED = False
AUTH_LDAP_TLS_CACERTFILE = '/opt/airflow/ldap/ldap_ca.crt'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #15451:
URL: https://github.com/apache/airflow/issues/15451
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] KIRY4 edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
KIRY4 edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-829278278
@potiuk fixed! now everything works perfect! thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-824001785
You can rebuild your image with `--force-pull-image` flag and it should work fine now @KIRY4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-823849690
@KIRY4 - can you see if you have error in your logs similar to:
```
ERROR - LDAP search for ........ returned multiple results
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-823182515
This I think can be ignored -- it is the webserver trying to load all the providers and it's failing on one, but this is not fatal and shouldn't stop the Webserver working.
Did 2.0.1 (or 2.0.0) have this same behaviour?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-823245238
Thanks @ClassyLion.
It's a bug (harmless) in definition of the google provider 2.2.0 in fact:
In `provider.yaml`:
`airflow.providers.google.common.hooks.leveldb.LevelDBHook`
should be:
`airflow.providers.google.leveldb.hooks.LevelDBHook`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-836703445
It is solved in the most recent version of the google provider (3.0.0). Currently when you install 2.0.2 you get the previous version of provider installed, but you can manually upgrade it - by extending the image and adding
`pip install --upgrade apache-airflow-providers-google`
It will be solved by out-of-the-box installation when we release `2.0.3` and `2.1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] maryampashmi edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
maryampashmi edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-836684818
> airflow.providers.google.leveldb.hooks.LevelDBHook
This is not solved. I am getting the same error. Just tried to create a docker to install the module separately and this is the error I am getting
```
=> ERROR [2/2] RUN pip install --upgrade "pip==21.1.1" && pip install airflow.providers.google.leveldb.hooks.LevelDBHook && pip install 7.7s
------
> [2/2] RUN pip install --upgrade "pip==21.1.1" && pip install airflow.providers.google.leveldb.hooks.LevelDBHook && pip install apache-airflow-providers-oracle==1.1.0 && pip install apache-airflow-providers-mysql==1.1.0 && pip install azure-storage-blob:
#5 0.265 + pip install --upgrade pip==21.1.1
#5 0.990 Requirement already satisfied: pip==21.1.1 in /home/airflow/.local/lib/python3.6/site-packages (21.1.1)
#5 6.744 + pip install airflow.providers.google.leveldb.hooks.LevelDBHook
#5 7.610 ERROR: Could not find a version that satisfies the requirement airflow.providers.google.leveldb.hooks.LevelDBHook (from versions: none)
#5 7.610 ERROR: No matching distribution found for airflow.providers.google.leveldb.hooks.LevelDBHook
------
executor failed running [/bin/bash -o pipefail -e -u -x -c pip install --upgrade "pip==21.1.1" && pip install airflow.providers.google.leveldb.hooks.LevelDBHook && pip install apache-airflow-providers-oracle==1.1.0 && pip install apache-airflow-providers-mysql==1.1.0 && pip install azure-storage-blob]: exit code: 1
```
What I suggest is to install complete package
pip install apache-airflow-providers-google
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] maryampashmi commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
maryampashmi commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-836841894
Using --upgrade, solved the error. I haven't get it anymore. Thanks for your help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] KIRY4 edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
KIRY4 edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-823726044
I'm not sure if it related with mentioned error. But now I'm trying to login to freshly installed airflow 2.0.2 with LDAP integration and I can't login with my credentials. Log of web pod full of mentioned error/warning:
```
WARNING - Exception when importing 'airflow.providers.google.common.hooks.leveldb.LevelDBHook' from 'apache-airflow-providers-google' package: No module named 'airflow.providers.google.common.hooks.leveldb'.
```
```
airflow@airflow-web-749bb5988d-hv8xj:/opt/airflow$ airflow version
2.0.2
```
Everything works fine for me with 2.0.0 and 2.0.1 with my credentials from AD. My Airflow running on AKS if it matters.
Here is my webserver_config.py (maybe it also will be useful):
```
import os
from airflow import configuration as conf
from flask_appbuilder.security.manager import AUTH_LDAP
basedir = os.path.abspath(os.path.dirname(__file__))
SQLALCHEMY_DATABASE_URI = conf.get('core', 'SQL_ALCHEMY_CONN')
CSRF_ENABLED = True
AUTH_TYPE = AUTH_LDAP
AUTH_ROLE_ADMIN = 'Admin'
AUTH_USER_REGISTRATION = True
#AUTH_USER_REGISTRATION_ROLE = "Admin"
AUTH_USER_REGISTRATION_ROLE = "User"
AUTH_LDAP_SERVER = 'ldaps://***.com:636'
AUTH_LDAP_SEARCH = "DC=***,DC=com"
AUTH_LDAP_BIND_USER = '***'
AUTH_LDAP_BIND_PASSWORD = '***'
AUTH_LDAP_UID_FIELD = '***'
AUTH_LDAP_USE_TLS = False
AUTH_LDAP_ALLOW_SELF_SIGNED = False
AUTH_LDAP_TLS_CACERTFILE = '/opt/airflow/ldap/ldap_ca.crt'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-823849690
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-829334514
> WARNING - Exception when importing 'airflow.providers.google.common.hooks.leveldb.LevelDBHook' from 'apache-airflow-providers-google' package: No module named 'airflow.providers.google.common.hooks.leveldb'
Yep. This will fixed when we release next Google Provider and you update either the provider or, Airlfow (we plan to release 2.0.3 very soon addressing a small subset of changes + PIP 21 installability). PR here: https://github.com/apache/airflow/pull/15576
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] KIRY4 edited a comment on issue #15451: No module named 'airflow.providers.google.common.hooks.leveldb'
Posted by GitBox <gi...@apache.org>.
KIRY4 edited a comment on issue #15451:
URL: https://github.com/apache/airflow/issues/15451#issuecomment-823726044
I'm not sure if it related with mentioned error. But now I'm trying to login to freshly installed airflow 2.0.2 with LDAP integration and I can't login with my credentials. Log of web pod full of mentioned error/warning:
`WARNING - Exception when importing 'airflow.providers.google.common.hooks.leveldb.LevelDBHook' from 'apache-airflow-providers-google' package: No module named 'airflow.providers.google.common.hooks.leveldb'.`
Everything works fine for me with 2.0.0 and 2.0.1 with my credentials from AD. My Airflow deployed on AKS if it matters.
Here is my webserver_config.py (maybe it also will be useful):
```
import os
from airflow import configuration as conf
from flask_appbuilder.security.manager import AUTH_LDAP
basedir = os.path.abspath(os.path.dirname(__file__))
SQLALCHEMY_DATABASE_URI = conf.get('core', 'SQL_ALCHEMY_CONN')
CSRF_ENABLED = True
AUTH_TYPE = AUTH_LDAP
AUTH_ROLE_ADMIN = 'Admin'
AUTH_USER_REGISTRATION = True
#AUTH_USER_REGISTRATION_ROLE = "Admin"
AUTH_USER_REGISTRATION_ROLE = "User"
AUTH_LDAP_SERVER = 'ldaps://***.com:636'
AUTH_LDAP_SEARCH = "DC=***,DC=com"
AUTH_LDAP_BIND_USER = '***'
AUTH_LDAP_BIND_PASSWORD = '***'
AUTH_LDAP_UID_FIELD = '***'
AUTH_LDAP_USE_TLS = False
AUTH_LDAP_ALLOW_SELF_SIGNED = False
AUTH_LDAP_TLS_CACERTFILE = '/opt/airflow/ldap/ldap_ca.crt'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org