You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/03/02 13:07:22 UTC
[GitHub] [airflow] vidyadhar821 opened a new issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
vidyadhar821 opened a new issue #14561:
URL: https://github.com/apache/airflow/issues/14561
Hi,
I have installed package apache-airflow-providers-snowflake on airflow on docker and i am getting error as "No module named Snowflake". After certain analysis i found that the the snowflake hook operator source code is getting saved with file name snowflake.py and within the source code it tries to import snowflake module. So basically it is trying to import itself.
Has anyone encountered this issue ? and please suggest possible solution for same
Please refer the error
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #14561:
URL: https://github.com/apache/airflow/issues/14561
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #14561:
URL: https://github.com/apache/airflow/issues/14561#issuecomment-822092344
This issue has been closed because it has not received response from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] xinbinhuang commented on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
Posted by GitBox <gi...@apache.org>.
xinbinhuang commented on issue #14561:
URL: https://github.com/apache/airflow/issues/14561#issuecomment-789324241
Can you share a bit more details on how you start the webserver and docker? It seems to be missing the `snowflake` dependencies (and that package is also named `snowflake` FYI)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vidyadhar821 commented on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
Posted by GitBox <gi...@apache.org>.
vidyadhar821 commented on issue #14561:
URL: https://github.com/apache/airflow/issues/14561#issuecomment-789411770
I have used airflow 2.0.1 docker image 2.0.1 and installed the dependencies using cli once the airflow container is up.
see the packages installed below (apache-airflow-providers-snowflake, snowflake-connector-python, snowflake-sqlalchemy is installed)
adal==1.2.6
aiohttp==3.7.3
alembic==1.5.4
amqp==2.6.1
apache-airflow==2.0.1
apache-airflow-providers-amazon==1.1.0
apache-airflow-providers-celery==1.0.1
apache-airflow-providers-cncf-kubernetes==1.0.1
apache-airflow-providers-docker==1.0.1
apache-airflow-providers-elasticsearch==1.0.1
apache-airflow-providers-ftp==1.0.1
apache-airflow-providers-google==1.0.0
apache-airflow-providers-grpc==1.0.1
apache-airflow-providers-hashicorp==1.0.1
apache-airflow-providers-http==1.1.0
apache-airflow-providers-imap==1.0.1
apache-airflow-providers-microsoft-azure==1.1.0
apache-airflow-providers-mysql==1.0.1
apache-airflow-providers-postgres==1.0.1
apache-airflow-providers-redis==1.0.1
apache-airflow-providers-sendgrid==1.0.1
apache-airflow-providers-sftp==1.1.0
apache-airflow-providers-slack==2.0.0
apache-airflow-providers-snowflake==1.1.0
apache-airflow-providers-sqlite==1.0.1
apache-airflow-providers-ssh==1.1.0
apispec==3.3.2
appdirs==1.4.4
argcomplete==1.12.2
asn1crypto==1.4.0
async-timeout==3.0.1
attrs==20.3.0
azure-batch==10.0.0
azure-common==1.1.26
azure-core==1.11.0
azure-cosmos==3.2.0
azure-datalake-store==0.0.51
azure-identity==1.5.0
azure-keyvault==4.1.0
azure-keyvault-certificates==4.2.1
azure-keyvault-keys==4.3.1
azure-keyvault-secrets==4.2.0
azure-kusto-data==0.0.45
azure-mgmt-containerinstance==1.5.0
azure-mgmt-core==1.2.2
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==0.5.0
azure-mgmt-nspkg==3.0.2
azure-mgmt-resource==15.0.0
azure-nspkg==3.0.2
azure-storage-blob==12.8.0
azure-storage-common==2.1.0
azure-storage-file==2.1.0
Babel==2.9.0
bcrypt==3.2.0
billiard==3.6.3.0
boto3==1.15.18
botocore==1.18.18
cached-property==1.5.2
cachetools==4.2.1
cattrs==1.0.0
celery==4.4.7
certifi==2020.12.5
cffi==1.14.5
chardet==3.0.4
click==7.1.2
clickclick==20.10.2
cloudpickle==1.4.1
colorama==0.4.4
colorlog==4.7.2
commonmark==0.9.1
connexion==2.7.0
contextvars==2.4
croniter==0.3.37
cryptography==3.4.5
dask==2021.2.0
dataclasses==0.7
defusedxml==0.6.0
dill==0.3.3
distlib==0.3.1
distributed==2.19.0
dnspython==1.16.0
docker==3.7.3
docker-pycreds==0.4.0
docutils==0.16
elasticsearch==7.5.1
elasticsearch-dbapi==0.1.0
elasticsearch-dsl==7.3.0
email-validator==1.1.2
eventlet==0.30.1
filelock==3.0.12
Flask==1.1.2
Flask-AppBuilder==3.1.1
Flask-Babel==1.0.0
Flask-Caching==1.9.0
Flask-JWT-Extended==3.25.1
Flask-Login==0.4.1
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.4.4
Flask-WTF==0.14.3
flower==0.9.7
gevent==21.1.2
google-ads==4.0.0
google-api-core==1.26.0
google-api-python-client==1.12.8
google-auth==1.26.1
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.2
google-cloud-automl==1.0.1
google-cloud-bigquery==2.8.0
google-cloud-bigquery-datatransfer==1.1.1
google-cloud-bigquery-storage==2.2.1
google-cloud-bigtable==1.7.0
google-cloud-container==1.0.1
google-cloud-core==1.6.0
google-cloud-datacatalog==0.7.0
google-cloud-dataproc==1.1.1
google-cloud-dlp==1.0.0
google-cloud-kms==1.4.0
google-cloud-language==1.3.0
google-cloud-logging==1.15.1
google-cloud-memcache==0.3.0
google-cloud-monitoring==1.1.0
google-cloud-os-login==1.0.0
google-cloud-pubsub==1.7.0
google-cloud-redis==1.0.0
google-cloud-secret-manager==1.0.0
google-cloud-spanner==1.19.1
google-cloud-speech==1.3.2
google-cloud-storage==1.36.0
google-cloud-tasks==1.5.0
google-cloud-texttospeech==1.0.1
google-cloud-translate==1.7.0
google-cloud-videointelligence==1.16.1
google-cloud-vision==1.0.0
google-crc32c==1.1.2
google-resumable-media==1.2.0
googleapis-common-protos==1.52.0
graphviz==0.16
greenlet==1.0.0
grpc-google-iam-v1==0.12.3
grpcio==1.35.0
grpcio-gcp==0.2.2
gunicorn==19.10.0
HeapDict==1.0.1
httplib2==0.19.0
humanize==3.2.0
hvac==0.10.8
idna==2.10
idna-ssl==1.1.0
immutables==0.15
importlib-metadata==1.7.0
importlib-resources==1.5.0
inflection==0.5.1
iso8601==0.1.14
isodate==0.6.0
itsdangerous==1.1.0
Jinja2==2.11.3
jmespath==0.10.0
json-merge-patch==0.2
jsonschema==3.2.0
kombu==4.6.11
kubernetes==11.0.0
lazy-object-proxy==1.4.3
ldap3==2.9
libcst==0.3.17
lockfile==0.12.2
Mako==1.1.4
Markdown==3.3.3
MarkupSafe==1.1.1
marshmallow==3.10.0
marshmallow-enum==1.5.1
marshmallow-oneofschema==2.1.0
marshmallow-sqlalchemy==0.23.1
msal==1.9.0
msal-extensions==0.3.0
msgpack==1.0.2
msrest==0.6.21
msrestazure==0.6.4
multidict==5.1.0
mypy-extensions==0.4.3
mysql-connector-python==8.0.22
mysqlclient==1.3.14
natsort==7.1.1
numpy==1.19.5
oauthlib==2.1.0
openapi-spec-validator==0.2.9
oscrypto==1.2.1
packaging==20.9
pandas==1.1.5
pandas-gbq==0.14.1
paramiko==2.7.2
pendulum==2.1.2
pep562==1.0
portalocker==1.7.1
prison==0.1.3
prometheus-client==0.8.0
proto-plus==1.13.0
protobuf==3.14.0
psutil==5.8.0
psycopg2-binary==2.8.6
pyarrow==0.17.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
pycryptodomex==3.10.1
pydata-google-auth==1.1.0
Pygments==2.8.0
PyJWT==1.7.1
PyNaCl==1.4.0
pyOpenSSL==19.1.0
pyparsing==2.4.7
pyrsistent==0.17.3
pysftp==0.2.9
python-daemon==2.2.4
python-dateutil==2.8.1
python-editor==1.0.4
python-http-client==3.3.2
python-ldap==3.3.1
python-nvd3==0.15.0
python-slugify==4.0.1
python3-openid==3.2.0
pytz==2020.5
pytzdata==2020.1
PyYAML==5.4.1
redis==3.5.3
requests==2.25.1
requests-oauthlib==1.1.0
rich==9.2.0
rsa==4.7
s3transfer==0.3.4
sendgrid==6.6.0
setproctitle==1.2.2
six==1.15.0
slack-sdk==3.3.2
slackclient==2.9.3
snowflake-connector-python==2.3.10
snowflake-sqlalchemy==1.2.4
sortedcontainers==2.3.0
SQLAlchemy==1.3.23
SQLAlchemy-JSONField==1.0.0
SQLAlchemy-Utils==0.36.8
sshtunnel==0.1.5
starkbank-ecdsa==1.1.0
statsd==3.3.0
swagger-ui-bundle==0.0.8
tabulate==0.8.7
tblib==1.7.0
tenacity==6.2.0
termcolor==1.1.0
text-unidecode==1.3
toolz==0.11.1
tornado==6.1
typing==3.7.4.3
typing-extensions==3.7.4.3
typing-inspect==0.6.0
unicodecsv==0.14.1
uritemplate==3.0.1
urllib3==1.25.11
vine==1.3.0
virtualenv==20.4.2
watchtower==0.7.3
websocket-client==0.57.0
Werkzeug==1.0.1
WTForms==2.3.3
yarl==1.6.3
zict==2.0.0
zipp==3.4.0
zope.event==4.5.0
zope.interface==5.2.0
$ pip list | grep snowflake
apache-airflow-providers-snowflake 1.1.0
snowflake-connector-python 2.3.10
snowflake-sqlalchemy 1.2.4
Also check the dag attached where issue is observed
[snowflake_poc.txt](https://github.com/apache/airflow/files/6073490/snowflake_poc.txt)
I guess the issue might be due to below reason
The snowflake hook is located at
/home/airflow/.local/lib/python3.6/site-packages/airflow/providers/snowflake/hooks/snowflake.py
and in the python script "snowflake" module is imported as seen below
Since the snowflake hook file is named as snowflake.py , hence it is trying to import itself and due to this reason its giving the mentioned error.
Thanks
Vidyadhar
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #14561:
URL: https://github.com/apache/airflow/issues/14561#issuecomment-817220897
This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] xinbinhuang edited a comment on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
Posted by GitBox <gi...@apache.org>.
xinbinhuang edited a comment on issue #14561:
URL: https://github.com/apache/airflow/issues/14561#issuecomment-789895391
I tried to reproduce on my end but I didn't get any errors.
```bash
# outside docker container
docker run -it --rm -v "${PWD}"/dags:/opt/airflow/dags apache/airflow:2.0.1 bash
# inside airflow container
pip install apache-airflow-providers-snowflake==1.1.0
airflow db init
python -c "from airflow.models import DagBag; print(DagBag().get_dag('snowflake_poc'))"
```
Please try out the above command and let us know if you still encounter any errors.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vidyadhar821 edited a comment on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
Posted by GitBox <gi...@apache.org>.
vidyadhar821 edited a comment on issue #14561:
URL: https://github.com/apache/airflow/issues/14561#issuecomment-789411770
@xinbinhuang , I have used airflow 2.0.1 docker image 2.0.1 and installed the dependencies using cli once the airflow container is up.
see the packages installed below (apache-airflow-providers-snowflake, snowflake-connector-python, snowflake-sqlalchemy is installed)
adal==1.2.6
aiohttp==3.7.3
alembic==1.5.4
amqp==2.6.1
apache-airflow==2.0.1
apache-airflow-providers-amazon==1.1.0
apache-airflow-providers-celery==1.0.1
apache-airflow-providers-cncf-kubernetes==1.0.1
apache-airflow-providers-docker==1.0.1
apache-airflow-providers-elasticsearch==1.0.1
apache-airflow-providers-ftp==1.0.1
apache-airflow-providers-google==1.0.0
apache-airflow-providers-grpc==1.0.1
apache-airflow-providers-hashicorp==1.0.1
apache-airflow-providers-http==1.1.0
apache-airflow-providers-imap==1.0.1
apache-airflow-providers-microsoft-azure==1.1.0
apache-airflow-providers-mysql==1.0.1
apache-airflow-providers-postgres==1.0.1
apache-airflow-providers-redis==1.0.1
apache-airflow-providers-sendgrid==1.0.1
apache-airflow-providers-sftp==1.1.0
apache-airflow-providers-slack==2.0.0
apache-airflow-providers-snowflake==1.1.0
apache-airflow-providers-sqlite==1.0.1
apache-airflow-providers-ssh==1.1.0
apispec==3.3.2
appdirs==1.4.4
argcomplete==1.12.2
asn1crypto==1.4.0
async-timeout==3.0.1
attrs==20.3.0
azure-batch==10.0.0
azure-common==1.1.26
azure-core==1.11.0
azure-cosmos==3.2.0
azure-datalake-store==0.0.51
azure-identity==1.5.0
azure-keyvault==4.1.0
azure-keyvault-certificates==4.2.1
azure-keyvault-keys==4.3.1
azure-keyvault-secrets==4.2.0
azure-kusto-data==0.0.45
azure-mgmt-containerinstance==1.5.0
azure-mgmt-core==1.2.2
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==0.5.0
azure-mgmt-nspkg==3.0.2
azure-mgmt-resource==15.0.0
azure-nspkg==3.0.2
azure-storage-blob==12.8.0
azure-storage-common==2.1.0
azure-storage-file==2.1.0
Babel==2.9.0
bcrypt==3.2.0
billiard==3.6.3.0
boto3==1.15.18
botocore==1.18.18
cached-property==1.5.2
cachetools==4.2.1
cattrs==1.0.0
celery==4.4.7
certifi==2020.12.5
cffi==1.14.5
chardet==3.0.4
click==7.1.2
clickclick==20.10.2
cloudpickle==1.4.1
colorama==0.4.4
colorlog==4.7.2
commonmark==0.9.1
connexion==2.7.0
contextvars==2.4
croniter==0.3.37
cryptography==3.4.5
dask==2021.2.0
dataclasses==0.7
defusedxml==0.6.0
dill==0.3.3
distlib==0.3.1
distributed==2.19.0
dnspython==1.16.0
docker==3.7.3
docker-pycreds==0.4.0
docutils==0.16
elasticsearch==7.5.1
elasticsearch-dbapi==0.1.0
elasticsearch-dsl==7.3.0
email-validator==1.1.2
eventlet==0.30.1
filelock==3.0.12
Flask==1.1.2
Flask-AppBuilder==3.1.1
Flask-Babel==1.0.0
Flask-Caching==1.9.0
Flask-JWT-Extended==3.25.1
Flask-Login==0.4.1
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.4.4
Flask-WTF==0.14.3
flower==0.9.7
gevent==21.1.2
google-ads==4.0.0
google-api-core==1.26.0
google-api-python-client==1.12.8
google-auth==1.26.1
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.2
google-cloud-automl==1.0.1
google-cloud-bigquery==2.8.0
google-cloud-bigquery-datatransfer==1.1.1
google-cloud-bigquery-storage==2.2.1
google-cloud-bigtable==1.7.0
google-cloud-container==1.0.1
google-cloud-core==1.6.0
google-cloud-datacatalog==0.7.0
google-cloud-dataproc==1.1.1
google-cloud-dlp==1.0.0
google-cloud-kms==1.4.0
google-cloud-language==1.3.0
google-cloud-logging==1.15.1
google-cloud-memcache==0.3.0
google-cloud-monitoring==1.1.0
google-cloud-os-login==1.0.0
google-cloud-pubsub==1.7.0
google-cloud-redis==1.0.0
google-cloud-secret-manager==1.0.0
google-cloud-spanner==1.19.1
google-cloud-speech==1.3.2
google-cloud-storage==1.36.0
google-cloud-tasks==1.5.0
google-cloud-texttospeech==1.0.1
google-cloud-translate==1.7.0
google-cloud-videointelligence==1.16.1
google-cloud-vision==1.0.0
google-crc32c==1.1.2
google-resumable-media==1.2.0
googleapis-common-protos==1.52.0
graphviz==0.16
greenlet==1.0.0
grpc-google-iam-v1==0.12.3
grpcio==1.35.0
grpcio-gcp==0.2.2
gunicorn==19.10.0
HeapDict==1.0.1
httplib2==0.19.0
humanize==3.2.0
hvac==0.10.8
idna==2.10
idna-ssl==1.1.0
immutables==0.15
importlib-metadata==1.7.0
importlib-resources==1.5.0
inflection==0.5.1
iso8601==0.1.14
isodate==0.6.0
itsdangerous==1.1.0
Jinja2==2.11.3
jmespath==0.10.0
json-merge-patch==0.2
jsonschema==3.2.0
kombu==4.6.11
kubernetes==11.0.0
lazy-object-proxy==1.4.3
ldap3==2.9
libcst==0.3.17
lockfile==0.12.2
Mako==1.1.4
Markdown==3.3.3
MarkupSafe==1.1.1
marshmallow==3.10.0
marshmallow-enum==1.5.1
marshmallow-oneofschema==2.1.0
marshmallow-sqlalchemy==0.23.1
msal==1.9.0
msal-extensions==0.3.0
msgpack==1.0.2
msrest==0.6.21
msrestazure==0.6.4
multidict==5.1.0
mypy-extensions==0.4.3
mysql-connector-python==8.0.22
mysqlclient==1.3.14
natsort==7.1.1
numpy==1.19.5
oauthlib==2.1.0
openapi-spec-validator==0.2.9
oscrypto==1.2.1
packaging==20.9
pandas==1.1.5
pandas-gbq==0.14.1
paramiko==2.7.2
pendulum==2.1.2
pep562==1.0
portalocker==1.7.1
prison==0.1.3
prometheus-client==0.8.0
proto-plus==1.13.0
protobuf==3.14.0
psutil==5.8.0
psycopg2-binary==2.8.6
pyarrow==0.17.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
pycryptodomex==3.10.1
pydata-google-auth==1.1.0
Pygments==2.8.0
PyJWT==1.7.1
PyNaCl==1.4.0
pyOpenSSL==19.1.0
pyparsing==2.4.7
pyrsistent==0.17.3
pysftp==0.2.9
python-daemon==2.2.4
python-dateutil==2.8.1
python-editor==1.0.4
python-http-client==3.3.2
python-ldap==3.3.1
python-nvd3==0.15.0
python-slugify==4.0.1
python3-openid==3.2.0
pytz==2020.5
pytzdata==2020.1
PyYAML==5.4.1
redis==3.5.3
requests==2.25.1
requests-oauthlib==1.1.0
rich==9.2.0
rsa==4.7
s3transfer==0.3.4
sendgrid==6.6.0
setproctitle==1.2.2
six==1.15.0
slack-sdk==3.3.2
slackclient==2.9.3
snowflake-connector-python==2.3.10
snowflake-sqlalchemy==1.2.4
sortedcontainers==2.3.0
SQLAlchemy==1.3.23
SQLAlchemy-JSONField==1.0.0
SQLAlchemy-Utils==0.36.8
sshtunnel==0.1.5
starkbank-ecdsa==1.1.0
statsd==3.3.0
swagger-ui-bundle==0.0.8
tabulate==0.8.7
tblib==1.7.0
tenacity==6.2.0
termcolor==1.1.0
text-unidecode==1.3
toolz==0.11.1
tornado==6.1
typing==3.7.4.3
typing-extensions==3.7.4.3
typing-inspect==0.6.0
unicodecsv==0.14.1
uritemplate==3.0.1
urllib3==1.25.11
vine==1.3.0
virtualenv==20.4.2
watchtower==0.7.3
websocket-client==0.57.0
Werkzeug==1.0.1
WTForms==2.3.3
yarl==1.6.3
zict==2.0.0
zipp==3.4.0
zope.event==4.5.0
zope.interface==5.2.0
$ pip list | grep snowflake
apache-airflow-providers-snowflake 1.1.0
snowflake-connector-python 2.3.10
snowflake-sqlalchemy 1.2.4
Also check the dag attached where issue is observed
[snowflake_poc.txt](https://github.com/apache/airflow/files/6073490/snowflake_poc.txt)
I guess the issue might be due to below reason
The snowflake hook is located at
/home/airflow/.local/lib/python3.6/site-packages/airflow/providers/snowflake/hooks/snowflake.py
and in the python script "snowflake" module is imported as seen below
Since the snowflake hook file is named as snowflake.py , hence it is trying to import itself and due to this reason its giving the mentioned error.
Thanks
Vidyadhar
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #14561:
URL: https://github.com/apache/airflow/issues/14561#issuecomment-789419393
This is how python behaved before python 3.0. As of python 3.0 you can easily have the same module name as package name because by default in python 3+ all imports are absolute (except those starting with .) https://www.python.org/dev/peps/pep-0328/ . We have many modules name smilarly as packages and it is not a problem.
You would have to have PYTHONPATH changed to point the snowflake operators/ folder.
But the most probable reason is the installation of extra packages. If you are using the prod image, from Airflow you need to install the packages with --user flag, (`pip install --user PACKAGE`) as 'airflow' user. My guess is that you switched to root and installed the snowflake packages without the --user flag.
In 2.0.2 the --user flag will be enabled by default in the image.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] xinbinhuang edited a comment on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
Posted by GitBox <gi...@apache.org>.
xinbinhuang edited a comment on issue #14561:
URL: https://github.com/apache/airflow/issues/14561#issuecomment-789324241
Can you share a bit more details on how you start the webserver and docker? It seems to be missing the `snowflake` dependencies (and that package is also named [`snowflake`](https://docs.snowflake.com/en/user-guide/python-connector-example.html))
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] xinbinhuang commented on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
Posted by GitBox <gi...@apache.org>.
xinbinhuang commented on issue #14561:
URL: https://github.com/apache/airflow/issues/14561#issuecomment-789895391
I tried to reproduce on my end but I didn't get any errors.
```bash
docker run -it --rm -v "${PWD}"/dags:/opt/airflow/dags apache/airflow:2.0.1 bash
pip install apache-airflow-providers-snowflake==1.1.0
airflow db init
python -c "from airflow.models import DagBag; print(DagBag().get_dag('snowflake_poc'))"
```
Please try out the above command and let us know if you still encounter any errors.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #14561:
URL: https://github.com/apache/airflow/issues/14561#issuecomment-788895689
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] xinbinhuang edited a comment on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
Posted by GitBox <gi...@apache.org>.
xinbinhuang edited a comment on issue #14561:
URL: https://github.com/apache/airflow/issues/14561#issuecomment-789324241
Can you share a bit more details on how you start the webserver and docker? It seems to be missing the `snowflake` dependencies (and that package is also named [`snowflake`](https://docs.snowflake.com/en/user-guide/python-connector-example.html)), Also, what's the version for `snowflake-connector-python`? (you can get it by running this `pip list | grep snowflake`)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] xinbinhuang edited a comment on issue #14561: Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)
Posted by GitBox <gi...@apache.org>.
xinbinhuang edited a comment on issue #14561:
URL: https://github.com/apache/airflow/issues/14561#issuecomment-789324241
Can you share a bit more details on how you start the webserver and docker? It seems to be missing the `snowflake` dependencies(i.e. `snowflake-connector-python` package is also named [`snowflake`](https://docs.snowflake.com/en/user-guide/python-connector-example.html), so it's not importing itself), Also, what's the version for `snowflake-connector-python`? (you can get it by running this `pip list | grep snowflake`)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org