You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/12 15:58:11 UTC
[GitHub] [airflow] gertnerbot opened a new issue #14208: Python 3.8 - Sqlite3 version error
gertnerbot opened a new issue #14208:
URL: https://github.com/apache/airflow/issues/14208
python 3.8 centos 7 Docker image
Can't update sqlite3. Tried Airflow 2.0.1 and 2.0.0. Same issue on Python 3.6 with Airflow 2.0.0. I was able to force the install on 2.0.0 but when running a task it failed because of the sqlite3 version mismatch.
Am I just stupid? #13496
#```
(app-root) airflow db init
Traceback (most recent call last):
File "/opt/app-root/bin/airflow", line 5, in <module>
from airflow.__main__ import main
File "/opt/app-root/lib64/python3.8/site-packages/airflow/__init__.py", line 34, in <module>
from airflow import settings
File "/opt/app-root/lib64/python3.8/site-packages/airflow/settings.py", line 37, in <module>
from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf # NOQA F401
File "/opt/app-root/lib64/python3.8/site-packages/airflow/configuration.py", line 1007, in <module>
conf.validate()
File "/opt/app-root/lib64/python3.8/site-packages/airflow/configuration.py", line 209, in validate
self._validate_config_dependencies()
File "/opt/app-root/lib64/python3.8/site-packages/airflow/configuration.py", line 246, in _validate_config_dependencies
raise AirflowConfigException(f"error: cannot use sqlite version < {min_sqlite_version}")
airflow.exceptions.AirflowConfigException: error: cannot use sqlite version < 3.15.0
(app-root) python -c "import sqlite3; print(sqlite3.sqlite_version)"
3.7.17
(app-root) python --version
Python 3.8.6
(app-root) pip install --upgrade sqlite3
ERROR: Could not find a version that satisfies the requirement sqlite3 (from versions: none)
ERROR: No matching distribution found for sqlite3
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14208: Python 3.8 - Sqlite3 version error
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #14208:
URL: https://github.com/apache/airflow/issues/14208#issuecomment-782913088
Maybe @subashcanapathy an/ @pgibert you could add a PR to guide people in https://github.com/apache/airflow/blob/master/docs/apache-airflow/howto/set-up-database.rst#setting-up-a-sqlite-database ?
Sounds like a useful "howto" part for people who use centos 7 / amazonlinux2 .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #14208: Python 3.8 - Sqlite3 version error
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #14208:
URL: https://github.com/apache/airflow/issues/14208
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #14208: Python 3.8 - Sqlite3 version error
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #14208:
URL: https://github.com/apache/airflow/issues/14208#issuecomment-778279682
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subashcanapathy commented on issue #14208: Python 3.8 - Sqlite3 version error
Posted by GitBox <gi...@apache.org>.
subashcanapathy commented on issue #14208:
URL: https://github.com/apache/airflow/issues/14208#issuecomment-782824716
I used the following on amazonlinux2 to get latest sqlite
```bash
# Pre-req
yum -y install wget tar gzip gcc make expect
wget https://www.sqlite.org/src/tarball/sqlite.tar.gz
tar xzf sqlite.tar.gz
cd sqlite/
export CFLAGS="-DSQLITE_ENABLE_FTS3 \
-DSQLITE_ENABLE_FTS3_PARENTHESIS \
-DSQLITE_ENABLE_FTS4 \
-DSQLITE_ENABLE_FTS5 \
-DSQLITE_ENABLE_JSON1 \
-DSQLITE_ENABLE_LOAD_EXTENSION \
-DSQLITE_ENABLE_RTREE \
-DSQLITE_ENABLE_STAT4 \
-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT \
-DSQLITE_SOUNDEX \
-DSQLITE_TEMP_STORE=3 \
-DSQLITE_USE_URI \
-O2 \
-fPIC"
export PREFIX="/usr/local"
LIBS="-lm" ./configure --disable-tcl --enable-shared --enable-tempstore=always --prefix="$PREFIX"
make
make install
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #14208: Python 3.8 - Sqlite3 version error
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #14208:
URL: https://github.com/apache/airflow/issues/14208#issuecomment-778291422
This is the version of the system sqlite library, not the python bindings.
I.e. `brew upgrade sqlite` or similar.
A PR to make that error message clearer would be happily merged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pgibert commented on issue #14208: Python 3.8 - Sqlite3 version error
Posted by GitBox <gi...@apache.org>.
pgibert commented on issue #14208:
URL: https://github.com/apache/airflow/issues/14208#issuecomment-781387974
_Hello close to your problem i think ... .. i got the same error with python 3.8 centos 7 but on a manual installation ( no docker)
resolved with the following steps_ :
1. wget https://www.sqlite.org/2021/sqlite-autoconf-3340100.tar.gz
2. tar xvf sqlite-autoconf-3340100.tar.gz
3. cd sqlite-autoconf-3340100
4. ./configure
5. make
6. sudo make install
7. ls /usr/local/lib
8. export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
9. airfow info
**Apache Airflow: 2.0.1
System info**
**.........
....... etc --> ok**
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14208: Python 3.8 - Sqlite3 version error
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #14208:
URL: https://github.com/apache/airflow/issues/14208#issuecomment-778371635
I've added a bit clearer documentation about it in #14209, added troubleshooting section (most likely you have an older version of sqlite library that airflow uses, and it is different than in case of your local "python" execution (LD_LIBRARY_PATH likely is a problem) . You need to remove older version of the libraries you have in your system (or better - upgrade your OS to something a bit "newer" . Centos 7 was relased in 2009 and Centos 8 has been released in 2011, so I believe it is highest time to upgrade.
If you are using such an old OS, you are unfortunately on your own and we will not be able to help, so upgrading OS is a good idea I think.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org