You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/05 17:47:14 UTC
[GitHub] [airflow] potiuk opened a new issue #13493: Fail when SQLite version is not high enough
potiuk opened a new issue #13493:
URL: https://github.com/apache/airflow/issues/13493
Seems that in Airflow 2.0 we use some features of sqlite3 which do not work with really old versions of sqlite (for sure for 3.11.0 which was released in 2016).
Following https://github.com/apache/airflow/issues/13397#issuecomment-754768930 and few other slack discussions, seems that there are enough systems out there (CentOS, RHEL7) that come with really old sqlite (RHEL7 with a 7 years old one!)
We have so far no formal requirement for sqlite3 but in fact there is one. We just do not know which. I propose to take as a base the one we have in CI in buster debian: `3.27.2`:
```
root@b8a8e73caa2c:/opt/airflow# python
Python 3.6.12 (default, Nov 25 2020, 03:59:00)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.27.2'
>>>
```
My propoosal is that we should fail Aiflow with appropriate message if sqlite is the db and the sqlite version is lower than that.
<!--
Welcome to Apache Airflow! For a smooth issue process, try to answer the following questions.
Don't worry if they're not all applicable; just try to include what you can :-)
If you need to include code snippets or logs, please put them in fenced code
blocks. If they're super-long, please use the details tag like
<details><summary>super-long log</summary> lots of stuff </details>
Please delete these comment blocks before submitting the issue.
-->
**Description**
<!-- A short description of your feature -->
**Use case / motivation**
<!-- What do you want to happen?
Rather than telling us how you might implement this solution, try to take a
step back and describe what you are trying to achieve.
-->
**Are you willing to submit a PR?**
<!--- We accept contributions! -->
**Related Issues**
<!-- Is there currently another issue associated with this? -->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #13493: Fail when SQLite version is not high enough
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #13493:
URL: https://github.com/apache/airflow/issues/13493#issuecomment-755205616
It worked for me when I tsted it. You must have airflow started in the way that uses still old version of sqlite. If you installed it wth the LD_LIBRARY_PATH modification, you need tto make sure your airflow also has the LD_LIBRARY_PATH set to point to the right sqlite library. Ideally - remove all other sqlite libraries to be sure airflow is using the right one.
What also might help if you start using some newer version of RedHat VM that already might have a version of sqlite which is not very old.
Sorry I can't help more. I tested it all and installing older/newer version of sqlite shows/solves the problem. You are on your own to make sure that the right version of sqlite is used. Here is the guide that might hep you.
https://unix.stackexchange.com/questions/434100/updating-the-sqlite-version-used-by-python-3-on-centos-7
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #13493: Fail when SQLite version is not high enough
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13493:
URL: https://github.com/apache/airflow/issues/13493#issuecomment-755205616
It worked for me when I tried. You must have airflow started in the way that uses still old version of sqlite. If you installed it wth the LD_LIBRARY_PATH modification, you need tto make sure your airflow also has the LD_LIBRARY_PATH set to point to the right sqlite library. Ideally - remove all other sqlite libraries to be sure airflow is using the right one.
What also might help if you start using some newer version of RedHat VM that already might have a version of sqlite which is not very old.
Sorry I can't help more. I tested it all and installing older/newer version of sqlite shows/solves the problem. You are on your own to make sure that the right version of sqlite is used. Here is the guide that might hep you.
https://unix.stackexchange.com/questions/434100/updating-the-sqlite-version-used-by-python-3-on-centos-7
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #13493: Fail when SQLite version is not high enough
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13493:
URL: https://github.com/apache/airflow/issues/13493#issuecomment-755290526
Closed by #13496
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #13493: Fail when SQLite version is not high enough
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #13493:
URL: https://github.com/apache/airflow/issues/13493
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] DuongPTIT commented on issue #13493: Fail when SQLite version is not high enough
Posted by GitBox <gi...@apache.org>.
DuongPTIT commented on issue #13493:
URL: https://github.com/apache/airflow/issues/13493#issuecomment-755186572
I upgrade sqlite3 to newest version, but it still throw error:
`Python 3.6.12 (default, Jan 6 2021, 11:20:09)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.34.0'
>>>
`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org