You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Chais (via GitHub)" <gi...@apache.org> on 2024/10/10 10:14:58 UTC
[I] unicodecsv dependency makes apache-airflow Python package un-installable [airflow]
Chais opened a new issue, #42892:
URL: https://github.com/apache/airflow/issues/42892
### Apache Airflow version
2.10.2
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
Airflow cannot be installed in a venv with any system that properly solves dependencies and version requirements.
This happens because the lowest supported Python version of `apache-airflow` is 3.8, while the highest supported version of `unicodecsv` is 3.5. So there is no Python version that would be supported by `apache-airflow` and all of its dependencies, making it un-installable.
### What you think should happen instead?
I know `unicodecsv` has been dropped as a dependency very recently, but no new version has been released since then.
### How to reproduce
I encountered the issue with Poetry, but any tool that ensures all requirements for dependency packages are met will encounter the same issue.
I encountered the issue targeting Python 3.8, but since there is no overlap between supported Python versions between `unicecsv` and `apache-airflow` it doesn't really matter.
Paste the following `pyproject.toml` into an empty directory:
```toml
[tool.poetry]
name = "airflow-dependency-test"
version = "0.1.0"
description = ""
authos = ["Your Name <yo...@example.com>"]
[tool.poetry.dependencies]
python = "~3.8"
apache-airflow = "2.10.2"
[tool.poetry.dev-dependencies]
[build-system]
requires = ["poetry-core >= 1.0.0"]
build-backend = "poetry-core.masonry.api"
```
In a terminal navigate to the directory and run `poetry install`
### Operating System
Ubuntu 22.04
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] unicodecsv dependency makes apache-airflow Python package un-installable [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #42892:
URL: https://github.com/apache/airflow/issues/42892#issuecomment-2404681915
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] unicodecsv dependency makes apache-airflow Python package un-installable [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #42892: unicodecsv dependency makes apache-airflow Python package un-installable
URL: https://github.com/apache/airflow/issues/42892
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] unicodecsv dependency makes apache-airflow Python package un-installable [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #42892:
URL: https://github.com/apache/airflow/issues/42892#issuecomment-2408790168
This is invalid.
Agree - we missed that we have an old and not really used package before, but it should not be prevented from being installed on higher airflow version.
While unicodecsv had indeed only specified classifiers for up to Python 3.5 it had no requirements set to be only used for Python < 3.5 and since it had no dependencies on its own and it did not have python_requires that prevented it from being installed.
What you are likely referring to when you mane are Trove Classifiers:
```
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: BSD License
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 2.6
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3.3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: Programming Language :: Python :: Implementation :: CPython
```
The trove classifiers are defined in https://peps.python.org/pep-0301/ and they are only there to classify projects when you search for them, but they should not be used by packaging tools to decide if they are installed for specific python version (`python_requires is the way to do that).
I am not sure what errors you had an which tools you used besides poetry - you missed to explain what kind of error you had when you tried to install airflow. But I am quite sure it is installable for higher python versio. In fact I just did it and it works (python 3.11 in this case):
```
[jarek:~/code/airflow] fix-dependencies-removal+ 1 ± python --version
Python 3.11.6
[jarek:~/code/airflow] fix-dependencies-removal+ ± pip install unicodecsv
DEPRECATION: Loading egg at /Users/jarek/.pyenv/versions/3.11.6/lib/python3.11/site-packages/Flask_OpenID-1.3.1-py3.11.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
DEPRECATION: Loading egg at /Users/jarek/.pyenv/versions/3.11.6/lib/python3.11/site-packages/defusedxml-0.8.0rc2-py3.11.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
DEPRECATION: Loading egg at /Users/jarek/.pyenv/versions/3.11.6/lib/python3.11/site-packages/python3_openid-3.2.0-py3.11.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
Collecting unicodecsv
Using cached unicodecsv-0.14.1-py3-none-any.whl
Installing collected packages: unicodecsv
Successfully installed unicodecsv-0.14.1
```
The setup.py of unicodecsv does not contain python_requires so generally no tool should be limited to installing it on higher python version than 3.5 (and since it is very simple code, it generally installs and works fine on higher versions).
This is setup.py of latest `unicode.csv` - there is no limitation of Python version:
```python
setup(
name='unicodecsv',
version=version,
description="Python2's stdlib csv module is nice, but it doesn't support unicode. This module is a drop-in replacement which *does*.",
long_description=open(os.path.join(os.path.dirname(__file__), 'README.rst'), 'rb').read().decode('utf-8'),
author='Jeremy Dunck',
author_email='jdunck@gmail.com',
url='https://github.com/jdunck/python-unicodecsv',
packages=find_packages(),
tests_require=['unittest2>=0.5.1'],
test_suite='runtests.get_suite',
license='BSD License',
classifiers=['Development Status :: 5 - Production/Stable',
'Intended Audience :: Developers',
'License :: OSI Approved :: BSD License',
'Natural Language :: English',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: Implementation :: PyPy',
'Programming Language :: Python :: Implementation :: CPython',],
```
I am not sure how you came to the conclusion that unicodecsv is not installable for Python 3.8+ - maybe you can explain what exactly error you experienced and what exactly you did in a follow-up discussion but I can assure you Airflow is instalable so this issue is invalid.
Converting to discussion if you need more. I will also make sure to check if unicodecsv removal is cherry-picked to 2-10 branch - so also we will remove it in the next release, but this is more of a cleanup than problem with installation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org