You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "srilman (via GitHub)" <gi...@apache.org> on 2023/02/04 19:06:08 UTC
[GitHub] [iceberg] srilman opened a new pull request, #6745: Python: Use Version Ranges for Various Dependencies
srilman opened a new pull request, #6745:
URL: https://github.com/apache/iceberg/pull/6745
As discussed in #6620, this PR uses version ranges for some dependencies. This will unfix the versions for certain dependencies and allow users to use older versions when install PyIceberg to use as a library. The dependencies changed are:
- PyArrow from `11.0.0` to `8.0.0-11.0.0`. It is an optional dependency used either for FileIO, conversion from Iceberg to Arrow (schema, expression, table, etc.), or a dep for Pandas or DuckDB. Made sure current tests pass with every major version.
- Pandas from `1.5.3` to `1.4.4-1.5.3`. It is only used to read Iceberg tables out to Pandas, which is current done through PyArrow anyways. Tested 1.4.4. PyIceberg used 1.5.2 until recently
- DuckDB from `0.6.1` to `0.6.0-0.6.1`. Similar to Pandas, and PyIceberg used 0.6.0 until recently
- FsSpec, S3Fs, and ADLSFs from `2023.1.0` to `2022.8.2-2023.1.0`. Used for FileIO, and I believe S3Fs and ADLSFs already depend on the same version of FsSpec. Tested with all major versions with success, and PyIceberg has used `2022.8.2`, `2022.10.0`, and `2022.11.2` in the past
- Requests from `2.28.2` to `2.28.1-2.28.2` since this is only a patch release and PyIceberg used `2.28.1` until recently.
- Rich from `13.3.1` to `13.0.0-13.3.1` since this is minor release changes with no significant differences
- PyYAML from `6.0.0` to `5.4.0-6.0.0` since `5.4` is still used commonly in other tools, and the only change in the major change in the major release was removing Python 2.7 support. Tested with `5.4.0`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] srilman commented on pull request #6745: Python: Use Version Ranges for Various Dependencies
Posted by "srilman (via GitHub)" <gi...@apache.org>.
srilman commented on PR #6745:
URL: https://github.com/apache/iceberg/pull/6745#issuecomment-1422714130
Ready for review. @Fokko or someone else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko merged pull request #6745: Python: Use Version Ranges for Various Dependencies
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko merged PR #6745:
URL: https://github.com/apache/iceberg/pull/6745
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on a diff in pull request #6745: Python: Use Version Ranges for Various Dependencies
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on code in PR #6745:
URL: https://github.com/apache/iceberg/pull/6745#discussion_r1100602879
##########
python/pyproject.toml:
##########
@@ -50,33 +50,33 @@ include = [
[tool.poetry.dependencies]
python = "^3.8"
mmhash3 = "3.0.1"
-requests = "2.28.2"
+requests = ">=2.28.1,<=2.28.2"
click = "8.1.3"
-rich = "13.3.1"
-pyyaml = "6.0.0"
+rich = ">=13.0.0,<=13.3.1"
+pyyaml = ">=5.4.0,<=6.0.0"
pydantic = "1.10.4"
-fsspec = "2023.1.0"
+fsspec = ">=2022.8.2,<=2023.1.0"
pyparsing = "3.0.9"
zstandard = "0.19.0"
-pyarrow = { version = "11.0.0", optional = true }
+pyarrow = { version = ">=8.0.0,<=11.0.0", optional = true }
-pandas = { version = "1.5.3", optional = true }
+pandas = { version = ">=1.4.4,<=1.5.3", optional = true }
-duckdb = { version = "0.6.1", optional = true }
+duckdb = { version = ">=0.6.0,<=0.6.1", optional = true }
python-snappy = { version = "0.6.1", optional = true }
thrift = { version = "0.16.0", optional = true }
-boto3 = {version = "1.24.59", optional = true}
+boto3 = { version = "1.24.59", optional = true }
Review Comment:
Would be cool if we can get some kind of TOML linter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org