You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tsfile.apache.org by "AxelNoun (via GitHub)" <gi...@apache.org> on 2026/08/26 04:29:50 UTC
[I] Python SDK: can pyarrow>=18,<20 on 3.10-3.13 be widened to overlap datasets (pyarrow>=21)? [tsfile]
AxelNoun opened a new issue, #921:
URL: https://github.com/apache/tsfile/issues/921
# Python SDK: can pyarrow>=18,<20 on 3.10–3.13 be widened?
**Status: hypothesis.** We do not know why this pin exists; asking whether it can be relaxed.
Measured on **2026-08-25** from `https://pypi.org/pypi/tsfile/json` (see
`pypi_tsfile_matrix.md`). Latest SDK: **2.4.0**. Pins identical for 2.3.0 / 2.3.1 / 2.4.0.
`datasets` (since 4.8.5) requires `pyarrow>=21`. The Apache TsFile Python SDK declares:
| Python | SDK `pyarrow` | Intersection with `datasets` |
|---|---|---|
| 3.9 | `>=16,<18` | n/a (`datasets` needs ≥ 3.10) |
| 3.10–3.13 | `>=18,<20` | empty → `ResolutionImpossible` |
| 3.14+ | `>=22,<24` | `[22, 24)` |
Empirical (2026-08-25): `pip install --dry-run "datasets==5.0.1" "tsfile==2.4.0"` on
Python 3.12 → `ResolutionImpossible`. Wheels exist for cp39–cp314; the hole is the pin.
The same SDK already accepts pyarrow 22/23 on 3.14, which *suggests* (does not prove)
that the `<20` bound on 3.10–3.13 may be leftover rather than a live ABI/API break.
**If** that bound can be widened to overlap `pyarrow>=21` on 3.10–3.13, then
`datasets[tsfile]` and dataset-viewer images on those Pythons become co-installable
with no change required in either Hugging Face repository. That is an effect
description, not an attribution of fault.
Question for upstream: is `pyarrow>=18,<20` on 3.10–3.13 motivated by a known
incompatibility, or can it be relaxed?
Context: co-install with Hugging Face datasets (requires pyarrow>=21) fails on Python 3.10–3.13 with ResolutionImpossible. Related discussion: https://github.com/huggingface/datasets/issues/8499
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tsfile.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Python SDK: can pyarrow>=18,<20 on 3.10-3.13 be widened to overlap datasets (pyarrow>=21)? [tsfile]
Posted by "ColinLeeo (via GitHub)" <gi...@apache.org>.
ColinLeeo commented on issue #921:
URL: https://github.com/apache/tsfile/issues/921#issuecomment-6160803785
Hi,thanks for your time and TsFile now has already release v2.5.0. You can now install tsfile 2.5.0 via pip directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tsfile.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Python SDK: can pyarrow>=18,<20 on 3.10-3.13 be widened to overlap datasets (pyarrow>=21)? [tsfile]
Posted by "AxelNoun (via GitHub)" <gi...@apache.org>.
AxelNoun commented on issue #921:
URL: https://github.com/apache/tsfile/issues/921#issuecomment-6036408307
Follow-up on the release status, checked on 2026-10-07.
- `datasets` 5.1.0, published on PyPI on 2026-10-05, requires `pyarrow>=24.0.0` (huggingface/datasets#8502).
- The latest stable `tsfile` release on PyPI is still 2.4.0, which requires `pyarrow<20` on Python 3.10 to 3.13 and `pyarrow<24` on Python 3.14. As a result, `datasets==5.1.0` and `tsfile==2.4.0` no longer resolve together on Python 3.14 either, although the bounds still overlapped there when this issue was opened. I checked this with `uv pip install --dry-run` on Python 3.12 and 3.14.
- The workaround given above first installs `pyarrow>=21,<24`, a range that `datasets` 5.1.0 no longer accepts. The Python 3.14 CI job of `datasets` currently installs `tsfile` with `--no-deps` alongside `pyarrow>=24.0.0`.
- The pre-releases 2.5.0.dev0 and 2.5.0.dev1 carry the relaxed bounds from #922, and `datasets==5.1.0` resolves with `tsfile==2.5.0.dev1` on both Python versions. Since pip and uv do not select pre-releases by default, a requirement such as `tsfile>=2.3.0` still fails to resolve against `datasets` 5.1.0.
Is there a target date for the 2.5.0 release?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tsfile.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Python SDK: can pyarrow>=18,<20 on 3.10-3.13 be widened to overlap datasets (pyarrow>=21)? [tsfile]
Posted by "ColinLeeo (via GitHub)" <gi...@apache.org>.
ColinLeeo commented on issue #921:
URL: https://github.com/apache/tsfile/issues/921#issuecomment-5421381266
Thank you for raising this issue.
The `pyarrow>=18,<20` constraint appears to be a historical leftover rather than a current compatibility requirement. We tested TsFile 2.4.0 with PyArrow 18 through 23, including Arrow read/write paths, and confirmed that it works directly with the newer versions.
As a temporary workaround before the next release, users can install the desired compatible PyArrow version first and then install TsFile without dependency resolution, for example:
```bash
pip install "pyarrow>=21,<24"
pip install --no-deps "tsfile==2.4.0"
```
TsFile 2.5.0 is planned for release soon, and the PyArrow constraint will be relaxed in its published package metadata so that normal installation works without this workaround.
Thanks again for identifying and reporting the problem. We will fix it promptly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tsfile.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Python SDK: can pyarrow>=18,<20 on 3.10-3.13 be widened to overlap datasets (pyarrow>=21)? [tsfile]
Posted by "ColinLeeo (via GitHub)" <gi...@apache.org>.
ColinLeeo closed issue #921: Python SDK: can pyarrow>=18,<20 on 3.10-3.13 be widened to overlap datasets (pyarrow>=21)?
URL: https://github.com/apache/tsfile/issues/921
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tsfile.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Python SDK: can pyarrow>=18,<20 on 3.10-3.13 be widened to overlap datasets (pyarrow>=21)? [tsfile]
Posted by "ColinLeeo (via GitHub)" <gi...@apache.org>.
ColinLeeo commented on issue #921:
URL: https://github.com/apache/tsfile/issues/921#issuecomment-6051041707
Thank you for following up and verifying compatibility with the pre-release.
TsFile 2.5.0 has completed internal testing, and we plan to start the release vote within the next two days. Assuming the vote passes, we expect to publish it to PyPI this weekend or early next week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tsfile.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org