You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "mgorny (via GitHub)" <gi...@apache.org> on 2024/02/20 15:55:58 UTC
[PR] GH-40153: [Python] Fix OverflowError in foreign_buffer on 32-bit platforms [arrow]
mgorny opened a new pull request, #40158:
URL: https://github.com/apache/arrow/pull/40158
Use `uintptr_t` rather than `intptr_t` to fix `OverflowError`, visible e.g. when running `tests/interchange/test_conversion.py` tests on 32-bit platforms.
### Rationale for this change
This fixes the `OverflowError`s from #40153, and makes `pyarrow/tests/interchange/` all pass on 32-bit x86.
### What changes are included in this PR?
- change the type used to store pointer from `intptr_t` to `uintptr_t` to provide coverage for pointers above `0x80000000`.
### Are these changes tested?
These changes are covered by the tests in `pyarrow/tests/interchange`.
### Are there any user-facing changes?
It fixes `OverflowError` that can be triggered by working with pandas data types, possibly more (though I'm not sure if this qualifies as a "crash").
**This PR contains a "Critical Fix".**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40153: [Python] Fix OverflowError in foreign_buffer on 32-bit platforms [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40158:
URL: https://github.com/apache/arrow/pull/40158#issuecomment-1954719894
Revision: d70772f3e12150a0e9770b120578c324279a1844
Submitted crossbow builds: [ursacomputing/crossbow @ actions-9e876b1e47](https://github.com/ursacomputing/crossbow/branches/all?query=actions-9e876b1e47)
|Task|Status|
|----|------|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685057/job/21781112373)|
|test-conda-python-3.10-cython2|[](https://github.com/ursacomputing/crossbow/actions/runs/7977686148/job/21781118978)|
|test-conda-python-3.10-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685507/job/21781115005)|
|test-conda-python-3.10-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685516/job/21781115001)|
|test-conda-python-3.10-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/7977686074/job/21781118746)|
|test-conda-python-3.10-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685362/job/21781113753)|
|test-conda-python-3.10-spark-v3.5.0|[](https://github.com/ursacomputing/crossbow/actions/runs/7977686080/job/21781118789)|
|test-conda-python-3.10-substrait|[](https://github.com/ursacomputing/crossbow/actions/runs/7977686104/job/21781118955)|
|test-conda-python-3.11|[](https://github.com/ursacomputing/crossbow/actions/runs/7977686224/job/21781119004)|
|test-conda-python-3.11-dask-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685478/job/21781115219)|
|test-conda-python-3.11-dask-upstream_devel|[](https://github.com/ursacomputing/crossbow/actions/runs/7977686182/job/21781118051)|
|test-conda-python-3.11-hypothesis|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685518/job/21781115273)|
|test-conda-python-3.11-pandas-upstream_devel|[](https://github.com/ursacomputing/crossbow/actions/runs/7977684989/job/21781112203)|
|test-conda-python-3.11-spark-master|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685521/job/21781115210)|
|test-conda-python-3.12|[](https://github.com/ursacomputing/crossbow/actions/runs/7977684864/job/21781111653)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685612/job/21781115478)|
|test-conda-python-3.8-pandas-1.0|[](https://github.com/ursacomputing/crossbow/actions/runs/7977686021/job/21781118739)|
|test-conda-python-3.8-spark-v3.5.0|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685983/job/21781117500)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685494/job/21781114599)|
|test-conda-python-3.9-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/7977684933/job/21781111961)|
|test-cuda-python|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685198/job/21781113476)|
|test-debian-11-python-3|[](https://github.com/ursacomputing/crossbow/runs/21781117080)|
|test-fedora-39-python-3|[](https://github.com/ursacomputing/crossbow/runs/21781117938)|
|test-ubuntu-20.04-python-3|[](https://github.com/ursacomputing/crossbow/runs/21781115941)|
|test-ubuntu-22.04-python-3|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685151/job/21781113330)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685176/job/21781113361)|
|wheel-macos-big-sur-cp311-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685377/job/21781113761)|
|wheel-macos-big-sur-cp312-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685359/job/21781113754)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685743/job/21781117034)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685663/job/21781116176)|
|wheel-macos-catalina-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685488/job/21781115258)|
|wheel-macos-catalina-cp311-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685458/job/21781115004)|
|wheel-macos-catalina-cp312-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685061/job/21781112385)|
|wheel-macos-catalina-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685763/job/21781117049)|
|wheel-macos-catalina-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685489/job/21781114492)|
|wheel-manylinux-2-28-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685145/job/21781113438)|
|wheel-manylinux-2-28-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685908/job/21781117816)|
|wheel-manylinux-2-28-cp311-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685383/job/21781113964)|
|wheel-manylinux-2-28-cp311-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685738/job/21781117515)|
|wheel-manylinux-2-28-cp312-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685637/job/21781116183)|
|wheel-manylinux-2-28-cp312-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685140/job/21781113154)|
|wheel-manylinux-2-28-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977684863/job/21781111639)|
|wheel-manylinux-2-28-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685103/job/21781112652)|
|wheel-manylinux-2-28-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977686440/job/21781153107)|
|wheel-manylinux-2-28-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685732/job/21781116169)|
|wheel-manylinux-2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977686112/job/21781118957)|
|wheel-manylinux-2014-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685998/job/21781118036)|
|wheel-manylinux-2014-cp311-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685568/job/21781115433)|
|wheel-manylinux-2014-cp311-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977684939/job/21781111958)|
|wheel-manylinux-2014-cp312-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977686073/job/21781118765)|
|wheel-manylinux-2014-cp312-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977686361/job/21781153125)|
|wheel-manylinux-2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685742/job/21781117829)|
|wheel-manylinux-2014-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685956/job/21781117894)|
|wheel-manylinux-2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977684888/job/21781111641)|
|wheel-manylinux-2014-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977684983/job/21781112310)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685206/job/21781113720)|
|wheel-windows-cp311-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685025/job/21781112559)|
|wheel-windows-cp312-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685962/job/21781117497)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977686259/job/21781119144)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/7977685981/job/21781117872)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40153: [Python] Fix OverflowError in foreign_buffer on 32-bit platforms [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40158:
URL: https://github.com/apache/arrow/pull/40158#issuecomment-1954522855
:warning: GitHub issue #40153 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40153: [Python] Fix OverflowError in foreign_buffer on 32-bit platforms [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #40158:
URL: https://github.com/apache/arrow/pull/40158#issuecomment-1954561231
@github-actions crossbow submit -g python -g wheel
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40153: [Python] Fix OverflowError in foreign_buffer on 32-bit platforms [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #40158:
URL: https://github.com/apache/arrow/pull/40158#issuecomment-1954715372
@github-actions crossbow submit -g python -g wheel
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40153: [Python] Fix OverflowError in foreign_buffer on 32-bit platforms [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou merged PR #40158:
URL: https://github.com/apache/arrow/pull/40158
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40153: [Python] Fix OverflowError in foreign_buffer on 32-bit platforms [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #40158:
URL: https://github.com/apache/arrow/pull/40158#issuecomment-1955091407
CI failures are expected. Will merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40153: [Python] Fix OverflowError in foreign_buffer on 32-bit platforms [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #40158:
URL: https://github.com/apache/arrow/pull/40158#issuecomment-1955881517
After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 29d2b168a5c43c2cc0bed65d93d08d83fc1ca80f.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/21800402377) has more details. It also includes information about 5 possible false positives for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40153: [Python] Fix OverflowError in foreign_buffer on 32-bit platforms [arrow]
Posted by "mgorny (via GitHub)" <gi...@apache.org>.
mgorny commented on PR #40158:
URL: https://github.com/apache/arrow/pull/40158#issuecomment-1954526036
Note that this doesn't address all issues from #40153, I've started with the one that seemed to have the biggest impact.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40153: [Python] Fix OverflowError in foreign_buffer on 32-bit platforms [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40158:
URL: https://github.com/apache/arrow/pull/40158#issuecomment-1954563928
```
parse() missing 1 required positional argument: 'config'
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/7976567301
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org