You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "raulcd (via GitHub)" <gi...@apache.org> on 2024/06/07 14:16:01 UTC
[PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
raulcd opened a new pull request, #42028:
URL: https://github.com/apache/arrow/pull/42028
### Rationale for this change
Removing unnecessary symbols for wheels will allow us to reduce the size of the wheels considerably.
### What changes are included in this PR?
TBF
### Are these changes tested?
Yes, via Archery.
### Are there any user-facing changes?
TBD, it will depend on the strip being used we might lose some debug symbols.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2163416506
After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit baf4089eaa84b2a5c2cb437f3a249f83577dbc29.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/26139109627) has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2154947028
Revision: 734c266115c60596fe18e4c97703f184136e8244
Submitted crossbow builds: [ursacomputing/crossbow @ actions-4644efaf3a](https://github.com/ursacomputing/crossbow/branches/all?query=actions-4644efaf3a)
|Task|Status|
|----|------|
|wheel-manylinux-2-28-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506664/job/25946144686)|
|wheel-manylinux-2-28-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506570/job/25946144623)|
|wheel-manylinux-2-28-cp311-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506208/job/25946142964)|
|wheel-manylinux-2-28-cp311-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506219/job/25946143004)|
|wheel-manylinux-2-28-cp312-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506776/job/25946144950)|
|wheel-manylinux-2-28-cp312-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506759/job/25946144877)|
|wheel-manylinux-2-28-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506330/job/25946143577)|
|wheel-manylinux-2-28-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506286/job/25946143360)|
|wheel-manylinux-2-28-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506541/job/25946144294)|
|wheel-manylinux-2-28-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506761/job/25946144843)|
|wheel-manylinux-2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506982/job/25946145727)|
|wheel-manylinux-2014-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506157/job/25946142983)|
|wheel-manylinux-2014-cp311-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506305/job/25946143445)|
|wheel-manylinux-2014-cp311-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506886/job/25946145400)|
|wheel-manylinux-2014-cp312-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506558/job/25946144355)|
|wheel-manylinux-2014-cp312-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506440/job/25946143682)|
|wheel-manylinux-2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506332/job/25946143765)|
|wheel-manylinux-2014-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506978/job/25946145695)|
|wheel-manylinux-2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506792/job/25946144994)|
|wheel-manylinux-2014-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9418506461/job/25946143914)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2157817083
Running `strip --strip-debug` manually seems to reduce a locally unzipped wheel from 131MB to 124MB
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2157992344
Revision: 837d11419ea0a7aefb3323609025ee8fa0364488
Submitted crossbow builds: [ursacomputing/crossbow @ actions-1d686e8f7f](https://github.com/ursacomputing/crossbow/branches/all?query=actions-1d686e8f7f)
|Task|Status|
|----|------|
|wheel-macos-catalina-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9446852335/job/26017380481)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #42028:
URL: https://github.com/apache/arrow/pull/42028#discussion_r1633179256
##########
ci/scripts/python_wheel_manylinux_build.sh:
##########
@@ -160,6 +160,26 @@ export CMAKE_PREFIX_PATH=/tmp/arrow-dist
pushd /arrow/python
python setup.py bdist_wheel
+echo "=== Strip symbols from wheel ==="
+mkdir dist/temp-fix-wheel
+mv dist/pyarrow-*.whl dist/temp-fix-wheel
+
+pushd dist/temp-fix-wheel
+wheel_name=$(ls pyarrow-*.whl)
+# Unzip and remove old wheel
+unzip $wheel_name
+rm $wheel_name
+for filename in $(ls pyarrow/*.so pyarrow/*.so.*); do
+ echo "Stripping debug symbols from: $filename";
+ strip --strip-debug $filename
+done
+# Zip wheel again after stripping symbols
+zip -r $wheel_name .
Review Comment:
Does this reuse the same compression settings? (is a wheel compressed at all?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2158088314
`strip -S $filename` doesn't seem to have any effect on macOS, the size of the wheels and the unpacked wheel is exactly the same with and without executing the command
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2155010966
This change reduces from [131MB locally unzipped wheel](https://github.com/ursacomputing/crossbow/actions/runs/9397680127) from the previous nightly build to [101MB locally unzipped wheel](https://github.com/ursacomputing/crossbow/actions/runs/9418506664) the wheel on wheel-manylinux-2-28-cp310-amd64.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2157986918
@github-actions crossbow submit wheel-macos-catalina-cp310-amd64
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2155231532
```
Failed to push updated references, potentially because of credential issues: ['refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp311-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp311-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp312-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp312-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp311-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp311-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp310-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp310-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp39-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp39-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp310-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp310-arm64', 'refs/heads/actions-3236717c81-github-wheel-manyl
inux-2014-cp312-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp312-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp312-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp312-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp311-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp311-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp310-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp310-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp312-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp312-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp310-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp310-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp39-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp39-arm64', 'refs/heads/actions-3236717c81-gi
thub-wheel-manylinux-2014-cp38-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp38-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp38-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp38-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp38-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp38-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp39-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp39-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp38-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp38-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp311-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp311-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp39-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp39-arm64', 'refs/heads/actions-3236717
c81']
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/9420748198
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2155233478
@github-actions crossbow submit wheel-manylinux-*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2154942557
@github-actions crossbow submit wheel-manylinux-*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2157825902
Revision: aeab32ea72328dd805789d3c4ee612b49067cdbb
Submitted crossbow builds: [ursacomputing/crossbow @ actions-8a67ea442e](https://github.com/ursacomputing/crossbow/branches/all?query=actions-8a67ea442e)
|Task|Status|
|----|------|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9445859608/job/26014308832)|
|wheel-macos-big-sur-cp311-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9445859460/job/26014308486)|
|wheel-macos-big-sur-cp312-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9445859506/job/26014308538)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9445859633/job/26014308824)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9445859673/job/26014308853)|
|wheel-macos-catalina-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9445859416/job/26014308103)|
|wheel-macos-catalina-cp311-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9445859349/job/26014308118)|
|wheel-macos-catalina-cp312-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9445859370/job/26014308085)|
|wheel-macos-catalina-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9445859565/job/26014308768)|
|wheel-macos-catalina-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9445859368/job/26014308098)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2157821053
@github-actions crossbow submit wheel-macos-*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2160279931
:+1: merging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2157926573
@github-actions crossbow submit wheel-macos-catalina-cp310-amd64
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2155025396
@jorisvandenbossche @pitrou would you prefer to:
- [1] Go with the current change just using `--strip` on `auditwheel repair`
- [2] Run `strip --strip-debug` manually
- [3] Wait until https://github.com/pypa/auditwheel/issues/491 is potentially done and released
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2155237220
Revision: 7dd1e283cb1fabd2edc325948a909bb5587e8fb3
Submitted crossbow builds: [ursacomputing/crossbow @ actions-f4cd718f1d](https://github.com/ursacomputing/crossbow/branches/all?query=actions-f4cd718f1d)
|Task|Status|
|----|------|
|wheel-manylinux-2-28-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799650/job/25953602273)|
|wheel-manylinux-2-28-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799660/job/25953602310)|
|wheel-manylinux-2-28-cp311-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799334/job/25953601118)|
|wheel-manylinux-2-28-cp311-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799372/job/25953601056)|
|wheel-manylinux-2-28-cp312-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799587/job/25953602683)|
|wheel-manylinux-2-28-cp312-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799563/job/25953601795)|
|wheel-manylinux-2-28-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799725/job/25953602662)|
|wheel-manylinux-2-28-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799695/job/25953602323)|
|wheel-manylinux-2-28-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799405/job/25953601036)|
|wheel-manylinux-2-28-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799820/job/25953603098)|
|wheel-manylinux-2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799765/job/25953603032)|
|wheel-manylinux-2014-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799762/job/25953603019)|
|wheel-manylinux-2014-cp311-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799346/job/25953601255)|
|wheel-manylinux-2014-cp311-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799341/job/25953601014)|
|wheel-manylinux-2014-cp312-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799530/job/25953601833)|
|wheel-manylinux-2014-cp312-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799458/job/25953601270)|
|wheel-manylinux-2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799569/job/25953601786)|
|wheel-manylinux-2014-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799537/job/25953601796)|
|wheel-manylinux-2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799567/job/25953602009)|
|wheel-manylinux-2014-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions/runs/9420799608/job/25953602317)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #42028:
URL: https://github.com/apache/arrow/pull/42028#discussion_r1633391718
##########
ci/scripts/python_wheel_manylinux_build.sh:
##########
@@ -160,6 +160,26 @@ export CMAKE_PREFIX_PATH=/tmp/arrow-dist
pushd /arrow/python
python setup.py bdist_wheel
+echo "=== Strip symbols from wheel ==="
+mkdir dist/temp-fix-wheel
+mv dist/pyarrow-*.whl dist/temp-fix-wheel
+
+pushd dist/temp-fix-wheel
+wheel_name=$(ls pyarrow-*.whl)
+# Unzip and remove old wheel
+unzip $wheel_name
+rm $wheel_name
+for filename in $(ls pyarrow/*.so pyarrow/*.so.*); do
+ echo "Stripping debug symbols from: $filename";
+ strip --strip-debug $filename
+done
+# Zip wheel again after stripping symbols
+zip -r $wheel_name .
Review Comment:
You're right, this does seem to work and the fact that the wheels are shrunk seems to indicate it's ok. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #42028:
URL: https://github.com/apache/arrow/pull/42028#discussion_r1633393336
##########
ci/scripts/python_wheel_manylinux_build.sh:
##########
@@ -160,6 +160,26 @@ export CMAKE_PREFIX_PATH=/tmp/arrow-dist
pushd /arrow/python
python setup.py bdist_wheel
+echo "=== Strip symbols from wheel ==="
+mkdir dist/temp-fix-wheel
Review Comment:
Let's avoid erroring out if the directory already exists?
```suggestion
mkdir -p dist/temp-fix-wheel
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2155229955
@github-actions crossbow submit wheel-manylinux-*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on code in PR #42028:
URL: https://github.com/apache/arrow/pull/42028#discussion_r1633383874
##########
ci/scripts/python_wheel_manylinux_build.sh:
##########
@@ -160,6 +160,26 @@ export CMAKE_PREFIX_PATH=/tmp/arrow-dist
pushd /arrow/python
python setup.py bdist_wheel
+echo "=== Strip symbols from wheel ==="
+mkdir dist/temp-fix-wheel
+mv dist/pyarrow-*.whl dist/temp-fix-wheel
+
+pushd dist/temp-fix-wheel
+wheel_name=$(ls pyarrow-*.whl)
+# Unzip and remove old wheel
+unzip $wheel_name
+rm $wheel_name
+for filename in $(ls pyarrow/*.so pyarrow/*.so.*); do
+ echo "Stripping debug symbols from: $filename";
+ strip --strip-debug $filename
+done
+# Zip wheel again after stripping symbols
+zip -r $wheel_name .
Review Comment:
Both wheels are compressed, a size comparison shows:
Initial one from CI from main:
```
-rw-r--r-- 1 raulcd raulcd 40M jun 6 08:46 pyarrow-17.0.0.dev260-cp310-cp310-manylinux_2_28_x86_64.whl
```
and new one generated from this PR
```
-rw-r--r-- 1 raulcd raulcd 38M jun 7 17:35 pyarrow-17.0.0.dev275-cp310-cp310-manylinux_2_28_x86_64.whl
```
The compression type for the files is exactly the same and all file sizes are exactly the same, I've tested with the following snippet:
```
import zipfile
with zipfile.ZipFile('pyarrow-17.0.0.dev260-cp38-cp38-manylinux_2_28_x86_64.whl', 'r') as old_wheel:
with zipfile.ZipFile('pyarrow-17.0.0.dev275-cp38-cp38-manylinux_2_28_x86_64.whl', 'r') as new_wheel:
for info_old, info_new in zip(sorted(old_wheel.infolist(), key=lambda x: x.filename), sorted(new_wheel.infolist(), key=lambda x: x.filename)):
if "dev260" not in info_old.filename:
assert info_old.filename == info_new.filename, info_old.filename + info_new.filename
assert info_old.compress_type == info_new.compress_type
if "RECORD" in info_old.filename:
continue
if not (info_old.filename.endswith('.so') or info_old.filename.endswith('.so.1700')):
assert info_old.compress_size == info_new.compress_size, f"old: {info_old.filename, info_old.compress_size}, new: {info_new.filename, info_new.compress_size}"
print(f'filename: {info_old.filename} ')
print(f'compress type: {info_old.compress_type}')
print(f'compress size: {info_new.compress_size}')
```
I am not showing the full output but this a an example of what I get:
```
...
filename: pyarrow/vendored/docscrape.py
compress type: 8
compress size: 6080
filename: pyarrow/vendored/version.py
compress type: 8
compress size: 3894
filename: scripts/
compress type: 0
compress size: 0
filename: scripts/test_imports.py
compress type: 8
compress size: 496
filename: scripts/test_leak.py
compress type: 8
compress size: 1422
```
I am not an expert on zip compression but all tests for the wheels are successful on the CI jobs and my validations seem to show the exact same level of compression and same size for all files in both wheels.
Any other test you think I should do or how I could check?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2157931569
Revision: e41c6541f44e3c453344cac420561ba917da6964
Submitted crossbow builds: [ursacomputing/crossbow @ actions-9df43f583e](https://github.com/ursacomputing/crossbow/branches/all?query=actions-9df43f583e)
|Task|Status|
|----|------|
|wheel-macos-catalina-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/9446465656/job/26016220402)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd merged PR #42028:
URL: https://github.com/apache/arrow/pull/42028
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #42028:
URL: https://github.com/apache/arrow/pull/42028#issuecomment-2155061012
> @jorisvandenbossche @pitrou would you prefer to:
> * [1] Go with the current change just using `--strip` on `auditwheel repair`
-1
> * [2] Run `strip --strip-debug` manually
+1
> * [3] Wait until [Allow repair to use `strip` with `--strip-debug` instead of only allowing `--strip-all` pypa/auditwheel#491](https://github.com/pypa/auditwheel/issues/491) is potentially done and released
+0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org