You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/07/13 21:35:21 UTC
[GitHub] [arrow] nealrichardson opened a new pull request #7741: ARROW-9449: [R] Strip arrow.so
nealrichardson opened a new pull request #7741:
URL: https://github.com/apache/arrow/pull/7741
cf. http://dirk.eddelbuettel.com/blog/2017/08/14/#009_compact_shared_libraries
cc @romainfrancois
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] xhochy commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
xhochy commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-657986173
Can you try passing `-g0` in `C(XX)FLAGS` instead / in addition of stripping? That's what we do in the Python wheels (through `multibuild`) so that the debug symbols themselves not even get build at all.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-658310822
On one of the linux images I've been testing locally, as well as on my mac, the R makevars added in that commit are setting `CXXFLAGS="-g -O2"`. Given that, it probably makes sense not to pass those flags to the libarrow build. But I still think we should strip because I can't control the fact that R is including that when it compiles the R bindings--that is, I could edit the makevars on my own system, but the package installation can't override whatever makevars R has on other people's systems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-658043657
On macOS it seems you may want `strip -Sx`, according to the [man page](https://www.unix.com/man-page/osx/1/strip/).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-657815754
This is an order of magnitude improvement. On the AMD64 Ubuntu 18.04 R 4.0 builds, which use an Arrow C++ shared library, arrow.so (so just the R bindings) is down from 24.3mb to 3.3mb. On the 4.0 centos7 build, which bundles static libs, it's down from 144.5mb (about 3x larger in the last couple of days) to 17.4mb.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-657876030
Apparently the macOS version of `strip` takes different args: https://travis-ci.org/github/ursa-labs/crossbow/builds/707793921#L1112
So we'll have to switch based on uname and/or make sure that this doesn't fail the build. Will pick up on this tomorrow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-657811450
https://issues.apache.org/jira/browse/ARROW-9449
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson closed pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
nealrichardson closed pull request #7741:
URL: https://github.com/apache/arrow/pull/7741
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-658393103
Revision: 343e87d2022a4c43b2e894a2b3d24a61f29f477e
Submitted crossbow builds: [ursa-labs/crossbow @ actions-419](https://github.com/ursa-labs/crossbow/branches/all?query=actions-419)
|Task|Status|
|----|------|
|test-r-rocker-r-base-latest|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-419-azure-test-r-rocker-r-base-latest)|
|test-r-rstudio-r-base-3.6-opensuse42|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-419-azure-test-r-rstudio-r-base-3.6-opensuse42)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
wesm commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-657891523
> On the 4.0 centos7 build, which bundles static libs, it's down from 144.5mb (about 3x larger in the last couple of days) to 17.4mb.
Whoa, that's kind of crazy. Which static libraries have debug symbols in them? I'm concerned that perhaps we might have a BUNDLED build that is being built in DEBUG mode instead of RELEASE which would also have negative performance implications
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
wesm commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-657902075
I only meant the thirdparty dependencies -- some projects have debug symbols by default even in release builds so it could just be that. At some point it might be worth going through and seeing which are the largest thirdparty static libraries
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-658334397
Revision: 343e87d2022a4c43b2e894a2b3d24a61f29f477e
Submitted crossbow builds: [ursa-labs/crossbow @ actions-418](https://github.com/ursa-labs/crossbow/branches/all?query=actions-418)
|Task|Status|
|----|------|
|homebrew-r-autobrew|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|test-conda-r-4.0|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-418-github-test-conda-r-4.0)|
|test-r-linux-as-cran|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-418-github-test-r-linux-as-cran)|
|test-r-rhub-ubuntu-gcc-release|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-418-azure-test-r-rhub-ubuntu-gcc-release)|
|test-r-rocker-r-base-latest|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-418-azure-test-r-rocker-r-base-latest)|
|test-r-rstudio-r-base-3.6-bionic|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-418-azure-test-r-rstudio-r-base-3.6-bionic)|
|test-r-rstudio-r-base-3.6-centos6|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-418-azure-test-r-rstudio-r-base-3.6-centos6)|
|test-r-rstudio-r-base-3.6-centos8|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-418-azure-test-r-rstudio-r-base-3.6-centos8)|
|test-r-rstudio-r-base-3.6-opensuse15|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-418-azure-test-r-rstudio-r-base-3.6-opensuse15)|
|test-r-rstudio-r-base-3.6-opensuse42|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-418-azure-test-r-rstudio-r-base-3.6-opensuse42)|
|test-ubuntu-18.04-r-sanitizer|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-418-azure-test-ubuntu-18.04-r-sanitizer)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-658332593
@github-actions crossbow submit -g r
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-657811833
Revision: d5fd33fdd260f44094e5a054e6795897f2565979
Submitted crossbow builds: [ursa-labs/crossbow @ actions-411](https://github.com/ursa-labs/crossbow/branches/all?query=actions-411)
|Task|Status|
|----|------|
|homebrew-cpp-autobrew|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|homebrew-r-autobrew|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-658392235
@github-actions crossbow submit test-r-rstudio-r-base-3.6-opensuse42 test-r-rocker-r-base-latest
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
wesm commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-657955297
As near as I can tell this is the patch that caused the size increase
https://github.com/apache/arrow/commit/ba1f6315aa4b4a85435e890bed948b537708d858
So I guess it's related to the CXX toolchain?
That's 2 commits after the dependency bundle match, which has 42MB of libs https://github.com/apache/arrow/runs/859650227
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-657900315
Cmake reports that it is a release build. You can see that in the logs. Maybe the change to the new bundled dependency library is no longer passing that to the external projects?
Either way, there's value in stripping even just the R bindings themselves.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-658392032
The two crossbow failures (other than conda-r, which we're ignoring) appear to be flakes, but I'll re-run them to be extra sure.
Re-review appreciated on the latest change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-657806099
@github-actions crossbow submit *autobrew*
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #7741: ARROW-9449: [R] Strip arrow.so
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on pull request #7741:
URL: https://github.com/apache/arrow/pull/7741#issuecomment-657915188
Something caused a major (3x) increase in the size of the libs from one day to the next. Could have been the bundled dependency patch or could have been one of several others, it was a busy day. I could bisect if we care to find out, but for the moment this seemed to be a good enough fix.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org