You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/06/17 15:55:33 UTC
[GitHub] [arrow] pitrou opened a new pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
pitrou opened a new pull request #7471:
URL: https://github.com/apache/arrow/pull/7471
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#discussion_r442025009
##########
File path: python/manylinux1/scripts/build_boost.sh
##########
@@ -20,10 +20,16 @@ BOOST_VERSION=1.73.0
BOOST_VERSION_UNDERSCORE=${BOOST_VERSION//\./_}
NCORES=$(($(grep -c ^processor /proc/cpuinfo) + 1))
-curl -sL https://sourceforge.net/projects/boost/files/boost/${BOOST_VERSION}/boost_${BOOST_VERSION_UNDERSCORE}.tar.bz2 -o /boost_${BOOST_VERSION_UNDERSCORE}.tar.bz2
-tar xf boost_${BOOST_VERSION_UNDERSCORE}.tar.bz2
+BASE_NAME=boost_${BOOST_VERSION_UNDERSCORE}
+ARCHIVE_NAME=boost_${BOOST_VERSION_UNDERSCORE}.tar.bz2
+
+# Bintray is much faster but can fail because of limitations
+curl -sL https://dl.bintray.com/boostorg/release/${BOOST_VERSION}/source/boost_${BOOST_VERSION_UNDERSCORE}.tar.bz2 -o /${ARCHIVE_NAME} \
+ || curl -sL https://sourceforge.net/projects/boost/files/boost/${BOOST_VERSION}/boost_${BOOST_VERSION_UNDERSCORE}.tar.bz2 -o /${ARCHIVE_NAME}
Review comment:
Nice!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#issuecomment-645461657
@github-actions crossbow submit -g wheel
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#issuecomment-645603358
> This grows the manylinux wheel sizes by about 2-3 MB (from ~13MB to ~15-16MB). It think it's reasonable.
Yes, I expected more.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#issuecomment-645560128
It seems like everything is working fine here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#issuecomment-645525436
@github-actions crossbow submit -g wheel
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#issuecomment-645466049
https://issues.apache.org/jira/browse/ARROW-9109
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#issuecomment-645507606
Revision: 282de25836ec86c707c34340c5a0f4907392e178
Submitted crossbow builds: [ursa-labs/crossbow @ actions-327](https://github.com/ursa-labs/crossbow/branches/all?query=actions-327)
|Task|Status|
|----|------|
|wheel-manylinux1-cp35m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-327-azure-wheel-manylinux1-cp35m)|
|wheel-manylinux1-cp36m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-327-azure-wheel-manylinux1-cp36m)|
|wheel-manylinux1-cp37m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-327-azure-wheel-manylinux1-cp37m)|
|wheel-manylinux1-cp38|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-327-azure-wheel-manylinux1-cp38)|
|wheel-manylinux2010-cp35m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-327-azure-wheel-manylinux2010-cp35m)|
|wheel-manylinux2010-cp36m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-327-azure-wheel-manylinux2010-cp36m)|
|wheel-manylinux2010-cp37m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-327-azure-wheel-manylinux2010-cp37m)|
|wheel-manylinux2010-cp38|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-327-azure-wheel-manylinux2010-cp38)|
|wheel-manylinux2014-cp35m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-327-azure-wheel-manylinux2014-cp35m)|
|wheel-manylinux2014-cp36m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-327-azure-wheel-manylinux2014-cp36m)|
|wheel-manylinux2014-cp37m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-327-azure-wheel-manylinux2014-cp37m)|
|wheel-manylinux2014-cp38|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-327-azure-wheel-manylinux2014-cp38)|
|wheel-osx-cp35m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-cp37m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-win-cp35m|[](https://ci.appveyor.com/project/ursa-labs/crossbow/history)|
|wheel-win-cp36m|[](https://ci.appveyor.com/project/ursa-labs/crossbow/history)|
|wheel-win-cp37m|[](https://ci.appveyor.com/project/ursa-labs/crossbow/history)|
|wheel-win-cp38|[](https://ci.appveyor.com/project/ursa-labs/crossbow/history)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#issuecomment-645562851
This grows the manylinux wheel sizes by about 2-3 MB (from ~13MB to ~15-16MB). It think it's reasonable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#issuecomment-645526238
Revision: f28701736f39939dc7eafb3db2e093612ed3c10c
Submitted crossbow builds: [ursa-labs/crossbow @ actions-328](https://github.com/ursa-labs/crossbow/branches/all?query=actions-328)
|Task|Status|
|----|------|
|wheel-manylinux1-cp35m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-328-azure-wheel-manylinux1-cp35m)|
|wheel-manylinux1-cp36m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-328-azure-wheel-manylinux1-cp36m)|
|wheel-manylinux1-cp37m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-328-azure-wheel-manylinux1-cp37m)|
|wheel-manylinux1-cp38|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-328-azure-wheel-manylinux1-cp38)|
|wheel-manylinux2010-cp35m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-328-azure-wheel-manylinux2010-cp35m)|
|wheel-manylinux2010-cp36m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-328-azure-wheel-manylinux2010-cp36m)|
|wheel-manylinux2010-cp37m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-328-azure-wheel-manylinux2010-cp37m)|
|wheel-manylinux2010-cp38|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-328-azure-wheel-manylinux2010-cp38)|
|wheel-manylinux2014-cp35m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-328-azure-wheel-manylinux2014-cp35m)|
|wheel-manylinux2014-cp36m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-328-azure-wheel-manylinux2014-cp36m)|
|wheel-manylinux2014-cp37m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-328-azure-wheel-manylinux2014-cp37m)|
|wheel-manylinux2014-cp38|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-328-azure-wheel-manylinux2014-cp38)|
|wheel-osx-cp35m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-cp37m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-win-cp35m|[](https://ci.appveyor.com/project/ursa-labs/crossbow/history)|
|wheel-win-cp36m|[](https://ci.appveyor.com/project/ursa-labs/crossbow/history)|
|wheel-win-cp37m|[](https://ci.appveyor.com/project/ursa-labs/crossbow/history)|
|wheel-win-cp38|[](https://ci.appveyor.com/project/ursa-labs/crossbow/history)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs edited a comment on pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
kszucs edited a comment on pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#issuecomment-645614037
Quickly went through it and it LGTM. I can give it a more thorough review tomorrow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a change in pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
kou commented on a change in pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#discussion_r442489766
##########
File path: python/manylinux1/scripts/build_boost.sh
##########
@@ -20,10 +20,16 @@ BOOST_VERSION=1.73.0
BOOST_VERSION_UNDERSCORE=${BOOST_VERSION//\./_}
NCORES=$(($(grep -c ^processor /proc/cpuinfo) + 1))
-curl -sL https://sourceforge.net/projects/boost/files/boost/${BOOST_VERSION}/boost_${BOOST_VERSION_UNDERSCORE}.tar.bz2 -o /boost_${BOOST_VERSION_UNDERSCORE}.tar.bz2
-tar xf boost_${BOOST_VERSION_UNDERSCORE}.tar.bz2
+BASE_NAME=boost_${BOOST_VERSION_UNDERSCORE}
+ARCHIVE_NAME=boost_${BOOST_VERSION_UNDERSCORE}.tar.bz2
+
+# Bintray is much faster but can fail because of limitations
+curl -sL https://dl.bintray.com/boostorg/release/${BOOST_VERSION}/source/boost_${BOOST_VERSION_UNDERSCORE}.tar.bz2 -o /${ARCHIVE_NAME} \
+ || curl -sL https://sourceforge.net/projects/boost/files/boost/${BOOST_VERSION}/boost_${BOOST_VERSION_UNDERSCORE}.tar.bz2 -o /${ARCHIVE_NAME}
Review comment:
FYI: I've created https://bintray.com/apache/arrow/boost-mirror and mirrored Boost 1.73.0 as https://apache.bintray.com/arrow/boost/1.73.0/boost_1_73_0.tar.bz2 .
We can use this to save the official Boost download resource.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
wesm commented on pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#issuecomment-645612814
Very reasonable. Thanks @pitrou!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#issuecomment-645502945
@github-actions crossbow submit -g wheel
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#issuecomment-645614037
Quickly went through it and LGTM. I can give it a more thorough review tomorrow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7471:
URL: https://github.com/apache/arrow/pull/7471#issuecomment-645462522
Revision: 0517d82791270284f4004ffe7812f08f2bb35235
Submitted crossbow builds: [ursa-labs/crossbow @ actions-326](https://github.com/ursa-labs/crossbow/branches/all?query=actions-326)
|Task|Status|
|----|------|
|wheel-manylinux1-cp35m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-326-azure-wheel-manylinux1-cp35m)|
|wheel-manylinux1-cp36m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-326-azure-wheel-manylinux1-cp36m)|
|wheel-manylinux1-cp37m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-326-azure-wheel-manylinux1-cp37m)|
|wheel-manylinux1-cp38|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-326-azure-wheel-manylinux1-cp38)|
|wheel-manylinux2010-cp35m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-326-azure-wheel-manylinux2010-cp35m)|
|wheel-manylinux2010-cp36m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-326-azure-wheel-manylinux2010-cp36m)|
|wheel-manylinux2010-cp37m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-326-azure-wheel-manylinux2010-cp37m)|
|wheel-manylinux2010-cp38|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-326-azure-wheel-manylinux2010-cp38)|
|wheel-manylinux2014-cp35m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-326-azure-wheel-manylinux2014-cp35m)|
|wheel-manylinux2014-cp36m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-326-azure-wheel-manylinux2014-cp36m)|
|wheel-manylinux2014-cp37m|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-326-azure-wheel-manylinux2014-cp37m)|
|wheel-manylinux2014-cp38|[](https://dev.azure.com/ursa-labs/crossbow/_build/latest?definitionId=1&branchName=actions-326-azure-wheel-manylinux2014-cp38)|
|wheel-osx-cp35m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-cp37m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-win-cp35m|[](https://ci.appveyor.com/project/ursa-labs/crossbow/history)|
|wheel-win-cp36m|[](https://ci.appveyor.com/project/ursa-labs/crossbow/history)|
|wheel-win-cp37m|[](https://ci.appveyor.com/project/ursa-labs/crossbow/history)|
|wheel-win-cp38|[](https://ci.appveyor.com/project/ursa-labs/crossbow/history)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed pull request #7471: ARROW-9109: [Python][Packaging] Enable S3 support in manylinux wheels
Posted by GitBox <gi...@apache.org>.
pitrou closed pull request #7471:
URL: https://github.com/apache/arrow/pull/7471
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org