You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/10/01 09:55:42 UTC
[GitHub] [arrow] kszucs opened a new pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
kszucs opened a new pull request #8315:
URL: https://github.com/apache/arrow/pull/8315
Depends on #8304
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-704925644
Revision: c40e6e4ada9a8ce7cc2a594eedf5d085a5b17fc8
Submitted crossbow builds: [ursa-labs/crossbow @ actions-611](https://github.com/ursa-labs/crossbow/branches/all?query=actions-611)
|Task|Status|
|----|------|
|wheel-osx-high-sierra-cp35m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp37m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp35m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp37m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-704176534
@github-actions crossbow submit wheel-osx-*
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-704922415
@github-actions crossbow submit wheel-osx-*
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-702033540
https://issues.apache.org/jira/browse/ARROW-9266
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-705024153
> Sadly we can't utilize travis' build cache since crossbow always creates new branches and the fallback default branch won't have any cache entries.
We *could* run something on `master` on the crossbow repo to warm the cache.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-704177368
Revision: 97b5b3248a2becd7bf86d89cb8bd71de2bf43e30
Submitted crossbow builds: [ursa-labs/crossbow @ actions-605](https://github.com/ursa-labs/crossbow/branches/all?query=actions-605)
|Task|Status|
|----|------|
|wheel-osx-high-sierra-cp35m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp37m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp35m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp37m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs closed pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kszucs closed pull request #8315:
URL: https://github.com/apache/arrow/pull/8315
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-703916845
Revision: 16a9f0916e8a9c5e10e0f66a086ad54c84676d34
Submitted crossbow builds: [ursa-labs/crossbow @ actions-604](https://github.com/ursa-labs/crossbow/branches/all?query=actions-604)
|Task|Status|
|----|------|
|wheel-osx-high-sierra-cp35m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp37m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp35m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp37m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] corleyma commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
corleyma commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-705020614
> The successful builds take between ~42-45 minutes but occasionally it exceeds the 50m build time limit. Sadly we can't utilize travis' build cache since crossbow always creates new branches and the fallback default branch won't have any cache entries.
Does this explain why some of the nightly builds seem to be intermittently missing certain macOS wheels? E.g., sometimes for example there's no cp38 wheel, only for that wheel to come back with the next nightly publish.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-702027646
Revision: 430ffdf472b9420cdbb5784946e2ba0edaf2f5c8
Submitted crossbow builds: [ursa-labs/crossbow @ actions-571](https://github.com/ursa-labs/crossbow/branches/all?query=actions-571)
|Task|Status|
|----|------|
|wheel-osx-high-sierra-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs edited a comment on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kszucs edited a comment on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-705168356
> > Sadly we can't utilize travis' build cache since crossbow always creates new branches and the fallback default branch won't have any cache entries.
>
> We _could_ run something on `master` on the crossbow repo to warm the cache.
We would need to merge the latests tasks (multiple .travis.yml files into a single one) generated by crossbow to master automatically which would be rather complicated. Crossbow also creates diverged branches from master at the moment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] corleyma commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
corleyma commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-705026109
I think @nealrichardson is right: my understanding is that travis caching will try to pull from default branch if no cache is available for current branch, so the warming strategy could work.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-702062031
@github-actions crossbow submit wheel-osx-*
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a change in pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kou commented on a change in pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#discussion_r499956145
##########
File path: dev/tasks/python-wheels/osx-build.sh
##########
@@ -67,6 +67,7 @@ function build_wheel {
-DARROW_GANDIVA=OFF \
-DARROW_GRPC_USE_SHARED=OFF \
-DARROW_HDFS=ON \
+ -DARROW_S3=${ARROW_S3} \
Review comment:
Could you keep alphabetical order?
##########
File path: dev/tasks/python-wheels/osx-build.sh
##########
@@ -103,6 +104,7 @@ function build_wheel {
export PYARROW_WITH_DATASET=1
export PYARROW_WITH_FLIGHT=1
export PYARROW_WITH_HDFS=1
+ export PYARROW_WITH_S3=${ARROW_S3}
Review comment:
Ditto.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs edited a comment on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kszucs edited a comment on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-705168356
> > Sadly we can't utilize travis' build cache since crossbow always creates new branches and the fallback default branch won't have any cache entries.
>
> We _could_ run something on `master` on the crossbow repo to warm the cache.
We would need to merge the latests tasks (multiple .travis.yml files into a single one) generated by crossbow to master automatically which would be rather complicated. Crossbow also creates diverged branches from master at the moment.
We can make a jira about supporting CI caches in crossbow though I can't see a generic solution at the moment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-703916030
@github-actions crossbow submit wheel-osx-*
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-702063070
Revision: 430ffdf472b9420cdbb5784946e2ba0edaf2f5c8
Submitted crossbow builds: [ursa-labs/crossbow @ actions-572](https://github.com/ursa-labs/crossbow/branches/all?query=actions-572)
|Task|Status|
|----|------|
|wheel-osx-high-sierra-cp35m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp37m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp35m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp37m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-703560971
Revision: 50aaa55f306fd492cb743f011c9968860dbab2d8
Submitted crossbow builds: [ursa-labs/crossbow @ actions-598](https://github.com/ursa-labs/crossbow/branches/all?query=actions-598)
|Task|Status|
|----|------|
|wheel-osx-high-sierra-cp35m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp37m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-high-sierra-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp35m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp36m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp37m|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|wheel-osx-mavericks-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-704978980
@github-actions crossbow submit wheel-osx-mavericks-cp38
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-702026510
@github-actions crossbow submit wheel-osx-mavericks-cp36m wheel-osx-high-sierra-cp36m
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-704987021
Revision: 7bcd83e9096bc4ff4078b5b908e8ca295b0c3192
Submitted crossbow builds: [ursa-labs/crossbow @ actions-613](https://github.com/ursa-labs/crossbow/branches/all?query=actions-613)
|Task|Status|
|----|------|
|wheel-osx-mavericks-cp38|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-704923912
The successful builds take between ~42-45 minutes but occasionally it exceeds the 50m build time limit. Sadly we can't utilize travis' build cache since crossbow always creates new branches and the fallback default branch won't have any cache entries.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-705168356
> > Sadly we can't utilize travis' build cache since crossbow always creates new branches and the fallback default branch won't have any cache entries.
>
> We _could_ run something on `master` on the crossbow repo to warm the cache.
We would need to merge the latests tasks (multiple .travis.yml files) generated by crossbow to master automatically which would be rather complicated. Crossbow also creates diverged branches from master at the moment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-703914601
@kszucs please rebase
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-703560112
@github-actions crossbow submit wheel-osx-*
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #8315:
URL: https://github.com/apache/arrow/pull/8315#issuecomment-705169726
> Does this explain why some of the nightly builds seem to be intermittently missing certain macOS wheels? E.g., sometimes for example there's no cp38 wheel, only for that wheel to come back with the next nightly publish.
Could you send me build links? I don't entirely see your question.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org