You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/01/31 14:59:34 UTC
[GitHub] [arrow] pitrou opened a new pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
pitrou opened a new pull request #12301:
URL: https://github.com/apache/arrow/pull/12301
Tests for optional components could be silently skipped if by accident the component would fail importing despite being built.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1027401421
Benchmark runs are scheduled for baseline = bf0ee3f9c50e10de1237bb426138b1734b7c934d and contender = 56e270fda7f5647a157acd1e428d9735d6399881. 56e270fda7f5647a157acd1e428d9735d6399881 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/c6d8e51dab46453d824268e1d3b37f09...2661ef88d7f14e00a19f686d82a70bdf/)
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/65554a70bc184fe8990d7e9579cf428c...a9f298d5793f485baed020f8dfeda553/)
[Finished :arrow_down:0.48% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/14749ae553b5411dac9944a0b02988e4...f3c8185d1f3444328676e5498a9c2f68/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1025845749
Revision: 9a38be5d52a2957cbe991b3282743b3ccf59c8a6
Submitted crossbow builds: [ursacomputing/crossbow @ actions-1550](https://github.com/ursacomputing/crossbow/branches/all?query=actions-1550)
|Task|Status|
|----|------|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-test-conda-python-3.9-spark-master)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-1550-azure-test-debian-11-python-3)|
|test-fedora-33-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-1550-azure-test-fedora-33-python-3)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-1550-azure-test-ubuntu-20.04-python-3)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-macos-big-sur-cp310-arm64)|
|wheel-macos-big-sur-cp310-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-macos-big-sur-cp310-universal2)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-macos-big-sur-cp38-arm64)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-macos-big-sur-cp39-arm64)|
|wheel-macos-big-sur-cp39-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-macos-big-sur-cp39-universal2)|
|wheel-macos-high-sierra-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-macos-high-sierra-cp310-amd64)|
|wheel-macos-high-sierra-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-macos-high-sierra-cp37-amd64)|
|wheel-macos-high-sierra-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-macos-high-sierra-cp38-amd64)|
|wheel-macos-high-sierra-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-macos-high-sierra-cp39-amd64)|
|wheel-macos-mavericks-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-macos-mavericks-cp310-amd64)|
|wheel-macos-mavericks-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-macos-mavericks-cp37-amd64)|
|wheel-macos-mavericks-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-macos-mavericks-cp38-amd64)|
|wheel-macos-mavericks-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-macos-mavericks-cp39-amd64)|
|wheel-manylinux2010-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-manylinux2010-cp310-amd64)|
|wheel-manylinux2010-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-manylinux2010-cp37-amd64)|
|wheel-manylinux2010-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-manylinux2010-cp38-amd64)|
|wheel-manylinux2010-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-manylinux2010-cp39-amd64)|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-windows-cp310-amd64)|
|wheel-windows-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-windows-cp37-amd64)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-windows-cp38-amd64)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1550-github-wheel-windows-cp39-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
pitrou closed pull request #12301:
URL: https://github.com/apache/arrow/pull/12301
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1026040795
Revision: 0838890f486e8d9f81e01fc83c90556729a8e444
Submitted crossbow builds: [ursacomputing/crossbow @ actions-1554](https://github.com/ursacomputing/crossbow/branches/all?query=actions-1554)
|Task|Status|
|----|------|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-test-conda-python-3.9-spark-master)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-1554-azure-test-debian-11-python-3)|
|test-fedora-33-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-1554-azure-test-fedora-33-python-3)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-1554-azure-test-ubuntu-20.04-python-3)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-macos-big-sur-cp310-arm64)|
|wheel-macos-big-sur-cp310-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-macos-big-sur-cp310-universal2)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-macos-big-sur-cp38-arm64)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-macos-big-sur-cp39-arm64)|
|wheel-macos-big-sur-cp39-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-macos-big-sur-cp39-universal2)|
|wheel-macos-high-sierra-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-macos-high-sierra-cp310-amd64)|
|wheel-macos-high-sierra-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-macos-high-sierra-cp37-amd64)|
|wheel-macos-high-sierra-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-macos-high-sierra-cp38-amd64)|
|wheel-macos-high-sierra-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-macos-high-sierra-cp39-amd64)|
|wheel-macos-mavericks-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-macos-mavericks-cp310-amd64)|
|wheel-macos-mavericks-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-macos-mavericks-cp37-amd64)|
|wheel-macos-mavericks-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-macos-mavericks-cp38-amd64)|
|wheel-macos-mavericks-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-macos-mavericks-cp39-amd64)|
|wheel-manylinux2010-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-manylinux2010-cp310-amd64)|
|wheel-manylinux2010-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-manylinux2010-cp37-amd64)|
|wheel-manylinux2010-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-manylinux2010-cp38-amd64)|
|wheel-manylinux2010-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-manylinux2010-cp39-amd64)|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-windows-cp310-amd64)|
|wheel-windows-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-windows-cp37-amd64)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-windows-cp38-amd64)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1554-github-wheel-windows-cp39-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1026848703
> @kszucs Do we lack macOS CI resources on x86-64? See Crossbow jobs above.
GHA limits the number of parallel macOS runners, see https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#discussion_r795771053
##########
File path: ci/scripts/python_test.sh
##########
@@ -30,4 +30,14 @@ export ARROW_GDB_SCRIPT=${arrow_dir}/cpp/gdb_arrow.py
# Enable some checks inside Python itself
export PYTHONDEVMODE=1
+# By default, force-test all optional components
+export PYARROW_TEST_CUDA=${PYARROW_TEST_ORC:-${ARROW_CUDA}}
+export PYARROW_TEST_DATASET=${PYARROW_TEST_ORC:-${ARROW_DATASET}}
+export PYARROW_TEST_FLIGHT=${PYARROW_TEST_ORC:-${ARROW_FLIGHT}}
+export PYARROW_TEST_GANDIVA=${PYARROW_TEST_ORC:-${ARROW_GANDIVA}}
+export PYARROW_TEST_HDFS=${PYARROW_TEST_ORC:-${ARROW_HDFS}}
+export PYARROW_TEST_ORC=${PYARROW_TEST_ORC:-${ARROW_ORC}}
+export PYARROW_TEST_PARQUET=${PYARROW_TEST_ORC:-${ARROW_PARQUET}}
+export PYARROW_TEST_S3=${PYARROW_TEST_ORC:-${ARROW_S3}}
Review comment:
Shouldn't these look like `export PYARROW_TEST_S3=${ARROW_S3:-ON}`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#discussion_r795796985
##########
File path: ci/scripts/python_test.sh
##########
@@ -30,4 +30,14 @@ export ARROW_GDB_SCRIPT=${arrow_dir}/cpp/gdb_arrow.py
# Enable some checks inside Python itself
export PYTHONDEVMODE=1
+# By default, force-test all optional components
+export PYARROW_TEST_CUDA=${PYARROW_TEST_ORC:-${ARROW_CUDA}}
+export PYARROW_TEST_DATASET=${PYARROW_TEST_ORC:-${ARROW_DATASET}}
+export PYARROW_TEST_FLIGHT=${PYARROW_TEST_ORC:-${ARROW_FLIGHT}}
+export PYARROW_TEST_GANDIVA=${PYARROW_TEST_ORC:-${ARROW_GANDIVA}}
+export PYARROW_TEST_HDFS=${PYARROW_TEST_ORC:-${ARROW_HDFS}}
+export PYARROW_TEST_ORC=${PYARROW_TEST_ORC:-${ARROW_ORC}}
+export PYARROW_TEST_PARQUET=${PYARROW_TEST_ORC:-${ARROW_PARQUET}}
+export PYARROW_TEST_S3=${PYARROW_TEST_ORC:-${ARROW_S3}}
Review comment:
Oh, yes, exactly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#discussion_r795786943
##########
File path: ci/scripts/python_test.sh
##########
@@ -30,4 +30,14 @@ export ARROW_GDB_SCRIPT=${arrow_dir}/cpp/gdb_arrow.py
# Enable some checks inside Python itself
export PYTHONDEVMODE=1
+# By default, force-test all optional components
+export PYARROW_TEST_CUDA=${PYARROW_TEST_ORC:-${ARROW_CUDA}}
+export PYARROW_TEST_DATASET=${PYARROW_TEST_ORC:-${ARROW_DATASET}}
+export PYARROW_TEST_FLIGHT=${PYARROW_TEST_ORC:-${ARROW_FLIGHT}}
+export PYARROW_TEST_GANDIVA=${PYARROW_TEST_ORC:-${ARROW_GANDIVA}}
+export PYARROW_TEST_HDFS=${PYARROW_TEST_ORC:-${ARROW_HDFS}}
+export PYARROW_TEST_ORC=${PYARROW_TEST_ORC:-${ARROW_ORC}}
+export PYARROW_TEST_PARQUET=${PYARROW_TEST_ORC:-${ARROW_PARQUET}}
+export PYARROW_TEST_S3=${PYARROW_TEST_ORC:-${ARROW_S3}}
Review comment:
Ohh, right: `: ${PYARROW_TEST_S3:=${ARROW_S3:-ON}}` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1026836899
Benchmark runs are scheduled for baseline = bf0ee3f9c50e10de1237bb426138b1734b7c934d and contender = 56e270fda7f5647a157acd1e428d9735d6399881. 56e270fda7f5647a157acd1e428d9735d6399881 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/c6d8e51dab46453d824268e1d3b37f09...b3c5aa6249d2460894716592294280bd/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/65554a70bc184fe8990d7e9579cf428c...6b8457ba724b4a75abf8c68b4b2144f9/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/14749ae553b5411dac9944a0b02988e4...165d44d2341746f7bfda0ca55f5242cc/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1026039729
@github-actions crossbow submit -g python -g wheel
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1026836899
Benchmark runs are scheduled for baseline = bf0ee3f9c50e10de1237bb426138b1734b7c934d and contender = 56e270fda7f5647a157acd1e428d9735d6399881. 56e270fda7f5647a157acd1e428d9735d6399881 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/c6d8e51dab46453d824268e1d3b37f09...b3c5aa6249d2460894716592294280bd/)
[Finished :arrow_down:1.43% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/65554a70bc184fe8990d7e9579cf428c...6b8457ba724b4a75abf8c68b4b2144f9/)
[Finished :arrow_down:0.04% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/14749ae553b5411dac9944a0b02988e4...165d44d2341746f7bfda0ca55f5242cc/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1026850363
> > @kszucs Do we lack macOS CI resources on x86-64? See Crossbow jobs above.
>
> GHA limits the number of parallel macOS runners, see https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits
I mean we seem to have our own M1 macOS runners, so we could have some for x86-64 macOS too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1027401421
Benchmark runs are scheduled for baseline = bf0ee3f9c50e10de1237bb426138b1734b7c934d and contender = 56e270fda7f5647a157acd1e428d9735d6399881. 56e270fda7f5647a157acd1e428d9735d6399881 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/c6d8e51dab46453d824268e1d3b37f09...2661ef88d7f14e00a19f686d82a70bdf/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/65554a70bc184fe8990d7e9579cf428c...a9f298d5793f485baed020f8dfeda553/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/14749ae553b5411dac9944a0b02988e4...f3c8185d1f3444328676e5498a9c2f68/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1025843606
@github-actions crossbow submit -g python -g wheel
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1025843674
https://issues.apache.org/jira/browse/ARROW-15504
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1026851499
Sure, we can configure self-hosted runners.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1027401421
Benchmark runs are scheduled for baseline = bf0ee3f9c50e10de1237bb426138b1734b7c934d and contender = 56e270fda7f5647a157acd1e428d9735d6399881. 56e270fda7f5647a157acd1e428d9735d6399881 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Scheduled] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/c6d8e51dab46453d824268e1d3b37f09...2661ef88d7f14e00a19f686d82a70bdf/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/65554a70bc184fe8990d7e9579cf428c...a9f298d5793f485baed020f8dfeda553/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/14749ae553b5411dac9944a0b02988e4...f3c8185d1f3444328676e5498a9c2f68/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1026108684
@kszucs Do we lack macOS CI resources on x86-64? See Crossbow jobs above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1026836899
Benchmark runs are scheduled for baseline = bf0ee3f9c50e10de1237bb426138b1734b7c934d and contender = 56e270fda7f5647a157acd1e428d9735d6399881. 56e270fda7f5647a157acd1e428d9735d6399881 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Scheduled] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/c6d8e51dab46453d824268e1d3b37f09...b3c5aa6249d2460894716592294280bd/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/65554a70bc184fe8990d7e9579cf428c...6b8457ba724b4a75abf8c68b4b2144f9/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/14749ae553b5411dac9944a0b02988e4...165d44d2341746f7bfda0ca55f5242cc/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1026836899
Benchmark runs are scheduled for baseline = bf0ee3f9c50e10de1237bb426138b1734b7c934d and contender = 56e270fda7f5647a157acd1e428d9735d6399881. 56e270fda7f5647a157acd1e428d9735d6399881 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/c6d8e51dab46453d824268e1d3b37f09...b3c5aa6249d2460894716592294280bd/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/65554a70bc184fe8990d7e9579cf428c...6b8457ba724b4a75abf8c68b4b2144f9/)
[Finished :arrow_down:0.04% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/14749ae553b5411dac9944a0b02988e4...165d44d2341746f7bfda0ca55f5242cc/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#discussion_r795783704
##########
File path: ci/scripts/python_test.sh
##########
@@ -30,4 +30,14 @@ export ARROW_GDB_SCRIPT=${arrow_dir}/cpp/gdb_arrow.py
# Enable some checks inside Python itself
export PYTHONDEVMODE=1
+# By default, force-test all optional components
+export PYARROW_TEST_CUDA=${PYARROW_TEST_ORC:-${ARROW_CUDA}}
+export PYARROW_TEST_DATASET=${PYARROW_TEST_ORC:-${ARROW_DATASET}}
+export PYARROW_TEST_FLIGHT=${PYARROW_TEST_ORC:-${ARROW_FLIGHT}}
+export PYARROW_TEST_GANDIVA=${PYARROW_TEST_ORC:-${ARROW_GANDIVA}}
+export PYARROW_TEST_HDFS=${PYARROW_TEST_ORC:-${ARROW_HDFS}}
+export PYARROW_TEST_ORC=${PYARROW_TEST_ORC:-${ARROW_ORC}}
+export PYARROW_TEST_PARQUET=${PYARROW_TEST_ORC:-${ARROW_PARQUET}}
+export PYARROW_TEST_S3=${PYARROW_TEST_ORC:-${ARROW_S3}}
Review comment:
No, we want to be able to override testing, I think? However, it seems I got the above wrong :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12301: ARROW-15504: [Python][CI] Ensure that optional components are tested
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12301:
URL: https://github.com/apache/arrow/pull/12301#issuecomment-1027401421
Benchmark runs are scheduled for baseline = bf0ee3f9c50e10de1237bb426138b1734b7c934d and contender = 56e270fda7f5647a157acd1e428d9735d6399881. 56e270fda7f5647a157acd1e428d9735d6399881 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/c6d8e51dab46453d824268e1d3b37f09...2661ef88d7f14e00a19f686d82a70bdf/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/65554a70bc184fe8990d7e9579cf428c...a9f298d5793f485baed020f8dfeda553/)
[Finished :arrow_down:0.48% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/14749ae553b5411dac9944a0b02988e4...f3c8185d1f3444328676e5498a9c2f68/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org