You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/10/06 13:56:03 UTC
[GitHub] [arrow] raulcd opened a new pull request, #14335: Arrow-17952: [Archery][CI] Fix archery error when running ubuntu-cuda-cpp
raulcd opened a new pull request, #14335:
URL: https://github.com/apache/arrow/pull/14335
As part of this PR I have also added a second commit to update the usages of shlex.split to shlex.shlex as it is deprecated. I am happy to move this commit to its own PR if we think is better to split it. https://docs.python.org/3/library/shlex.html#shlex.split
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #14335: ARROW-17952: [Archery][CI] Fix archery error when running ubuntu-cuda-cpp
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #14335:
URL: https://github.com/apache/arrow/pull/14335#discussion_r989103984
##########
dev/archery/archery/docker/core.py:
##########
@@ -378,7 +378,12 @@ def run(self, service_name, command=None, *, env=None, volumes=None,
else:
cmd = service.get('command', '')
if cmd:
- args.extend(shlex.split(cmd))
+ # service command might be already defined as a list
+ # on the docker-compose yaml file.
+ if isinstance(cmd, list):
+ cmd = shlex.join(cmd)
+ else:
Review Comment:
Why the "else"? You must still extend the args after joining the command list...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou merged pull request #14335: ARROW-17952: [Archery][CI] Fix archery error when running ubuntu-cuda-cpp
Posted by GitBox <gi...@apache.org>.
pitrou merged PR #14335:
URL: https://github.com/apache/arrow/pull/14335
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #14335: ARROW-17952: [Archery][CI] Fix archery error when running ubuntu-cuda-cpp
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #14335:
URL: https://github.com/apache/arrow/pull/14335#issuecomment-1270102601
https://issues.apache.org/jira/browse/ARROW-17952
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on a diff in pull request #14335: ARROW-17952: [Archery][CI] Fix archery error when running ubuntu-cuda-cpp
Posted by GitBox <gi...@apache.org>.
raulcd commented on code in PR #14335:
URL: https://github.com/apache/arrow/pull/14335#discussion_r989117682
##########
dev/archery/archery/docker/core.py:
##########
@@ -378,7 +378,12 @@ def run(self, service_name, command=None, *, env=None, volumes=None,
else:
cmd = service.get('command', '')
if cmd:
- args.extend(shlex.split(cmd))
+ # service command might be already defined as a list
+ # on the docker-compose yaml file.
+ if isinstance(cmd, list):
+ cmd = shlex.join(cmd)
+ else:
Review Comment:
you are correct. I missed to add the command because I couldn't test locally with GPU. I can see the full expanded command now:
```
docker run --rm --gpus all --shm-size 2147483648 -e ARROW_BUILD_STATIC=OFF -e ARROW_CUDA=ON -e ARROW_GANDIVA=OFF -e ARROW_GCS=OFF -e ARROW_ORC=OFF -e ARROW_S3=OFF -e ARROW_SUBSTRAIT=OFF -e ARROW_WITH_OPENTELEMETRY=OFF -e CCACHE_COMPILERCHECK=content -e CCACHE_COMPRESS=1 -e CCACHE_COMPRESSLEVEL=6 -e CCACHE_DIR=/ccache -e CCACHE_MAXSIZE=1G -v /home/raulcd/code/arrow:/arrow -v ubuntu-ccache:/ccache apache/arrow-dev:amd64-ubuntu-20.04-cuda-11.0.3-cpp /bin/bash -c
/arrow/ci/scripts/cpp_build.sh /arrow /build &&
/arrow/ci/scripts/integration_skyhook.sh /build &&
/arrow/ci/scripts/cpp_test.sh /arrow /build
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #14335: ARROW-17952: [Archery][CI] Fix archery error when running ubuntu-cuda-cpp
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #14335:
URL: https://github.com/apache/arrow/pull/14335#issuecomment-1270595499
Benchmark runs are scheduled for baseline = 418f11558d561cfca700891efb2a53e478d26a8e and contender = ed36fcd218d381bd7420f1b762a28c5feea4665f. ed36fcd218d381bd7420f1b762a28c5feea4665f is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/60d8242557b04691b79d0cfe3a188e98...49a8f0f6200b47978bc19e2a660fded2/)
[Failed :arrow_down:0.56% :arrow_up:0.0%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/a03a508ce7c34e6396e9a89e810301fd...437019d29dd74947b9e4d1da70114746/)
[Failed :arrow_down:0.82% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/e5433ba68e584bfdaa816b0c3bbf5bb3...fea142179e3e4c81800bdab4055f6c7f/)
[Finished :arrow_down:0.39% :arrow_up:0.04%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/a44194863b3e4cf8ab34d85dd91f46ac...b0336dfe4cca497e85a55f6c66bc5e18/)
Buildkite builds:
[Finished] [`ed36fcd2` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/1637)
[Failed] [`ed36fcd2` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/1656)
[Failed] [`ed36fcd2` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/1639)
[Finished] [`ed36fcd2` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/1650)
[Finished] [`418f1155` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/1636)
[Failed] [`418f1155` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/1655)
[Failed] [`418f1155` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/1638)
[Finished] [`418f1155` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/1649)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #14335: Arrow-17952: [Archery][CI] Fix archery error when running ubuntu-cuda-cpp
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #14335:
URL: https://github.com/apache/arrow/pull/14335#issuecomment-1270098853
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
Thanks for opening a pull request!
If this is not a [minor PR](https://github.com/apache/arrow/blob/master/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW
Opening JIRAs ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
Then could you also rename pull request title in the following format?
ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
See also:
* [Other pull requests](https://github.com/apache/arrow/pulls/)
* [Contribution Guidelines - How to contribute patches](https://arrow.apache.org/docs/developers/contributing.html#how-to-contribute-patches)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #14335: ARROW-17952: [Archery][CI] Fix archery error when running ubuntu-cuda-cpp
Posted by GitBox <gi...@apache.org>.
pitrou commented on PR #14335:
URL: https://github.com/apache/arrow/pull/14335#issuecomment-1270342596
`ubuntu-cuda-cpp` ran successfully on my local machine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org