You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "AntoinePrv (via GitHub)" <gi...@apache.org> on 2026/05/04 10:17:22 UTC
[I] [Archery] C++ Benchmark preserve improvements [arrow]
AntoinePrv opened a new issue, #49913:
URL: https://github.com/apache/arrow/issues/49913
### Describe the enhancement requested
With `archery benchmark [run|diff] --preserve`, the build and source folders are preserved into a folder like the following. In my personal use, I found several improvements that would ease working with `archery benchmark`.
```
<TMP>/arrow-archery-xlzqaz4l/<GIT STR>/
- arrow/
- build/
```
### A - Set the preserve directory
On MacOS, the <TMP> directory ends up being something like this
```
/var/folders/9c/thrhbgqx2xb2xqvfk_2m6pgh0000gn/T/
```
Which is impossible to find again without parsing `archery` log.
On top of this, I'd sometime want to control more finely where the cache is stored, either for convenience of inspecting/using it, or because the path structure is not satisfying for some use case (e.g. baenchmarking same commit but with xsimd 14.1 and 14.2, with different compiler options...).
I propose adding an optional CLI argument `--preserve-dir <PATH>` to explicitly control where the preserve directory are stored (`<TMP>` in the above).
### B - Always preserve benchmark output with preserve option
When `--preserve` is set, I recommend we always store the benchmark timings JSON file in the preserve directory:
- It is relatively small compared the the size of the build directory, so we should be eager in saving it just in case it might be needed
- It helps keeping track of their name. Currently we have to think of an explicit name in the `--output`. Having a copy automatically with the build, it is associated with the commit name, the path of the `preserve-dir`, and we can retract the compilation context from the `build` directory (compiler flags used...).
This greatly reduce the cognitive load of having to choose name, track which file correspond to which settings, reduce the length of the `archery benchmark` commands we type.
This would be independent from `--output`, which would still work as before.
There is also more information we should store, such as the invocation command.
### C - Resolve git string (breaking)
Right now, with `archery benchmark run main` the path created is:
```
<TMP>/arrow-archery-xlzqaz4l/main/
```
I suggest replacing it automatically with
```
<TMP>/arrow-archery-xlzqaz4l/<GIT SHA>/
```
At first glance, it will be slightly harder looking at the folder that `main` was intended. Though in practice beleive this is more sneaky than helpful. `main` is a moving target and even with a day of work its meaning can change and we forget "which main". This is even more error-prone with a feature branch, and remote copies for bench-marking different platforms.
### Component(s)
Archery
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Archery] C++ Benchmark preserve improvements [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #49913:
URL: https://github.com/apache/arrow/issues/49913#issuecomment-4372329584
I'd say is still a nice improvement, as you suggest using main sounds like a false friend and is error-prone.
@rok do you know if conbench uses `archery benchmark run main` when running benchmarks in any scenario where the generated subfolder is relevant? See C above. If not, I think we should update it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Archery] C++ Benchmark preserve improvements [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on issue #49913:
URL: https://github.com/apache/arrow/issues/49913#issuecomment-4370641680
A and B are additions, I don't know if anyone is attached to C.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Archery] C++ Benchmark preserve improvements [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou closed issue #49913: [Archery] C++ Benchmark preserve improvements
URL: https://github.com/apache/arrow/issues/49913
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Archery] C++ Benchmark preserve improvements [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #49913:
URL: https://github.com/apache/arrow/issues/49913#issuecomment-4370270457
Those sound like reasonable improvements to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Archery] C++ Benchmark preserve improvements [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #49913:
URL: https://github.com/apache/arrow/issues/49913#issuecomment-4845189779
Issue resolved by pull request 50056
https://github.com/apache/arrow/pull/50056
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org