You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "boshek (via GitHub)" <gi...@apache.org> on 2023/06/26 23:53:14 UTC
[GitHub] [arrow] boshek opened a new pull request, #36315: GH-36236: [R][Python] Add performance comparison between Release Candidate and previous release
boshek opened a new pull request, #36315:
URL: https://github.com/apache/arrow/pull/36315
### Rationale for this change
This is WIP automated reporting product that compares two commits. The intent here is that this will compare a RC and a previous release via the conbench API and then summarize that information graphically. Ideally this is run automatically as part of a release candidate process.
It is described well in #36236
### What changes are included in this PR?
This PR adds a quarto document which can be rendered into a static html file. To specify which commits should be compared one can set the `BASELINE_GIT_COMMIT` and `CONTENDER_GIT_COMMIT` variables.
@raulcd has offered to determine a process to auto-detect the correct shas.
I am happy to work on the CI to render this. I think an R + renv + quarto action workflow would work pretty well here and ensure some stability for R package versions. However, I have not checked in a renv lockfile as of yet so as not to pre-suppose any solutions.
Right now we are only looking at comparisons with the `ursa-i9-9960x` machine. Let me know if we want to expand beyond that. I am attaching a zipped html file which contains an example static html doc. In the future hopefully we will have a better process to share rendering artifacts:
[arrow-release-report.html.zip](https://github.com/apache/arrow/files/11875579/arrow-release-report.html.zip)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on pull request #36315: GH-36236: [R][Python] Add performance comparison between Release Candidate and previous release
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on PR #36315:
URL: https://github.com/apache/arrow/pull/36315#issuecomment-1670551575
> create a repository dispatch workflow
That would be the "normal" way to do this but I think in this case it would be better to create a crossbow job to do it. That way we can also make it an option to only render the report and upload it as an artifact without publishing it. e.g. to review easier on changes to the report.
The windows vcpkg job would be a simple example to follow: https://github.com/apache/arrow/blob/main/dev/tasks/tasks.yml#L1538
We can also pair on this if you'd like :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on pull request #36315: GH-36236: [R][Python] Add performance comparison between Release Candidate and previous release
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on PR #36315:
URL: https://github.com/apache/arrow/pull/36315#issuecomment-1644932341
I think it would make sense to make this into a crossbow job that is triggered during the rc preparation process as we already have everything setup over there to be able to publish static pages via s3:
- trigger conbench on rc commit (via comment on rc PR?)
- start job in a later step (e.g. after binaries are prepared/signed) with benchmarked commit and commit from last-release-tag (insert bash :magic_wand: here)
- insert link to rendered perf report into rc vote message
What do you think @raulcd ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] boshek closed pull request #36315: GH-36236: [R][Python] Add performance comparison between Release Candidate and previous release
Posted by "boshek (via GitHub)" <gi...@apache.org>.
boshek closed pull request #36315: GH-36236: [R][Python] Add performance comparison between Release Candidate and previous release
URL: https://github.com/apache/arrow/pull/36315
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #36315: GH-36236: [R][Python] Add performance comparison between Release Candidate and previous release
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36315:
URL: https://github.com/apache/arrow/pull/36315#issuecomment-1608483831
:warning: GitHub issue #36236 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] boshek commented on pull request #36315: GH-36236: [R][Python] Add performance comparison between Release Candidate and previous release
Posted by "boshek (via GitHub)" <gi...@apache.org>.
boshek commented on PR #36315:
URL: https://github.com/apache/arrow/pull/36315#issuecomment-1670236130
> I think it would make sense to make this into a crossbow job that is triggered during the rc preparation process as we already have everything setup over there to be able to publish static pages via s3:
>
> * trigger conbench on rc commit (via comment on rc PR?)
> * start job in a later step (e.g. after binaries are prepared/signed) with benchmarked commit and commit from last-release-tag (insert bash 🪄 here)
> * insert link to rendered perf report into rc vote message
>
> What do you think @raulcd ?
@assignUser I was thinking I could create a repository dispatch workflow over in crossbow which renders this report. I don't know if this was also implied but I also think we _could_ move this raw file into the crossbow repo as well (though that obviously isn't required). What do you think? I'd have some cycles to start on that so this step:
> * start job in a later step (e.g. after binaries are prepared/signed) with benchmarked commit and commit from last-release-tag (insert bash 🪄 here)
just has to construct the payload and trigger the rendering.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] boshek commented on pull request #36315: GH-36236: [R][Python] Add performance comparison between Release Candidate and previous release
Posted by "boshek (via GitHub)" <gi...@apache.org>.
boshek commented on PR #36315:
URL: https://github.com/apache/arrow/pull/36315#issuecomment-1675477027
Closing this PR is favour of implementing this over in crossbow: https://github.com/ursacomputing/crossbow/pull/61
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org