You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/05/16 13:11:59 UTC
[GitHub] [arrow] raulcd opened a new pull request, #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
raulcd opened a new pull request, #13165:
URL: https://github.com/apache/arrow/pull/13165
This PR will allow us to generate a CSV with the nightly builds status. This will be used in order to generate a static page with the different builds information and will allow us to analyze historical information from the different builds like since when they have been failing, last successful build commit, some tags, etcetera.
An example of a CSV generated can be seen on the following gist:
https://gist.github.com/raulcd/1799c8499ef228dfdf1ab0d986367d2a
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13165:
URL: https://github.com/apache/arrow/pull/13165#issuecomment-1127656107
:warning: Ticket **has not been started in JIRA**, please click 'Start Progress'.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on PR #13165:
URL: https://github.com/apache/arrow/pull/13165#issuecomment-1129673595
> I am not generating a header row for the CSV.
It wouldn't be that hard to do, I think, and could improve usability a bit given that there are quite some columns in there? (eg another attribute `rows_header` that returns a single list that can be written first to the csv file)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #13165:
URL: https://github.com/apache/arrow/pull/13165#discussion_r875558440
##########
dev/archery/archery/crossbow/reports.py:
##########
@@ -83,6 +84,29 @@ def tasks(self):
def show(self):
raise NotImplementedError()
+ @property
+ def rows(self):
+ """
+ Produces a generator that allow us to iterate over
+ the job tasks as a list of rows.
+ """
+ for task_name, task in sorted(self.job.tasks.items()):
+ task_status = task.status()
+ row = [
+ task_name,
+ task_status.combined_state,
+ task_status.build_links,
+ self.branch_url(task.branch),
+ task.ci,
+ # We want this to be serialized as a dict instead
+ # of an orderedict.
+ {k: v for k, v in task.params.items()},
+ task.template,
+ # Arrow repository commit
+ self.job.target.head
Review Comment:
I asked about a commit, but I see here that it is already included! It seems that I missed that there was more content in https://gist.github.com/raulcd/1799c8499ef228dfdf1ab0d986367d2a (only the first three columns are visible ;))
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on a diff in pull request #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
Posted by GitBox <gi...@apache.org>.
raulcd commented on code in PR #13165:
URL: https://github.com/apache/arrow/pull/13165#discussion_r876884582
##########
dev/archery/archery/crossbow/reports.py:
##########
@@ -30,6 +30,17 @@
# TODO(kszucs): use archery.report.JinjaReport instead
class Report:
+ ROW_HEADERS = [
+ "Task name",
Review Comment:
done! Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #13165:
URL: https://github.com/apache/arrow/pull/13165#issuecomment-1129832643
> It wouldn't be that hard to do, I think, and could improve usability a bit given that there are quite some columns in there? (eg another attribute `rows_header` that returns a single list that can be written first to the csv file)
I have added the CSV header and have updated the gist with the updated outcome of running the command locally:
```
$ archery crossbow -t $GITHUB_TOKEN report-csv --no-fetch --save nightly-packaging-2022-04-25-0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on PR #13165:
URL: https://github.com/apache/arrow/pull/13165#issuecomment-1129663643
> * I am not storing the date as part of the CSV because the filename generated will be something like: `nightly-packaging-2022-04-25-0.csv `.
I think that is fine. But I am wondering, it might be useful to keep track of the exact commit that is being tested in the build? For example in case you want to check if a certain fix was already included in the run or not (currently only the date is not that informative, given a date is still a broad range and depending on when it is exactly scheduled on that day)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #13165:
URL: https://github.com/apache/arrow/pull/13165#issuecomment-1127690095
@amol- @jorisvandenbossche I have created this initial PR to be able to generate individual CSV files with the nightly status reports. The idea is to follow up by updating the current report workflows to generate the CSV and commit them to the repository. We can then have a scheduled workflow task that analyses the data and generates the static dashboard with consuming the information from all the CSV files.
A couple of things for consideration on the PR:
* I am not storing the date as part of the CSV because the filename generated will be something like: `nightly-packaging-2022-04-25-0.csv `.
* I am not generating a header row for the CSV.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on a diff in pull request #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
Posted by GitBox <gi...@apache.org>.
raulcd commented on code in PR #13165:
URL: https://github.com/apache/arrow/pull/13165#discussion_r875694382
##########
dev/archery/archery/crossbow/reports.py:
##########
@@ -83,6 +84,29 @@ def tasks(self):
def show(self):
raise NotImplementedError()
+ @property
+ def rows(self):
+ """
+ Produces a generator that allow us to iterate over
+ the job tasks as a list of rows.
+ """
+ for task_name, task in sorted(self.job.tasks.items()):
+ task_status = task.status()
+ row = [
+ task_name,
+ task_status.combined_state,
+ task_status.build_links,
+ self.branch_url(task.branch),
+ task.ci,
+ # We want this to be serialized as a dict instead
+ # of an orderedict.
+ {k: v for k, v in task.params.items()},
+ task.template,
+ # Arrow repository commit
+ self.job.target.head
Review Comment:
yes, the view on the gist for CSV is not great :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #13165:
URL: https://github.com/apache/arrow/pull/13165#discussion_r876803516
##########
dev/archery/archery/crossbow/reports.py:
##########
@@ -30,6 +30,17 @@
# TODO(kszucs): use archery.report.JinjaReport instead
class Report:
+ ROW_HEADERS = [
+ "Task name",
Review Comment:
We could also use names like `"task_name"` instead (lowercase, no spaces), which are generally easier to use when afterwards playing with the data with pandas or R or something alike
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13165:
URL: https://github.com/apache/arrow/pull/13165#issuecomment-1127656064
https://issues.apache.org/jira/browse/ARROW-16358
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #13165:
URL: https://github.com/apache/arrow/pull/13165#issuecomment-1132945843
Benchmark runs are scheduled for baseline = 71737eae6b4d7d341f928838fe403c5fdaaefa76 and contender = c5a175dd61b3fa94a38b918c1cc3d7b4ffaefba7. c5a175dd61b3fa94a38b918c1cc3d7b4ffaefba7 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/aa8b6e9b78a348a19ced25203138e8a3...33ade4f827b841aba6cb2c6b49296751/)
[Failed :arrow_down:0.55% :arrow_up:0.0%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/cb16ef0469b743939747222bbc1c72fa...2b876a603d08471e99fc4ab2e2391c49/)
[Failed :arrow_down:0.74% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/25de8e5c61524fdda601431f268282f0...266b9facb58d450d9381a64db5bb792c/)
[Finished :arrow_down:0.08% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/c3aa3cdaffd64d73b961f62263f293d1...0f74c66afb8a41a79d4ad199a4e70cdd/)
Buildkite builds:
[Finished] [`c5a175dd` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/802)
[Failed] [`c5a175dd` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/799)
[Failed] [`c5a175dd` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/789)
[Finished] [`c5a175dd` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/805)
[Finished] [`71737eae` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/801)
[Failed] [`71737eae` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/798)
[Failed] [`71737eae` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/788)
[Finished] [`71737eae` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/804)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche closed pull request #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche closed pull request #13165: ARROW-16358: [CI][Dev] Allow archery crossbow to generate a CSV report for nightly builds
URL: https://github.com/apache/arrow/pull/13165
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org