You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Joe McDonnell (Code Review)" <ge...@cloudera.org> on 2026/10/21 22:46:55 UTC
[Impala-ASF-CR] IMPALA-15556 (part 3): Add --results csv file option to run-workload.py
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/25178
Change subject: IMPALA-15556 (part 3): Add --results_csv_file option to run-workload.py
......................................................................
IMPALA-15556 (part 3): Add --results_csv_file option to run-workload.py
Adds a --results_csv_file option to bin/run-workload.py that writes the
query run times as CSV, for easier comparison in A/B runs. There is a row
per table format, query, client, and workload iteration, and a column
per query iteration (iter_1..iter_N). Failed executions are left blank.
To make this easier, Scheduler now records workload_iteration and
query_iteration on each result. These are also available in the results
JSON. write_profiles() now asserts that its per-client execution counter
agrees with these iteration numbers.
bin/single_node_perf_run.py has been modified to always generate the
CSV output.
Testing:
- Added tests in infra/test_run_workload.py
Change-Id: Iac7c871b32d223bea0ebc7a2d1eb8ad6c0944509
Assisted-by: Claude Sonnet 5.5 (Claude Code)
---
M bin/run-workload.py
M bin/single_node_perf_run.py
M tests/infra/test_run_workload.py
M tests/performance/query.py
M tests/performance/scheduler.py
5 files changed, 90 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/25178/1
--
To view, visit http://gerrit.cloudera.org:8080/25178
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iac7c871b32d223bea0ebc7a2d1eb8ad6c0944509
Gerrit-Change-Number: 25178
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-15556 (part 3): Add --results csv file option to run-workload.py
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/25178 )
Change subject: IMPALA-15556 (part 3): Add --results_csv_file option to run-workload.py
......................................................................
Patch Set 2: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/25178
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iac7c871b32d223bea0ebc7a2d1eb8ad6c0944509
Gerrit-Change-Number: 25178
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Thu, 22 Oct 2026 04:57:54 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-15556 (part 3): Add --results csv file option to run-workload.py
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/25178 )
Change subject: IMPALA-15556 (part 3): Add --results_csv_file option to run-workload.py
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/24633/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/25178
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iac7c871b32d223bea0ebc7a2d1eb8ad6c0944509
Gerrit-Change-Number: 25178
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 21 Oct 2026 23:43:01 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-15556 (part 3): Add --results csv file option to run-workload.py
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/25178 )
Change subject: IMPALA-15556 (part 3): Add --results_csv_file option to run-workload.py
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/14310/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/25178
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iac7c871b32d223bea0ebc7a2d1eb8ad6c0944509
Gerrit-Change-Number: 25178
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Thu, 22 Oct 2026 00:13:02 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-15556 (part 3): Add --results csv file option to run-workload.py
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/25178 )
Change subject: IMPALA-15556 (part 3): Add --results_csv_file option to run-workload.py
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/25178/1/bin/run-workload.py
File bin/run-workload.py:
http://gerrit.cloudera.org:8080/#/c/25178/1/bin/run-workload.py@262
PS1, Line 262: +
> flake8: W504 line break after binary operator
Done
--
To view, visit http://gerrit.cloudera.org:8080/25178
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iac7c871b32d223bea0ebc7a2d1eb8ad6c0944509
Gerrit-Change-Number: 25178
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 21 Oct 2026 23:15:54 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-15556 (part 3): Add --results csv file option to run-workload.py
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/25178
to look at the new patch set (#2).
Change subject: IMPALA-15556 (part 3): Add --results_csv_file option to run-workload.py
......................................................................
IMPALA-15556 (part 3): Add --results_csv_file option to run-workload.py
Adds a --results_csv_file option to bin/run-workload.py that writes the
query run times as CSV, for easier comparison in A/B runs. There is a row
per table format, query, client, and workload iteration, and a column
per query iteration (iter_1..iter_N). Failed executions are left blank.
To make this easier, Scheduler now records workload_iteration and
query_iteration on each result. These are also available in the results
JSON. write_profiles() now asserts that its per-client execution counter
agrees with these iteration numbers.
bin/single_node_perf_run.py has been modified to always generate the
CSV output.
Testing:
- Added tests in infra/test_run_workload.py
Change-Id: Iac7c871b32d223bea0ebc7a2d1eb8ad6c0944509
Assisted-by: Claude Sonnet 5.5 (Claude Code)
---
M bin/run-workload.py
M bin/single_node_perf_run.py
M tests/infra/test_run_workload.py
M tests/performance/query.py
M tests/performance/scheduler.py
5 files changed, 90 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/25178/2
--
To view, visit http://gerrit.cloudera.org:8080/25178
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iac7c871b32d223bea0ebc7a2d1eb8ad6c0944509
Gerrit-Change-Number: 25178
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-15556 (part 3): Add --results csv file option to run-workload.py
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/25178 )
Change subject: IMPALA-15556 (part 3): Add --results_csv_file option to run-workload.py
......................................................................
Patch Set 1:
(1 comment)
gerrit-auto-critic failed. You can reproduce it locally using command:
python3 bin/jenkins/critique-gerrit-review.py --dryrun
To run it, you might need a virtual env with Python3's venv installed.
http://gerrit.cloudera.org:8080/#/c/25178/1/bin/run-workload.py
File bin/run-workload.py:
http://gerrit.cloudera.org:8080/#/c/25178/1/bin/run-workload.py@262
PS1, Line 262: +
flake8: W504 line break after binary operator
--
To view, visit http://gerrit.cloudera.org:8080/25178
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iac7c871b32d223bea0ebc7a2d1eb8ad6c0944509
Gerrit-Change-Number: 25178
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 21 Oct 2026 23:06:38 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-15556 (part 3): Add --results csv file option to run-workload.py
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/25178 )
Change subject: IMPALA-15556 (part 3): Add --results_csv_file option to run-workload.py
......................................................................
Patch Set 1:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/24632/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/25178
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iac7c871b32d223bea0ebc7a2d1eb8ad6c0944509
Gerrit-Change-Number: 25178
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 21 Oct 2026 23:03:41 +0000
Gerrit-HasComments: No