You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Joe McDonnell (Code Review)" <ge...@cloudera.org> on 2025/08/08 05:30:10 UTC
[Impala-ASF-CR] IMPALA-14294: Fix bin/single node perf run.py to work with compression levels
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23270
Change subject: IMPALA-14294: Fix bin/single_node_perf_run.py to work with compression levels
......................................................................
IMPALA-14294: Fix bin/single_node_perf_run.py to work with compression levels
bin/single_node_perf_run.py and the supporting scripts currently don't
support compression levels. It is useful to be able to compare compression
levels for these benchmarks, so this adds that support. There are several
parts:
- Add support for parsing the compression codecs with a level (test_dimension.py)
- Plumb through the compression level to the compression_codec query option
(generate-schema-statements.py)
- Adjust filenames / database names to incorporate the compression level
to avoid collisions (test_dimension.py, generate-schema-statements.py)
For example, the database for tpcds parquet/zstd:6 would be
tpcds42_parquet/zstd_level6. Database names can't contain "-", so this
specifies negative compression levels with negX. e.g. tpcds42_parquet_zstd_levelneg1
This keeps the naming the same if the compression level is not specified.
- Unify the logic around the table format string to simplify the handling
(report_benchmark_results.py / query.py / test_file_parser.py)
Testing:
- Ran perf-AB-test with zstd:6, lz4:3, zstd:-1, and without specifying
compression levels.
Change-Id: If4d3e22f5ac0ff1a41f67e002b72f15523363cc2
---
M testdata/bin/generate-schema-statements.py
M tests/benchmark/report_benchmark_results.py
M tests/common/test_dimensions.py
M tests/performance/query.py
M tests/util/test_file_parser.py
5 files changed, 86 insertions(+), 62 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/23270/1
--
To view, visit http://gerrit.cloudera.org:8080/23270
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: If4d3e22f5ac0ff1a41f67e002b72f15523363cc2
Gerrit-Change-Number: 23270
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-14294: Fix bin/single node perf run.py to work with compression levels
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23270
to look at the new patch set (#2).
Change subject: IMPALA-14294: Fix bin/single_node_perf_run.py to work with compression levels
......................................................................
IMPALA-14294: Fix bin/single_node_perf_run.py to work with compression levels
bin/single_node_perf_run.py and the supporting scripts currently don't
support compression levels. It is useful to be able to compare compression
levels for these benchmarks, so this adds that support. There are several
parts:
- Add support for parsing the compression codecs with a level (test_dimension.py)
- Plumb through the compression level to the compression_codec query option
(generate-schema-statements.py)
- Adjust filenames / database names to incorporate the compression level
to avoid collisions (test_dimension.py, generate-schema-statements.py)
For example, the database for tpcds parquet/zstd:6 would be
tpcds42_parquet/zstd_level6. Database names can't contain "-", so this
specifies negative compression levels with negX. e.g. tpcds42_parquet_zstd_levelneg1
This keeps the naming the same if the compression level is not specified.
- Unify the logic around the table format string to simplify the handling
(report_benchmark_results.py / query.py / test_file_parser.py)
Testing:
- Ran perf-AB-test with zstd:6, lz4:3, zstd:-1, and without specifying
compression levels.
Change-Id: If4d3e22f5ac0ff1a41f67e002b72f15523363cc2
---
M testdata/bin/generate-schema-statements.py
M tests/benchmark/report_benchmark_results.py
M tests/common/test_dimensions.py
M tests/infra/test_run_workload.py
M tests/performance/query.py
M tests/util/test_file_parser.py
6 files changed, 88 insertions(+), 64 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/23270/2
--
To view, visit http://gerrit.cloudera.org:8080/23270
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If4d3e22f5ac0ff1a41f67e002b72f15523363cc2
Gerrit-Change-Number: 23270
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-14294: Fix bin/single node perf run.py to work with compression levels
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23270 )
Change subject: IMPALA-14294: Fix bin/single_node_perf_run.py to work with compression levels
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/20099/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23270
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If4d3e22f5ac0ff1a41f67e002b72f15523363cc2
Gerrit-Change-Number: 23270
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Sat, 09 Aug 2025 00:54:47 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14294: Fix bin/single node perf run.py to work with compression levels
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23270 )
Change subject: IMPALA-14294: Fix bin/single_node_perf_run.py to work with compression levels
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/20093/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23270
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If4d3e22f5ac0ff1a41f67e002b72f15523363cc2
Gerrit-Change-Number: 23270
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 Aug 2025 05:58:49 +0000
Gerrit-HasComments: No