You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Joe McDonnell (Code Review)" <ge...@cloudera.org> on 2023/04/29 02:14:47 UTC
[Impala-ASF-CR] IMPALA-11603: Build against Cloudflare ZLIB by default
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19207
Change subject: IMPALA-11603: Build against Cloudflare ZLIB by default
......................................................................
IMPALA-11603: Build against Cloudflare ZLIB by default
Cloudflare Zlib is a fork of the Zlib codebase that
has been optimized to take advantage of CPU SIMD
instructions and other platform-specific optimizations.
It has the same license as regular Zlib. Amazon has
touted this as a major speedup over regular Zlib:
https://aws.amazon.com/blogs/opensource/improving-zlib-cloudflare-and-comparing-performance-with-other-zlib-forks/
This adds the IMPALA_USE_CLOUDFLARE_ZLIB environment
variable which allows Impala to be built against
Cloudflare Zlib. This defaults to true. If set to
any other value, it will build against regular Zlib.
Cloudflare Zlib shows a clear performance benefit
over regular Zlib on TPC-H ORC/deflate benchmark:
+----------+-------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+-------------------+---------+------------+------------+----------------+
| TPCH(42) | orc / def / block | 4.18 | -6.43% | 3.29 | -6.74% |
+----------+-------------------+---------+------------+------------+----------------+
Testing:
- Ran GVO tests and exhaustive release tests
Change-Id: I82c480890726da0fa5bdc2a646022554eec181f4
---
M CMakeLists.txt
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
3 files changed, 18 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/19207/5
--
To view, visit http://gerrit.cloudera.org:8080/19207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I82c480890726da0fa5bdc2a646022554eec181f4
Gerrit-Change-Number: 19207
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-11603: Build against Cloudflare ZLIB by default
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19207 )
Change subject: IMPALA-11603: Build against Cloudflare ZLIB by default
......................................................................
Patch Set 5: Code-Review+2
Carry +1 from Michael.
--
To view, visit http://gerrit.cloudera.org:8080/19207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I82c480890726da0fa5bdc2a646022554eec181f4
Gerrit-Change-Number: 19207
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Mon, 01 May 2023 17:35:39 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11603: Build against Cloudflare ZLIB by default
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19207 )
Change subject: IMPALA-11603: Build against Cloudflare ZLIB by default
......................................................................
IMPALA-11603: Build against Cloudflare ZLIB by default
Cloudflare Zlib is a fork of the Zlib codebase that
has been optimized to take advantage of CPU SIMD
instructions and other platform-specific optimizations.
It has the same license as regular Zlib. Amazon has
touted this as a major speedup over regular Zlib:
https://aws.amazon.com/blogs/opensource/improving-zlib-cloudflare-and-comparing-performance-with-other-zlib-forks/
This adds the IMPALA_USE_CLOUDFLARE_ZLIB environment
variable which allows Impala to be built against
Cloudflare Zlib. This defaults to true. If set to
any other value, it will build against regular Zlib.
Cloudflare Zlib shows a clear performance benefit
over regular Zlib on TPC-H ORC/deflate benchmark:
+----------+-------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+-------------------+---------+------------+------------+----------------+
| TPCH(42) | orc / def / block | 4.18 | -6.43% | 3.29 | -6.74% |
+----------+-------------------+---------+------------+------------+----------------+
Testing:
- Ran GVO tests and exhaustive release tests
Change-Id: I82c480890726da0fa5bdc2a646022554eec181f4
Reviewed-on: http://gerrit.cloudera.org:8080/19207
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Michael Smith <mi...@cloudera.com>
Reviewed-by: Wenzhe Zhou <wz...@cloudera.com>
---
M CMakeLists.txt
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
3 files changed, 18 insertions(+), 4 deletions(-)
Approvals:
Impala Public Jenkins: Verified
Michael Smith: Looks good to me, but someone else must approve
Wenzhe Zhou: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/19207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I82c480890726da0fa5bdc2a646022554eec181f4
Gerrit-Change-Number: 19207
Gerrit-PatchSet: 6
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
[Impala-ASF-CR] IMPALA-11603: Build against Cloudflare ZLIB by default
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19207 )
Change subject: IMPALA-11603: Build against Cloudflare ZLIB by default
......................................................................
Patch Set 5: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/19207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I82c480890726da0fa5bdc2a646022554eec181f4
Gerrit-Change-Number: 19207
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 01 May 2023 17:32:21 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11603: Build against Cloudflare ZLIB by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19207 )
Change subject: IMPALA-11603: Build against Cloudflare ZLIB by default
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12896/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19207
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I82c480890726da0fa5bdc2a646022554eec181f4
Gerrit-Change-Number: 19207
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Sat, 29 Apr 2023 02:35:56 +0000
Gerrit-HasComments: No