You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Joe McDonnell (Code Review)" <ge...@cloudera.org> on 2024/08/24 01:30:32 UTC
[native-toolchain-CR] IMPALA-13308: Build GCC with profile guided optimization
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21721
Change subject: IMPALA-13308: Build GCC with profile guided optimization
......................................................................
IMPALA-13308: Build GCC with profile guided optimization
GCC has an option to build itself using profile guided
optimization. It collects a profile while building itself
and then uses that to optimize the binary. This seems to
be enough to speed up Impala compilation by about 5%:
Current toolchain GCC (LTO):
real 4m58.416s
user 70m41.989s
sys 6m41.961s
GCC with PGO, no LTO:
real 4m45.360s
user 67m12.081s
sys 6m40.376s
LTO shows little improvement over the PGO numbers, so this
switches back to a non-LTO build to save build time. Avoiding
LTO and using a slightly faster GCC reduces the total build
time by 35 minutes on an internal job.
Testing:
- Ran a build, then used it to build Impala
Change-Id: I9f5850999b05207ddc008d839de2783579eaaf31
---
M source/gcc/build.sh
1 file changed, 3 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/21/21721/1
--
To view, visit http://gerrit.cloudera.org:8080/21721
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I9f5850999b05207ddc008d839de2783579eaaf31
Gerrit-Change-Number: 21721
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
[native-toolchain-CR] IMPALA-13308: Build GCC with profile guided optimization
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21721 )
Change subject: IMPALA-13308: Build GCC with profile guided optimization
......................................................................
IMPALA-13308: Build GCC with profile guided optimization
GCC has an option to build itself using profile guided
optimization. It collects a profile while building itself
and then uses that to optimize the binary. This seems to
be enough to speed up Impala compilation by about 5%:
Current toolchain GCC (LTO):
real 4m58.416s
user 70m41.989s
sys 6m41.961s
GCC with PGO, no LTO:
real 4m45.360s
user 67m12.081s
sys 6m40.376s
LTO shows little improvement over the PGO numbers, so this
switches back to a non-LTO build to save build time. Avoiding
LTO and using a slightly faster GCC reduces the total build
time by 35 minutes on an internal job.
Testing:
- Ran a build, then used it to build Impala
Change-Id: I9f5850999b05207ddc008d839de2783579eaaf31
Reviewed-on: http://gerrit.cloudera.org:8080/21721
Reviewed-by: Michael Smith <mi...@cloudera.com>
Tested-by: Joe McDonnell <jo...@cloudera.com>
---
M source/gcc/build.sh
1 file changed, 3 insertions(+), 2 deletions(-)
Approvals:
Michael Smith: Looks good to me, approved
Joe McDonnell: Verified
--
To view, visit http://gerrit.cloudera.org:8080/21721
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I9f5850999b05207ddc008d839de2783579eaaf31
Gerrit-Change-Number: 21721
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
[native-toolchain-CR] IMPALA-13308: Build GCC with profile guided optimization
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/21721 )
Change subject: IMPALA-13308: Build GCC with profile guided optimization
......................................................................
Patch Set 1: Verified+1
Builds succeed on x86_64 and ARM and everything works, so going ahead with this
--
To view, visit http://gerrit.cloudera.org:8080/21721
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9f5850999b05207ddc008d839de2783579eaaf31
Gerrit-Change-Number: 21721
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 27 Aug 2024 16:58:43 +0000
Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-13308: Build GCC with profile guided optimization
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21721 )
Change subject: IMPALA-13308: Build GCC with profile guided optimization
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/21721
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9f5850999b05207ddc008d839de2783579eaaf31
Gerrit-Change-Number: 21721
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 26 Aug 2024 16:33:13 +0000
Gerrit-HasComments: No