[jira] [Commented] (IMPALA-11511) Provide an option to build with compressed debug info

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-11511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17609617#comment-17609617 ] 

ASF subversion and git services commented on IMPALA-11511:
----------------------------------------------------------

Commit 10c19b1a5730a898e17cc653be6bd19f0dc3340e in impala's branch refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=10c19b1a5 ]

IMPALA-11511: Add build options for reducing binary sizes

Impala's build produces dozens of C++ binaries
that link in all Impala libraries. Each binary is
hundreds of megabytes, leading to 10s of gigabytes
of disk space. A large proportion of this (~80%) is debug
information. The debug information increases in newer
versions of GCC such as GCC 10.

This introduces two options for reducing the size
of debug information:
 - IMPALA_MINIMAL_DEBUG_INFO=true builds Impala with
   minimal debug information (-g1). This contains line tables
   and can resolve backtraces, but it does not contain
   variable information and restricts further debugging.
 - IMPALA_COMPRESSED_DEBUG_INFO=true builds Impala with
   compressed debug information (-gz). This does not change
   the debug information included, but the compression saves
   significant disk space. gdb is known to work with
   compressed debug information, but other tools may not
   support it. The dump_breakpad_symbols.py script has been
   adjusted to handle these binaries.
These are disabled by default.

Release impalad binary sizes:
Configuration                  | Size (bytes) | % reduction over base
Base                           | 707834808    | N/A
Stripped                       |  83351664    | 88%
Minimal debuginfo              | 215924096    | 69%
Compressed debuginfo           | 301619286    | 57%
Minimal + compressed debuginfo | 120886705    | 83%

Testing:
 - Generated minidumps and resolved them
 - Verified this is disabled by default

Change-Id: I04a20258a86053d8f3972b9c7c81cd5bec1bbb66
Reviewed-on: http://gerrit.cloudera.org:8080/18962
Reviewed-by: Michael Smith <mi...@cloudera.com>
Reviewed-by: Wenzhe Zhou <wz...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Provide an option to build with compressed debug info
> -----------------------------------------------------
>
>                 Key: IMPALA-11511
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11511
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>    Affects Versions: Impala 4.2.0
>            Reporter: Joe McDonnell
>            Priority: Major
>
> For builds with debug information, the debug information is often a large portion of the binary size. There is a feature that compresses the debug info using ZLIB via the "-gz" compilation flag. It makes a very large difference to the size of our binaries:
> {noformat}
> GCC 7.5:
> debug: 726767520
> debug with -gz: 325970776
> release: 707911496
> with with -gz: 301671026
> GCC 10.4:
> debug: 870378832
> debug with -gz: 351442253
> release: 974600536
> release with -gz: 367938487{noformat}
> The size reduction would be useful for developers, but support in other tools is mixed. gdb has support and seems to work fine. breakpad does not have support. However, it is easy to convert a binary with compressed debug symbols to one with normal debug symbols using objcopy:
> {noformat}
> objcopy --decompress-debug-sections [in_binary] [out_binary]{noformat}
> Given a minidump, it is possible to run objcopy to decompress the debug symbols for the original binary, dump the breakpad symbols, and then process the minidump successfully. So, it should be possible to modify bin/dump_breakpad_symbols.py to do this automatically.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org