You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Cody Maloney <co...@mesosphere.io> on 2014/10/08 00:38:51 UTC
Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/
-----------------------------------------------------------
Review request for mesos, Benjamin Hindman and Timothy St. Clair.
Repository: mesos-git
Description
-------
Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
Main benefits:
1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
Diffs
-----
configure.ac da1c82db31583fc81de658574b9a95628cb84dbc
Diff: https://reviews.apache.org/r/26426/diff/
Testing
-------
Built with both --enable-debug and without, checking that the flags get passed through correctly.
Thanks,
Cody Maloney
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review55742
-----------------------------------------------------------
Patch looks great!
Reviews applied: [26426]
All tests passed.
- Mesos ReviewBot
On Oct. 7, 2014, 10:38 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 7, 2014, 10:38 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac da1c82db31583fc81de658574b9a95628cb84dbc
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by Ben Mahler <be...@gmail.com>.
> On Oct. 14, 2014, 9:06 p.m., Timothy St. Clair wrote:
> > configure.ac, line 281
> > <https://reviews.apache.org/r/26426/diff/1/?file=714874#file714874line281>
> >
> > Is there a reason you want to leave debug symbols out of optimized builds?
> >
> > cmake has the pattern correct imho:
> > Release
> > Debug
> > ReleaseWithDebug
> >
> > A ReleaseWithDebug allows packagers, such as myself, to build w/debugsymbols that are stripped out into a .debuginfo package which can be used by developers for tracing "When bears attack". Granted that it is tenuous debugging at best, but it's better then nothing.
> >
> > So I think we want all three modes, stripping all debug information is not really idea.
>
> Cody Maloney wrote:
> My main motivation is to shrink the size of libmesos. Yesterday sugis in #mesos had one which was 213M. For the buildbot internally, full debian packages (which are compressed) of mesos weign in at 165M a piece (Yes, stripping post-build would help a lot, but why build it to begin with?). Most of this is debug info. Also, we build a bunch of different ways, and when libmesos is as big as it is, a decent amount of time ends up being spent on disk I/O reading / writing all the debug info when we are really just trying to ensure it builds on all the different platforms (Not to mention storage and file size shipping things around the network to centralized repositories).
>
> The simple toggle between debug and release, removing the legacy logic gets us most of the benefit.
>
> If you have a good place to point me for what is needed to get a 'ReleaseWithDebug' info build up and running I can definitely work on adding that as well.
>
> Cody Maloney wrote:
> It is also entirely possible to specify custom CXXFLAGS + CFLAGS with this patch to get the old "optimized debug" build. The flags set by --enable-debug (or not having it), to get back the "optimized debug" build from before.
>
> Timothy St. Clair wrote:
> I suppose rpm builds default CXXFLAGS to '-O2 -g ...', so I can buy the argument of just making it easier on the average person.
Re-opening because conflating debug and optimization under a single flag seems a bit confusing to me.
In practice, we run mesos **with optimizations** (for free performance win), and **with debugging symbols** (for meaningful backtraces and core dumps), which was previously the default but now requires the setting of CXXFLAGS to obtain. :(
For development, I think by default we need debugging symbols turned on. Otherwise most people will forget to configure with `--enable-debug` and consequently will need to recompile everything when they encounter a backtrace or need to debug. CI jobs will encounter this issue as well.
If there are use cases that merit no debugging symbols, seems nice to make that explicit given it runs the risk of making debugging difficult.
Thoughts?
- Ben
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56573
-----------------------------------------------------------
On Oct. 14, 2014, 11:07 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2014, 11:07 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by Ben Mahler <be...@gmail.com>.
> On Oct. 14, 2014, 9:06 p.m., Timothy St. Clair wrote:
> > configure.ac, line 281
> > <https://reviews.apache.org/r/26426/diff/1/?file=714874#file714874line281>
> >
> > Is there a reason you want to leave debug symbols out of optimized builds?
> >
> > cmake has the pattern correct imho:
> > Release
> > Debug
> > ReleaseWithDebug
> >
> > A ReleaseWithDebug allows packagers, such as myself, to build w/debugsymbols that are stripped out into a .debuginfo package which can be used by developers for tracing "When bears attack". Granted that it is tenuous debugging at best, but it's better then nothing.
> >
> > So I think we want all three modes, stripping all debug information is not really idea.
>
> Cody Maloney wrote:
> My main motivation is to shrink the size of libmesos. Yesterday sugis in #mesos had one which was 213M. For the buildbot internally, full debian packages (which are compressed) of mesos weign in at 165M a piece (Yes, stripping post-build would help a lot, but why build it to begin with?). Most of this is debug info. Also, we build a bunch of different ways, and when libmesos is as big as it is, a decent amount of time ends up being spent on disk I/O reading / writing all the debug info when we are really just trying to ensure it builds on all the different platforms (Not to mention storage and file size shipping things around the network to centralized repositories).
>
> The simple toggle between debug and release, removing the legacy logic gets us most of the benefit.
>
> If you have a good place to point me for what is needed to get a 'ReleaseWithDebug' info build up and running I can definitely work on adding that as well.
>
> Cody Maloney wrote:
> It is also entirely possible to specify custom CXXFLAGS + CFLAGS with this patch to get the old "optimized debug" build. The flags set by --enable-debug (or not having it), to get back the "optimized debug" build from before.
>
> Timothy St. Clair wrote:
> I suppose rpm builds default CXXFLAGS to '-O2 -g ...', so I can buy the argument of just making it easier on the average person.
>
> Ben Mahler wrote:
> Re-opening because conflating debug and optimization under a single flag seems a bit confusing to me.
>
> In practice, we run mesos **with optimizations** (for free performance win), and **with debugging symbols** (for meaningful backtraces and core dumps), which was previously the default but now requires the setting of CXXFLAGS to obtain. :(
>
> For development, I think by default we need debugging symbols turned on. Otherwise most people will forget to configure with `--enable-debug` and consequently will need to recompile everything when they encounter a backtrace or need to debug. CI jobs will encounter this issue as well.
>
> If there are use cases that merit no debugging symbols, seems nice to make that explicit given it runs the risk of making debugging difficult.
>
> Thoughts?
>
> Cody Maloney wrote:
> Note that the two were conflated under one flag previously. This just changes how we do it to be a little more standard way of conflating them. Generally people talk about `Debug` or `Relase` builds. Not the specific compiler flags that get switched. The general switch is -O2 -> -O0 + -g and back (Possibly with a couple extra warning flags). Autotools projects which implement a debug vs. release build almost always do '--enable-debug' / the default is 'optimized' build.
>
> https://gcc.gnu.org/wiki/DebugFission - that artical has some good explanations why debug info gets problematic as programs get larger and larger, which is only going to get worse over time. Already mesos debug info is over 100MB. As the codebase grows, this will get worse than it already is, increasing the minimum machine specs you need to develop / compile Mesos.
>
> Compiler implementors generally give that '-O2' should be used to generate an optimal binary. '-g' should be used when you are planning to attach GDB and step through a program line by line. General development generally people want nice backtraces, which is inbetween those. See **Backtraces**. The section **Coredumps** talks about how we can give all the info in coredumps without running full debug builds ('-g') everywhere.
>
> **Backtraces** should generally be reasonably meaningful as long as you don't strip the binary. Most of the symbols are still there (You just lose inlined things). You don't get file names and line numbers, but all the functions in mesos should be uniquely named (C++'s ODR rule), so you get out a mangled C++ name which you can demangle with `c++filt` (Shipped with GCC) and you will get the full namespace prefixed name of the C++ function. Find the function with your favorite text editor search, life is happy.
>
> Pretty much every binary which is shipped on your machine via a distribution is sent like this. The backtraces + coredumps that are gathered come from this.
>
> [Clang has a option -gline-tables-only](http://clang.llvm.org/docs/UsersManual.html#cmdoption-gline-tables-only) which adds just the file name and line numbers to all functions in the debug info (Which can be done with minimal space overhead. [Gcc has '-g1'](https://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html#Debugging-Options), which would mean that you would get line number + file names in back traces with less overhead.
>
> Those options would make it so what people see printed by the build bot is exactly what they see now. I think that should be an easily accessible level, but not default build.
>
>
> **Coredumps**: It is possible to make coredumps just as inspectable without including debug information in all the shipping binaries. In fact, you generally probably don't want to run gdb on the binary on the production host anyways. For this, what typically happens is that
>
> 1. People actually only look at the backtrace to figure out what went wrong
> 2. You can generate debug information which lives in seperate files from the target executable (-gsplit-dwarf on new versions of GCC, requires a relatively new toolchain. I believe there is a way to do it on older tooling where you split it out later, but am not familiar with it). This debug info would generally live in a '-dbg' package. Which you can install / run on your local machine to inspect whatever variables remain directly, as well as global state.
>
>
> **OVERALL**
> I don't see anywhere where this makes debugging considerably more difficult than it is already. Yes, you can't attach a GDB to any build of mesos without thinking about it. I think most of the time most developers aren't attaching GDB.
>
> CI jobs I think can be easily switched over where they care. There aren't that many of them around. I agree it is something people need to be aware of, but not a hard level of change. Most CI systems don't let people connect in and step through the program with GDB to where it failed a test. It just gives a backtrace. In the case of a buildbot it isn't hard to set CXXFLAGS to just the level you want to get that ('-g1'). No major loss there with this change.
>
> The only case this makes considerably more difficult is that if you want to attach GDB to inspect variabels you need to have specified --enable-debug at configure time. I don't think that is a major loss, or something which is in the standard compile/test loop of most most Mesos contributors / developers (Although we can collect evidence if desired)
Thanks for being thorough! Having `-g1` and `-gline-tables-only` sound good to me for release builds, any reason not to include those?
As for local development, I would _personally_ be fine with optimization turned off and `-g1` and `-gline-tables-only`, if the smaller debug information has a measurable improvement on compile speed or memory consumption (let's measure!). I tend to only need backtrace, or at most gdb thread backtraces from a dump.
These two use-cases seem to suggest leaving the existing `--disable-optimize` and merely tweaking the debug flags to avoid dumping all debug information into release builds.
How about for this change, we tackle the following:
Default is Release: -O2 -g1 (or -gline-tables-only)
--disable-optimize: -O0 -g1 (or -gline-tables-only)
In a subsequent patch, we can add an orthogonal debug flag for those who want to the full set of debug information:
--enable-debug: turns on -g
Thoughts? It seems we can acheive Debug / Release builds even if we don't map the distinction to a single binary configure flag.
- Ben
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56573
-----------------------------------------------------------
On Oct. 14, 2014, 11:07 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2014, 11:07 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by "Timothy St. Clair" <ts...@redhat.com>.
> On Oct. 14, 2014, 9:06 p.m., Timothy St. Clair wrote:
> > configure.ac, line 281
> > <https://reviews.apache.org/r/26426/diff/1/?file=714874#file714874line281>
> >
> > Is there a reason you want to leave debug symbols out of optimized builds?
> >
> > cmake has the pattern correct imho:
> > Release
> > Debug
> > ReleaseWithDebug
> >
> > A ReleaseWithDebug allows packagers, such as myself, to build w/debugsymbols that are stripped out into a .debuginfo package which can be used by developers for tracing "When bears attack". Granted that it is tenuous debugging at best, but it's better then nothing.
> >
> > So I think we want all three modes, stripping all debug information is not really idea.
>
> Cody Maloney wrote:
> My main motivation is to shrink the size of libmesos. Yesterday sugis in #mesos had one which was 213M. For the buildbot internally, full debian packages (which are compressed) of mesos weign in at 165M a piece (Yes, stripping post-build would help a lot, but why build it to begin with?). Most of this is debug info. Also, we build a bunch of different ways, and when libmesos is as big as it is, a decent amount of time ends up being spent on disk I/O reading / writing all the debug info when we are really just trying to ensure it builds on all the different platforms (Not to mention storage and file size shipping things around the network to centralized repositories).
>
> The simple toggle between debug and release, removing the legacy logic gets us most of the benefit.
>
> If you have a good place to point me for what is needed to get a 'ReleaseWithDebug' info build up and running I can definitely work on adding that as well.
>
> Cody Maloney wrote:
> It is also entirely possible to specify custom CXXFLAGS + CFLAGS with this patch to get the old "optimized debug" build. The flags set by --enable-debug (or not having it), to get back the "optimized debug" build from before.
I suppose rpm builds default CXXFLAGS to '-O2 -g ...', so I can buy the argument of just making it easier on the average person.
- Timothy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56573
-----------------------------------------------------------
On Oct. 14, 2014, 11:07 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2014, 11:07 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by "Timothy St. Clair" <ts...@redhat.com>.
> On Oct. 14, 2014, 9:06 p.m., Timothy St. Clair wrote:
> > configure.ac, line 281
> > <https://reviews.apache.org/r/26426/diff/1/?file=714874#file714874line281>
> >
> > Is there a reason you want to leave debug symbols out of optimized builds?
> >
> > cmake has the pattern correct imho:
> > Release
> > Debug
> > ReleaseWithDebug
> >
> > A ReleaseWithDebug allows packagers, such as myself, to build w/debugsymbols that are stripped out into a .debuginfo package which can be used by developers for tracing "When bears attack". Granted that it is tenuous debugging at best, but it's better then nothing.
> >
> > So I think we want all three modes, stripping all debug information is not really idea.
>
> Cody Maloney wrote:
> My main motivation is to shrink the size of libmesos. Yesterday sugis in #mesos had one which was 213M. For the buildbot internally, full debian packages (which are compressed) of mesos weign in at 165M a piece (Yes, stripping post-build would help a lot, but why build it to begin with?). Most of this is debug info. Also, we build a bunch of different ways, and when libmesos is as big as it is, a decent amount of time ends up being spent on disk I/O reading / writing all the debug info when we are really just trying to ensure it builds on all the different platforms (Not to mention storage and file size shipping things around the network to centralized repositories).
>
> The simple toggle between debug and release, removing the legacy logic gets us most of the benefit.
>
> If you have a good place to point me for what is needed to get a 'ReleaseWithDebug' info build up and running I can definitely work on adding that as well.
>
> Cody Maloney wrote:
> It is also entirely possible to specify custom CXXFLAGS + CFLAGS with this patch to get the old "optimized debug" build. The flags set by --enable-debug (or not having it), to get back the "optimized debug" build from before.
>
> Timothy St. Clair wrote:
> I suppose rpm builds default CXXFLAGS to '-O2 -g ...', so I can buy the argument of just making it easier on the average person.
>
> Ben Mahler wrote:
> Re-opening because conflating debug and optimization under a single flag seems a bit confusing to me.
>
> In practice, we run mesos **with optimizations** (for free performance win), and **with debugging symbols** (for meaningful backtraces and core dumps), which was previously the default but now requires the setting of CXXFLAGS to obtain. :(
>
> For development, I think by default we need debugging symbols turned on. Otherwise most people will forget to configure with `--enable-debug` and consequently will need to recompile everything when they encounter a backtrace or need to debug. CI jobs will encounter this issue as well.
>
> If there are use cases that merit no debugging symbols, seems nice to make that explicit given it runs the risk of making debugging difficult.
>
> Thoughts?
>
> Cody Maloney wrote:
> Note that the two were conflated under one flag previously. This just changes how we do it to be a little more standard way of conflating them. Generally people talk about `Debug` or `Relase` builds. Not the specific compiler flags that get switched. The general switch is -O2 -> -O0 + -g and back (Possibly with a couple extra warning flags). Autotools projects which implement a debug vs. release build almost always do '--enable-debug' / the default is 'optimized' build.
>
> https://gcc.gnu.org/wiki/DebugFission - that artical has some good explanations why debug info gets problematic as programs get larger and larger, which is only going to get worse over time. Already mesos debug info is over 100MB. As the codebase grows, this will get worse than it already is, increasing the minimum machine specs you need to develop / compile Mesos.
>
> Compiler implementors generally give that '-O2' should be used to generate an optimal binary. '-g' should be used when you are planning to attach GDB and step through a program line by line. General development generally people want nice backtraces, which is inbetween those. See **Backtraces**. The section **Coredumps** talks about how we can give all the info in coredumps without running full debug builds ('-g') everywhere.
>
> **Backtraces** should generally be reasonably meaningful as long as you don't strip the binary. Most of the symbols are still there (You just lose inlined things). You don't get file names and line numbers, but all the functions in mesos should be uniquely named (C++'s ODR rule), so you get out a mangled C++ name which you can demangle with `c++filt` (Shipped with GCC) and you will get the full namespace prefixed name of the C++ function. Find the function with your favorite text editor search, life is happy.
>
> Pretty much every binary which is shipped on your machine via a distribution is sent like this. The backtraces + coredumps that are gathered come from this.
>
> [Clang has a option -gline-tables-only](http://clang.llvm.org/docs/UsersManual.html#cmdoption-gline-tables-only) which adds just the file name and line numbers to all functions in the debug info (Which can be done with minimal space overhead. [Gcc has '-g1'](https://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html#Debugging-Options), which would mean that you would get line number + file names in back traces with less overhead.
>
> Those options would make it so what people see printed by the build bot is exactly what they see now. I think that should be an easily accessible level, but not default build.
>
>
> **Coredumps**: It is possible to make coredumps just as inspectable without including debug information in all the shipping binaries. In fact, you generally probably don't want to run gdb on the binary on the production host anyways. For this, what typically happens is that
>
> 1. People actually only look at the backtrace to figure out what went wrong
> 2. You can generate debug information which lives in seperate files from the target executable (-gsplit-dwarf on new versions of GCC, requires a relatively new toolchain. I believe there is a way to do it on older tooling where you split it out later, but am not familiar with it). This debug info would generally live in a '-dbg' package. Which you can install / run on your local machine to inspect whatever variables remain directly, as well as global state.
>
>
> **OVERALL**
> I don't see anywhere where this makes debugging considerably more difficult than it is already. Yes, you can't attach a GDB to any build of mesos without thinking about it. I think most of the time most developers aren't attaching GDB.
>
> CI jobs I think can be easily switched over where they care. There aren't that many of them around. I agree it is something people need to be aware of, but not a hard level of change. Most CI systems don't let people connect in and step through the program with GDB to where it failed a test. It just gives a backtrace. In the case of a buildbot it isn't hard to set CXXFLAGS to just the level you want to get that ('-g1'). No major loss there with this change.
>
> The only case this makes considerably more difficult is that if you want to attach GDB to inspect variabels you need to have specified --enable-debug at configure time. I don't think that is a major loss, or something which is in the standard compile/test loop of most most Mesos contributors / developers (Although we can collect evidence if desired)
>
> Ben Mahler wrote:
> Thanks for being thorough! Having `-g1` and `-gline-tables-only` sound good to me for release builds, any reason not to include those?
>
> As for local development, I would _personally_ be fine with optimization turned off and `-g1` and `-gline-tables-only`, if the smaller debug information has a measurable improvement on compile speed or memory consumption (let's measure!). I tend to only need backtrace, or at most gdb thread backtraces from a dump.
>
> These two use-cases seem to suggest leaving the existing `--disable-optimize` and merely tweaking the debug flags to avoid dumping all debug information into release builds.
> How about for this change, we tackle the following:
>
> Default is Release: -O2 -g1 (or -gline-tables-only)
> --disable-optimize: -O0 -g1 (or -gline-tables-only)
>
> In a subsequent patch, we can add an orthogonal debug flag for those who want to the full set of debug information:
>
> --enable-debug: turns on -g
>
> Thoughts? It seems we can acheive Debug / Release builds even if we don't map the distinction to a single binary configure flag.
>
> Cody Maloney wrote:
> I'm going to run some tests:
> CXXFLAGS=-O2
> CXXFLAGS=-O0 -g1
> CXXFLAGS=-O0 -g
> CXXFLAGS=-O2 -g
>
> I'm going to measure compile time resulting size, of the build directory, and size of the install directory. Will provide some feedback after that.
>
> Note '-g' and '-g1' probably override eachother (they are actually the same flag), So if we want to do that, we need to make it swap that flag which will take a little bit more complex logic. The simplest/cleanest thing to do if people want a debug is to say 'Specify your own CFLAGS'
>
> Cody Maloney wrote:
> Ran the tests, results posted to mesos dev mailing list: http://www.mail-archive.com/dev@mesos.apache.org/msg22097.html
>
> I think we should use just -O0 for default dev build, don't touch CXXFLAGS at all if they are passed (which means we get the right behavior in packaging systems for various distros).
>
> I can add a --optimize flag to set CXXFLAGS to -O2.
> I can add a --debug flag to set CXXFLAGS to -g2.
>
> Those are useful if you are doing a full build and then afterwards turning it into a distro package / tarball which you then ship around (If you use a distro pacakging system, those specify the packaging guideliene flags to configure via environment variables)
>
> I think people are better off specifying those manually then having flags which wrap them in. Those flags aren't independent toggles how autotools projects are setup, so doing it right is really really hard / there will be several edge cases just not properly handled when people hand-specify some CXXFLAGS.
>
> In the longer run a different build system (CMake?) would be the cleaner if we want explicit specification.
I like it, but again I think we should check with BenM 1st.
+1 to cmake.
- Timothy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56573
-----------------------------------------------------------
On Oct. 14, 2014, 11:07 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2014, 11:07 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by Cody Maloney <co...@mesosphere.io>.
> On Oct. 14, 2014, 9:06 p.m., Timothy St. Clair wrote:
> > configure.ac, line 281
> > <https://reviews.apache.org/r/26426/diff/1/?file=714874#file714874line281>
> >
> > Is there a reason you want to leave debug symbols out of optimized builds?
> >
> > cmake has the pattern correct imho:
> > Release
> > Debug
> > ReleaseWithDebug
> >
> > A ReleaseWithDebug allows packagers, such as myself, to build w/debugsymbols that are stripped out into a .debuginfo package which can be used by developers for tracing "When bears attack". Granted that it is tenuous debugging at best, but it's better then nothing.
> >
> > So I think we want all three modes, stripping all debug information is not really idea.
>
> Cody Maloney wrote:
> My main motivation is to shrink the size of libmesos. Yesterday sugis in #mesos had one which was 213M. For the buildbot internally, full debian packages (which are compressed) of mesos weign in at 165M a piece (Yes, stripping post-build would help a lot, but why build it to begin with?). Most of this is debug info. Also, we build a bunch of different ways, and when libmesos is as big as it is, a decent amount of time ends up being spent on disk I/O reading / writing all the debug info when we are really just trying to ensure it builds on all the different platforms (Not to mention storage and file size shipping things around the network to centralized repositories).
>
> The simple toggle between debug and release, removing the legacy logic gets us most of the benefit.
>
> If you have a good place to point me for what is needed to get a 'ReleaseWithDebug' info build up and running I can definitely work on adding that as well.
>
> Cody Maloney wrote:
> It is also entirely possible to specify custom CXXFLAGS + CFLAGS with this patch to get the old "optimized debug" build. The flags set by --enable-debug (or not having it), to get back the "optimized debug" build from before.
>
> Timothy St. Clair wrote:
> I suppose rpm builds default CXXFLAGS to '-O2 -g ...', so I can buy the argument of just making it easier on the average person.
>
> Ben Mahler wrote:
> Re-opening because conflating debug and optimization under a single flag seems a bit confusing to me.
>
> In practice, we run mesos **with optimizations** (for free performance win), and **with debugging symbols** (for meaningful backtraces and core dumps), which was previously the default but now requires the setting of CXXFLAGS to obtain. :(
>
> For development, I think by default we need debugging symbols turned on. Otherwise most people will forget to configure with `--enable-debug` and consequently will need to recompile everything when they encounter a backtrace or need to debug. CI jobs will encounter this issue as well.
>
> If there are use cases that merit no debugging symbols, seems nice to make that explicit given it runs the risk of making debugging difficult.
>
> Thoughts?
Note that the two were conflated under one flag previously. This just changes how we do it to be a little more standard way of conflating them. Generally people talk about `Debug` or `Relase` builds. Not the specific compiler flags that get switched. The general switch is -O2 -> -O0 + -g and back (Possibly with a couple extra warning flags). Autotools projects which implement a debug vs. release build almost always do '--enable-debug' / the default is 'optimized' build.
https://gcc.gnu.org/wiki/DebugFission - that artical has some good explanations why debug info gets problematic as programs get larger and larger, which is only going to get worse over time. Already mesos debug info is over 100MB. As the codebase grows, this will get worse than it already is, increasing the minimum machine specs you need to develop / compile Mesos.
Compiler implementors generally give that '-O2' should be used to generate an optimal binary. '-g' should be used when you are planning to attach GDB and step through a program line by line. General development generally people want nice backtraces, which is inbetween those. See **Backtraces**. The section **Coredumps** talks about how we can give all the info in coredumps without running full debug builds ('-g') everywhere.
**Backtraces** should generally be reasonably meaningful as long as you don't strip the binary. Most of the symbols are still there (You just lose inlined things). You don't get file names and line numbers, but all the functions in mesos should be uniquely named (C++'s ODR rule), so you get out a mangled C++ name which you can demangle with `c++filt` (Shipped with GCC) and you will get the full namespace prefixed name of the C++ function. Find the function with your favorite text editor search, life is happy.
Pretty much every binary which is shipped on your machine via a distribution is sent like this. The backtraces + coredumps that are gathered come from this.
[Clang has a option -gline-tables-only](http://clang.llvm.org/docs/UsersManual.html#cmdoption-gline-tables-only) which adds just the file name and line numbers to all functions in the debug info (Which can be done with minimal space overhead. [Gcc has '-g1'](https://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html#Debugging-Options), which would mean that you would get line number + file names in back traces with less overhead.
Those options would make it so what people see printed by the build bot is exactly what they see now. I think that should be an easily accessible level, but not default build.
**Coredumps**: It is possible to make coredumps just as inspectable without including debug information in all the shipping binaries. In fact, you generally probably don't want to run gdb on the binary on the production host anyways. For this, what typically happens is that
1. People actually only look at the backtrace to figure out what went wrong
2. You can generate debug information which lives in seperate files from the target executable (-gsplit-dwarf on new versions of GCC, requires a relatively new toolchain. I believe there is a way to do it on older tooling where you split it out later, but am not familiar with it). This debug info would generally live in a '-dbg' package. Which you can install / run on your local machine to inspect whatever variables remain directly, as well as global state.
**OVERALL**
I don't see anywhere where this makes debugging considerably more difficult than it is already. Yes, you can't attach a GDB to any build of mesos without thinking about it. I think most of the time most developers aren't attaching GDB.
CI jobs I think can be easily switched over where they care. There aren't that many of them around. I agree it is something people need to be aware of, but not a hard level of change. Most CI systems don't let people connect in and step through the program with GDB to where it failed a test. It just gives a backtrace. In the case of a buildbot it isn't hard to set CXXFLAGS to just the level you want to get that ('-g1'). No major loss there with this change.
The only case this makes considerably more difficult is that if you want to attach GDB to inspect variabels you need to have specified --enable-debug at configure time. I don't think that is a major loss, or something which is in the standard compile/test loop of most most Mesos contributors / developers (Although we can collect evidence if desired)
- Cody
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56573
-----------------------------------------------------------
On Oct. 14, 2014, 11:07 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2014, 11:07 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by Cody Maloney <co...@mesosphere.io>.
> On Oct. 14, 2014, 9:06 p.m., Timothy St. Clair wrote:
> > configure.ac, line 281
> > <https://reviews.apache.org/r/26426/diff/1/?file=714874#file714874line281>
> >
> > Is there a reason you want to leave debug symbols out of optimized builds?
> >
> > cmake has the pattern correct imho:
> > Release
> > Debug
> > ReleaseWithDebug
> >
> > A ReleaseWithDebug allows packagers, such as myself, to build w/debugsymbols that are stripped out into a .debuginfo package which can be used by developers for tracing "When bears attack". Granted that it is tenuous debugging at best, but it's better then nothing.
> >
> > So I think we want all three modes, stripping all debug information is not really idea.
>
> Cody Maloney wrote:
> My main motivation is to shrink the size of libmesos. Yesterday sugis in #mesos had one which was 213M. For the buildbot internally, full debian packages (which are compressed) of mesos weign in at 165M a piece (Yes, stripping post-build would help a lot, but why build it to begin with?). Most of this is debug info. Also, we build a bunch of different ways, and when libmesos is as big as it is, a decent amount of time ends up being spent on disk I/O reading / writing all the debug info when we are really just trying to ensure it builds on all the different platforms (Not to mention storage and file size shipping things around the network to centralized repositories).
>
> The simple toggle between debug and release, removing the legacy logic gets us most of the benefit.
>
> If you have a good place to point me for what is needed to get a 'ReleaseWithDebug' info build up and running I can definitely work on adding that as well.
>
> Cody Maloney wrote:
> It is also entirely possible to specify custom CXXFLAGS + CFLAGS with this patch to get the old "optimized debug" build. The flags set by --enable-debug (or not having it), to get back the "optimized debug" build from before.
>
> Timothy St. Clair wrote:
> I suppose rpm builds default CXXFLAGS to '-O2 -g ...', so I can buy the argument of just making it easier on the average person.
>
> Ben Mahler wrote:
> Re-opening because conflating debug and optimization under a single flag seems a bit confusing to me.
>
> In practice, we run mesos **with optimizations** (for free performance win), and **with debugging symbols** (for meaningful backtraces and core dumps), which was previously the default but now requires the setting of CXXFLAGS to obtain. :(
>
> For development, I think by default we need debugging symbols turned on. Otherwise most people will forget to configure with `--enable-debug` and consequently will need to recompile everything when they encounter a backtrace or need to debug. CI jobs will encounter this issue as well.
>
> If there are use cases that merit no debugging symbols, seems nice to make that explicit given it runs the risk of making debugging difficult.
>
> Thoughts?
>
> Cody Maloney wrote:
> Note that the two were conflated under one flag previously. This just changes how we do it to be a little more standard way of conflating them. Generally people talk about `Debug` or `Relase` builds. Not the specific compiler flags that get switched. The general switch is -O2 -> -O0 + -g and back (Possibly with a couple extra warning flags). Autotools projects which implement a debug vs. release build almost always do '--enable-debug' / the default is 'optimized' build.
>
> https://gcc.gnu.org/wiki/DebugFission - that artical has some good explanations why debug info gets problematic as programs get larger and larger, which is only going to get worse over time. Already mesos debug info is over 100MB. As the codebase grows, this will get worse than it already is, increasing the minimum machine specs you need to develop / compile Mesos.
>
> Compiler implementors generally give that '-O2' should be used to generate an optimal binary. '-g' should be used when you are planning to attach GDB and step through a program line by line. General development generally people want nice backtraces, which is inbetween those. See **Backtraces**. The section **Coredumps** talks about how we can give all the info in coredumps without running full debug builds ('-g') everywhere.
>
> **Backtraces** should generally be reasonably meaningful as long as you don't strip the binary. Most of the symbols are still there (You just lose inlined things). You don't get file names and line numbers, but all the functions in mesos should be uniquely named (C++'s ODR rule), so you get out a mangled C++ name which you can demangle with `c++filt` (Shipped with GCC) and you will get the full namespace prefixed name of the C++ function. Find the function with your favorite text editor search, life is happy.
>
> Pretty much every binary which is shipped on your machine via a distribution is sent like this. The backtraces + coredumps that are gathered come from this.
>
> [Clang has a option -gline-tables-only](http://clang.llvm.org/docs/UsersManual.html#cmdoption-gline-tables-only) which adds just the file name and line numbers to all functions in the debug info (Which can be done with minimal space overhead. [Gcc has '-g1'](https://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html#Debugging-Options), which would mean that you would get line number + file names in back traces with less overhead.
>
> Those options would make it so what people see printed by the build bot is exactly what they see now. I think that should be an easily accessible level, but not default build.
>
>
> **Coredumps**: It is possible to make coredumps just as inspectable without including debug information in all the shipping binaries. In fact, you generally probably don't want to run gdb on the binary on the production host anyways. For this, what typically happens is that
>
> 1. People actually only look at the backtrace to figure out what went wrong
> 2. You can generate debug information which lives in seperate files from the target executable (-gsplit-dwarf on new versions of GCC, requires a relatively new toolchain. I believe there is a way to do it on older tooling where you split it out later, but am not familiar with it). This debug info would generally live in a '-dbg' package. Which you can install / run on your local machine to inspect whatever variables remain directly, as well as global state.
>
>
> **OVERALL**
> I don't see anywhere where this makes debugging considerably more difficult than it is already. Yes, you can't attach a GDB to any build of mesos without thinking about it. I think most of the time most developers aren't attaching GDB.
>
> CI jobs I think can be easily switched over where they care. There aren't that many of them around. I agree it is something people need to be aware of, but not a hard level of change. Most CI systems don't let people connect in and step through the program with GDB to where it failed a test. It just gives a backtrace. In the case of a buildbot it isn't hard to set CXXFLAGS to just the level you want to get that ('-g1'). No major loss there with this change.
>
> The only case this makes considerably more difficult is that if you want to attach GDB to inspect variabels you need to have specified --enable-debug at configure time. I don't think that is a major loss, or something which is in the standard compile/test loop of most most Mesos contributors / developers (Although we can collect evidence if desired)
>
> Ben Mahler wrote:
> Thanks for being thorough! Having `-g1` and `-gline-tables-only` sound good to me for release builds, any reason not to include those?
>
> As for local development, I would _personally_ be fine with optimization turned off and `-g1` and `-gline-tables-only`, if the smaller debug information has a measurable improvement on compile speed or memory consumption (let's measure!). I tend to only need backtrace, or at most gdb thread backtraces from a dump.
>
> These two use-cases seem to suggest leaving the existing `--disable-optimize` and merely tweaking the debug flags to avoid dumping all debug information into release builds.
> How about for this change, we tackle the following:
>
> Default is Release: -O2 -g1 (or -gline-tables-only)
> --disable-optimize: -O0 -g1 (or -gline-tables-only)
>
> In a subsequent patch, we can add an orthogonal debug flag for those who want to the full set of debug information:
>
> --enable-debug: turns on -g
>
> Thoughts? It seems we can acheive Debug / Release builds even if we don't map the distinction to a single binary configure flag.
>
> Cody Maloney wrote:
> I'm going to run some tests:
> CXXFLAGS=-O2
> CXXFLAGS=-O0 -g1
> CXXFLAGS=-O0 -g
> CXXFLAGS=-O2 -g
>
> I'm going to measure compile time resulting size, of the build directory, and size of the install directory. Will provide some feedback after that.
>
> Note '-g' and '-g1' probably override eachother (they are actually the same flag), So if we want to do that, we need to make it swap that flag which will take a little bit more complex logic. The simplest/cleanest thing to do if people want a debug is to say 'Specify your own CFLAGS'
Ran the tests, results posted to mesos dev mailing list: http://www.mail-archive.com/dev@mesos.apache.org/msg22097.html
I think we should use just -O0 for default dev build, don't touch CXXFLAGS at all if they are passed (which means we get the right behavior in packaging systems for various distros).
I can add a --optimize flag to set CXXFLAGS to -O2.
I can add a --debug flag to set CXXFLAGS to -g2.
Those are useful if you are doing a full build and then afterwards turning it into a distro package / tarball which you then ship around (If you use a distro pacakging system, those specify the packaging guideliene flags to configure via environment variables)
I think people are better off specifying those manually then having flags which wrap them in. Those flags aren't independent toggles how autotools projects are setup, so doing it right is really really hard / there will be several edge cases just not properly handled when people hand-specify some CXXFLAGS.
In the longer run a different build system (CMake?) would be the cleaner if we want explicit specification.
- Cody
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56573
-----------------------------------------------------------
On Oct. 14, 2014, 11:07 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2014, 11:07 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by Cody Maloney <co...@mesosphere.io>.
> On Oct. 14, 2014, 9:06 p.m., Timothy St. Clair wrote:
> > configure.ac, line 281
> > <https://reviews.apache.org/r/26426/diff/1/?file=714874#file714874line281>
> >
> > Is there a reason you want to leave debug symbols out of optimized builds?
> >
> > cmake has the pattern correct imho:
> > Release
> > Debug
> > ReleaseWithDebug
> >
> > A ReleaseWithDebug allows packagers, such as myself, to build w/debugsymbols that are stripped out into a .debuginfo package which can be used by developers for tracing "When bears attack". Granted that it is tenuous debugging at best, but it's better then nothing.
> >
> > So I think we want all three modes, stripping all debug information is not really idea.
>
> Cody Maloney wrote:
> My main motivation is to shrink the size of libmesos. Yesterday sugis in #mesos had one which was 213M. For the buildbot internally, full debian packages (which are compressed) of mesos weign in at 165M a piece (Yes, stripping post-build would help a lot, but why build it to begin with?). Most of this is debug info. Also, we build a bunch of different ways, and when libmesos is as big as it is, a decent amount of time ends up being spent on disk I/O reading / writing all the debug info when we are really just trying to ensure it builds on all the different platforms (Not to mention storage and file size shipping things around the network to centralized repositories).
>
> The simple toggle between debug and release, removing the legacy logic gets us most of the benefit.
>
> If you have a good place to point me for what is needed to get a 'ReleaseWithDebug' info build up and running I can definitely work on adding that as well.
>
> Cody Maloney wrote:
> It is also entirely possible to specify custom CXXFLAGS + CFLAGS with this patch to get the old "optimized debug" build. The flags set by --enable-debug (or not having it), to get back the "optimized debug" build from before.
>
> Timothy St. Clair wrote:
> I suppose rpm builds default CXXFLAGS to '-O2 -g ...', so I can buy the argument of just making it easier on the average person.
>
> Ben Mahler wrote:
> Re-opening because conflating debug and optimization under a single flag seems a bit confusing to me.
>
> In practice, we run mesos **with optimizations** (for free performance win), and **with debugging symbols** (for meaningful backtraces and core dumps), which was previously the default but now requires the setting of CXXFLAGS to obtain. :(
>
> For development, I think by default we need debugging symbols turned on. Otherwise most people will forget to configure with `--enable-debug` and consequently will need to recompile everything when they encounter a backtrace or need to debug. CI jobs will encounter this issue as well.
>
> If there are use cases that merit no debugging symbols, seems nice to make that explicit given it runs the risk of making debugging difficult.
>
> Thoughts?
>
> Cody Maloney wrote:
> Note that the two were conflated under one flag previously. This just changes how we do it to be a little more standard way of conflating them. Generally people talk about `Debug` or `Relase` builds. Not the specific compiler flags that get switched. The general switch is -O2 -> -O0 + -g and back (Possibly with a couple extra warning flags). Autotools projects which implement a debug vs. release build almost always do '--enable-debug' / the default is 'optimized' build.
>
> https://gcc.gnu.org/wiki/DebugFission - that artical has some good explanations why debug info gets problematic as programs get larger and larger, which is only going to get worse over time. Already mesos debug info is over 100MB. As the codebase grows, this will get worse than it already is, increasing the minimum machine specs you need to develop / compile Mesos.
>
> Compiler implementors generally give that '-O2' should be used to generate an optimal binary. '-g' should be used when you are planning to attach GDB and step through a program line by line. General development generally people want nice backtraces, which is inbetween those. See **Backtraces**. The section **Coredumps** talks about how we can give all the info in coredumps without running full debug builds ('-g') everywhere.
>
> **Backtraces** should generally be reasonably meaningful as long as you don't strip the binary. Most of the symbols are still there (You just lose inlined things). You don't get file names and line numbers, but all the functions in mesos should be uniquely named (C++'s ODR rule), so you get out a mangled C++ name which you can demangle with `c++filt` (Shipped with GCC) and you will get the full namespace prefixed name of the C++ function. Find the function with your favorite text editor search, life is happy.
>
> Pretty much every binary which is shipped on your machine via a distribution is sent like this. The backtraces + coredumps that are gathered come from this.
>
> [Clang has a option -gline-tables-only](http://clang.llvm.org/docs/UsersManual.html#cmdoption-gline-tables-only) which adds just the file name and line numbers to all functions in the debug info (Which can be done with minimal space overhead. [Gcc has '-g1'](https://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html#Debugging-Options), which would mean that you would get line number + file names in back traces with less overhead.
>
> Those options would make it so what people see printed by the build bot is exactly what they see now. I think that should be an easily accessible level, but not default build.
>
>
> **Coredumps**: It is possible to make coredumps just as inspectable without including debug information in all the shipping binaries. In fact, you generally probably don't want to run gdb on the binary on the production host anyways. For this, what typically happens is that
>
> 1. People actually only look at the backtrace to figure out what went wrong
> 2. You can generate debug information which lives in seperate files from the target executable (-gsplit-dwarf on new versions of GCC, requires a relatively new toolchain. I believe there is a way to do it on older tooling where you split it out later, but am not familiar with it). This debug info would generally live in a '-dbg' package. Which you can install / run on your local machine to inspect whatever variables remain directly, as well as global state.
>
>
> **OVERALL**
> I don't see anywhere where this makes debugging considerably more difficult than it is already. Yes, you can't attach a GDB to any build of mesos without thinking about it. I think most of the time most developers aren't attaching GDB.
>
> CI jobs I think can be easily switched over where they care. There aren't that many of them around. I agree it is something people need to be aware of, but not a hard level of change. Most CI systems don't let people connect in and step through the program with GDB to where it failed a test. It just gives a backtrace. In the case of a buildbot it isn't hard to set CXXFLAGS to just the level you want to get that ('-g1'). No major loss there with this change.
>
> The only case this makes considerably more difficult is that if you want to attach GDB to inspect variabels you need to have specified --enable-debug at configure time. I don't think that is a major loss, or something which is in the standard compile/test loop of most most Mesos contributors / developers (Although we can collect evidence if desired)
>
> Ben Mahler wrote:
> Thanks for being thorough! Having `-g1` and `-gline-tables-only` sound good to me for release builds, any reason not to include those?
>
> As for local development, I would _personally_ be fine with optimization turned off and `-g1` and `-gline-tables-only`, if the smaller debug information has a measurable improvement on compile speed or memory consumption (let's measure!). I tend to only need backtrace, or at most gdb thread backtraces from a dump.
>
> These two use-cases seem to suggest leaving the existing `--disable-optimize` and merely tweaking the debug flags to avoid dumping all debug information into release builds.
> How about for this change, we tackle the following:
>
> Default is Release: -O2 -g1 (or -gline-tables-only)
> --disable-optimize: -O0 -g1 (or -gline-tables-only)
>
> In a subsequent patch, we can add an orthogonal debug flag for those who want to the full set of debug information:
>
> --enable-debug: turns on -g
>
> Thoughts? It seems we can acheive Debug / Release builds even if we don't map the distinction to a single binary configure flag.
I'm going to run some tests:
CXXFLAGS=-O2
CXXFLAGS=-O0 -g1
CXXFLAGS=-O0 -g
CXXFLAGS=-O2 -g
I'm going to measure compile time resulting size, of the build directory, and size of the install directory. Will provide some feedback after that.
Note '-g' and '-g1' probably override eachother (they are actually the same flag), So if we want to do that, we need to make it swap that flag which will take a little bit more complex logic. The simplest/cleanest thing to do if people want a debug is to say 'Specify your own CFLAGS'
- Cody
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56573
-----------------------------------------------------------
On Oct. 14, 2014, 11:07 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2014, 11:07 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by Cody Maloney <co...@mesosphere.io>.
> On Oct. 14, 2014, 9:06 p.m., Timothy St. Clair wrote:
> > configure.ac, line 281
> > <https://reviews.apache.org/r/26426/diff/1/?file=714874#file714874line281>
> >
> > Is there a reason you want to leave debug symbols out of optimized builds?
> >
> > cmake has the pattern correct imho:
> > Release
> > Debug
> > ReleaseWithDebug
> >
> > A ReleaseWithDebug allows packagers, such as myself, to build w/debugsymbols that are stripped out into a .debuginfo package which can be used by developers for tracing "When bears attack". Granted that it is tenuous debugging at best, but it's better then nothing.
> >
> > So I think we want all three modes, stripping all debug information is not really idea.
My main motivation is to shrink the size of libmesos. Yesterday sugis in #mesos had one which was 213M. For the buildbot internally, full debian packages (which are compressed) of mesos weign in at 165M a piece (Yes, stripping post-build would help a lot, but why build it to begin with?). Most of this is debug info. Also, we build a bunch of different ways, and when libmesos is as big as it is, a decent amount of time ends up being spent on disk I/O reading / writing all the debug info when we are really just trying to ensure it builds on all the different platforms (Not to mention storage and file size shipping things around the network to centralized repositories).
The simple toggle between debug and release, removing the legacy logic gets us most of the benefit.
If you have a good place to point me for what is needed to get a 'ReleaseWithDebug' info build up and running I can definitely work on adding that as well.
- Cody
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56573
-----------------------------------------------------------
On Oct. 7, 2014, 10:38 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 7, 2014, 10:38 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac da1c82db31583fc81de658574b9a95628cb84dbc
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by Cody Maloney <co...@mesosphere.io>.
> On Oct. 14, 2014, 9:06 p.m., Timothy St. Clair wrote:
> > configure.ac, line 281
> > <https://reviews.apache.org/r/26426/diff/1/?file=714874#file714874line281>
> >
> > Is there a reason you want to leave debug symbols out of optimized builds?
> >
> > cmake has the pattern correct imho:
> > Release
> > Debug
> > ReleaseWithDebug
> >
> > A ReleaseWithDebug allows packagers, such as myself, to build w/debugsymbols that are stripped out into a .debuginfo package which can be used by developers for tracing "When bears attack". Granted that it is tenuous debugging at best, but it's better then nothing.
> >
> > So I think we want all three modes, stripping all debug information is not really idea.
>
> Cody Maloney wrote:
> My main motivation is to shrink the size of libmesos. Yesterday sugis in #mesos had one which was 213M. For the buildbot internally, full debian packages (which are compressed) of mesos weign in at 165M a piece (Yes, stripping post-build would help a lot, but why build it to begin with?). Most of this is debug info. Also, we build a bunch of different ways, and when libmesos is as big as it is, a decent amount of time ends up being spent on disk I/O reading / writing all the debug info when we are really just trying to ensure it builds on all the different platforms (Not to mention storage and file size shipping things around the network to centralized repositories).
>
> The simple toggle between debug and release, removing the legacy logic gets us most of the benefit.
>
> If you have a good place to point me for what is needed to get a 'ReleaseWithDebug' info build up and running I can definitely work on adding that as well.
It is also entirely possible to specify custom CXXFLAGS + CFLAGS with this patch to get the old "optimized debug" build. The flags set by --enable-debug (or not having it), to get back the "optimized debug" build from before.
- Cody
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56573
-----------------------------------------------------------
On Oct. 14, 2014, 11:07 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2014, 11:07 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by "Timothy St. Clair" <ts...@redhat.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56573
-----------------------------------------------------------
configure.ac
<https://reviews.apache.org/r/26426/#comment96918>
Typical pattern is yes/no vs. true/false.
Not that it really matters, more for consistency.
configure.ac
<https://reviews.apache.org/r/26426/#comment96919>
Is there a reason you want to leave debug symbols out of optimized builds?
cmake has the pattern correct imho:
Release
Debug
ReleaseWithDebug
A ReleaseWithDebug allows packagers, such as myself, to build w/debugsymbols that are stripped out into a .debuginfo package which can be used by developers for tracing "When bears attack". Granted that it is tenuous debugging at best, but it's better then nothing.
So I think we want all three modes, stripping all debug information is not really idea.
- Timothy St. Clair
On Oct. 7, 2014, 10:38 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 7, 2014, 10:38 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac da1c82db31583fc81de658574b9a95628cb84dbc
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by "Timothy St. Clair" <ts...@redhat.com>.
> On Oct. 14, 2014, 11:11 p.m., Dominic Hamon wrote:
> > configure.ac, line 281
> > <https://reviews.apache.org/r/26426/diff/2/?file=721143#file721143line281>
> >
> > aside: you might want to consider Os for release. It'll keep the size down and will often be as performant, even without rigourous optimizations, due to cache friendliness.
You almost always want -O2 over -Os, only typically in embedded situations does that change.
- Timothy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56598
-----------------------------------------------------------
On Oct. 14, 2014, 11:07 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2014, 11:07 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by Dominic Hamon <dh...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56598
-----------------------------------------------------------
configure.ac
<https://reviews.apache.org/r/26426/#comment96963>
aside: you might want to consider Os for release. It'll keep the size down and will often be as performant, even without rigourous optimizations, due to cache friendliness.
- Dominic Hamon
On Oct. 14, 2014, 4:07 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2014, 4:07 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56661
-----------------------------------------------------------
Patch looks great!
Reviews applied: [26426]
All tests passed.
- Mesos ReviewBot
On Oct. 14, 2014, 11:07 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2014, 11:07 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by "Timothy St. Clair" <ts...@redhat.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review56692
-----------------------------------------------------------
Ship it!
I'll wait a couple of days for any more comments before a push.
- Timothy St. Clair
On Oct. 14, 2014, 11:07 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2014, 11:07 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
>
> This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
> Main benefits:
> 1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
> 2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Built with both --enable-debug and without, checking that the flags get passed through correctly.
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug and --enable-optimize
flag for controlling building debug and optimized verisons of mesos
Posted by "Timothy St. Clair" <ts...@redhat.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review58108
-----------------------------------------------------------
Ship it!
Just noting that in our code base we've about picked either [action-if-not-given], or [action-if-give] but we should probably do both.
- Timothy St. Clair
On Oct. 23, 2014, 8:59 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 23, 2014, 8:59 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks buiding mesos in a "debug" vs. a "release" configuration. By default, mesos is built in a developer-centric setup (No optimizations, minimal debug info), in order to maximize developer productivity
>
> None: '-O0 -g1'
> --enable-optimize == '-O2'
> --enable-debug == '-g'
> --enable-optimize --enable-debug == '-O2 -g'
>
> If a user / developer passes CXXFLAGS or CFLAGS manually, then they are not changed / touched at all. This is important so that Mesos is a good citizen when being built for various distributions (As well as making it so specialized one-off groupings of flags are feasible to use).
>
> Adds two defines for accessing what mode things are being built in: 'DEBUG' and 'OPTIMIZE' which can be hooked into later to enable extra logging and the like. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Tested all four combinations of flags, making sure they set the right CXXFLAGS. Also specified
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug and --enable-optimize
flag for controlling building debug and optimized verisons of mesos
Posted by Cody Maloney <co...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/
-----------------------------------------------------------
(Updated Oct. 24, 2014, 9:35 p.m.)
Review request for mesos, Benjamin Hindman, Ben Mahler, and Timothy St. Clair.
Bugs: MESOS-1985
https://issues.apache.org/jira/browse/MESOS-1985
Repository: mesos-git
Description
-------
Reworks buiding mesos in a "debug" vs. a "release" configuration. By default, mesos is built in a developer-centric setup (No optimizations, minimal debug info), in order to maximize developer productivity
None: '-O0 -g1'
--enable-optimize == '-O2'
--enable-debug == '-g'
--enable-optimize --enable-debug == '-O2 -g'
If a user / developer passes CXXFLAGS or CFLAGS manually, then they are not changed / touched at all. This is important so that Mesos is a good citizen when being built for various distributions (As well as making it so specialized one-off groupings of flags are feasible to use).
Adds two defines for accessing what mode things are being built in: 'DEBUG' and 'OPTIMIZE' which can be hooked into later to enable extra logging and the like. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
Diffs
-----
configure.ac 2b372e06006250b5230956ef096473e98f3fa590
Diff: https://reviews.apache.org/r/26426/diff/
Testing
-------
Tested all four combinations of flags, making sure they set the right CXXFLAGS. Also specified
Thanks,
Cody Maloney
Re: Review Request 26426: Add --enable-debug and --enable-optimize
flag for controlling building debug and optimized verisons of mesos
Posted by Cody Maloney <co...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/
-----------------------------------------------------------
(Updated Oct. 24, 2014, 9:29 p.m.)
Review request for mesos, Benjamin Hindman, Ben Mahler, and Timothy St. Clair.
Changes
-------
Update per Dominic Hamon and Ben Mahler's comments.
Repository: mesos-git
Description
-------
Reworks buiding mesos in a "debug" vs. a "release" configuration. By default, mesos is built in a developer-centric setup (No optimizations, minimal debug info), in order to maximize developer productivity
None: '-O0 -g1'
--enable-optimize == '-O2'
--enable-debug == '-g'
--enable-optimize --enable-debug == '-O2 -g'
If a user / developer passes CXXFLAGS or CFLAGS manually, then they are not changed / touched at all. This is important so that Mesos is a good citizen when being built for various distributions (As well as making it so specialized one-off groupings of flags are feasible to use).
Adds two defines for accessing what mode things are being built in: 'DEBUG' and 'OPTIMIZE' which can be hooked into later to enable extra logging and the like. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
Diffs (updated)
-----
configure.ac 2b372e06006250b5230956ef096473e98f3fa590
Diff: https://reviews.apache.org/r/26426/diff/
Testing
-------
Tested all four combinations of flags, making sure they set the right CXXFLAGS. Also specified
Thanks,
Cody Maloney
Re: Review Request 26426: Add --enable-debug and --enable-optimize
flag for controlling building debug and optimized verisons of mesos
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review58330
-----------------------------------------------------------
Ship it!
Thanks Cody! Just some trivial comments below and we can get this committed!
configure.ac
<https://reviews.apache.org/r/26426/#comment99282>
Can you wrap after the comma to be consistent with the other AC_ARG_ENABLE blocks? Any reason to do it differently here?
configure.ac
<https://reviews.apache.org/r/26426/#comment99283>
How about s/$debug/$debug_flags/ and s/$optimize/$optimize_flags/ ?
- Ben Mahler
On Oct. 23, 2014, 9:20 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 23, 2014, 9:20 p.m.)
>
>
> Review request for mesos, Benjamin Hindman, Ben Mahler, and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks buiding mesos in a "debug" vs. a "release" configuration. By default, mesos is built in a developer-centric setup (No optimizations, minimal debug info), in order to maximize developer productivity
>
> None: '-O0 -g1'
> --enable-optimize == '-O2'
> --enable-debug == '-g'
> --enable-optimize --enable-debug == '-O2 -g'
>
> If a user / developer passes CXXFLAGS or CFLAGS manually, then they are not changed / touched at all. This is important so that Mesos is a good citizen when being built for various distributions (As well as making it so specialized one-off groupings of flags are feasible to use).
>
> Adds two defines for accessing what mode things are being built in: 'DEBUG' and 'OPTIMIZE' which can be hooked into later to enable extra logging and the like. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Tested all four combinations of flags, making sure they set the right CXXFLAGS. Also specified
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug and --enable-optimize
flag for controlling building debug and optimized verisons of mesos
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review58161
-----------------------------------------------------------
Patch looks great!
Reviews applied: [26426]
All tests passed.
- Mesos ReviewBot
On Oct. 23, 2014, 9:20 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 23, 2014, 9:20 p.m.)
>
>
> Review request for mesos, Benjamin Hindman, Ben Mahler, and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks buiding mesos in a "debug" vs. a "release" configuration. By default, mesos is built in a developer-centric setup (No optimizations, minimal debug info), in order to maximize developer productivity
>
> None: '-O0 -g1'
> --enable-optimize == '-O2'
> --enable-debug == '-g'
> --enable-optimize --enable-debug == '-O2 -g'
>
> If a user / developer passes CXXFLAGS or CFLAGS manually, then they are not changed / touched at all. This is important so that Mesos is a good citizen when being built for various distributions (As well as making it so specialized one-off groupings of flags are feasible to use).
>
> Adds two defines for accessing what mode things are being built in: 'DEBUG' and 'OPTIMIZE' which can be hooked into later to enable extra logging and the like. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Tested all four combinations of flags, making sure they set the right CXXFLAGS. Also specified
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug and --enable-optimize
flag for controlling building debug and optimized verisons of mesos
Posted by Cody Maloney <co...@mesosphere.io>.
> On Oct. 23, 2014, 9:50 p.m., Dominic Hamon wrote:
> > configure.ac, line 289
> > <https://reviews.apache.org/r/26426/diff/3/?file=731214#file731214line289>
> >
> > might be simpler to set a default then override:
> >
> > debug="-g1"
> > if ...
> > debug="-g"
> > else if ...
> > debug=""
> > fi
> >
> > optimize="-O0"
> > if ...
> > optimize="-O2"
> > fi
Updated for the debug case where there is a defualt then a "in other cases do this, in other cases the default should be different".
For optimize I like that I see I will definitively get either -O0 or -O2 based on which way the if goes.
- Cody
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review58118
-----------------------------------------------------------
On Oct. 24, 2014, 9:29 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 9:29 p.m.)
>
>
> Review request for mesos, Benjamin Hindman, Ben Mahler, and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks buiding mesos in a "debug" vs. a "release" configuration. By default, mesos is built in a developer-centric setup (No optimizations, minimal debug info), in order to maximize developer productivity
>
> None: '-O0 -g1'
> --enable-optimize == '-O2'
> --enable-debug == '-g'
> --enable-optimize --enable-debug == '-O2 -g'
>
> If a user / developer passes CXXFLAGS or CFLAGS manually, then they are not changed / touched at all. This is important so that Mesos is a good citizen when being built for various distributions (As well as making it so specialized one-off groupings of flags are feasible to use).
>
> Adds two defines for accessing what mode things are being built in: 'DEBUG' and 'OPTIMIZE' which can be hooked into later to enable extra logging and the like. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Tested all four combinations of flags, making sure they set the right CXXFLAGS. Also specified
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug and --enable-optimize
flag for controlling building debug and optimized verisons of mesos
Posted by Dominic Hamon <dh...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/#review58118
-----------------------------------------------------------
configure.ac
<https://reviews.apache.org/r/26426/#comment99023>
might be simpler to set a default then override:
debug="-g1"
if ...
debug="-g"
else if ...
debug=""
fi
optimize="-O0"
if ...
optimize="-O2"
fi
- Dominic Hamon
On Oct. 23, 2014, 2:20 p.m., Cody Maloney wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26426/
> -----------------------------------------------------------
>
> (Updated Oct. 23, 2014, 2:20 p.m.)
>
>
> Review request for mesos, Benjamin Hindman, Ben Mahler, and Timothy St. Clair.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Reworks buiding mesos in a "debug" vs. a "release" configuration. By default, mesos is built in a developer-centric setup (No optimizations, minimal debug info), in order to maximize developer productivity
>
> None: '-O0 -g1'
> --enable-optimize == '-O2'
> --enable-debug == '-g'
> --enable-optimize --enable-debug == '-O2 -g'
>
> If a user / developer passes CXXFLAGS or CFLAGS manually, then they are not changed / touched at all. This is important so that Mesos is a good citizen when being built for various distributions (As well as making it so specialized one-off groupings of flags are feasible to use).
>
> Adds two defines for accessing what mode things are being built in: 'DEBUG' and 'OPTIMIZE' which can be hooked into later to enable extra logging and the like. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
>
>
> Diffs
> -----
>
> configure.ac 2b372e06006250b5230956ef096473e98f3fa590
>
> Diff: https://reviews.apache.org/r/26426/diff/
>
>
> Testing
> -------
>
> Tested all four combinations of flags, making sure they set the right CXXFLAGS. Also specified
>
>
> Thanks,
>
> Cody Maloney
>
>
Re: Review Request 26426: Add --enable-debug and --enable-optimize
flag for controlling building debug and optimized verisons of mesos
Posted by Cody Maloney <co...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/
-----------------------------------------------------------
(Updated Oct. 23, 2014, 9:20 p.m.)
Review request for mesos, Benjamin Hindman, Ben Mahler, and Timothy St. Clair.
Changes
-------
Adding bmahler to review.
Repository: mesos-git
Description
-------
Reworks buiding mesos in a "debug" vs. a "release" configuration. By default, mesos is built in a developer-centric setup (No optimizations, minimal debug info), in order to maximize developer productivity
None: '-O0 -g1'
--enable-optimize == '-O2'
--enable-debug == '-g'
--enable-optimize --enable-debug == '-O2 -g'
If a user / developer passes CXXFLAGS or CFLAGS manually, then they are not changed / touched at all. This is important so that Mesos is a good citizen when being built for various distributions (As well as making it so specialized one-off groupings of flags are feasible to use).
Adds two defines for accessing what mode things are being built in: 'DEBUG' and 'OPTIMIZE' which can be hooked into later to enable extra logging and the like. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
Diffs
-----
configure.ac 2b372e06006250b5230956ef096473e98f3fa590
Diff: https://reviews.apache.org/r/26426/diff/
Testing
-------
Tested all four combinations of flags, making sure they set the right CXXFLAGS. Also specified
Thanks,
Cody Maloney
Re: Review Request 26426: Add --enable-debug and --enable-optimize
flag for controlling building debug and optimized verisons of mesos
Posted by Cody Maloney <co...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/
-----------------------------------------------------------
(Updated Oct. 23, 2014, 8:59 p.m.)
Review request for mesos, Benjamin Hindman and Timothy St. Clair.
Changes
-------
Largely a complete rewrite. See updated description.
Summary (updated)
-----------------
Add --enable-debug and --enable-optimize flag for controlling building debug and optimized verisons of mesos
Repository: mesos-git
Description (updated)
-------
Reworks buiding mesos in a "debug" vs. a "release" configuration. By default, mesos is built in a developer-centric setup (No optimizations, minimal debug info), in order to maximize developer productivity
None: '-O0 -g1'
--enable-optimize == '-O2'
--enable-debug == '-g'
--enable-optimize --enable-debug == '-O2 -g'
If a user / developer passes CXXFLAGS or CFLAGS manually, then they are not changed / touched at all. This is important so that Mesos is a good citizen when being built for various distributions (As well as making it so specialized one-off groupings of flags are feasible to use).
Adds two defines for accessing what mode things are being built in: 'DEBUG' and 'OPTIMIZE' which can be hooked into later to enable extra logging and the like. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
Diffs (updated)
-----
configure.ac 2b372e06006250b5230956ef096473e98f3fa590
Diff: https://reviews.apache.org/r/26426/diff/
Testing (updated)
-------
Tested all four combinations of flags, making sure they set the right CXXFLAGS. Also specified
Thanks,
Cody Maloney
Re: Review Request 26426: Add --enable-debug flag to ./configure for
controlling emission of debug information
Posted by Cody Maloney <co...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26426/
-----------------------------------------------------------
(Updated Oct. 14, 2014, 11:07 p.m.)
Review request for mesos, Benjamin Hindman and Timothy St. Clair.
Changes
-------
Switch to yes/no
Repository: mesos-git
Description
-------
Reworks building mesos in "debug" vs. "release". By default, mesos is now built in release (no debug info, optimized build). If '--enable-debug' is specified to configure, than optimization will be turned off, and debug info will be turned on.
This also adds a variable 'DEBUG' to the build environment, which people can use in code to see if mesos is built with debugging to enable extra assertions / checks. For release builds we may want to set 'NDEBUG' which removes assert()'s, but that is a seperate discussion.
Main benefits:
1) Getting a build to include/exclude debug information at will is feasible. Before some things like using clang would forcibly enable debug info in all cases
2) libmesos.so and the other binaries which get packaged up for use in distributions shrink considerably without manually stripping post-build (Improves build time, makes packaging cleaner)
Diffs (updated)
-----
configure.ac 2b372e06006250b5230956ef096473e98f3fa590
Diff: https://reviews.apache.org/r/26426/diff/
Testing
-------
Built with both --enable-debug and without, checking that the flags get passed through correctly.
Thanks,
Cody Maloney