You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Fraser Adams <fr...@blueyonder.co.uk> on 2014/01/25 17:05:32 UTC
Qpid's move to CMake and effect on performance.
Qpid has now deprecated the automake build process and moved to CMake,
but I've got a nagging feeling.........
I've not *actually* got round to doing any benchmark/performance test
type things since I moved to using CMake myself and I'm wondering to
myself whether as part of Qpid/Proton release cycle anyone is has been
doing regression tests and in particular systematic performance tests?
One of my colleagues mentioned to me that he reckons that there was a
performance drop off between (I think) 0.12 and 0.18 but I've got no
evidence other than anecdotal from him and certainly no reproducer.
It did get me thinking about CMake though (though my colleague insists
he was using automake so there might still be an issue).
Am I correct in thinking that with CMake there are different "build
types" and that in "idiomatic" CMake usage it's "generally" via the use
of different build types that compiler optimisation flags get set?
In the automake build system I think that things were set to "-O3", but
I'm less convinced for CMake, indeed I *think* when at one point I made
Proton using make VERBOSE=1 I made a mental note that I didn't see any
optimisation flags.
The install instructions tend to say things along the lines of
mkdir build
cd build
cmake ..
make all
make install
CMAKE_BUILD_TYPE=Release
But I'm wondering if the third step really should be:
cmake CMAKE_BUILD_TYPE=Release ..
or somesuch.
I'm not much of a CMake expert (barely beyond a N00b) but if we do need
something like that to build with optimisation flags then I reckon we
*really* need to make sure that the Qpid and Proton build/install guides
get updated to reflect this otherwise users might be seeing
unrepresentative performance figures.
This might be second nature to CMake experts, but possibly/probably
isn't for everyone.
Thoughts?
Frase
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid's move to CMake and effect on performance.
Posted by Alan Conway <ac...@redhat.com>.
On Sat, 2014-01-25 at 16:05 +0000, Fraser Adams wrote:
> Qpid has now deprecated the automake build process and moved to CMake,
> but I've got a nagging feeling.........
>
> I've not *actually* got round to doing any benchmark/performance test
> type things since I moved to using CMake myself and I'm wondering to
> myself whether as part of Qpid/Proton release cycle anyone is has been
> doing regression tests and in particular systematic performance tests?
>
> One of my colleagues mentioned to me that he reckons that there was a
> performance drop off between (I think) 0.12 and 0.18 but I've got no
> evidence other than anecdotal from him and certainly no reproducer.
>
> It did get me thinking about CMake though (though my colleague insists
> he was using automake so there might still be an issue).
>
> Am I correct in thinking that with CMake there are different "build
> types" and that in "idiomatic" CMake usage it's "generally" via the use
> of different build types that compiler optimisation flags get set?
>
> In the automake build system I think that things were set to "-O3", but
> I'm less convinced for CMake, indeed I *think* when at one point I made
> Proton using make VERBOSE=1 I made a mental note that I didn't see any
> optimisation flags.
>
> The install instructions tend to say things along the lines of
>
> mkdir build
> cd build
> cmake ..
> make all
> make install
>
> CMAKE_BUILD_TYPE=Release
> But I'm wondering if the third step really should be:
> cmake CMAKE_BUILD_TYPE=Release ..
>
> or somesuch.
>
> I'm not much of a CMake expert (barely beyond a N00b) but if we do need
> something like that to build with optimisation flags then I reckon we
> *really* need to make sure that the Qpid and Proton build/install guides
> get updated to reflect this otherwise users might be seeing
> unrepresentative performance figures.
>
> This might be second nature to CMake experts, but possibly/probably
> isn't for everyone.
>
> Thoughts?
>
I am easily confused and often switch between debug and release builds
for debugging vs. profiling work so I always put my builds in
directories called debug*, release* or reldbg* and I have my run-cmake
script like this:
case $(basename $PWD) in
debug*) BUILD_TYPE=Debug;;
release*) BUILD_TYPE=Release;;
reldbg*) BUILD_TYPE=RelWithDebInfo;;
esac
BUILD_TYPE=${1:-$BUILD_TYPE}
test -n "$BUILD_TYPE" || { echo "Specify build type."; exit 1; }
# And later...
cmake ... -DCMAKE_BUILD_TYPE:STRING=$BUILD_TYPE
This has saved me many hours (days) of trying to figure out why my
performance results are suddenly in the toilet because I forgot which
build flags I set.
Cheers,
Alan.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid's move to CMake and effect on performance.
Posted by Andrew Stitcher <as...@redhat.com>.
On Mon, 2014-01-27 at 15:43 -0500, Alan Conway wrote:
> ...
>
> "Release" is: -O3 -DNDEBUG
I stand corrected - The non lazy version is (ie I actually looked at the
CMake files):
Debug -g
Release -O3 -DNDEBUG
RelWithDebInfo -O2 -g -DNDEBUG
MinSizeRel -Os -DNDEBUG
BTW it's not definite that -O3 gives better performance than -Os or even
-O2 in all cases. You should benchmark.
-O3 does tend to be hard to get useful debug info out of though.
-Os often gives you better performance (or at least did) because smaller
code may fit in cache better. -O3 will make the code bigger in many/most
cases (unrolling loops etc.) and so may lose performance by not fitting
into cache as well.
[FWIW (ie not a lot) I tend to prefer -Os -g for release flags. In other
words always have debug info with the smallest code and don't turn
asserts off]
Andrew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid's move to CMake and effect on performance.
Posted by Alan Conway <ac...@redhat.com>.
On Mon, 2014-01-27 at 20:25 +0000, Fraser Adams wrote:
> On 27/01/14 15:06, Andrew Stitcher wrote:
> > I think that (too lazy to actually go look) in autotools the build
> > default was -O2 and the Fedora/RHEL packaging set -O3.
> >
> > It is completely true that with cmake we've opted to follow the usual
> > cmake practice (which is standard on Windows too) of having the build
> > configuration dictate the optimisation/debug flags. I think that in the
> > Fedora/RHEL packages we now use the RelWithDebInfo build configuration
> > which corresponds to -O2 -g.
> >
> > It is true that if you specify no CMAKE_BUILD_TYPE then you will get no
> > optimisation or debug flags set which is perhaps a rather bad idea.
> >
> > What do you think the default build type should be? (Out of Debug = -g,
> > Release = -O2 -DNDEBUG, RelWithDebInfo -O2 -DNDEBUG -g, MinSizeRel = -Os
> > -DNDEBUG)
> >
> > It's not clear to me that we should prioritise performance over
> > debugability in our default build.
> >
> > Andrew
> >
> Thanks for the response Andrew - glad I'm not going mad :-D
>
> I think that I'm *relatively* agnostic as to what the default should be,
> but I definitely think that this stuff should be included in the
> build/install instructions for qpid/proton/dispatch/etc. If you've got
> familiarity with CMake it might be second nature, but I don't think we
> should assume this (I only figured it out by accident, having only
> fairly rudimentary CMake experience).
>
> Out of curiosity looking at the list above I don't see an option that
> does "-O3" what would that be? I'd like to eke the maximum performance
> out of the components I'm running operationally - we've got some very
> high throughput use cases so I'm keen to make sure that we don't hit
> performance regressions as we upgrade from earlier Qpid versions.
>
"Release" is: -O3 -DNDEBUG
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid's move to CMake and effect on performance.
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
On 27/01/14 15:06, Andrew Stitcher wrote:
> I think that (too lazy to actually go look) in autotools the build
> default was -O2 and the Fedora/RHEL packaging set -O3.
>
> It is completely true that with cmake we've opted to follow the usual
> cmake practice (which is standard on Windows too) of having the build
> configuration dictate the optimisation/debug flags. I think that in the
> Fedora/RHEL packages we now use the RelWithDebInfo build configuration
> which corresponds to -O2 -g.
>
> It is true that if you specify no CMAKE_BUILD_TYPE then you will get no
> optimisation or debug flags set which is perhaps a rather bad idea.
>
> What do you think the default build type should be? (Out of Debug = -g,
> Release = -O2 -DNDEBUG, RelWithDebInfo -O2 -DNDEBUG -g, MinSizeRel = -Os
> -DNDEBUG)
>
> It's not clear to me that we should prioritise performance over
> debugability in our default build.
>
> Andrew
>
Thanks for the response Andrew - glad I'm not going mad :-D
I think that I'm *relatively* agnostic as to what the default should be,
but I definitely think that this stuff should be included in the
build/install instructions for qpid/proton/dispatch/etc. If you've got
familiarity with CMake it might be second nature, but I don't think we
should assume this (I only figured it out by accident, having only
fairly rudimentary CMake experience).
Out of curiosity looking at the list above I don't see an option that
does "-O3" what would that be? I'd like to eke the maximum performance
out of the components I'm running operationally - we've got some very
high throughput use cases so I'm keen to make sure that we don't hit
performance regressions as we upgrade from earlier Qpid versions.
Cheers,
Frase
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid's move to CMake and effect on performance.
Posted by Andrew Stitcher <as...@redhat.com>.
On Mon, 2014-01-27 at 11:08 -0500, Shearer, Davin wrote:
> Right, it will create a *separate RPM* with symbols, so if folks need to
> debug, they use the debuginfo-install tool and if you ship a debuginfo RPM,
> it will install the symbols. For most folks, this arrangement is fine.
The point being that you *must* use compile flags with -g in them or
there will be no (useful) debugging info to populate the -debuginfo rpm.
So in this regard Release and RelWithDebInfo are actually quite
different.
A
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid's move to CMake and effect on performance.
Posted by "Shearer, Davin" <ds...@novetta.com>.
Right, it will create a *separate RPM* with symbols, so if folks need to
debug, they use the debuginfo-install tool and if you ship a debuginfo RPM,
it will install the symbols. For most folks, this arrangement is fine.
Re: Qpid's move to CMake and effect on performance.
Posted by Andrew Stitcher <as...@redhat.com>.
On Mon, 2014-01-27 at 10:15 -0500, Shearer, Davin wrote:
> FYI -- by default rpmbuild strips symbols, effectively making Release and
> RelWIthDebinfo the same thing.
I not sure that's true. IIRC rpmbuild creates a separate file (set of
files) with the debugging info in it. This can be installed separately
but still used for debugging. This is how the -debuginfo packages are
created.
At least that's what it does in stock Fedora and RHEL builds. I don't
think anything special has to be done in the spec file to make this
work. Although I expect there are some special rpm macros to make this
work.
A
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid's move to CMake and effect on performance.
Posted by "Shearer, Davin" <ds...@novetta.com>.
FYI -- by default rpmbuild strips symbols, effectively making Release and
RelWIthDebinfo the same thing.
On Mon, Jan 27, 2014 at 10:06 AM, Andrew Stitcher <as...@redhat.com>wrote:
> On Sat, 2014-01-25 at 16:05 +0000, Fraser Adams wrote:
>
> > ...
> > One of my colleagues mentioned to me that he reckons that there was a
> > performance drop off between (I think) 0.12 and 0.18 but I've got no
> > evidence other than anecdotal from him and certainly no reproducer.
>
> I doubt this has anything to do with the change of build system.
>
> >
> > It did get me thinking about CMake though (though my colleague insists
> > he was using automake so there might still be an issue).
> >
> > Am I correct in thinking that with CMake there are different "build
> > types" and that in "idiomatic" CMake usage it's "generally" via the use
> > of different build types that compiler optimisation flags get set?
>
> I think that (too lazy to actually go look) in autotools the build
> default was -O2 and the Fedora/RHEL packaging set -O3.
>
> It is completely true that with cmake we've opted to follow the usual
> cmake practice (which is standard on Windows too) of having the build
> configuration dictate the optimisation/debug flags. I think that in the
> Fedora/RHEL packages we now use the RelWithDebInfo build configuration
> which corresponds to -O2 -g.
>
> It is true that if you specify no CMAKE_BUILD_TYPE then you will get no
> optimisation or debug flags set which is perhaps a rather bad idea.
>
> What do you think the default build type should be? (Out of Debug = -g,
> Release = -O2 -DNDEBUG, RelWithDebInfo -O2 -DNDEBUG -g, MinSizeRel = -Os
> -DNDEBUG)
>
> It's not clear to me that we should prioritise performance over
> debugability in our default build.
>
> Andrew
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
--
Davin Shearer
Engineer
8830 Stanford Blvd, Suite 306
Columbia, MD 21045
443-741-4517
Re: Qpid's move to CMake and effect on performance.
Posted by Andrew Stitcher <as...@redhat.com>.
On Sat, 2014-01-25 at 16:05 +0000, Fraser Adams wrote:
> ...
> One of my colleagues mentioned to me that he reckons that there was a
> performance drop off between (I think) 0.12 and 0.18 but I've got no
> evidence other than anecdotal from him and certainly no reproducer.
I doubt this has anything to do with the change of build system.
>
> It did get me thinking about CMake though (though my colleague insists
> he was using automake so there might still be an issue).
>
> Am I correct in thinking that with CMake there are different "build
> types" and that in "idiomatic" CMake usage it's "generally" via the use
> of different build types that compiler optimisation flags get set?
I think that (too lazy to actually go look) in autotools the build
default was -O2 and the Fedora/RHEL packaging set -O3.
It is completely true that with cmake we've opted to follow the usual
cmake practice (which is standard on Windows too) of having the build
configuration dictate the optimisation/debug flags. I think that in the
Fedora/RHEL packages we now use the RelWithDebInfo build configuration
which corresponds to -O2 -g.
It is true that if you specify no CMAKE_BUILD_TYPE then you will get no
optimisation or debug flags set which is perhaps a rather bad idea.
What do you think the default build type should be? (Out of Debug = -g,
Release = -O2 -DNDEBUG, RelWithDebInfo -O2 -DNDEBUG -g, MinSizeRel = -Os
-DNDEBUG)
It's not clear to me that we should prioritise performance over
debugability in our default build.
Andrew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org