You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Kenneth Giusti <kg...@apache.org> on 2011/10/20 15:26:52 UTC
Review Request: Performance tweaks for message group code.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2466/
-----------------------------------------------------------
Review request for qpid, Andrew Stitcher, Gordon Sim, and Chug Rolke.
Summary
-------
Two performance-related tweaks to message group code:
1) cache the last group lookup
2) use tr1::unordered_map rather than std::map for group identifier lookup - no need to keep the group id's in order.
Probably the most visible change would be for #2: the include path for unordered_map differs between gnu and windows - please see the included sys/unordered_map.h header for my "clever" work-around.
Simple benchmark before patch:
[root@mrg44 kgiusti]# qpid-cpp-benchmark -m 1000000 --content-size=8 --repeat=3 --summarize --send-arg="--group-key=GROUP-ID" --send-arg="--group-size=11" --receive-option "node:{x-declare:{arguments:{qpid.group_header_key:GROUP-ID,qpid.shared_msg_group:1}}}" --senders=2 --receivers=2 --send-arg="--group-interleave=3"
send-tp recv-tp l-min l-max l-avg
52863 52752 0.47 62.65 18.52
52437 52440 0.53 79.35 19.10
52042 52070 0.36 87.69 19.26
after patch:
[root@mrg44 kgiusti]# qpid-cpp-benchmark -m 1000000 --content-size=8 --repeat=3 --summarize --send-arg="--group-key=GROUP-ID" --send-arg="--group-size=11" --receive-option "node:{x-declare:{arguments:{qpid.group_header_key:GROUP-ID,qpid.shared_msg_group:1}}}" --senders=2 --receivers=2 --send-arg="--group-interleave=3"
send-tp recv-tp l-min l-max l-avg
55888 55842 0.68 84.53 17.37
55810 55784 0.46 85.09 17.23
54776 54750 0.36 77.25 17.46
This addresses bug qpid-3346.
https://issues.apache.org/jira/browse/qpid-3346
Diffs
-----
/trunk/qpid/cpp/src/Makefile.am 1186491
/trunk/qpid/cpp/src/qpid/broker/MessageGroupManager.h 1186491
/trunk/qpid/cpp/src/qpid/broker/MessageGroupManager.cpp 1186491
/trunk/qpid/cpp/src/qpid/sys/unordered_map.h PRE-CREATION
Diff: https://reviews.apache.org/r/2466/diff
Testing
-------
unit & benchmarking.
Thanks,
Kenneth
Re: Review Request: Performance tweaks for message group code.
Posted by Kenneth Giusti <kg...@apache.org>.
> On 2011-10-22 23:36:00, Andrew Stitcher wrote:
> > /trunk/qpid/cpp/src/qpid/sys/unordered_map.h, line 24
> > <https://reviews.apache.org/r/2466/diff/1/?file=51487#file51487line24>
> >
> > This is almost certainly the wrong test. You should be testing the compiler being used not the platform being compiled for.
> > (_WIN32 will be true for the mingw compile, but it uses the gcc compiler)
Thanks Andrew - I didn't think of that.
I'll replace the _WIN32 check with a check for the definition of _MSC_VER instead: as per http://msdn.microsoft.com/en-us/library/b0084kay.aspx
- Kenneth
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2466/#review2782
-----------------------------------------------------------
On 2011-10-20 13:26:52, Kenneth Giusti wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2466/
> -----------------------------------------------------------
>
> (Updated 2011-10-20 13:26:52)
>
>
> Review request for qpid, Andrew Stitcher, Gordon Sim, and Chug Rolke.
>
>
> Summary
> -------
>
> Two performance-related tweaks to message group code:
> 1) cache the last group lookup
> 2) use tr1::unordered_map rather than std::map for group identifier lookup - no need to keep the group id's in order.
>
> Probably the most visible change would be for #2: the include path for unordered_map differs between gnu and windows - please see the included sys/unordered_map.h header for my "clever" work-around.
>
>
> Simple benchmark before patch:
>
> [root@mrg44 kgiusti]# qpid-cpp-benchmark -m 1000000 --content-size=8 --repeat=3 --summarize --send-arg="--group-key=GROUP-ID" --send-arg="--group-size=11" --receive-option "node:{x-declare:{arguments:{qpid.group_header_key:GROUP-ID,qpid.shared_msg_group:1}}}" --senders=2 --receivers=2 --send-arg="--group-interleave=3"
> send-tp recv-tp l-min l-max l-avg
> 52863 52752 0.47 62.65 18.52
> 52437 52440 0.53 79.35 19.10
> 52042 52070 0.36 87.69 19.26
>
> after patch:
>
> [root@mrg44 kgiusti]# qpid-cpp-benchmark -m 1000000 --content-size=8 --repeat=3 --summarize --send-arg="--group-key=GROUP-ID" --send-arg="--group-size=11" --receive-option "node:{x-declare:{arguments:{qpid.group_header_key:GROUP-ID,qpid.shared_msg_group:1}}}" --senders=2 --receivers=2 --send-arg="--group-interleave=3"
> send-tp recv-tp l-min l-max l-avg
> 55888 55842 0.68 84.53 17.37
> 55810 55784 0.46 85.09 17.23
> 54776 54750 0.36 77.25 17.46
>
>
> This addresses bug qpid-3346.
> https://issues.apache.org/jira/browse/qpid-3346
>
>
> Diffs
> -----
>
> /trunk/qpid/cpp/src/Makefile.am 1186491
> /trunk/qpid/cpp/src/qpid/broker/MessageGroupManager.h 1186491
> /trunk/qpid/cpp/src/qpid/broker/MessageGroupManager.cpp 1186491
> /trunk/qpid/cpp/src/qpid/sys/unordered_map.h PRE-CREATION
>
> Diff: https://reviews.apache.org/r/2466/diff
>
>
> Testing
> -------
>
> unit & benchmarking.
>
>
> Thanks,
>
> Kenneth
>
>
Re: Review Request: Performance tweaks for message group code.
Posted by Andrew Stitcher <as...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2466/#review2782
-----------------------------------------------------------
/trunk/qpid/cpp/src/qpid/sys/unordered_map.h
<https://reviews.apache.org/r/2466/#comment6243>
This is almost certainly the wrong test. You should be testing the compiler being used not the platform being compiled for.
(_WIN32 will be true for the mingw compile, but it uses the gcc compiler)
- Andrew
On 2011-10-20 13:26:52, Kenneth Giusti wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2466/
> -----------------------------------------------------------
>
> (Updated 2011-10-20 13:26:52)
>
>
> Review request for qpid, Andrew Stitcher, Gordon Sim, and Chug Rolke.
>
>
> Summary
> -------
>
> Two performance-related tweaks to message group code:
> 1) cache the last group lookup
> 2) use tr1::unordered_map rather than std::map for group identifier lookup - no need to keep the group id's in order.
>
> Probably the most visible change would be for #2: the include path for unordered_map differs between gnu and windows - please see the included sys/unordered_map.h header for my "clever" work-around.
>
>
> Simple benchmark before patch:
>
> [root@mrg44 kgiusti]# qpid-cpp-benchmark -m 1000000 --content-size=8 --repeat=3 --summarize --send-arg="--group-key=GROUP-ID" --send-arg="--group-size=11" --receive-option "node:{x-declare:{arguments:{qpid.group_header_key:GROUP-ID,qpid.shared_msg_group:1}}}" --senders=2 --receivers=2 --send-arg="--group-interleave=3"
> send-tp recv-tp l-min l-max l-avg
> 52863 52752 0.47 62.65 18.52
> 52437 52440 0.53 79.35 19.10
> 52042 52070 0.36 87.69 19.26
>
> after patch:
>
> [root@mrg44 kgiusti]# qpid-cpp-benchmark -m 1000000 --content-size=8 --repeat=3 --summarize --send-arg="--group-key=GROUP-ID" --send-arg="--group-size=11" --receive-option "node:{x-declare:{arguments:{qpid.group_header_key:GROUP-ID,qpid.shared_msg_group:1}}}" --senders=2 --receivers=2 --send-arg="--group-interleave=3"
> send-tp recv-tp l-min l-max l-avg
> 55888 55842 0.68 84.53 17.37
> 55810 55784 0.46 85.09 17.23
> 54776 54750 0.36 77.25 17.46
>
>
> This addresses bug qpid-3346.
> https://issues.apache.org/jira/browse/qpid-3346
>
>
> Diffs
> -----
>
> /trunk/qpid/cpp/src/Makefile.am 1186491
> /trunk/qpid/cpp/src/qpid/broker/MessageGroupManager.h 1186491
> /trunk/qpid/cpp/src/qpid/broker/MessageGroupManager.cpp 1186491
> /trunk/qpid/cpp/src/qpid/sys/unordered_map.h PRE-CREATION
>
> Diff: https://reviews.apache.org/r/2466/diff
>
>
> Testing
> -------
>
> unit & benchmarking.
>
>
> Thanks,
>
> Kenneth
>
>