You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stdcxx.apache.org by Martin Sebor <ms...@gmail.com> on 2012/10/01 00:18:57 UTC
Re: Fwd: Re: STDCXX-1071 numpunct facet defect
I see you did a 64-bit build while I did a 32-bit one. so
I tried 64-bits. The cached version (i.e., the one compiled
with -UNO_USE_NUMPUNCT_CACHE) is still about twice as fast
as the non-cached one (compiled with -DNO_USE_NUMPUNCT_CACHE).
I had made one change to the test program that I thought might
account for the difference: I removed the call to abort from
the thread function since it was causing the process to exit
prematurely in some of my tests. But since you used the
modified program for your latest measurements that couldn't
be it.
I can't explain the differences. They just don't make sense
to me. Your results should be the other way around. Can you
post the disassembly of function f() for each of the two
configurations of the test?
Martin
On 09/30/2012 03:30 PM, Liviu Nicoara wrote:
> On 9/30/12 2:21 PM, Liviu Nicoara wrote:
>> Forwarding with the attachment.
>>
>> -------- Original Message --------
>> Subject: Re: STDCXX-1071 numpunct facet defect
>> Date: Sun, 30 Sep 2012 12:09:10 -0600
>> From: Martin Sebor <ms...@gmail.com>
>> To: Liviu Nicoara <ni...@hates.ms>
>>
>>> On 9/27/12 8:27 PM, Martin Sebor wrote:
>>
>> Here are my timings for library-reduction.cpp when compiled
>> GCC 4.5.3 on Solaris 10 (4 SPARCV9 CPUs). I had to make a small
>> number of trivial changes to get it to compile:
>>
>> With cache No cache
>> real 1m38.332s 8m58.568s
>> user 6m30.244s 34m25.942s
>> sys 0m0.060s 0m3.922s
>>
>> I also experimented with the program on Linux (CEL 4 with 16
>> CPUs). Initially, I saw no differences between the two versions.
>> So I modified it a bit to make it closer to the library (the
>> modified program is attached). With those changes the timings
>
> I see the difference -- your program has a virtual function it calls
> from the inline grouping function.
>
>> are below:
>>
>> With cache No cache
>> real 0m 1.107s 0m 5.669s
>> user 0m17.204s 0m 5.669s
>> sys 0m 0.000s 0m22.347s
>>
>> I also recompiled and re-ran the test on Solaris. To speed
>> things along, I set the number threads and loops to 8 and
>> 1000000. The numbers are as follows:
>>
>> With cache No cache
>> real 0m3.341s 0m26.333s
>> user 0m13.052s 1m37.470s
>> sys 0m0.009s 0m0.132s
>>
>> The numbers match my expectation. The overhead without the
>> "numpunct cache" is considerable.
>
> I have done another (smaller) round of measurements, this time using the
> test program you posted. Here are the results:
>
> * iMac, 4x Intel, 12S:
>
> 16, 10000000:
>
> Cached Not cached
> real 0m9.300s 0m5.224s
> user 0m36.441s 0m20.523s
> sys 0m0.043s 0m0.068s
>
> * iMac, 4x Intel, 12D:
>
> Cached Not cached
> real 0m9.012s 0m5.774s
> user 0m35.343s 0m20.997s
> sys 0m0.045s 0m0.183s
>
> * Linux Slackware, 16x AMD Opteron, 12S:
>
> 16, 10000000:
>
> Cached Not cached
> real 0m29.798s 0m3.278s
> user 0m48.662s 0m47.338s
> sys 6m18.525s 0m3.298s
>
>>
>> Somewhat unexpectedly, the test with the cache didn't crash.
>
> On my iMac it did not crash for me either (gcc 4.5.4), this time. On the
> other box (gcc 4.5.2) crashed every time with caching, so I had to add a
> call to fac.grouping outside the thread function to initialize the "facet".
>
> Liviu
Re: Fwd: Re: STDCXX-1071 numpunct facet defect
Posted by Liviu Nicoara <ni...@hates.ms>.
On 10/03/12 11:10, Martin Sebor wrote:
> [...]
> I was just thinking of a few simple loops along the lines of:
>
tl;dr: I consider the results of the multi-threaded performance tests (12S, Intel/AMD multicores) as coming from heavy contention in copying of reference-counted std::string objects.
The long version: I went back to the original test case (attached, t.cpp). As a reminder, the program is a simplification of the library numpunct facet creation process, which can be compiled either with data caching enabled or disabled.
I ran the perf tool (https://perf.wiki.kernel.org/index.php/Main_Page) on the program, and I noticed marked differences in the number of instructions and cycles (an order of magnitude) and also in the number of CPU-migrations and context-switches (way larger for the cached version):
Performance counter stats for './t':
121168.026301 task-clock-msecs # 3.778 CPUs
6133377 context-switches # 0.051 M/sec
35457 CPU-migrations # 0.000 M/sec
298 page-faults # 0.000 M/sec
338006437277 cycles # 2789.568 M/sec
136554855096 instructions # 0.404 IPC
31298346444 branches # 258.305 M/sec
2305761092 branch-misses # 7.367 %
1592157186 cache-references # 13.140 M/sec
1729755 cache-misses # 0.014 M/sec
32.070381363 seconds time elapsed
vs.
Performance counter stats for './t':
16194.235282 task-clock-msecs # 3.883 CPUs
7491 context-switches # 0.000 M/sec
47 CPU-migrations # 0.000 M/sec
311 page-faults # 0.000 M/sec
45210281987 cycles # 2791.752 M/sec
76784904688 instructions # 1.698 IPC
18891711522 branches # 1166.570 M/sec
3162251552 branch-misses # 16.739 %
3332839 cache-references # 0.206 M/sec
119424 cache-misses # 0.007 M/sec
4.170904279 seconds time elapsed
Then, I have run more detailed reports on the runs. I noticed that the report for the cached version contains what appears to be thread synchronization overhead, whereas the non-cached version doesn't:
# Events: 134K cycles
#
# Overhead Command Shared Object Symbol
# ........ ....... .................. ...................................
#
50.69% t [kernel.kallsyms] [k] _raw_spin_lock
5.46% t libpthread-2.13.so [.] pthread_mutex_lock
4.16% t libpthread-2.13.so [.] __pthread_mutex_unlock
3.65% t libpthread-2.13.so [.] __lll_lock_wait
3.35% t [kernel.kallsyms] [k] copy_user_generic_string
3.09% t 40395b [.] 40395b
2.91% t [kernel.kallsyms] [k] hash_futex
2.87% t [kernel.kallsyms] [k] futex_wake
2.67% t t [.] f
2.37% t [kernel.kallsyms] [k] futex_wait_setup
1.61% t [kernel.kallsyms] [k] system_call
1.00% t [kernel.kallsyms] [k] system_call_after_swapgs
0.98% t [kernel.kallsyms] [k] __switch_to
0.96% t libpthread-2.13.so [.] __lll_unlock_wake
0.95% t [kernel.kallsyms] [k] get_futex_key
0.92% t libc-2.13.so [.] __strlen_sse42
[...]
vs.
# Events: 15K cycles
#
# Overhead Command Shared Object Symbol
# ........ ....... .................. ..............................................................................................................................
#
49.55% :29571 7f1ed3830a86 [.] 7f1ed3830a86
8.67% t libc-2.13.so [.] __malloc
8.03% t libc-2.13.so [.] cfree
4.88% t t [.] std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)
4.67% t libc-2.13.so [.] __strlen_sse42
4.14% t libpthread-2.13.so [.] __pthread_mutex_unlock
3.62% t libpthread-2.13.so [.] pthread_mutex_lock
2.64% t t [.] f
2.40% t libc-2.13.so [.] _int_malloc
2.10% t libc-2.13.so [.] _int_free
1.53% t libpthread-2.13.so [.] __pthread_mutex_init_internal
1.33% t t [.] operator new(unsigned long)
1.08% t t [.] X::numpunct::do_grouping() const
1.04% t libc-2.13.so [.] __memcpy_sse2
0.89% t t [.] X::get_data(X::facet*)
0.51% t t [.] __rw::__rw_allocate(unsigned long, int)
0.45% t t [.] strlen@plt
0.33% t t [.] __rw::__rw_deallocate(void*, unsigned long, int)
0.32% t libpthread-2.13.so [.] pthread_mutex_destroy
0.21% t t [.] pthread_mutex_unlock@plt
0.21% t t [.] pthread_mutex_init@plt
0.18% t t [.] pthread_mutex_destroy@plt
0.17% t t [.] operator delete(void*)
0.17% t t [.] malloc@plt
0.16% t t [.] memcpy@plt
0.15% :29571 [kernel.kallsyms] [k] hpet_msi_next_event
0.12% t libc-2.13.so [.] __strrchr_sse2
0.12% t t [.] pthread_mutex_lock@plt
0.11% t t [.] free@plt
[...]
Now, if you remember from previous measurements, the performance of the two versions of the program was reversed for the case of one thread, i.e., one-threaded programs performed better in the cached version. Here is the report for a run with one thread, the cached version (running one thread in 1000M loops, otherwise it does not register anything):
# Events: 36K cycles
#
# Overhead Command Shared Object Symbol
# ........ ....... .................. ........................
#
37.68% t libpthread-2.13.so [.] __pthread_mutex_unlock
31.04% t libpthread-2.13.so [.] pthread_mutex_lock
11.87% t t [.] f
9.64% :27362 7fd94e2ebd6d [.] 7fd94e2ebd6d
3.21% t libc-2.13.so [.] __strlen_sse42
2.38% t t [.] pthread_mutex_unlock@plt
2.27% t t [.] pthread_mutex_lock@plt
1.61% t t [.] strlen@plt
[...]
The one-threaded run for the non-cached version yields a report identical with the multi-threaded run.
There are two things which I take from this round of measurements: first, that performance suffers on multicores in heavy contention, possibly as a direct result of std::string object copying and I'll follow up with direct measurements. Second, that perhaps we should not focus on this particular use model for the numpunct facet, because it may not be very representative for `normal' usage.
HTH
Liviu
Re: Fwd: Re: STDCXX-1071 numpunct facet defect
Posted by Liviu Nicoara <ni...@hates.ms>.
On 10/04/12 22:41, Liviu Nicoara wrote:
> On 10/4/12 10:10 PM, Liviu Nicoara wrote:
>>
>> On 10/3/12 11:10 AM, Martin Sebor wrote:
>>> On 10/03/2012 07:01 AM, Liviu Nicoara wrote:
>>>>
>>> void* thread_func (void*) {
>>> for (int i = 0; i < N; ++)
>>> test 1: do some simple stuff inline
>>> test 2: call a virtual function to do the same stuff
>>> test 3: lock and unlock a mutex and do the same stuff
>>> }
>>>
>>> Test 1 should be the fastest and test 3 the slowest. This should
>>> hold regardless of what "simple stuff" is (eventually, even when
>>> it's getting numpunct::grouping() data).
>>
>> That is expected; I attached test case x.cpp and results-x.txt.
>>
>> I did not find it too interesting in its own, though. The difference
>> between the
>> cached and non-cached data is that in the case of the cached data the
>> copying of
>> the string involves nothing more than a bump in the reference counter,
>> whereas
>> in the non-cached version a string object is constructed anew, and
>> memory gets
>> allocated for its body. Yet, in my measurements, the cached version is
>> the one
>> which shows the worse performance.
>>
>> So, I extracted the std::string class and simplified it down and put
>> it in
>> another test case. That would be u.cpp and the results are
>> results-u.txt. The
>> results show the same performance trends although the absolute values
>> have
>> skewed. Will get back to this after I digest the results a bit more.
This discussion has stagnated lately. I hope to allocate some time this
week-end to go over the results, re-verify them and analyze the
disassembly. In the meantime, any thoughts?
Liviu
Re: Fwd: Re: STDCXX-1071 numpunct facet defect
Posted by Liviu Nicoara <ni...@hates.ms>.
On 10/4/12 10:10 PM, Liviu Nicoara wrote:
>
> On 10/3/12 11:10 AM, Martin Sebor wrote:
>> On 10/03/2012 07:01 AM, Liviu Nicoara wrote:
>>>
>>> I am gathering some more measurements along these lines but it's time
>>> consuming. I estimate I will have some ready for review later today or
>>> tomorrow. In the meantime could you please post your kernel, glibc and
>>> compiler versions?
>>
>> I was just thinking of a few simple loops along the lines of:
>>
>> void* thread_func (void*) {
>> for (int i = 0; i < N; ++)
>> test 1: do some simple stuff inline
>> test 2: call a virtual function to do the same stuff
>> test 3: lock and unlock a mutex and do the same stuff
>> }
>>
>> Test 1 should be the fastest and test 3 the slowest. This should
>> hold regardless of what "simple stuff" is (eventually, even when
>> it's getting numpunct::grouping() data).
>
> That is expected; I attached test case x.cpp and results-x.txt.
>
> I did not find it too interesting in its own, though. The difference between the
> cached and non-cached data is that in the case of the cached data the copying of
> the string involves nothing more than a bump in the reference counter, whereas
> in the non-cached version a string object is constructed anew, and memory gets
> allocated for its body. Yet, in my measurements, the cached version is the one
> which shows the worse performance.
>
> So, I extracted the std::string class and simplified it down and put it in
> another test case. That would be u.cpp and the results are results-u.txt. The
> results show the same performance trends although the absolute values have
> skewed. Will get back to this after I digest the results a bit more.
Attached some incomplete files previously. Corrected.
Liviu
Re: Fwd: Re: STDCXX-1071 numpunct facet defect
Posted by Liviu Nicoara <ni...@hates.ms>.
On 10/3/12 11:10 AM, Martin Sebor wrote:
> On 10/03/2012 07:01 AM, Liviu Nicoara wrote:
>>
>> I am gathering some more measurements along these lines but it's time
>> consuming. I estimate I will have some ready for review later today or
>> tomorrow. In the meantime could you please post your kernel, glibc and
>> compiler versions?
>
> I was just thinking of a few simple loops along the lines of:
>
> void* thread_func (void*) {
> for (int i = 0; i < N; ++)
> test 1: do some simple stuff inline
> test 2: call a virtual function to do the same stuff
> test 3: lock and unlock a mutex and do the same stuff
> }
>
> Test 1 should be the fastest and test 3 the slowest. This should
> hold regardless of what "simple stuff" is (eventually, even when
> it's getting numpunct::grouping() data).
That is expected; I attached test case x.cpp and results-x.txt.
I did not find it too interesting in its own, though. The difference between the
cached and non-cached data is that in the case of the cached data the copying of
the string involves nothing more than a bump in the reference counter, whereas
in the non-cached version a string object is constructed anew, and memory gets
allocated for its body. Yet, in my measurements, the cached version is the one
which shows the worse performance.
So, I extracted the std::string class and simplified it down and put it in
another test case. That would be u.cpp and the results are results-u.txt. The
results show the same performance trends although the absolute values have
skewed. Will get back to this after I digest the results a bit more.
Liviu
Re: Fwd: Re: STDCXX-1071 numpunct facet defect
Posted by Liviu Nicoara <ni...@hates.ms>.
On 10/26/12 18:50, Martin Sebor wrote:
> On 10/26/2012 06:50 AM, Liviu Nicoara wrote:
>> [...]
>> tl;dr: removing the facet data cache is a priority. All else can be put
>> on the back-burner.
>> [...]
>> // facet data accessor
>> ...
>> if (0 == _C_impsize) { // 1
>> mutex_lock ();
>> if (_C_impsize)
>> return _C_data;
>> _C_data = get_facet_data (); // 2
>> ?? // 3
>> _C_impsize = 1; // 4
>> mutex_unlock ();
>> }
>> ?? // 5
>> return _C_data; // 6
>> [...]
>> Various compilers provide these features in various forms, but at the
>> moment we don't have a unified STDCXX API to implement this.
>>
>
> [...]
> I suggested moving the body of the outer if from the header
> into a .cpp file in the library where we could implement
> ugly, bloated locking without the risk of breaking things
> if we removed/replaced it in the future. That's when we ran
> into questions about how exactly to do this cleanly, etc.
> It didn't seem to be doable very cleanly but I still think
> it's a viable approach.
Just making sure we are talking about the same thing. My argument (and the code oversimplification above) was about the initialization of the std::facet data, not the cached numpunct data. I think we should get rid of the numpunct cache right away because it makes the facet unusable in MT builds.
As for the DCII I was talking about in the previous post, I would tackle it but I only have access to x86(_64). Does the foundation have shared dev machines?
Liviu
Re: Fwd: Re: STDCXX-1071 numpunct facet defect
Posted by Martin Sebor <ms...@gmail.com>.
On 10/26/2012 06:50 AM, Liviu Nicoara wrote:
> On 10/03/12 11:10, Martin Sebor wrote:
>> [...]
>> I was just thinking of a few simple loops along the lines of:
>>
>> void* thread_func (void*) {
>> for (int i = 0; i < N; ++)
>> test 1: do some simple stuff inline
>> test 2: call a virtual function to do the same stuff
>> test 3: lock and unlock a mutex and do the same stuff
>> }
>>
>> Test 1 should be the fastest and test 3 the slowest. This should
>> hold regardless of what "simple stuff" is (eventually, even when
>> it's getting numpunct::grouping() data).
>
> tl;dr: removing the facet data cache is a priority. All else can be put
> on the back-burner.
>
> Conflicting test results aside, there still is the case of the incorrect
> handling of the cached data in the facet. I don't think there is a
> disagreement on that. Considering that the std::string is moving in the
> direction of dropping the handle-body implementation, simply getting rid
> of the cache is a step in the same direction.
>
> I think that we should preserve the lock-free reading of the facet data,
> as a benign race, but making it benign is perhaps more complicated than
> previously suggested.
>
> As a reminder, the core of the facet access and initialization code
> essentially looks like this (pseudocode-ish):
>
>
> // facet data accessor
> ...
> if (0 == _C_impsize) { // 1
> mutex_lock ();
> if (_C_impsize)
> return _C_data;
> _C_data = get_facet_data (); // 2
> ?? // 3
> _C_impsize = 1; // 4
> mutex_unlock ();
> }
> ?? // 5
> return _C_data; // 6
> ...
>
>
> with question marks for missing, necessary fixes. The compiler needs to
> be prevented from re-ordering both 2-4 and 1-6. Just for the sake of
> argument I can imagine an optimization that reorders the reads in 1-6:
>
> register x = _C_data;
> if (_C_impsize)
> return x;
>
> and if the loads are executed in this order, the caller will see a stale
> _C_data.
>
> First, the 2-4 writes need to be executed in the program order. This
> needs both a compiler barrier and a store-store memory barrier that will
> keep the writes ordered.
>
> Then, the reads in 1-6 need to be ordered such that _C_data is read
> after _C_impsize, via a compiler barrier and a load-load memory barrier
> that will preserve the program order of the loads.
>
> Various compilers provide these features in various forms, but at the
> moment we don't have a unified STDCXX API to implement this.
>
> Of course, I might be wrong. Input is appreciated.
You're right (we don't have an API for it, and it is the double
checked locking problem mentioned earlier on in this thread).
Implementing it here would bloat the function (just in terms
of source code, not necessarily object code), and potentially
introduce a binary compatibility dependency on the new code
(i.e., we wouldn't be able to remove the fix in the future
w/o braking it).
I suggested moving the body of the outer if from the header
into a .cpp file in the library where we could implement
ugly, bloated locking without the risk of breaking things
if we removed/replaced it in the future. That's when we ran
into questions about how exactly to do this cleanly, etc.
It didn't seem to be doable very cleanly but I still think
it's a viable approach.
Martin
>
> Thanks,
> Liviu
>
Re: Fwd: Re: STDCXX-1071 numpunct facet defect
Posted by Liviu Nicoara <ni...@hates.ms>.
On 10/03/12 11:10, Martin Sebor wrote:
> [...]
> I was just thinking of a few simple loops along the lines of:
>
> void* thread_func (void*) {
> for (int i = 0; i < N; ++)
> test 1: do some simple stuff inline
> test 2: call a virtual function to do the same stuff
> test 3: lock and unlock a mutex and do the same stuff
> }
>
> Test 1 should be the fastest and test 3 the slowest. This should
> hold regardless of what "simple stuff" is (eventually, even when
> it's getting numpunct::grouping() data).
tl;dr: removing the facet data cache is a priority. All else can be put
on the back-burner.
Conflicting test results aside, there still is the case of the incorrect
handling of the cached data in the facet. I don't think there is a
disagreement on that. Considering that the std::string is moving in the
direction of dropping the handle-body implementation, simply getting rid
of the cache is a step in the same direction.
I think that we should preserve the lock-free reading of the facet data,
as a benign race, but making it benign is perhaps more complicated than
previously suggested.
As a reminder, the core of the facet access and initialization code
essentially looks like this (pseudocode-ish):
// facet data accessor
...
if (0 == _C_impsize) { // 1
mutex_lock ();
if (_C_impsize)
return _C_data;
_C_data = get_facet_data (); // 2
?? // 3
_C_impsize = 1; // 4
mutex_unlock ();
}
?? // 5
return _C_data; // 6
...
with question marks for missing, necessary fixes. The compiler needs to
be prevented from re-ordering both 2-4 and 1-6. Just for the sake of
argument I can imagine an optimization that reorders the reads in 1-6:
register x = _C_data;
if (_C_impsize)
return x;
and if the loads are executed in this order, the caller will see a stale
_C_data.
First, the 2-4 writes need to be executed in the program order. This
needs both a compiler barrier and a store-store memory barrier that will
keep the writes ordered.
Then, the reads in 1-6 need to be ordered such that _C_data is read
after _C_impsize, via a compiler barrier and a load-load memory barrier
that will preserve the program order of the loads.
Various compilers provide these features in various forms, but at the
moment we don't have a unified STDCXX API to implement this.
Of course, I might be wrong. Input is appreciated.
Thanks,
Liviu
Re: Fwd: Re: STDCXX-1071 numpunct facet defect
Posted by Martin Sebor <ms...@gmail.com>.
On 10/03/2012 07:01 AM, Liviu Nicoara wrote:
> On 10/02/12 10:41, Martin Sebor wrote:
>> I haven't had time to look at this since my last email on
>> Sunday. I also forgot about the string mutex. I don't think
>> I'll have time to spend on this until later in the week.
>> Unless the disassembly reveals the smoking gun, I think we
>> might need to simplify the test to get to the bottom of the
>> differences in our measurements. (I.e., eliminate the library
>> and measure the runtime of a simple thread loop, with and
>> without locking.) We should also look at the GLIBC and
>> kernel versions on our systems, on the off chance that
>> there has been a change that could explain the discrepancy
>> between my numbers and yours. I suspect my system (RHEL 4.8)
>> is much older than yours (I don't remember now if you posted
>> your details).
>
> I am gathering some more measurements along these lines but it's time
> consuming. I estimate I will have some ready for review later today or
> tomorrow. In the meantime could you please post your kernel, glibc and
> compiler versions?
I was just thinking of a few simple loops along the lines of:
void* thread_func (void*) {
for (int i = 0; i < N; ++)
test 1: do some simple stuff inline
test 2: call a virtual function to do the same stuff
test 3: lock and unlock a mutex and do the same stuff
}
Test 1 should be the fastest and test 3 the slowest. This should
hold regardless of what "simple stuff" is (eventually, even when
it's getting numpunct::grouping() data).
For the Linux tests I used a 16 CPU (Xeon X5570 @ 3GHz) box with
RHEL 4.8 with 2.6.9-89.0.11.ELlargesmp, GLIBC version is 2.3.4,
and GCC 3.4.6.
Martin
>
> Liviu
>
>>
>> Martin
>>
>> On 10/02/2012 06:22 AM, Liviu Nicoara wrote:
>>> On 09/30/12 18:18, Martin Sebor wrote:
>>>> I see you did a 64-bit build while I did a 32-bit one. so
>>>> I tried 64-bits. The cached version (i.e., the one compiled
>>>> with -UNO_USE_NUMPUNCT_CACHE) is still about twice as fast
>>>> as the non-cached one (compiled with -DNO_USE_NUMPUNCT_CACHE).
>>>>
>>>> I had made one change to the test program that I thought might
>>>> account for the difference: I removed the call to abort from
>>>> the thread function since it was causing the process to exit
>>>> prematurely in some of my tests. But since you used the
>>>> modified program for your latest measurements that couldn't
>>>> be it.
>>>>
>>>> I can't explain the differences. They just don't make sense
>>>> to me. Your results should be the other way around. Can you
>>>> post the disassembly of function f() for each of the two
>>>> configurations of the test?
>>>
>>> The first thing that struck me in the cached `f' was that __string_ref
>>> class uses a mutex for synchronizing access to the ref counter. It turns
>>> out, for Linux on AMD64 we explicitly use a mutex instead of the atomic
>>> ops on the ref counter, via a block in rw/_config.h:
>>>
>>> # if _RWSTD_VER_MAJOR < 5
>>> # ifdef _RWSTD_OS_LINUX
>>> // on Linux/AMD64, unless explicitly requested, disable the use
>>> // of atomic operations in string for binary compatibility with
>>> // stdcxx 4.1.x
>>> # ifndef _RWSTD_USE_STRING_ATOMIC_OPS
>>> # define _RWSTD_NO_STRING_ATOMIC_OPS
>>> # endif // _RWSTD_USE_STRING_ATOMIC_OPS
>>> # endif // _WIN32
>>> # endif // stdcxx < 5.0
>>>
>>>
>>> That is not the cause for the performance difference, though. Even after
>>> building with __RWSTD_USE_STRING_ATOMIC_OPS I get the same better
>>> performance with the non-cached version.
>>>
>>> Liviu
>
Re: Fwd: Re: STDCXX-1071 numpunct facet defect
Posted by Liviu Nicoara <ni...@hates.ms>.
On 10/02/12 10:41, Martin Sebor wrote:
> I haven't had time to look at this since my last email on
> Sunday. I also forgot about the string mutex. I don't think
> I'll have time to spend on this until later in the week.
> Unless the disassembly reveals the smoking gun, I think we
> might need to simplify the test to get to the bottom of the
> differences in our measurements. (I.e., eliminate the library
> and measure the runtime of a simple thread loop, with and
> without locking.) We should also look at the GLIBC and
> kernel versions on our systems, on the off chance that
> there has been a change that could explain the discrepancy
> between my numbers and yours. I suspect my system (RHEL 4.8)
> is much older than yours (I don't remember now if you posted
> your details).
I am gathering some more measurements along these lines but it's time
consuming. I estimate I will have some ready for review later today or
tomorrow. In the meantime could you please post your kernel, glibc and
compiler versions?
Liviu
>
> Martin
>
> On 10/02/2012 06:22 AM, Liviu Nicoara wrote:
>> On 09/30/12 18:18, Martin Sebor wrote:
>>> I see you did a 64-bit build while I did a 32-bit one. so
>>> I tried 64-bits. The cached version (i.e., the one compiled
>>> with -UNO_USE_NUMPUNCT_CACHE) is still about twice as fast
>>> as the non-cached one (compiled with -DNO_USE_NUMPUNCT_CACHE).
>>>
>>> I had made one change to the test program that I thought might
>>> account for the difference: I removed the call to abort from
>>> the thread function since it was causing the process to exit
>>> prematurely in some of my tests. But since you used the
>>> modified program for your latest measurements that couldn't
>>> be it.
>>>
>>> I can't explain the differences. They just don't make sense
>>> to me. Your results should be the other way around. Can you
>>> post the disassembly of function f() for each of the two
>>> configurations of the test?
>>
>> The first thing that struck me in the cached `f' was that __string_ref
>> class uses a mutex for synchronizing access to the ref counter. It turns
>> out, for Linux on AMD64 we explicitly use a mutex instead of the atomic
>> ops on the ref counter, via a block in rw/_config.h:
>>
>> # if _RWSTD_VER_MAJOR < 5
>> # ifdef _RWSTD_OS_LINUX
>> // on Linux/AMD64, unless explicitly requested, disable the use
>> // of atomic operations in string for binary compatibility with
>> // stdcxx 4.1.x
>> # ifndef _RWSTD_USE_STRING_ATOMIC_OPS
>> # define _RWSTD_NO_STRING_ATOMIC_OPS
>> # endif // _RWSTD_USE_STRING_ATOMIC_OPS
>> # endif // _WIN32
>> # endif // stdcxx < 5.0
>>
>>
>> That is not the cause for the performance difference, though. Even after
>> building with __RWSTD_USE_STRING_ATOMIC_OPS I get the same better
>> performance with the non-cached version.
>>
>> Liviu
Re: Fwd: Re: STDCXX-1071 numpunct facet defect
Posted by Martin Sebor <ms...@gmail.com>.
I haven't had time to look at this since my last email on
Sunday. I also forgot about the string mutex. I don't think
I'll have time to spend on this until later in the week.
Unless the disassembly reveals the smoking gun, I think we
might need to simplify the test to get to the bottom of the
differences in our measurements. (I.e., eliminate the library
and measure the runtime of a simple thread loop, with and
without locking.) We should also look at the GLIBC and
kernel versions on our systems, on the off chance that
there has been a change that could explain the discrepancy
between my numbers and yours. I suspect my system (RHEL 4.8)
is much older than yours (I don't remember now if you posted
your details).
Martin
On 10/02/2012 06:22 AM, Liviu Nicoara wrote:
> On 09/30/12 18:18, Martin Sebor wrote:
>> I see you did a 64-bit build while I did a 32-bit one. so
>> I tried 64-bits. The cached version (i.e., the one compiled
>> with -UNO_USE_NUMPUNCT_CACHE) is still about twice as fast
>> as the non-cached one (compiled with -DNO_USE_NUMPUNCT_CACHE).
>>
>> I had made one change to the test program that I thought might
>> account for the difference: I removed the call to abort from
>> the thread function since it was causing the process to exit
>> prematurely in some of my tests. But since you used the
>> modified program for your latest measurements that couldn't
>> be it.
>>
>> I can't explain the differences. They just don't make sense
>> to me. Your results should be the other way around. Can you
>> post the disassembly of function f() for each of the two
>> configurations of the test?
>
> The first thing that struck me in the cached `f' was that __string_ref
> class uses a mutex for synchronizing access to the ref counter. It turns
> out, for Linux on AMD64 we explicitly use a mutex instead of the atomic
> ops on the ref counter, via a block in rw/_config.h:
>
> # if _RWSTD_VER_MAJOR < 5
> # ifdef _RWSTD_OS_LINUX
> // on Linux/AMD64, unless explicitly requested, disable the use
> // of atomic operations in string for binary compatibility with
> // stdcxx 4.1.x
> # ifndef _RWSTD_USE_STRING_ATOMIC_OPS
> # define _RWSTD_NO_STRING_ATOMIC_OPS
> # endif // _RWSTD_USE_STRING_ATOMIC_OPS
> # endif // _WIN32
> # endif // stdcxx < 5.0
>
>
> That is not the cause for the performance difference, though. Even after
> building with __RWSTD_USE_STRING_ATOMIC_OPS I get the same better
> performance with the non-cached version.
>
> Liviu
Re: Fwd: Re: STDCXX-1071 numpunct facet defect
Posted by Liviu Nicoara <ni...@hates.ms>.
On 09/30/12 18:18, Martin Sebor wrote:
> I see you did a 64-bit build while I did a 32-bit one. so
> I tried 64-bits. The cached version (i.e., the one compiled
> with -UNO_USE_NUMPUNCT_CACHE) is still about twice as fast
> as the non-cached one (compiled with -DNO_USE_NUMPUNCT_CACHE).
>
> I had made one change to the test program that I thought might
> account for the difference: I removed the call to abort from
> the thread function since it was causing the process to exit
> prematurely in some of my tests. But since you used the
> modified program for your latest measurements that couldn't
> be it.
>
> I can't explain the differences. They just don't make sense
> to me. Your results should be the other way around. Can you
> post the disassembly of function f() for each of the two
> configurations of the test?
The first thing that struck me in the cached `f' was that __string_ref
class uses a mutex for synchronizing access to the ref counter. It turns
out, for Linux on AMD64 we explicitly use a mutex instead of the atomic
ops on the ref counter, via a block in rw/_config.h:
# if _RWSTD_VER_MAJOR < 5
# ifdef _RWSTD_OS_LINUX
// on Linux/AMD64, unless explicitly requested, disable the use
// of atomic operations in string for binary compatibility with
// stdcxx 4.1.x
# ifndef _RWSTD_USE_STRING_ATOMIC_OPS
# define _RWSTD_NO_STRING_ATOMIC_OPS
# endif // _RWSTD_USE_STRING_ATOMIC_OPS
# endif // _WIN32
# endif // stdcxx < 5.0
That is not the cause for the performance difference, though. Even after
building with __RWSTD_USE_STRING_ATOMIC_OPS I get the same better
performance with the non-cached version.
Liviu