You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stdcxx.apache.org by Martin Sebor <se...@roguewave.com> on 2008/04/22 20:38:46 UTC
Re: 22.locale.codecvt.length on AIX
Martin Sebor wrote:
> Travis Vitek wrote:
>> These tests may have binary compatibility issues...
>>
>> 22.locale.codecvt.length
>
> I see some unexpected output with this test when I compile
> it with XLC++ 9.0 in 15d.
Even more strange, it looks like the test passes in nightly
builds on AIX...
> First, note the "Error: bad_alloc:
> out of memory" I think it might actually be coming out of
> the localedef utility. But then the test seems to hang:
>
> $ TOPDIR=~/stdcxx-4.2.x ./22.locale.codecvt.length
> # INFO (S1) (10 lines):
> # TEXT:
> # COMPILER: IBM VisualAge C++, __IBMCPP__ = 900
> # ENVIRONMENT: powerpc running aix-5.3
> # FILE: 22.locale.codecvt.length.cpp
> # COMPILED: Apr 22 2008, 12:15:12
> # COMMENT:
> ############################################################
>
> # CLAUSE: lib.locale.codecvt.virtuals
>
> # NOTE (S2) (5 lines):
> # TEXT: executing "rm -rf /tmp/tmpfile-elUaaa"
> # CLAUSE: lib.locale.codecvt.virtuals
> # FILE: process.cpp
> # LINE: 276
>
> # NOTE (S2) (5 lines):
> # TEXT: executing "mkdir /tmp/tmpfile-elUaaa"
> # CLAUSE: lib.locale.codecvt.virtuals
> # FILE: process.cpp
> # LINE: 276
>
> # INFO (S1) (3 lines):
> # TEXT: std::codecvt<char, char, mbstate_t>::length (state_type&, const
> extern_type*, const extern_type*, size_t)
> # CLAUSE: lib.locale.codecvt.virtuals
>
> # INFO (S1) (3 lines):
> # TEXT: std::codecvt_byname<char, char, mbstate_t>::length (state_type&,
> const extern_type*, const extern_type*, size_t)
> # CLAUSE: lib.locale.codecvt.virtuals
>
> # INFO (S1) (3 lines):
> # TEXT: std::codecvt<wchar_t, char, mbstate_t>::length (state_type&,
> const extern_type*, const extern_type*, size_t)
> # CLAUSE: lib.locale.codecvt.virtuals
>
> # INFO (S1) (3 lines):
> # TEXT: std::codecvt_byname<wchar_t, char, mbstate_t>::length
> (state_type&, const extern_type*, const extern_type*, size_t)
> # CLAUSE: lib.locale.codecvt.virtuals
>
> # INFO (S1) (3 lines):
> # TEXT: locale ("UTF-8@UCS") [algorithmic encoding]
> # CLAUSE: lib.locale.codecvt.virtuals
>
> # NOTE (S2) (5 lines):
> # TEXT: executing "../bin/localedef -w701 -w702 -w706 -c -f
> /tmp/tmpfile-elUaaa/charmap -i /tmp/tmpfile-elUaaa/source
> /tmp/tmpfile-elUaaa/mb_cur_max-9"
> # CLAUSE: lib.locale.codecvt.virtuals
> # FILE: process.cpp
> # LINE: 276
>
> Error: bad_alloc: out of memory
> # ERROR (S8) (5 lines):
> # TEXT: the command "../bin/localedef -w701 -w702 -w706 -c -f
> /tmp/tmpfile-elUaaa/charmap -i /tmp/tmpfile-elUaaa/source
> /tmp/tmpfile-elUaaa/mb_cur_max-9" exited with status 4
> # CLAUSE: lib.locale.codecvt.virtuals
> # FILE: process.cpp
> # LINE: 306
>
> # ERROR (S8) (4 lines):
> # TEXT: failed to create a locale database
> # CLAUSE: lib.locale.codecvt.virtuals
> # LINE: 368
>
> # NOTE (S2) (5 lines):
> # TEXT: executing "locale -a > /tmp/tmpfile-ftUaab"
> # CLAUSE: lib.locale.codecvt.virtuals
> # FILE: process.cpp
> # LINE: 276
>
>
> When I run it in the debugger, I don't get the bad_alloc and
> the test runs to completion with this output:
>
> # NOTE (S2) (5 lines):
> # TEXT: failed to find 3-byte characters in locale "Zh_CN" with
> MB_CUR_MAX = 4
> # CLAUSE: lib.locale.codecvt.virtuals
> # FILE: locale.cpp
> # LINE: 631
>
> # NOTE (S2) (5 lines):
> # TEXT: failed to find 3-byte characters in locale "Zh_CN.GB18030" with
> MB_CUR_MAX = 4
> # CLAUSE: lib.locale.codecvt.virtuals
> # FILE: locale.cpp
> # LINE: 631
>
> # NOTE (S2) (5 lines):
> # TEXT: failed to find 3-byte characters in locale "zh_TW" with
> MB_CUR_MAX = 4
> # CLAUSE: lib.locale.codecvt.virtuals
> # FILE: locale.cpp
> # LINE: 631
>
> # NOTE (S2) (5 lines):
> # TEXT: failed to find 3-byte characters in locale "zh_TW.IBM-eucTW"
> with MB_CUR_MAX = 4
> # CLAUSE: lib.locale.codecvt.virtuals
> # FILE: locale.cpp
> # LINE: 631
>
> # INFO (S1) (3 lines):
> # TEXT: locale ("EN_GB.UTF-8@euro") [libc-based encoding, single-byte
> characters]
> # CLAUSE: lib.locale.codecvt.virtuals
>
> # INFO (S1) (3 lines):
> # TEXT: locale ("EN_GB.UTF-8@euro") [libc-based encoding, MB_CUR_MAX =
> 3, multi-byte characters: "\x01", "\xc2\x80", "\xe0\xa0\x80"]
> # CLAUSE: lib.locale.codecvt.virtuals
>
> # +-----------------------+----------+----------+----------+
> # | DIAGNOSTIC | ACTIVE | TOTAL | INACTIVE |
> # +-----------------------+----------+----------+----------+
> # | (S1) INFO | 9 | 9 | 0% |
> # | (S2) NOTE | 8 | 612 | 98% |
> # | (S5) WARNING | 0 | 2 | 100% |
> # | (S7) ASSERTION | 0 | 382 | 100% |
> # | (S8) ERROR | 0 | 5 | 100% |
> # +-----------------------+----------+----------+----------+
> /amd/devco/sebor/stdcxx-4.2.x/tests/src/driver.cpp:1528: rw_note():
> warning: test finished, cannot call
>
> execution completed
>
>
>> 22.locale.ctype.scan
>> 22.locale.num.put
>>
>> I'm also concerned with this test because it does not fail in nightly
>> builds...
>> 22.locale.messages.mt
>>
>> Travis
>>
>>
>> aix-5.3-vacpp-8.0 SRC: 4.2.0 4.2.0 4.2.1 4.2.1
>> LIB: 4.2.0 4.2.1 4.2.0 4.2.1
>> 0.printf SEGV SEGV A1 A1
>> 0.process SEGV SEGV 0 0
>> 18.setjmp HUP HUP A1 A1
>> 18.support.rtti N/A N/A A7 A7
>> 21.cwchar A11 A11 A5 A5
>> 21.string.assign A360 A360 A60 A60
>> 21.string.insert A840 A840 A180 A180
>> 21.string.replace A1060 A1060 A240 A240
>> 22.locale.codecvt.length 0 ABRT 0 0
>> 22.locale.codecvt.out N/A N/A SEGV SEGV
>> 22.locale.codecvt.stdcxx-435 N/A N/A ABRT NOUT
>> 22.locale.ctype.scan HUP ABRT 0 0
>> 22.locale.messages HUP A106 HUP 0
>> 22.locale.messages.mt SEGV SEGV SEGV SEGV
>> 22.locale.money.get A20 A20 A20 A20
>> 22.locale.num.get A96 A96 A96 A96
>> 22.locale.num.put 0 A24 A26 0
>> 22.locale.num.put.stdcxx-2 N/A N/A ABRT NOUT
>> 22.locale.time.get A46 A46 SEGV SEGV
>> 23.deque.special A1536 A1536 0 0
>> 23.vector.bool.stdcxx-235 ABRT ABRT NOUT NOUT
>> 25.heap A31 A31 0 0
>> 25.partial.sort A2603 A2603 0 0
>> 25.random.shuffle A1 A1 0 0
>> 26.class.gslice A162 A162 A162 A162
>> 26.valarray.cassign 0 0 COMP COMP
>> 27.basic.ios.copyfmt.stdcxx-766 N/A N/A ABRT NOUT
>> 27.ostream A2 A2 0 0
>> 8.cstdint A3 A3 0 0
>>
>
Re: 22.locale.codecvt.length on AIX
Posted by Martin Sebor <se...@roguewave.com>.
Btw., the test attempts to create a locale from a generated charmap
file. The charmap file defines mb_cur_max to be 9 which is pretty
much guaranteed to exceed the value of MB_LEN_MAX (on AIX it's 4).
Unless the localedef utility is careful about buffer overflow it
might easily corrupt memory. I just corrected one potential problem
in this area pointed out in STDCXX-753 but I'm not sure that was
the only one. One way confirm the hypothesis that the excessively
long multibyte characters are causing trouble on AIX would be to
reduce their length and re-run the test.
http://issues.apache.org/jira/browse/STDCXX-753
Martin Sebor wrote:
> Martin Sebor wrote:
>> Travis Vitek wrote:
>>> These tests may have binary compatibility issues...
>>>
>>> 22.locale.codecvt.length
>>
>> I see some unexpected output with this test when I compile
>> it with XLC++ 9.0 in 15d.
>
> Even more strange, it looks like the test passes in nightly
> builds on AIX...
>
>> First, note the "Error: bad_alloc:
>> out of memory" I think it might actually be coming out of
>> the localedef utility. But then the test seems to hang:
>>
>> $ TOPDIR=~/stdcxx-4.2.x ./22.locale.codecvt.length
>> # INFO (S1) (10 lines):
>> # TEXT:
>> # COMPILER: IBM VisualAge C++, __IBMCPP__ = 900
>> # ENVIRONMENT: powerpc running aix-5.3
>> # FILE: 22.locale.codecvt.length.cpp
>> # COMPILED: Apr 22 2008, 12:15:12
>> # COMMENT:
>> ############################################################
>>
>> # CLAUSE: lib.locale.codecvt.virtuals
>>
>> # NOTE (S2) (5 lines):
>> # TEXT: executing "rm -rf /tmp/tmpfile-elUaaa"
>> # CLAUSE: lib.locale.codecvt.virtuals
>> # FILE: process.cpp
>> # LINE: 276
>>
>> # NOTE (S2) (5 lines):
>> # TEXT: executing "mkdir /tmp/tmpfile-elUaaa"
>> # CLAUSE: lib.locale.codecvt.virtuals
>> # FILE: process.cpp
>> # LINE: 276
>>
>> # INFO (S1) (3 lines):
>> # TEXT: std::codecvt<char, char, mbstate_t>::length (state_type&,
>> const extern_type*, const extern_type*, size_t)
>> # CLAUSE: lib.locale.codecvt.virtuals
>>
>> # INFO (S1) (3 lines):
>> # TEXT: std::codecvt_byname<char, char, mbstate_t>::length
>> (state_type&, const extern_type*, const extern_type*, size_t)
>> # CLAUSE: lib.locale.codecvt.virtuals
>>
>> # INFO (S1) (3 lines):
>> # TEXT: std::codecvt<wchar_t, char, mbstate_t>::length (state_type&,
>> const extern_type*, const extern_type*, size_t)
>> # CLAUSE: lib.locale.codecvt.virtuals
>>
>> # INFO (S1) (3 lines):
>> # TEXT: std::codecvt_byname<wchar_t, char, mbstate_t>::length
>> (state_type&, const extern_type*, const extern_type*, size_t)
>> # CLAUSE: lib.locale.codecvt.virtuals
>>
>> # INFO (S1) (3 lines):
>> # TEXT: locale ("UTF-8@UCS") [algorithmic encoding]
>> # CLAUSE: lib.locale.codecvt.virtuals
>>
>> # NOTE (S2) (5 lines):
>> # TEXT: executing "../bin/localedef -w701 -w702 -w706 -c -f
>> /tmp/tmpfile-elUaaa/charmap -i /tmp/tmpfile-elUaaa/source
>> /tmp/tmpfile-elUaaa/mb_cur_max-9"
>> # CLAUSE: lib.locale.codecvt.virtuals
>> # FILE: process.cpp
>> # LINE: 276
>>
>> Error: bad_alloc: out of memory
>> # ERROR (S8) (5 lines):
>> # TEXT: the command "../bin/localedef -w701 -w702 -w706 -c -f
>> /tmp/tmpfile-elUaaa/charmap -i /tmp/tmpfile-elUaaa/source
>> /tmp/tmpfile-elUaaa/mb_cur_max-9" exited with status 4
>> # CLAUSE: lib.locale.codecvt.virtuals
>> # FILE: process.cpp
>> # LINE: 306
>>
>> # ERROR (S8) (4 lines):
>> # TEXT: failed to create a locale database
>> # CLAUSE: lib.locale.codecvt.virtuals
>> # LINE: 368
>>
>> # NOTE (S2) (5 lines):
>> # TEXT: executing "locale -a > /tmp/tmpfile-ftUaab"
>> # CLAUSE: lib.locale.codecvt.virtuals
>> # FILE: process.cpp
>> # LINE: 276
>>
>>
>> When I run it in the debugger, I don't get the bad_alloc and
>> the test runs to completion with this output:
>>
>> # NOTE (S2) (5 lines):
>> # TEXT: failed to find 3-byte characters in locale "Zh_CN" with
>> MB_CUR_MAX = 4
>> # CLAUSE: lib.locale.codecvt.virtuals
>> # FILE: locale.cpp
>> # LINE: 631
>>
>> # NOTE (S2) (5 lines):
>> # TEXT: failed to find 3-byte characters in locale "Zh_CN.GB18030"
>> with MB_CUR_MAX = 4
>> # CLAUSE: lib.locale.codecvt.virtuals
>> # FILE: locale.cpp
>> # LINE: 631
>>
>> # NOTE (S2) (5 lines):
>> # TEXT: failed to find 3-byte characters in locale "zh_TW" with
>> MB_CUR_MAX = 4
>> # CLAUSE: lib.locale.codecvt.virtuals
>> # FILE: locale.cpp
>> # LINE: 631
>>
>> # NOTE (S2) (5 lines):
>> # TEXT: failed to find 3-byte characters in locale "zh_TW.IBM-eucTW"
>> with MB_CUR_MAX = 4
>> # CLAUSE: lib.locale.codecvt.virtuals
>> # FILE: locale.cpp
>> # LINE: 631
>>
>> # INFO (S1) (3 lines):
>> # TEXT: locale ("EN_GB.UTF-8@euro") [libc-based encoding, single-byte
>> characters]
>> # CLAUSE: lib.locale.codecvt.virtuals
>>
>> # INFO (S1) (3 lines):
>> # TEXT: locale ("EN_GB.UTF-8@euro") [libc-based encoding, MB_CUR_MAX =
>> 3, multi-byte characters: "\x01", "\xc2\x80", "\xe0\xa0\x80"]
>> # CLAUSE: lib.locale.codecvt.virtuals
>>
>> # +-----------------------+----------+----------+----------+
>> # | DIAGNOSTIC | ACTIVE | TOTAL | INACTIVE |
>> # +-----------------------+----------+----------+----------+
>> # | (S1) INFO | 9 | 9 | 0% |
>> # | (S2) NOTE | 8 | 612 | 98% |
>> # | (S5) WARNING | 0 | 2 | 100% |
>> # | (S7) ASSERTION | 0 | 382 | 100% |
>> # | (S8) ERROR | 0 | 5 | 100% |
>> # +-----------------------+----------+----------+----------+
>> /amd/devco/sebor/stdcxx-4.2.x/tests/src/driver.cpp:1528: rw_note():
>> warning: test finished, cannot call
>>
>> execution completed
>>
>>
>>> 22.locale.ctype.scan
>>> 22.locale.num.put
>>>
>>> I'm also concerned with this test because it does not fail in nightly
>>> builds...
>>> 22.locale.messages.mt
>>>
>>> Travis
>>>
>>>
>>> aix-5.3-vacpp-8.0 SRC: 4.2.0 4.2.0 4.2.1 4.2.1
>>> LIB: 4.2.0 4.2.1 4.2.0 4.2.1
>>> 0.printf SEGV SEGV A1 A1
>>> 0.process SEGV SEGV 0 0
>>> 18.setjmp HUP HUP A1 A1
>>> 18.support.rtti N/A N/A A7 A7
>>> 21.cwchar A11 A11 A5 A5
>>> 21.string.assign A360 A360 A60 A60
>>> 21.string.insert A840 A840 A180 A180
>>> 21.string.replace A1060 A1060 A240 A240
>>> 22.locale.codecvt.length 0 ABRT 0 0
>>> 22.locale.codecvt.out N/A N/A SEGV SEGV
>>> 22.locale.codecvt.stdcxx-435 N/A N/A ABRT NOUT
>>> 22.locale.ctype.scan HUP ABRT 0 0
>>> 22.locale.messages HUP A106 HUP 0
>>> 22.locale.messages.mt SEGV SEGV SEGV SEGV
>>> 22.locale.money.get A20 A20 A20 A20
>>> 22.locale.num.get A96 A96 A96 A96
>>> 22.locale.num.put 0 A24 A26 0
>>> 22.locale.num.put.stdcxx-2 N/A N/A ABRT NOUT
>>> 22.locale.time.get A46 A46 SEGV SEGV
>>> 23.deque.special A1536 A1536 0 0
>>> 23.vector.bool.stdcxx-235 ABRT ABRT NOUT NOUT
>>> 25.heap A31 A31 0 0
>>> 25.partial.sort A2603 A2603 0 0
>>> 25.random.shuffle A1 A1 0 0
>>> 26.class.gslice A162 A162 A162 A162
>>> 26.valarray.cassign 0 0 COMP COMP
>>> 27.basic.ios.copyfmt.stdcxx-766 N/A N/A ABRT NOUT
>>> 27.ostream A2 A2 0 0
>>> 8.cstdint A3 A3 0 0
>>>
>>
>