You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Gregory Shimansky <gs...@gmail.com> on 2007/04/17 20:23:22 UTC
Re: [jira] Commented: (HARMONY-2669) [classlib][charsets] Working with charset ISO-2022-CN crashes DRLVM
Hello
Below is my evaluation of the bug HARMONY-2669. I think it is in ICU4JNI
native code. The possible solution for it is to recompile ICUInterface34
libraries from patched sources and file a bug on ICU meanwhile.
I would like to know, what sources were used to compile ICU4JNI for windows
and other operation systems? I found today that ICU4JNI 3.4 contains quite a
lot of memory leaks, and there is a patch to fix them in v3.4 [1]. Was it
used to compile our binaries (from what I learned while analyzing this bug on
windows 32 it *was* used)? I know that I didn't use it when I compiled ICU
for IPF because I learned about its existence only today. Are there any other
patches for ICU code that we need to use for version 3.4?
[1]
ftp://ftp.software.ibm.com/software/globalization/icu/icu4j/icu4jni/3.4/icu4jni_3_4_patch-01.zip
On Tuesday 17 April 2007 20:22 Gregory Shimansky (JIRA) wrote:
> [
> https://issues.apache.org/jira/browse/HARMONY-2669?page=com.atlassian.jira.
>plugin.system.issuetabpanels:comment-tabpanel#action_12489454 ]
>
> Gregory Shimansky commented on HARMONY-2669:
> --------------------------------------------
>
> The bug appears to be in this code of
> Java_com_ibm_icu4jni_converters_NativeConverter_setSubstitutionBytes (see
> icu4jni, the patched version for memory leaks):
>
> jbyte* u_subChars =
> (*env)->GetPrimitiveArrayCritical(env,subChars,NULL); if(u_subChars){
> char* mySubChars= (char*)malloc(sizeof(char)*length);
> toChars((UChar*)u_subChars,&mySubChars[0],length);
> ucnv_setSubstChars(cnv,mySubChars, (char)length,&errorCode);
> if(U_FAILURE(errorCode)){
> /* !!! */
> (*env)->ReleasePrimitiveArrayCritical(env,subChars,mySubChars /* Here
> should be u_subChars */ ,0); /* !!! */ return errorCode;
> }
> free(mySubChars);
> }
>
> ICU mistakenly passes a pointer to an array allocated in this function. But
> on windows in debug mode MSVC has some paranoiac checks that heap pointer
> allocated in one DLL (ICU) cannot be freed in another DLL (DRLVM). So when
> VM tries to free mySubChars it fails paranoiac MSVC check.
>
> I checked ICU 3.6 and the code is still the same. Probably we could use
> patched ICUInterface34.dll and report a bug on ICU. I am not sure how many
> other places in ICU share the same pattern.
>
> > [classlib][charsets] Working with charset ISO-2022-CN crashes DRLVM
> > -------------------------------------------------------------------
> >
> > Key: HARMONY-2669
> > URL: https://issues.apache.org/jira/browse/HARMONY-2669
> > Project: Harmony
> > Issue Type: Bug
> > Components: Classlib
> > Environment: Windows XP i686
> > Reporter: Salikh Zakirov
> > Assigned To: Gregory Shimansky
> > Attachments: Enc.java
> >
> >
> > The attached test crashes debug version of DRLVM on a heap pointer
> > validity check on free(). With the following stack:
> > harmonyvm.dll!_CrtIsValidHeapPointer(const void * pUserData=0x02876958)
> > Line 1807 C harmonyvm.dll!_free_dbg_lk(void * pUserData=0x02876958, int
> > nBlockUse=0x00000001) Line 1132 + 0x9 bytes C
> > harmonyvm.dll!_free_dbg(void * pUserData=0x02876958, int
> > nBlockUse=0x00000001) Line 1070 + 0xd bytes C harmonyvm.dll!free(void *
> > pUserData=0x02876958) Line 1025 + 0xb bytes C
> > harmonyvm.dll!ReleaseByteArrayElements(JNIEnv_External * env=0x011c2cc0,
> > _jobject * array=0x0013f40c, char * elems=0x02876958, int
> > mode=0x00000000) Line 908 + 0x9 bytes C++
> >
> > > harmonyvm.dll!ReleasePrimitiveArrayCritical(JNIEnv_External *
> > > jenv=0x011c2cc0, _jobject * array=0x0013f40c, void * carray=0x02876958,
> > > int mode=0x00000000) Line 1305 + 0x15 bytes C++
> >
> > ICUInterface34.dll!0285211e()
> > Running the same test with J9 produces
> > Exception in thread "main" java.lang.IllegalArgumentException:
> > U_ILLEGAL_ARGUMENT_ERROR at
> > com.ibm.icu4jni.common.ErrorCode.getException(Unknown Source) at
> > com.ibm.icu4jni.charset.CharsetEncoderICU.implReplaceWith(Unknown Source)
> > at java.nio.charset.CharsetEncoder.replaceWith(CharsetEncoder.java:856)
> > at Enc.main(Enc.java:9)
--
Gregory
Re: [jira] Commented: (HARMONY-2669) [classlib][charsets] Working with charset ISO-2022-CN crashes DRLVM
Posted by Tony Wu <wu...@gmail.com>.
Sorry for replying late.
I found another bug of icu4jni's native code too, as described below.
in src/native/converter/ConverterInterface.c
jint count =32;
UChar invalidUChars[32];
ucnv_getInvalidUChars(cnv,invalidUChars,(int8_t*)&count,&errorCode);
The cast from jint* to int8_t* will lead to some problem in big-endian
machine, usually get zero. It was fixed in 3.6 but no patch for 3.4.
As far I know, Richard is not feeling well these days. actually I'm
preparing for upgrading both icu4jni and icu4j to 3.6.
For icu4j, it is much easier, what we need to do is just replace the
original jar file.
It is a little complex with icu4jni36. the library name is
libInterface36 rather than libInterface34, IIRC, it was hardcoded in
vm. So we may need modify and rebuild vm.
On 4/19/07, Tim Ellison <t....@gmail.com> wrote:
> Gregory Shimansky wrote:
> > I think there was a suggestion to move to version 3.6 but I don't know
> > what the decision was when the discussion has finished. Could someone
> > remind me?
>
> I was waiting for Richard to respond, since IIRC he was the one that
> looked at it. I think he decided upgrading was the right thing to do
> (but after the milestone build please<g>).
>
> Regards,
> Tim
>
>
>
--
Tony Wu
China Software Development Lab, IBM
Re: [jira] Commented: (HARMONY-2669) [classlib][charsets] Working
with charset ISO-2022-CN crashes DRLVM
Posted by Tim Ellison <t....@gmail.com>.
Gregory Shimansky wrote:
> I think there was a suggestion to move to version 3.6 but I don't know
> what the decision was when the discussion has finished. Could someone
> remind me?
I was waiting for Richard to respond, since IIRC he was the one that
looked at it. I think he decided upgrading was the right thing to do
(but after the milestone build please<g>).
Regards,
Tim
Re: [jira] Commented: (HARMONY-2669) [classlib][charsets] Working
with charset ISO-2022-CN crashes DRLVM
Posted by Gregory Shimansky <gs...@gmail.com>.
Tim Ellison wrote:
> Gregory Shimansky wrote:
>> On Wednesday 18 April 2007 00:26 Tim Ellison wrote:
>>> Gregory Shimansky wrote:
>>>> Below is my evaluation of the bug HARMONY-2669. I think it is in ICU4JNI
>>>> native code. The possible solution for it is to recompile ICUInterface34
>>>> libraries from patched sources and file a bug on ICU meanwhile.
>>>>
>>>> I would like to know, what sources were used to compile ICU4JNI for
>>>> windows and other operation systems? I found today that ICU4JNI 3.4
>>>> contains quite a lot of memory leaks, and there is a patch to fix them in
>>>> v3.4 [1]. Was it used to compile our binaries (from what I learned while
>>>> analyzing this bug on windows 32 it *was* used)? I know that I didn't use
>>>> it when I compiled ICU for IPF because I learned about its existence only
>>>> today. Are there any other patches for ICU code that we need to use for
>>>> version 3.4?
>>>>
>>>> [1]
>>>> ftp://ftp.software.ibm.com/software/globalization/icu/icu4j/icu4jni/3.4/i
>>>> cu4jni_3_4_patch-01.zip
>>> Sorry that you had to find it the hard way Gregory. The ICU4JNI code
>>> does have the patch applied, as described in the readme in the depends'
>>> ICU4JNI directory[1].
>> I didn't find it a hard way, I just didn't look for any patches when I created
>> IPF binaries some time ago. I the memory leaks patch when I went to ICU site
>> to download its sources to see what's going on when it calls VM JNI code.
>> What I found out is that it calls VM JNI code with a wrong elems pointer and
>> to fix the bug we need to patch ICU. The bug is present it 3.6 as well.
>>
>> So my question was actually, do we need any patches other than memory leaks
>> patch to build ICU binaries that we have in SVN currently?
>
> No, the code is just 3.4 with patch 1.
>
> I'd have no objection to moving to 3.6, and asking the ICU team for an
> official patch for the elems bug. We could use a temporary patch while
> we wait.
I think there was a suggestion to move to version 3.6 but I don't know
what the decision was when the discussion has finished. Could someone
remind me?
--
Gregory
Re: [jira] Commented: (HARMONY-2669) [classlib][charsets] Working
with charset ISO-2022-CN crashes DRLVM
Posted by Tim Ellison <t....@gmail.com>.
Gregory Shimansky wrote:
> On Wednesday 18 April 2007 00:26 Tim Ellison wrote:
>> Gregory Shimansky wrote:
>>> Below is my evaluation of the bug HARMONY-2669. I think it is in ICU4JNI
>>> native code. The possible solution for it is to recompile ICUInterface34
>>> libraries from patched sources and file a bug on ICU meanwhile.
>>>
>>> I would like to know, what sources were used to compile ICU4JNI for
>>> windows and other operation systems? I found today that ICU4JNI 3.4
>>> contains quite a lot of memory leaks, and there is a patch to fix them in
>>> v3.4 [1]. Was it used to compile our binaries (from what I learned while
>>> analyzing this bug on windows 32 it *was* used)? I know that I didn't use
>>> it when I compiled ICU for IPF because I learned about its existence only
>>> today. Are there any other patches for ICU code that we need to use for
>>> version 3.4?
>>>
>>> [1]
>>> ftp://ftp.software.ibm.com/software/globalization/icu/icu4j/icu4jni/3.4/i
>>> cu4jni_3_4_patch-01.zip
>> Sorry that you had to find it the hard way Gregory. The ICU4JNI code
>> does have the patch applied, as described in the readme in the depends'
>> ICU4JNI directory[1].
>
> I didn't find it a hard way, I just didn't look for any patches when I created
> IPF binaries some time ago. I the memory leaks patch when I went to ICU site
> to download its sources to see what's going on when it calls VM JNI code.
> What I found out is that it calls VM JNI code with a wrong elems pointer and
> to fix the bug we need to patch ICU. The bug is present it 3.6 as well.
>
> So my question was actually, do we need any patches other than memory leaks
> patch to build ICU binaries that we have in SVN currently?
No, the code is just 3.4 with patch 1.
I'd have no objection to moving to 3.6, and asking the ICU team for an
official patch for the elems bug. We could use a temporary patch while
we wait.
Regards,
Tim
Re: [jira] Commented: (HARMONY-2669) [classlib][charsets] Working with charset ISO-2022-CN crashes DRLVM
Posted by Gregory Shimansky <gs...@gmail.com>.
On Wednesday 18 April 2007 00:26 Tim Ellison wrote:
> Gregory Shimansky wrote:
> > Below is my evaluation of the bug HARMONY-2669. I think it is in ICU4JNI
> > native code. The possible solution for it is to recompile ICUInterface34
> > libraries from patched sources and file a bug on ICU meanwhile.
> >
> > I would like to know, what sources were used to compile ICU4JNI for
> > windows and other operation systems? I found today that ICU4JNI 3.4
> > contains quite a lot of memory leaks, and there is a patch to fix them in
> > v3.4 [1]. Was it used to compile our binaries (from what I learned while
> > analyzing this bug on windows 32 it *was* used)? I know that I didn't use
> > it when I compiled ICU for IPF because I learned about its existence only
> > today. Are there any other patches for ICU code that we need to use for
> > version 3.4?
> >
> > [1]
> > ftp://ftp.software.ibm.com/software/globalization/icu/icu4j/icu4jni/3.4/i
> >cu4jni_3_4_patch-01.zip
>
> Sorry that you had to find it the hard way Gregory. The ICU4JNI code
> does have the patch applied, as described in the readme in the depends'
> ICU4JNI directory[1].
I didn't find it a hard way, I just didn't look for any patches when I created
IPF binaries some time ago. I the memory leaks patch when I went to ICU site
to download its sources to see what's going on when it calls VM JNI code.
What I found out is that it calls VM JNI code with a wrong elems pointer and
to fix the bug we need to patch ICU. The bug is present it 3.6 as well.
So my question was actually, do we need any patches other than memory leaks
patch to build ICU binaries that we have in SVN currently?
--
Gregory
Re: [jira] Commented: (HARMONY-2669) [classlib][charsets] Working
with charset ISO-2022-CN crashes DRLVM
Posted by Tim Ellison <t....@gmail.com>.
Gregory Shimansky wrote:
> Below is my evaluation of the bug HARMONY-2669. I think it is in ICU4JNI
> native code. The possible solution for it is to recompile ICUInterface34
> libraries from patched sources and file a bug on ICU meanwhile.
>
> I would like to know, what sources were used to compile ICU4JNI for windows
> and other operation systems? I found today that ICU4JNI 3.4 contains quite a
> lot of memory leaks, and there is a patch to fix them in v3.4 [1]. Was it
> used to compile our binaries (from what I learned while analyzing this bug on
> windows 32 it *was* used)? I know that I didn't use it when I compiled ICU
> for IPF because I learned about its existence only today. Are there any other
> patches for ICU code that we need to use for version 3.4?
>
> [1]
> ftp://ftp.software.ibm.com/software/globalization/icu/icu4j/icu4jni/3.4/icu4jni_3_4_patch-01.zip
Sorry that you had to find it the hard way Gregory. The ICU4JNI code
does have the patch applied, as described in the readme in the depends'
ICU4JNI directory[1].
[1]
http://svn.apache.org/viewvc/harmony/enhanced/classlib/trunk/depends/jars/icu4jni_3.4/README.txt?revision=492523&view=markup
Regards,
Tim