You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-java@ibatis.apache.org by John Seer <pu...@yahoo.com> on 2010/03/05 22:44:21 UTC
Problem with Character Encoding
Hello,
We am currently using Oracle DB with default utf-8 encoding, we have two
application one is using hibernate other one ibatis
If I am running hibernate on windows or on linux I am seeing every character
which is returned is encoded correctly.
But when I am using ibatis I can see that some of my characters were replace
by question mark others with replacement character when I run it on linux.
Is there any way to force connection encoding for ibatis?
I googled this only reference about setting connection encoding is for
ibatis 3. I am currently using 2.3.4.726.
John
--
View this message in context: http://old.nabble.com/Problem-with-Character-Encoding-tp27799549p27799549.html
Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
Re: Problem with Character Encoding
Posted by François Schiettecatte <fs...@gmail.com>.
I don't have any experience with Oracle specifically, but when I see this in MySQL or indeed on the web in general, there is usually a character encoding disconnect. I suspect that the layer connecting iBatis to Oracle most likely is not being told that it should expect utf8. I need to tell the MySQL JDBC driver that is is talking utf8 to the server by adding 'characterEncoding=UTF-8&useUnicode=true' to the JDBC MySQL url, assuming you are using JDBC of course.
You might also want to check for any encoding 'munging' going the other way, for example seeing what the orable server sees when you send it "SELECT * FROM names WHERE name = 'françois'".
François
On Mar 12, 2010, at 6:49 PM, John Seer wrote:
>
> for example if I have long dash, I will see "?" if I have é I will see
> replacement character.
> I see it in debug/sysout/jsp
>
>
>
> Larry Meadors wrote:
>>
>> Where are you seeing the "?" characters?
>>
>> On Fri, Mar 12, 2010 at 3:14 PM, John Seer <pu...@yahoo.com> wrote:
>>>
>>> Hello,
>>>
>>> I am not sure where is problem...
>>> If I am using hibernate I am getting data back with out problems...
>>> Only combination is ibatis and oracle. I am currently using JNDI to
>>> connect
>>> and saw in documentation for ibatis 3 that I can set
>>> env.encoding=UTF-8... I
>>> tried to use it no difference. :(
>>>
>>>
>>> Ellis Miller wrote:
>>>>
>>>> If you think it's on the Oracle side start by checking NLS_LANG
>>>> settings:
>>>>
>>>> http://www.oracle.com/technology/tech/globalization/htdocs/nls_lang%20faq.htm#_Toc110410543
>>>>
>>>> Would try an Oracle Alter Session for NLS_LANG and see if that helps:
>>>>
>>>> http://forums.oracle.com/forums/thread.jspa?threadID=436286&tstart=360
>>>>
>>>> Ellis R. Miller
>>>> 937.830.8242
>>>> 937.830.6027
>>>>
>>>> --- @ WiseStamp
>>>> Signature<http://my.wisestamp.com/link?u=2hxhdfd4p76bkhcm&site=www.wisestamp.com/email-install>.
>>>> Get it
>>>> now<http://my.wisestamp.com/link?u=2hxhdfd4p76bkhcm&site=www.wisestamp.com/email-install>
>>>>
>>>>
>>>> On Fri, Mar 5, 2010 at 6:27 PM, Larry Meadors
>>>> <la...@gmail.com>wrote:
>>>>
>>>>> I'd look for info on the oracle connection string, it's not really an
>>>>> ibatis thing.
>>>>>
>>>>> I think that it happens that way by default though.
>>>>>
>>>>> Where are you seeing the "?" characters?
>>>>>
>>>>> Larry
>>>>>
>>>>>
>>>>> On Fri, Mar 5, 2010 at 2:44 PM, John Seer <pu...@yahoo.com> wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> We am currently using Oracle DB with default utf-8 encoding, we have
>>>>> two
>>>>>> application one is using hibernate other one ibatis
>>>>>>
>>>>>> If I am running hibernate on windows or on linux I am seeing every
>>>>> character
>>>>>> which is returned is encoded correctly.
>>>>>> But when I am using ibatis I can see that some of my characters were
>>>>> replace
>>>>>> by question mark others with replacement character when I run it on
>>>>> linux.
>>>>>>
>>>>>> Is there any way to force connection encoding for ibatis?
>>>>>>
>>>>>> I googled this only reference about setting connection encoding is
>>>>> for
>>>>>> ibatis 3. I am currently using 2.3.4.726.
>>>>>>
>>>>>> John
>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>> http://old.nabble.com/Problem-with-Character-Encoding-tp27799549p27799549.html
>>>>>> Sent from the iBATIS - User - Java mailing list archive at
>>>>> Nabble.com.
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>>>>>> For additional commands, e-mail: user-java-help@ibatis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>>>>> For additional commands, e-mail: user-java-help@ibatis.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/Problem-with-Character-Encoding-tp27799549p27883501.html
>>> Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>>> For additional commands, e-mail: user-java-help@ibatis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>> For additional commands, e-mail: user-java-help@ibatis.apache.org
>>
>>
>>
>
> --
> View this message in context: http://old.nabble.com/Problem-with-Character-Encoding-tp27799549p27884132.html
> Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> For additional commands, e-mail: user-java-help@ibatis.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
Re: Problem with Character Encoding
Posted by Larry Meadors <la...@gmail.com>.
I just happen to have spent the last month wrestling with Oracle and
UTF-8 encoding. I had to deal with BLOBs, but Strings should be
easier.
Oracle stores VARCHAR2, VARCHAR, and CLOB fields as unicode and when
the driver fetches the value, it goes into a String which is just an
array of unicode characters, so that should be OK. Regardless of the
database encoding, you should be getting unicode Strings back.
Have you verified that the encoding of the jsp (and browser) are set to utf-8?
Do you have a unit test that shows the problem?
It might be worth examining the characters in the values that are
boogered up to verify that they are coming through correctly. Look at
the String in a debugger and you should be able to see the array of
characters that make it up.
For example, the "é" character should be "\u00E9". If you convert it
to bytes (yourString.getBytes("UTF-8")), you should see 2 byte values
for that character: 0xC3 and 0xA9 - since those are bytes, they may
show up in your debugger as negative values: -61 and -87.
http://www.fileformat.info/info/unicode/char/00e9/index.htm
Let us know what you find out. :)
Larry
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
Re: Problem with Character Encoding
Posted by John Seer <pu...@yahoo.com>.
for example if I have long dash, I will see "?" if I have é I will see
replacement character.
I see it in debug/sysout/jsp
Larry Meadors wrote:
>
> Where are you seeing the "?" characters?
>
> On Fri, Mar 12, 2010 at 3:14 PM, John Seer <pu...@yahoo.com> wrote:
>>
>> Hello,
>>
>> I am not sure where is problem...
>> If I am using hibernate I am getting data back with out problems...
>> Only combination is ibatis and oracle. I am currently using JNDI to
>> connect
>> and saw in documentation for ibatis 3 that I can set
>> env.encoding=UTF-8... I
>> tried to use it no difference. :(
>>
>>
>> Ellis Miller wrote:
>>>
>>> If you think it's on the Oracle side start by checking NLS_LANG
>>> settings:
>>>
>>> http://www.oracle.com/technology/tech/globalization/htdocs/nls_lang%20faq.htm#_Toc110410543
>>>
>>> Would try an Oracle Alter Session for NLS_LANG and see if that helps:
>>>
>>> http://forums.oracle.com/forums/thread.jspa?threadID=436286&tstart=360
>>>
>>> Ellis R. Miller
>>> 937.830.8242
>>> 937.830.6027
>>>
>>> --- @ WiseStamp
>>> Signature<http://my.wisestamp.com/link?u=2hxhdfd4p76bkhcm&site=www.wisestamp.com/email-install>.
>>> Get it
>>> now<http://my.wisestamp.com/link?u=2hxhdfd4p76bkhcm&site=www.wisestamp.com/email-install>
>>>
>>>
>>> On Fri, Mar 5, 2010 at 6:27 PM, Larry Meadors
>>> <la...@gmail.com>wrote:
>>>
>>>> I'd look for info on the oracle connection string, it's not really an
>>>> ibatis thing.
>>>>
>>>> I think that it happens that way by default though.
>>>>
>>>> Where are you seeing the "?" characters?
>>>>
>>>> Larry
>>>>
>>>>
>>>> On Fri, Mar 5, 2010 at 2:44 PM, John Seer <pu...@yahoo.com> wrote:
>>>> >
>>>> > Hello,
>>>> >
>>>> > We am currently using Oracle DB with default utf-8 encoding, we have
>>>> two
>>>> > application one is using hibernate other one ibatis
>>>> >
>>>> > If I am running hibernate on windows or on linux I am seeing every
>>>> character
>>>> > which is returned is encoded correctly.
>>>> > But when I am using ibatis I can see that some of my characters were
>>>> replace
>>>> > by question mark others with replacement character when I run it on
>>>> linux.
>>>> >
>>>> > Is there any way to force connection encoding for ibatis?
>>>> >
>>>> > I googled this only reference about setting connection encoding is
>>>> for
>>>> > ibatis 3. I am currently using 2.3.4.726.
>>>> >
>>>> > John
>>>> >
>>>> >
>>>> > --
>>>> > View this message in context:
>>>> http://old.nabble.com/Problem-with-Character-Encoding-tp27799549p27799549.html
>>>> > Sent from the iBATIS - User - Java mailing list archive at
>>>> Nabble.com.
>>>> >
>>>> >
>>>> > ---------------------------------------------------------------------
>>>> > To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>>>> > For additional commands, e-mail: user-java-help@ibatis.apache.org
>>>> >
>>>> >
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>>>> For additional commands, e-mail: user-java-help@ibatis.apache.org
>>>>
>>>>
>>>
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Problem-with-Character-Encoding-tp27799549p27883501.html
>> Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>> For additional commands, e-mail: user-java-help@ibatis.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> For additional commands, e-mail: user-java-help@ibatis.apache.org
>
>
>
--
View this message in context: http://old.nabble.com/Problem-with-Character-Encoding-tp27799549p27884132.html
Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
Re: Problem with Character Encoding
Posted by Larry Meadors <la...@gmail.com>.
Where are you seeing the "?" characters?
On Fri, Mar 12, 2010 at 3:14 PM, John Seer <pu...@yahoo.com> wrote:
>
> Hello,
>
> I am not sure where is problem...
> If I am using hibernate I am getting data back with out problems...
> Only combination is ibatis and oracle. I am currently using JNDI to connect
> and saw in documentation for ibatis 3 that I can set env.encoding=UTF-8... I
> tried to use it no difference. :(
>
>
> Ellis Miller wrote:
>>
>> If you think it's on the Oracle side start by checking NLS_LANG settings:
>>
>> http://www.oracle.com/technology/tech/globalization/htdocs/nls_lang%20faq.htm#_Toc110410543
>>
>> Would try an Oracle Alter Session for NLS_LANG and see if that helps:
>>
>> http://forums.oracle.com/forums/thread.jspa?threadID=436286&tstart=360
>>
>> Ellis R. Miller
>> 937.830.8242
>> 937.830.6027
>>
>> --- @ WiseStamp
>> Signature<http://my.wisestamp.com/link?u=2hxhdfd4p76bkhcm&site=www.wisestamp.com/email-install>.
>> Get it
>> now<http://my.wisestamp.com/link?u=2hxhdfd4p76bkhcm&site=www.wisestamp.com/email-install>
>>
>>
>> On Fri, Mar 5, 2010 at 6:27 PM, Larry Meadors
>> <la...@gmail.com>wrote:
>>
>>> I'd look for info on the oracle connection string, it's not really an
>>> ibatis thing.
>>>
>>> I think that it happens that way by default though.
>>>
>>> Where are you seeing the "?" characters?
>>>
>>> Larry
>>>
>>>
>>> On Fri, Mar 5, 2010 at 2:44 PM, John Seer <pu...@yahoo.com> wrote:
>>> >
>>> > Hello,
>>> >
>>> > We am currently using Oracle DB with default utf-8 encoding, we have
>>> two
>>> > application one is using hibernate other one ibatis
>>> >
>>> > If I am running hibernate on windows or on linux I am seeing every
>>> character
>>> > which is returned is encoded correctly.
>>> > But when I am using ibatis I can see that some of my characters were
>>> replace
>>> > by question mark others with replacement character when I run it on
>>> linux.
>>> >
>>> > Is there any way to force connection encoding for ibatis?
>>> >
>>> > I googled this only reference about setting connection encoding is for
>>> > ibatis 3. I am currently using 2.3.4.726.
>>> >
>>> > John
>>> >
>>> >
>>> > --
>>> > View this message in context:
>>> http://old.nabble.com/Problem-with-Character-Encoding-tp27799549p27799549.html
>>> > Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>>> > For additional commands, e-mail: user-java-help@ibatis.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>>> For additional commands, e-mail: user-java-help@ibatis.apache.org
>>>
>>>
>>
>>
>
> --
> View this message in context: http://old.nabble.com/Problem-with-Character-Encoding-tp27799549p27883501.html
> Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> For additional commands, e-mail: user-java-help@ibatis.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
Re: Problem with Character Encoding
Posted by John Seer <pu...@yahoo.com>.
Hello,
I am not sure where is problem...
If I am using hibernate I am getting data back with out problems...
Only combination is ibatis and oracle. I am currently using JNDI to connect
and saw in documentation for ibatis 3 that I can set env.encoding=UTF-8... I
tried to use it no difference. :(
Ellis Miller wrote:
>
> If you think it's on the Oracle side start by checking NLS_LANG settings:
>
> http://www.oracle.com/technology/tech/globalization/htdocs/nls_lang%20faq.htm#_Toc110410543
>
> Would try an Oracle Alter Session for NLS_LANG and see if that helps:
>
> http://forums.oracle.com/forums/thread.jspa?threadID=436286&tstart=360
>
> Ellis R. Miller
> 937.830.8242
> 937.830.6027
>
> --- @ WiseStamp
> Signature<http://my.wisestamp.com/link?u=2hxhdfd4p76bkhcm&site=www.wisestamp.com/email-install>.
> Get it
> now<http://my.wisestamp.com/link?u=2hxhdfd4p76bkhcm&site=www.wisestamp.com/email-install>
>
>
> On Fri, Mar 5, 2010 at 6:27 PM, Larry Meadors
> <la...@gmail.com>wrote:
>
>> I'd look for info on the oracle connection string, it's not really an
>> ibatis thing.
>>
>> I think that it happens that way by default though.
>>
>> Where are you seeing the "?" characters?
>>
>> Larry
>>
>>
>> On Fri, Mar 5, 2010 at 2:44 PM, John Seer <pu...@yahoo.com> wrote:
>> >
>> > Hello,
>> >
>> > We am currently using Oracle DB with default utf-8 encoding, we have
>> two
>> > application one is using hibernate other one ibatis
>> >
>> > If I am running hibernate on windows or on linux I am seeing every
>> character
>> > which is returned is encoded correctly.
>> > But when I am using ibatis I can see that some of my characters were
>> replace
>> > by question mark others with replacement character when I run it on
>> linux.
>> >
>> > Is there any way to force connection encoding for ibatis?
>> >
>> > I googled this only reference about setting connection encoding is for
>> > ibatis 3. I am currently using 2.3.4.726.
>> >
>> > John
>> >
>> >
>> > --
>> > View this message in context:
>> http://old.nabble.com/Problem-with-Character-Encoding-tp27799549p27799549.html
>> > Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>> > For additional commands, e-mail: user-java-help@ibatis.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
>> For additional commands, e-mail: user-java-help@ibatis.apache.org
>>
>>
>
>
--
View this message in context: http://old.nabble.com/Problem-with-Character-Encoding-tp27799549p27883501.html
Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org
Re: Problem with Character Encoding
Posted by Ellis Miller <ou...@gmail.com>.
If you think it's on the Oracle side start by checking NLS_LANG settings:
http://www.oracle.com/technology/tech/globalization/htdocs/nls_lang%20faq.htm#_Toc110410543
Would try an Oracle Alter Session for NLS_LANG and see if that helps:
http://forums.oracle.com/forums/thread.jspa?threadID=436286&tstart=360
Ellis R. Miller
937.830.8242
937.830.6027
--- @ WiseStamp
Signature<http://my.wisestamp.com/link?u=2hxhdfd4p76bkhcm&site=www.wisestamp.com/email-install>.
Get it now<http://my.wisestamp.com/link?u=2hxhdfd4p76bkhcm&site=www.wisestamp.com/email-install>
On Fri, Mar 5, 2010 at 6:27 PM, Larry Meadors <la...@gmail.com>wrote:
> I'd look for info on the oracle connection string, it's not really an
> ibatis thing.
>
> I think that it happens that way by default though.
>
> Where are you seeing the "?" characters?
>
> Larry
>
>
> On Fri, Mar 5, 2010 at 2:44 PM, John Seer <pu...@yahoo.com> wrote:
> >
> > Hello,
> >
> > We am currently using Oracle DB with default utf-8 encoding, we have two
> > application one is using hibernate other one ibatis
> >
> > If I am running hibernate on windows or on linux I am seeing every
> character
> > which is returned is encoded correctly.
> > But when I am using ibatis I can see that some of my characters were
> replace
> > by question mark others with replacement character when I run it on
> linux.
> >
> > Is there any way to force connection encoding for ibatis?
> >
> > I googled this only reference about setting connection encoding is for
> > ibatis 3. I am currently using 2.3.4.726.
> >
> > John
> >
> >
> > --
> > View this message in context:
> http://old.nabble.com/Problem-with-Character-Encoding-tp27799549p27799549.html
> > Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> > For additional commands, e-mail: user-java-help@ibatis.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> For additional commands, e-mail: user-java-help@ibatis.apache.org
>
>
Re: Problem with Character Encoding
Posted by Larry Meadors <la...@gmail.com>.
I'd look for info on the oracle connection string, it's not really an
ibatis thing.
I think that it happens that way by default though.
Where are you seeing the "?" characters?
Larry
On Fri, Mar 5, 2010 at 2:44 PM, John Seer <pu...@yahoo.com> wrote:
>
> Hello,
>
> We am currently using Oracle DB with default utf-8 encoding, we have two
> application one is using hibernate other one ibatis
>
> If I am running hibernate on windows or on linux I am seeing every character
> which is returned is encoded correctly.
> But when I am using ibatis I can see that some of my characters were replace
> by question mark others with replacement character when I run it on linux.
>
> Is there any way to force connection encoding for ibatis?
>
> I googled this only reference about setting connection encoding is for
> ibatis 3. I am currently using 2.3.4.726.
>
> John
>
>
> --
> View this message in context: http://old.nabble.com/Problem-with-Character-Encoding-tp27799549p27799549.html
> Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> For additional commands, e-mail: user-java-help@ibatis.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org