You are viewing a plain text version of this content. The canonical link for it is here.
Posted to taglibs-user@tomcat.apache.org by ilanchezhian ganesamurthy <co...@yahoo.co.in> on 2006/03/22 10:43:46 UTC
Problem with
Hello,
I have problem while displaying of japanese UTF-8 character.
I set response header character encoding to UTF-8 by <%@ page contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" %> then I read the UTF-8 japanese character by following statement
<fmt:bundle basename="com.xxx.myBundle">
<fmt:message key="k1"/>
</fmt:bundle>
Even though i set response character encoding to UTF-8 but when compiler start executing <fmt:bundle> it automatically chages response character encoding to 'Shift_JIS' from 'UTF-8' I can notice this by printing response character encoding before and after <fmt:bundle>. Due to this form is not displayed properly. I dont have issue with locale, I have only issue with response character encoding.
Resource bundle contains UTF-8 character
I have followig jsp
<%@ page contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" %>
<%@ taglib uri="fmt" prefix="fmt" %>
<html>
<head>
</head>
<%System.out.println("CharacterEncoding before calling fmt = "+response.getCharacterEncoding() );%>
<fmt:bundle basename="com.xxx.myBundle">
<%System.out.println("CharacterEncoding after calling fmt = "+response.getCharacterEncoding() );%>
<fmt:message key="k1"/>
</fmt:bundle>
</html>
You can see result o/p to
CharacterEncoding before calling fmt = UTF-8
CharacterEncoding after calling fmt = Shift_JIS
Cheers
ilan
---------------------------------
Jiyo cricket on Yahoo! India cricket
Yahoo! Messenger Mobile Stay in touch with your buddies all the time.
Re: Problem with
Posted by ilanchezhian ganesamurthy <co...@yahoo.co.in>.
THanks Murray, I perfectly understood (atlest from my perspective) what you have said and appreciate your kind patient in explaining it detailly...
I really dont sure what ever point you mentioned happens only with tomcat. I use JRUN it happens to me :-). So it looks in most of the server we can notice this issue.
Again my assumption is, it is not just issue with Tomcat as i am encoutering same issue in JRUN, I think it is problem with <fmt:bundle> tag, probably some kind of bug (or some feature missing in thistag)...
Any thoughts from your side...
Murray Steele <mu...@peoplesarchive.com> wrote:
My answers are below.
On 22 Mar 2006, at 10:31, ilanchezhian ganesamurthy wrote:
> Thanks for your reply... My answer & questions below
>
> Murray Steele wrote: Hi,
>
> I ran up against this problem in my application. After some digging I
> boiled it down to the fact that the servlet spec / tomcat versions I
> was using didn't allow me to provide a Locale to character encoding
> mapping, and when the fmt tags do pretty much anything,
>
> Sorry I realy dint understand what you mean by proving locale
> to character encoding.
>
> they end up
> setting the Locale, which in turn sets the character encoding. I think
> I read somewhere that some version of the servlet spec allows you to
> specify Locale to character encoding mappings in the web.xml file, but
>
> My understanding is character encoding and Local are
> independent. In my case Locale is already set to Japanese ( which is
> set from user browser request) and in jsp page i forcefully set
> response character encoding to UTF-8, but still why tag
> should change this to 'Shift_JIS'.
>
You are right that they *should* be independent, but at least in Tomcat
they aren't. When the tags do anything they set the Locale of
your response and when you set the Locale of a response object in
Tomcat one of the things that happens is that the Character Encoding is
also set according to what Character Encoding Tomcat thinks is
appropriate for that Locale. In Tomcat there is a one-to-one mapping
between Locale's and Character Encodings (it's defined in some resource
file in the tomcat jar files I think). The only way to prevent this
from happening is to already have committed the response which starts
writing the headers to the browser (and from which point you can't
change them). As you say below you've already worked this out by
calling flush().
> for what I'm using (2.3 on tomcat 4.1) what I had to do was create a
> custom CharSetMapper and specify it on the tag in my
> server.xml.
>
> My charset mapper ignores whatever locale you ask for a mapping for and
> returns UTF-8 as the charset every the time. It seemed extreme, but
> it's what I wanted. I compiled this and put it in
> /server/classes/
>
> UTF8CharSetMapper
> --
> package com.peoplesarchive.tomcat;
>
> import java.util.Locale;
>
> import org.apache.catalina.util.CharsetMapper;
>
> public class UTF8CharsetMapper extends CharsetMapper {
> public UTF8CharsetMapper() {
> super();
> }
>
> public UTF8CharsetMapper(String name) {
> super(name);
> }
>
> public String getCharset(Locale locale) {
> return "utf-8";
> }
> }
>
> server.xml fragment
> --
> charsetMapperClass="com.peoplesarchive.tomcat.UTF8CharsetMapper"
> path="" docBase="ROOT" debug="0" reloadable="true" />
>
> hope this helps
> If I use japanese I should be either use 'Shift_JIS' or 'UTF-8'
> or any other japanese character encoding. Incase of French I should be
> either able to use 'ISO-8859-1' or UTF-8' I should not able to lock
> particular local to particular character encoding. My understanding is
> now this class locks particular charactter encoding to particular
> locale. This is my understanding applogize me if i am not correct.
>
You're right, you should be able to choose to use whichever chaacter
Encoding you want for your Locale, but in Tomcat (4.1 at least) you
can't. You have to map each Locale to a specific Character Encoding.
The class that I extended actually does what you say e.g. lock a
particular character encoding to a particular locale, my class is worse
in that respect, it maps *all* Locales to one Character Encoding;
UTF-8. (In my defense,utf-8 is all that I cared about and it was the
easiest fix).
> More over i find one work to avoid changing character
> encoding to 'Shift_JIS'. After setting response character encoding
> thru call 'response.getWriter().flush();', this ensures
> response character encoding is always set to UTF-8.
>
As I said above, this will work properly, but you'd have to be pretty
sure that you don't want to set any other headers (cookies,
content-encoding, cache) before you call flush. I think I'm correct in
saying that once you call flush you can't add or change any other
headers (that's why altho the tags are still setting the Locale
of your response, and Tomcat is still trying to then set the Character
Encoding from the Local it's being ignored as the headers have been
committed.
Hope this clears things up a little.
Muz
> Muz
>
> On 22 Mar 2006, at 09:43, ilanchezhian ganesamurthy wrote:
>
>> Hello,
>>
>> I have problem while displaying of japanese UTF-8 character.
>> I set response header character encoding to UTF-8 by >
>> contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" %>then I
>> read the UTF-8 japanese character by following statement
>>
>>
>>
>>
>>
>> Even though i set response character encoding to UTF-8 but when
>> compiler start executing it automatically chages response
>> character encoding to 'Shift_JIS' from 'UTF-8' I can notice this by
>> printing response character encoding before and after .
>> Due to this form is not displayed properly. I dont have issue with
>> locale, I have only issue with response character encoding.
>>
>> Resource bundle contains UTF-8 character
>> I have followig jsp
>>
>>> pageEncoding="UTF-8" %>
>>
>>
>>
>>
>>> "+response.getCharacterEncoding() );%>
>>
>>
>>> "+response.getCharacterEncoding() );%>
>>
>>
>>
>>
>>
>> You can see result o/p to
>>
>> CharacterEncoding before calling fmt = UTF-8
>> CharacterEncoding after calling fmt = Shift_JIS
>>
>> Cheers
>> ilan
>>
>>
>>
>>
>> ---------------------------------
>> Jiyo cricket on Yahoo! India cricket
>> Yahoo! Messenger Mobile Stay in touch with your buddies all the time.
> --
> Murray Steele
> Head of Development
>
> Peoples Archive
> w: http://www.peoplesarchive.com
> t: 0207 323 0323
> d: 0207 631 9147
>
> This email has been scanned by Postini.
> For more information please visit http://www.postini.com
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: taglibs-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: taglibs-user-help@jakarta.apache.org
>
>
>
>
>
> ---------------------------------
> Jiyo cricket on Yahoo! India cricket
> Yahoo! Messenger Mobile Stay in touch with your buddies all the time.
--
Murray Steele
Head of Development
Peoples Archive
w: http://www.peoplesarchive.com
t: 0207 323 0323
d: 0207 631 9147
This email has been scanned by Postini.
For more information please visit http://www.postini.com
---------------------------------------------------------------------
To unsubscribe, e-mail: taglibs-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: taglibs-user-help@jakarta.apache.org
---------------------------------
Jiyo cricket on Yahoo! India cricket
Yahoo! Messenger Mobile Stay in touch with your buddies all the time.
Re: Problem with
Posted by Murray Steele <mu...@peoplesarchive.com>.
My answers are below.
On 22 Mar 2006, at 10:31, ilanchezhian ganesamurthy wrote:
> Thanks for your reply... My answer & questions below
>
> Murray Steele <mu...@peoplesarchive.com> wrote: Hi,
>
> I ran up against this problem in my application. After some digging I
> boiled it down to the fact that the servlet spec / tomcat versions I
> was using didn't allow me to provide a Locale to character encoding
> mapping, and when the fmt tags do pretty much anything,
>
> <Ilan> Sorry I realy dint understand what you mean by proving locale
> to character encoding.
>
> they end up
> setting the Locale, which in turn sets the character encoding. I think
> I read somewhere that some version of the servlet spec allows you to
> specify Locale to character encoding mappings in the web.xml file, but
>
> <Ilan> My understanding is character encoding and Local are
> independent. In my case Locale is already set to Japanese ( which is
> set from user browser request) and in jsp page i forcefully set
> response character encoding to UTF-8, but still why <fmt:bundle> tag
> should change this to 'Shift_JIS'.
>
You are right that they *should* be independent, but at least in Tomcat
they aren't. When the <fmt> tags do anything they set the Locale of
your response and when you set the Locale of a response object in
Tomcat one of the things that happens is that the Character Encoding is
also set according to what Character Encoding Tomcat thinks is
appropriate for that Locale. In Tomcat there is a one-to-one mapping
between Locale's and Character Encodings (it's defined in some resource
file in the tomcat jar files I think). The only way to prevent this
from happening is to already have committed the response which starts
writing the headers to the browser (and from which point you can't
change them). As you say below you've already worked this out by
calling flush().
> for what I'm using (2.3 on tomcat 4.1) what I had to do was create a
> custom CharSetMapper and specify it on the tag in my
> server.xml.
>
> My charset mapper ignores whatever locale you ask for a mapping for and
> returns UTF-8 as the charset every the time. It seemed extreme, but
> it's what I wanted. I compiled this and put it in
> /server/classes/
>
> UTF8CharSetMapper
> --
> package com.peoplesarchive.tomcat;
>
> import java.util.Locale;
>
> import org.apache.catalina.util.CharsetMapper;
>
> public class UTF8CharsetMapper extends CharsetMapper {
> public UTF8CharsetMapper() {
> super();
> }
>
> public UTF8CharsetMapper(String name) {
> super(name);
> }
>
> public String getCharset(Locale locale) {
> return "utf-8";
> }
> }
>
> server.xml fragment
> --
> charsetMapperClass="com.peoplesarchive.tomcat.UTF8CharsetMapper"
> path="" docBase="ROOT" debug="0" reloadable="true" />
>
> hope this helps
> <Ilan> If I use japanese I should be either use 'Shift_JIS' or 'UTF-8'
> or any other japanese character encoding. Incase of French I should be
> either able to use 'ISO-8859-1' or UTF-8' I should not able to lock
> particular local to particular character encoding. My understanding is
> now this class locks particular charactter encoding to particular
> locale. This is my understanding applogize me if i am not correct.
>
You're right, you should be able to choose to use whichever chaacter
Encoding you want for your Locale, but in Tomcat (4.1 at least) you
can't. You have to map each Locale to a specific Character Encoding.
The class that I extended actually does what you say e.g. lock a
particular character encoding to a particular locale, my class is worse
in that respect, it maps *all* Locales to one Character Encoding;
UTF-8. (In my defense,utf-8 is all that I cared about and it was the
easiest fix).
> More over i find one work to avoid <fmt:bundle> changing character
> encoding to 'Shift_JIS'. After setting response character encoding
> thru <%Page > call 'response.getWriter().flush();', this ensures
> response character encoding is always set to UTF-8.
>
As I said above, this will work properly, but you'd have to be pretty
sure that you don't want to set any other headers (cookies,
content-encoding, cache) before you call flush. I think I'm correct in
saying that once you call flush you can't add or change any other
headers (that's why altho the <fmt> tags are still setting the Locale
of your response, and Tomcat is still trying to then set the Character
Encoding from the Local it's being ignored as the headers have been
committed.
Hope this clears things up a little.
Muz
> Muz
>
> On 22 Mar 2006, at 09:43, ilanchezhian ganesamurthy wrote:
>
>> Hello,
>>
>> I have problem while displaying of japanese UTF-8 character.
>> I set response header character encoding to UTF-8 by >
>> contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" %>then I
>> read the UTF-8 japanese character by following statement
>>
>>
>>
>>
>>
>> Even though i set response character encoding to UTF-8 but when
>> compiler start executing it automatically chages response
>> character encoding to 'Shift_JIS' from 'UTF-8' I can notice this by
>> printing response character encoding before and after .
>> Due to this form is not displayed properly. I dont have issue with
>> locale, I have only issue with response character encoding.
>>
>> Resource bundle contains UTF-8 character
>> I have followig jsp
>>
>>> pageEncoding="UTF-8" %>
>>
>>
>>
>>
>>> "+response.getCharacterEncoding() );%>
>>
>>
>>> "+response.getCharacterEncoding() );%>
>>
>>
>>
>>
>>
>> You can see result o/p to
>>
>> CharacterEncoding before calling fmt = UTF-8
>> CharacterEncoding after calling fmt = Shift_JIS
>>
>> Cheers
>> ilan
>>
>>
>>
>>
>> ---------------------------------
>> Jiyo cricket on Yahoo! India cricket
>> Yahoo! Messenger Mobile Stay in touch with your buddies all the time.
> --
> Murray Steele
> Head of Development
>
> Peoples Archive
> w: http://www.peoplesarchive.com
> t: 0207 323 0323
> d: 0207 631 9147
>
> This email has been scanned by Postini.
> For more information please visit http://www.postini.com
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: taglibs-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: taglibs-user-help@jakarta.apache.org
>
>
>
>
>
> ---------------------------------
> Jiyo cricket on Yahoo! India cricket
> Yahoo! Messenger Mobile Stay in touch with your buddies all the time.
--
Murray Steele
Head of Development
Peoples Archive
w: http://www.peoplesarchive.com
t: 0207 323 0323
d: 0207 631 9147
This email has been scanned by Postini.
For more information please visit http://www.postini.com
---------------------------------------------------------------------
To unsubscribe, e-mail: taglibs-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: taglibs-user-help@jakarta.apache.org
Re: Problem with
Posted by ilanchezhian ganesamurthy <co...@yahoo.co.in>.
Thanks for your reply... My answer & questions below
Murray Steele <mu...@peoplesarchive.com> wrote: Hi,
I ran up against this problem in my application. After some digging I
boiled it down to the fact that the servlet spec / tomcat versions I
was using didn't allow me to provide a Locale to character encoding
mapping, and when the fmt tags do pretty much anything,
<Ilan> Sorry I realy dint understand what you mean by proving locale to character encoding.
they end up
setting the Locale, which in turn sets the character encoding. I think
I read somewhere that some version of the servlet spec allows you to
specify Locale to character encoding mappings in the web.xml file, but
<Ilan> My understanding is character encoding and Local are independent. In my case Locale is already set to Japanese ( which is set from user browser request) and in jsp page i forcefully set response character encoding to UTF-8, but still why <fmt:bundle> tag should change this to 'Shift_JIS'.
for what I'm using (2.3 on tomcat 4.1) what I had to do was create a
custom CharSetMapper and specify it on the tag in my
server.xml.
My charset mapper ignores whatever locale you ask for a mapping for and
returns UTF-8 as the charset every the time. It seemed extreme, but
it's what I wanted. I compiled this and put it in
/server/classes/
UTF8CharSetMapper
--
package com.peoplesarchive.tomcat;
import java.util.Locale;
import org.apache.catalina.util.CharsetMapper;
public class UTF8CharsetMapper extends CharsetMapper {
public UTF8CharsetMapper() {
super();
}
public UTF8CharsetMapper(String name) {
super(name);
}
public String getCharset(Locale locale) {
return "utf-8";
}
}
server.xml fragment
--
charsetMapperClass="com.peoplesarchive.tomcat.UTF8CharsetMapper"
path="" docBase="ROOT" debug="0" reloadable="true" />
hope this helps
<Ilan> If I use japanese I should be either use 'Shift_JIS' or 'UTF-8' or any other japanese character encoding. Incase of French I should be either able to use 'ISO-8859-1' or UTF-8' I should not able to lock particular local to particular character encoding. My understanding is now this class locks particular charactter encoding to particular locale. This is my understanding applogize me if i am not correct.
More over i find one work to avoid <fmt:bundle> changing character encoding to 'Shift_JIS'. After setting response character encoding thru <%Page > call 'response.getWriter().flush();', this ensures response character encoding is always set to UTF-8.
Muz
On 22 Mar 2006, at 09:43, ilanchezhian ganesamurthy wrote:
> Hello,
>
> I have problem while displaying of japanese UTF-8 character.
> I set response header character encoding to UTF-8 by > contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" %>then I
> read the UTF-8 japanese character by following statement
>
>
>
>
>
> Even though i set response character encoding to UTF-8 but when
> compiler start executing it automatically chages response
> character encoding to 'Shift_JIS' from 'UTF-8' I can notice this by
> printing response character encoding before and after .
> Due to this form is not displayed properly. I dont have issue with
> locale, I have only issue with response character encoding.
>
> Resource bundle contains UTF-8 character
> I have followig jsp
>
> > pageEncoding="UTF-8" %>
>
>
>
>
> > "+response.getCharacterEncoding() );%>
>
>
> > "+response.getCharacterEncoding() );%>
>
>
>
>
>
> You can see result o/p to
>
> CharacterEncoding before calling fmt = UTF-8
> CharacterEncoding after calling fmt = Shift_JIS
>
> Cheers
> ilan
>
>
>
>
> ---------------------------------
> Jiyo cricket on Yahoo! India cricket
> Yahoo! Messenger Mobile Stay in touch with your buddies all the time.
--
Murray Steele
Head of Development
Peoples Archive
w: http://www.peoplesarchive.com
t: 0207 323 0323
d: 0207 631 9147
This email has been scanned by Postini.
For more information please visit http://www.postini.com
---------------------------------------------------------------------
To unsubscribe, e-mail: taglibs-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: taglibs-user-help@jakarta.apache.org
---------------------------------
Jiyo cricket on Yahoo! India cricket
Yahoo! Messenger Mobile Stay in touch with your buddies all the time.
Re: Problem with
Posted by Murray Steele <mu...@peoplesarchive.com>.
Hi,
I ran up against this problem in my application. After some digging I
boiled it down to the fact that the servlet spec / tomcat versions I
was using didn't allow me to provide a Locale to character encoding
mapping, and when the fmt tags do pretty much anything, they end up
setting the Locale, which in turn sets the character encoding. I think
I read somewhere that some version of the servlet spec allows you to
specify Locale to character encoding mappings in the web.xml file, but
for what I'm using (2.3 on tomcat 4.1) what I had to do was create a
custom CharSetMapper and specify it on the <context> tag in my
server.xml.
My charset mapper ignores whatever locale you ask for a mapping for and
returns UTF-8 as the charset every the time. It seemed extreme, but
it's what I wanted. I compiled this and put it in
<tomcat_home>/server/classes/
UTF8CharSetMapper
--
package com.peoplesarchive.tomcat;
import java.util.Locale;
import org.apache.catalina.util.CharsetMapper;
public class UTF8CharsetMapper extends CharsetMapper {
public UTF8CharsetMapper() {
super();
}
public UTF8CharsetMapper(String name) {
super(name);
}
public String getCharset(Locale locale) {
return "utf-8";
}
}
server.xml fragment
--
<Context
charsetMapperClass="com.peoplesarchive.tomcat.UTF8CharsetMapper"
path="" docBase="ROOT" debug="0" reloadable="true" />
hope this helps
Muz
On 22 Mar 2006, at 09:43, ilanchezhian ganesamurthy wrote:
> Hello,
>
> I have problem while displaying of japanese UTF-8 character.
> I set response header character encoding to UTF-8 by <%@ page
> contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" %> then I
> read the UTF-8 japanese character by following statement
>
> <fmt:bundle basename="com.xxx.myBundle">
> <fmt:message key="k1"/>
> </fmt:bundle>
>
> Even though i set response character encoding to UTF-8 but when
> compiler start executing <fmt:bundle> it automatically chages response
> character encoding to 'Shift_JIS' from 'UTF-8' I can notice this by
> printing response character encoding before and after <fmt:bundle>.
> Due to this form is not displayed properly. I dont have issue with
> locale, I have only issue with response character encoding.
>
> Resource bundle contains UTF-8 character
> I have followig jsp
>
> <%@ page contentType="text/html; charset=UTF-8"
> pageEncoding="UTF-8" %>
> <%@ taglib uri="fmt" prefix="fmt" %>
> <html>
> <head>
> </head>
> <%System.out.println("CharacterEncoding before calling fmt =
> "+response.getCharacterEncoding() );%>
>
> <fmt:bundle basename="com.xxx.myBundle">
> <%System.out.println("CharacterEncoding after calling fmt =
> "+response.getCharacterEncoding() );%>
>
> <fmt:message key="k1"/>
> </fmt:bundle>
> </html>
>
> You can see result o/p to
>
> CharacterEncoding before calling fmt = UTF-8
> CharacterEncoding after calling fmt = Shift_JIS
>
> Cheers
> ilan
>
>
>
>
> ---------------------------------
> Jiyo cricket on Yahoo! India cricket
> Yahoo! Messenger Mobile Stay in touch with your buddies all the time.
--
Murray Steele
Head of Development
Peoples Archive
w: http://www.peoplesarchive.com
t: 0207 323 0323
d: 0207 631 9147
This email has been scanned by Postini.
For more information please visit http://www.postini.com
---------------------------------------------------------------------
To unsubscribe, e-mail: taglibs-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: taglibs-user-help@jakarta.apache.org