You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by "James H. H. Lampert" <ja...@touchtonecorp.com> on 2016/08/24 16:43:40 UTC
Character encoding issues
Ladies and Gentlemen of the Tomcat and Midrange-Java communities:
We're having a weird problem with character encoding in a Tomcat webapp.
We've added an interface to GMail to our webapp, and we've got, just for
our own development, testing, and production use, instances of that
webapp running in three different Tomcat 7 servers: one running on an
IBM Midrange box (an E4A, running V6R1, with Tomcat running on a Java 6
JVM), one running on a Windows box, and the third running on a Linux
(CentOS) box.
On the Midrange box, the traffic between us and GMail is getting garbled
(Chinese characters appear, seemingly at random), with an apparent
character encoding conflict. On the Linux box, it isn't. Not sure about
the Windows box.
Now, on the Midrange box, it's a fairly straightforward process to look
at the Java System Properties for a JVM. For the JVM Tomcat is running
in, "Initial Java System Properties" shows
> file.encoding 'ISO8859_1'
and "Current Java System Properties" shows
> os.encoding 'ISO8859-1'
> sun.jnu.encoding 'ISO8859-1'
> sun.io.unicode.encoding 'UnicodeBig'
> ibm.system.encoding 'ISO8859-1'
> file.encoding 'ISO8859_1'
I found JConsole and JVisualVM on the Linux box, and while I couldn't
find system properties in JConsole, I could in JVisualVM. I have:
> file.encoding=UTF-8
> sun.jnu.encoding=UTF-8
> sun.io.unicode.encoding=UnicodeLittle
Can somebody enlighten me on whether this is the cause of the encoding
issue with Google, and what to do about it?
--
JHHL
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Character encoding issues
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
James,
On 8/24/16 3:46 PM, James H. H. Lampert wrote:
> On 8/24/16, 12:36 PM, Mark Thomas wrote:
>
>> At a guess, something in the web application is using the
>> platform default encoding rather than an explicit encoding. Given
>> that the Linux box is OK, it looks like the app should be
>> explicitly using UTF-8 everywhere.
>
> Based on a response I got on the Midrange Java List, and on what
> I'd found since I entered the query, I would agree.
>
> What's the best way to accomplish this?
http://wiki.apache.org/tomcat/FAQ/CharacterEncoding#Q8
If you also want to change the default platform encoding, you'll have
to do the following (or equivalent on your E4A):
1. Edit CATALINA_BASE/bin/setenv.sh (or create it)
2. Add this line:
export CATALINA_OPTS="$CATALINA_OPTS -Dfile.encoding=UTF-8"
The above should change the values on the E4A's JVM so that everything
reports UTF-8 and nothing reports ISO-8859-1. The "UnicodeBig" is the
internal encoding that the JVM uses to represent Java primitive "char"
type and will probably only work with the value you've got there...
don't try to mess with that! :)
FWIW, the system properties of each JVM are somewhat interesting, but
probably won't help you debug anything. It might not even fix anything.
Every HTTP request/response is defined to have a character encoding:
1. As specified in the Content-Type header
2. To be ISO-8859-1 as the default in case no header exists
"Most" client software these days actually defaults to sending
requests without a Content-Type character encoding and instead just
using "whatever character encoding the server sent this page to me
using" as the character encoding. That's usually not the case with
back-end software, which is likely the case with your app.
Presumably, your application uses IMAP or similar to contact the gmail
server? In that case, HTTP isn't in use and it's possible that the
system properties defining the system character encoding are in use.
It all depends upon how the software works under the hood.
If HTTP is in use, here, then the problem exists in some component not
following the spec. Tomcat isn't part of the the problem, there. If
some other protocol is in use, it's entirely possible that default
"platform" (as defined by system properties) encoding is being used.
The only solution there would be to change the file.encoding property
as I've described above.
Let us know how it goes.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJXvhPQAAoJEBzwKT+lPKRY97kQAKPNSiS+6ZHWd/XvfgQMsGMB
zF/+gD75swub3qjz93vYYwG/OCykNE7ljpZJva3VNXdHwfnPkPe9xL4Kbn5uH/0i
CA4zztQW89Mkdhe9tGa4LDCv4tDWQGLhvKiGu3moFzcjKMSHTyIQe6wByX4SUk5N
HCbHf39avr6So60G55i7vWBPkeU9Du8Oa0T/X3NOlAlBIoSiDm2HKdvwb+3Fmeqn
W7JytdcRxoS5VwkIJFa3lsFt77Rz3ROV7KnCl7wrCVaxPs0RIM7DI9ojzzbMLt2s
S+nArR4gKwR0A5js+nRGJ/H53m1qiqUGvpb6HmUUz2pVSpTejGQFwVANwf54+IUY
uQKxud5XkB+JDN4f7+7ZKUn2l4kgrtYJxxyr2bzzYmHu3Z0AAMAqt78ZI7DYdCBZ
B0Gpdx6DPV0czsQs4g/usmF3M3hbAhkozYi7U5tzZfmUg2rIBfHKo4bX0GMEznJ9
5HvVJpRyLUPnXkA85wPi3aJwuvavFb9r51Kg17Vhuj74qcEacH4RwydE2vPRmVm1
WovpPjP0rwIpmJJlYq+RzzSXkYShiOZftqOKOeH/XSO+IwpQS2MlYpFUNiLpU4Y7
7qhatQQMcbmBHEFJ7jI1gJs/jkChm3iUWicOwju0XWoTshg0wEA3tEhGgZe8laN9
kLp4YiKoxMDPDX5uFQvO
=uvDy
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Character encoding issues
Posted by "James H. H. Lampert" <ja...@touchtonecorp.com>.
On 8/24/16, 12:36 PM, Mark Thomas wrote:
> At a guess, something in the web application is using the platform
> default encoding rather than an explicit encoding. Given that the Linux
> box is OK, it looks like the app should be explicitly using UTF-8
> everywhere.
Based on a response I got on the Midrange Java List, and on what I'd
found since I entered the query, I would agree.
What's the best way to accomplish this?
--
JHHL
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Character encoding issues
Posted by Mark Thomas <ma...@apache.org>.
On 24/08/2016 17:43, James H. H. Lampert wrote:
> Ladies and Gentlemen of the Tomcat and Midrange-Java communities:
>
> We're having a weird problem with character encoding in a Tomcat webapp.
>
> We've added an interface to GMail to our webapp, and we've got, just for
> our own development, testing, and production use, instances of that
> webapp running in three different Tomcat 7 servers: one running on an
> IBM Midrange box (an E4A, running V6R1, with Tomcat running on a Java 6
> JVM), one running on a Windows box, and the third running on a Linux
> (CentOS) box.
>
> On the Midrange box, the traffic between us and GMail is getting garbled
> (Chinese characters appear, seemingly at random), with an apparent
> character encoding conflict. On the Linux box, it isn't. Not sure about
> the Windows box.
>
> Now, on the Midrange box, it's a fairly straightforward process to look
> at the Java System Properties for a JVM. For the JVM Tomcat is running
> in, "Initial Java System Properties" shows
>> file.encoding 'ISO8859_1'
> and "Current Java System Properties" shows
>> os.encoding 'ISO8859-1'
>> sun.jnu.encoding 'ISO8859-1'
>> sun.io.unicode.encoding 'UnicodeBig'
>> ibm.system.encoding 'ISO8859-1'
>> file.encoding 'ISO8859_1'
>
> I found JConsole and JVisualVM on the Linux box, and while I couldn't
> find system properties in JConsole, I could in JVisualVM. I have:
>> file.encoding=UTF-8
>> sun.jnu.encoding=UTF-8
>> sun.io.unicode.encoding=UnicodeLittle
>
> Can somebody enlighten me on whether this is the cause of the encoding
> issue with Google, and what to do about it?
At a guess, something in the web application is using the platform
default encoding rather than an explicit encoding. Given that the Linux
box is OK, it looks like the app should be explicitly using UTF-8
everywhere.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org