[OT] unicode

[ma...@belgium.agfa.com]

hi,

I am fairly new to this unicode-topic, and I know this is not the list for it,
but I know a lot of you guys have dealt with the matter, so I ask it here...

I am writing a program on Windows that uses Xerces-c and that must support a
wide number of different languages. So Unicode is the answer here for internal
character and string manipulations. That much is clear to me (I think). I see
that the winNT API uses unicode and there is also support for it , for example
through the TCHAR header file. But then I see people using ICU library from
IBM. Why is that ?

I believe Xerces uses Unicode UTF-8 internally. Is this in a format I can use
right away, or does it need conversion of some sort to the internal unicode
(wide) characters, i.e. can I use XMLCh in a wstring, or should I transcode it
first to something ? What is the proper use of Xerces regarding this ?

And the other way around, when I generate an XML file, what should I print in
the encoding tag ? say I have a wstring and I send it to a file 'output.xml',
what encoding is used ? Can I choose the encoding that is used (UTF-8,
ISO-LATIN-1, ISO-LATIN-5,...) and what is the most appropriate to send it to a
server which uses Xerces-java to parse my XML ?


Sorry about all this questions in one mail, but this is all very confusing (and
new) to me... I would appreciate if someone can shed some light on it.

--manu--

Re: [OT] unicode

[Dean Roddey <dr...@charmedquark.com>]

Xerces (as of the most recent code base) uses UTF-16 exclusively for its
internal representation. On NT, you can pass that straight to OS system
calls.  When you generate the output file, what you put in the encoding
depends on what you put into the output file. If you use a wide character
stream to write it out, then you can put UTF-16 as the encoding and it
should be fine, since the wide char stream on NT would just pass the UTF-16
data straight through.

Or, you can transcode it to some other encoding and set the encoding string
appropriately. For storage or interchange purposes, encodings like UTF-16
and UTF-8 are best, because they can reprsent any Unicode data. Using any
other encoidng means that you will have to escape any chars that are not
supported in that encoding.

If you do need to output to encodings other than UTF-8, UTF-16, or
UCS-4/UTF-32, then you can use the XMLFormatter class to help do the right
thing.

--------------------------
Dean Roddey
The CIDLib C++ Frameworks
Charmed Quark Software
droddey@charmedquark.com
http://www.charmedquark.com

"It takes two buttocks to make friction"
    - African Proverb


----- Original Message -----
From: <ma...@belgium.agfa.com>
To: <xe...@xml.apache.org>
Sent: Tuesday, November 28, 2000 10:57 PM
Subject: [OT] unicode


> hi,
>
> I am fairly new to this unicode-topic, and I know this is not the list for
it,
> but I know a lot of you guys have dealt with the matter, so I ask it
here...
>
> I am writing a program on Windows that uses Xerces-c and that must support
a
> wide number of different languages. So Unicode is the answer here for
internal
> character and string manipulations. That much is clear to me (I think). I
see
> that the winNT API uses unicode and there is also support for it , for
example
> through the TCHAR header file. But then I see people using ICU library
from
> IBM. Why is that ?
>