You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-dev@xerces.apache.org by Jon Smirl <jo...@mediaone.net> on 2000/08/11 08:11:19 UTC
When is transcoding used?
Xalan is taking an encoded document and transcoding it to Unicode and then
firing off events. Xalan takes these events and transforms them. Xalan then
transcodes the Unicode to the output charset.
In practice, does anyone output from Xalan in a different charset than the
input document? I know this is a cool feature, but does anyone actually use
it? In reality do people actually mix charsets with xml:lang?
The question is: could Xalan/Xerces process the document in it's input
charset and only transcode when the document requires it? You could also
think of this as lazy transcoding. This may be too hard because of needing
things like substring() for an arbitrary charset. But, if you needed to do
substring() you could just transcode it first and then mark the string as
Unicode.
This is making me think that a Latin1 only version of Xalan/Xerces would be
valuable for environments that never deal in the other character sets.
Jon Smirl
jonsmirl@mediaone.net
Re: When is transcoding used?
Posted by Andy Heninger <an...@jtcsv.com>.
----- Original Message -----
From: "Andy Heninger" <an...@jtcsv.com>
To: <xe...@xml.apache.org>
Sent: Tuesday, August 15, 2000 3:05 PM
Subject: Re: When is transcoding used?
> All of this is pretty much idle speculation, though, at least for the
> development group at IBM. The >>>utf-8<<< encoded unicode based parser
> meets our needs.
>
Oooops, make that utf-16
>
> Andy Heninger
> IBM XML Technology Group, Cupertino, CA
> heninger@us.ibm.com
>
Re: When is transcoding used?
Posted by Andy Heninger <an...@jtcsv.com>.
"Jon Smirl" <jo...@mediaone.net> wrote
>
> The question is: could Xalan/Xerces process the document in it's input
> charset and only transcode when the document requires it? You could also
> think of this as lazy transcoding. This may be too hard because of
needing
> things like substring() for an arbitrary charset. But, if you needed to
do
> substring() you could just transcode it first and then mark the string
as
> Unicode.
>
> This is making me think that a Latin1 only version of Xalan/Xerces would
be
> valuable for environments that never deal in the other character sets.
>
The Xerces-J parser actually uses lazy transcoding. It works,
but introduces a great deal of complexity into the code base,
making it difficult to understand and maintain. And it's still
unclear whether there actually is a performance advantage.
An 8-bit ascii family parser would be a more interesting possibility.
All of the special characters and reserved kewywords are in the 7 bit
subset that is common to ASCII, latin-1, utf-8, etc, so parsing
could happen in a pure eight bit world with reasonably sane code.
EBCDIC and some ill-behaved multi-byte encodings would still need
upfront transcoding, but most could just go through as-is.
All of this is pretty much idle speculation, though, at least for the
development group at IBM. The utf-8 encoded unicode based parser
meets our needs.
Andy Heninger
IBM XML Technology Group, Cupertino, CA
heninger@us.ibm.com
Re: When is transcoding used?
Posted by Dean Roddey <dr...@charmedquark.com>.
Probably ain't gonna happen. The parser is inherently Unicode oriented. This
topic has been beaten to death already. And this is another one that
theoretically could be done conditionally, but the ramifications and
complexities it would add would be far more than its worth. It would also
require that a separate version be built and tested on every platform and
compiler.
--------------------------
Dean Roddey
The CIDLib C++ Frameworks
Charmed Quark Software
droddey@charmedquark.com
http://www.charmedquark.com
"You young, and you gotcha health. Whatchoo wanna job fer?"
----- Original Message -----
From: "Jon Smirl" <jo...@mediaone.net>
To: "xerces" <xe...@xml.apache.org>
Sent: Thursday, August 10, 2000 11:11 PM
Subject: When is transcoding used?
> Xalan is taking an encoded document and transcoding it to Unicode and then
> firing off events. Xalan takes these events and transforms them. Xalan
then
> transcodes the Unicode to the output charset.
>
> In practice, does anyone output from Xalan in a different charset than the
> input document? I know this is a cool feature, but does anyone actually
use
> it? In reality do people actually mix charsets with xml:lang?
>
> The question is: could Xalan/Xerces process the document in it's input
> charset and only transcode when the document requires it? You could also
> think of this as lazy transcoding. This may be too hard because of needing
> things like substring() for an arbitrary charset. But, if you needed to
do
> substring() you could just transcode it first and then mark the string as
> Unicode.
>
> This is making me think that a Latin1 only version of Xalan/Xerces would
be
> valuable for environments that never deal in the other character sets.
>
> Jon Smirl
> jonsmirl@mediaone.net
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-c-dev-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-c-dev-help@xml.apache.org
>
Re: When is transcoding used?
Posted by Jon Smirl <jo...@mediaone.net>.
From: "David E. Cleary" <da...@progress.com>
> In web based applications where you want to support the client's encoding,
I
> can see where you would use different encodings for the input and output
> documents.
This is the theoretical need, but when does this occur in practice? For
example how often is a Chinese speaker going to look at an English web page
and not have English fonts installed? IE will trigger an auto download of
English fonts if you try to do this. The user would need to cancel the font
download in order to have a need for a transcode.
It doesn't make any sense to return a page in English to machine that only
has Cyrillic or Arabic fonts installed. Transcoding doesn't do any good
because the English characters aren't in the destination charset.
Jon Smirl
jonsmirl@mediaone.net
RE: When is transcoding used?
Posted by "David E. Cleary" <da...@progress.com>.
> In practice, does anyone output from Xalan in a different charset than the
> input document? I know this is a cool feature, but does anyone
> actually use
> it? In reality do people actually mix charsets with xml:lang?
In web based applications where you want to support the client's encoding, I
can see where you would use different encodings for the input and output
documents.
David Cleary
Progress Software