You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-users@xerces.apache.org by Giulio Troccoli <Gi...@uk.linedata.com> on 2009/08/13 12:19:10 UTC
Invalid byte 1 (£) of a 1-byte sequence
Hello everybody.
We have been using Xerces and Xalan for few years and recently I have upgraded to Xerces 2.8 and Xalan 1.10. I have personally built both on Windows and AIX.
One of our applications produces an XML file that another application processes. This second application throws the error in this email subject.
First of all I would like to make sure I have my facts right.
The XML specifies an encoding of UTF-8 with <?xml version="1.0" encoding="UTF-8"?> in the first line. However, I don't think it's been saved in UTF-8 because if I open it as binary and go to where the £ is, I can see the following
BEFCE0: 20 A3 30 2E 35 E8 2E 3C 2F 6C 69 6E 65 3E 0A 20 £0.58.</line>.
I was expecting £ to be encoding with 2 bytes. Am I correct in assuming this?
If I am correct, then the error is correct too.
My question is about AIX. I don't have the error in AIX and the XML document is parsed correctly. Also I didn't have any problem with the pound sign with the old versions of Xerces and Xalan, 2.1 and 1.4 respectively, but that doesn't matter now.
Would any one be in a position to confirm that this is a bug in Xerces 2.8 on AIX?
If, of course, I change the encoding in the XML to ISO-8859-1 it works on Windows too, and that's probably what we will do, as it's the right thing to do. Still, I'd like to know whether there is a bug on AIX (so that I can say "it's a bug" when they ask me "why does it work on AIX then?")
Thanks
Giulio
Linedata Services (UK) Ltd
Registered Office: Bishopsgate Court, 4-12 Norton Folgate, London, E1 6DB
Registered in England and Wales No 3027851 VAT Reg No 778499447
RE: Invalid byte 1 (£) of a 1-byte sequence
Posted by Giulio Troccoli <Gi...@uk.linedata.com>.
>
Linedata Services (UK) Ltd
Registered Office: Bishopsgate Court, 4-12 Norton Folgate, London, E1 6DB
Registered in England and Wales No 3027851 VAT Reg No 778499447
-----Original Message-----
> From: Dale Worley [mailto:dworley@nortel.com]
> Sent: 14 August 2009 15:08
> To: c-users@xerces.apache.org
> Subject: RE: Invalid byte 1 (£) of a 1-byte sequence
>
> On Fri, 2009-08-14 at 09:29 +0100, Giulio Troccoli wrote:
> > My XML document is not in UTF-8. The pound sign is just A3,
> not C2 A3.
> >
> > But I'm telling my application that the document IS in UTF-8 (using
> > the encoding="UTF-8" option).
> >
> > Windows correctly rejects it. AIX does not.
> >
> > When you say "make sure all of your processing is in
> UTF-8", I can't
> > do that. The XML is not in UTF-8 and I can not change that (it's
> > created by a C programme and I have no idea how to do that).
>
> One question to ask is why the contents of the file say that
> it is in UTF-8, but the file is not, in fact, in UTF-8.
> Whatever is generating the file is malfunctioning badly.
Mystery solved!!
Xerces does not have a bug. The reason why on AIX it was accepting the pound sign without problems is that we were still using the old Xerces 2.1 (which has never given us this problem on any platforms). As soon as I relinked the application with the new Xerces we had the same behaviour than on Windows.
The file is, as I said, created by a C programme, written 7-8 years ago. Since the beginning the XML files had the encoding="UTF-8" at the top without being actually UTF-8. But because Xerces didn't say anything (I'm sure we had the pound sign before) we didn't realise the inconsistency. Now that we have, I am going to change the programme to say that the XML is in fact ISO-8859-1.
Thanks to everyone for their help.
Giulio
RE: Invalid byte 1 (£) of a 1-byte sequence
Posted by Dale Worley <dw...@nortel.com>.
On Fri, 2009-08-14 at 09:29 +0100, Giulio Troccoli wrote:
> My XML document is not in UTF-8. The pound sign is just A3, not C2 A3.
>
> But I'm telling my application that the document IS in UTF-8 (using
> the encoding="UTF-8" option).
>
> Windows correctly rejects it. AIX does not.
>
> When you say "make sure all of your processing is in UTF-8", I can't
> do that. The XML is not in UTF-8 and I can not change that (it's
> created by a C programme and I have no idea how to do that).
One question to ask is why the contents of the file say that it is in
UTF-8, but the file is not, in fact, in UTF-8. Whatever is generating
the file is malfunctioning badly.
Dale
Re: Invalid byte 1 (£) of a 1-byte sequence
Posted by David Bertoni <db...@apache.org>.
Giulio Troccoli wrote:
>
>
> Linedata Services (UK) Ltd
> Registered Office: Bishopsgate Court, 4-12 Norton Folgate, London, E1 6DB
> Registered in England and Wales No 3027851 VAT Reg No 778499447
>
> -----Original Message-----
>
>
>> From: David Bertoni [mailto:dbertoni@apache.org]
>> Sent: 13 August 2009 18:38
>> To: c-users@xerces.apache.org
>> Subject: Re: Invalid byte 1 (£) of a 1-byte sequence
>>
>> Giulio Troccoli wrote:
>>> Well, I configure the built as follows
>>>
>>> runConfigure -paix -cxlc -xxlC_r
>>>
>>> So it should have used the 'native' for transcoding. I'm
>> afraid I don't know about this to be of more help.
>> What could be happening is there's local code page
>> transcoding somewhere in your processing stream, and on
>> Windows, you're getting a Windows code page (probably 1252),
>> not UTF-8.
>>
>> Note that Windows-1252 and ISO-8859-1 are not compatible, so
>> don't assume you can interchange them. Rather than hack
>> around your source files, you should make sure all of your
>> processing is in UTF-8.
>>
>> You should also run the locale command on your AIX machine to
>> verify what code page it's using. If it's UTF-8, that would
>> be further evidence your application is doing inappropriate
>> local code page transcoding.
>
> I'm a bit out of depth here so forgive me if I say soemthing really stupid (which is quite likely).
>
> My XML document is not in UTF-8. The pound sign is just A3, not C2 A3.
Yes, but are you sure you're processing the exact same byte stream on AIX?
> But I'm telling my application that the document IS in UTF-8 (using the encoding="UTF-8" option).
>
> Windows correctly rejects it. AIX does not.
This would be a major bug if you were processing identical documents. I
haven't used Xerces-C on AIX for a while, but when I used it, ill-formed
UTF-8 byte sequences were rejected correctly.
>
> When you say "make sure all of your processing is in UTF-8", I can't do that. The XML is not in UTF-8 and I can not change that (it's created by a C programme and I have no idea how to do that).
Then why is your program generating an incorrect encoding declaration?
It seems to me the program is assuming it's generating UTF-8, but it's
not. You need to fix that.
>
> I ran some locale commands on my AIX box and here's the result
>
> ibuild@kylie% locale
> LANG=en_US
> LC_COLLATE="en_US"
> LC_CTYPE="en_US"
> LC_MONETARY="en_US"
> LC_NUMERIC="en_US"
> LC_TIME="en_US"
> LC_MESSAGES="en_US"
> LC_ALL=
>
> ibuild@kylie% locale charmap
> ISO8859-1
>
> Is this why Xerces on AIX understands that A3 is in fact the pound sign?
Xerces-C doesn't rely on the locale settings, except when transcoding to
and from the local code, using XMLString::transcode(), or when the local
code page transcoder is used.
Perhaps you need to provide more of your code, so we can see how the
data is getting to the parser.
Dave
RE: Invalid byte 1 (£) of a 1-byte sequence
Posted by John Lilley <jl...@datalever.com>.
Either a bug in Xerces or the underlying transcoder, whatever that is... while it would be best to fix Xerces, assuming that is the problem, there may also be a workaround.
-----Original Message-----
From: Giulio Troccoli [mailto:Giulio.Troccoli@uk.linedata.com]
Sent: Friday, August 14, 2009 5:47 AM
To: c-users@xerces.apache.org
Subject: RE: Invalid byte 1 (£) of a 1-byte sequence
It won't be easy but I'll give it a go and report back.
In any case, that would be a bug in Xerces, wouldn't it? So the "approrpiate corrective action" would be to change my application to work around this bug, wouldn't it?
Giulio
RE: Invalid byte 1 (£) of a 1-byte sequence
Posted by Giulio Troccoli <Gi...@uk.linedata.com>.
>
Linedata Services (UK) Ltd
Registered Office: Bishopsgate Court, 4-12 Norton Folgate, London, E1 6DB
Registered in England and Wales No 3027851 VAT Reg No 778499447
-----Original Message-----
> From: John Lilley [mailto:jlilley@datalever.com]
> Sent: 14 August 2009 12:42
> To: c-users@xerces.apache.org
> Subject: RE: Invalid byte 1 (£) of a 1-byte sequence
>
> I will also quite likely say some thing stupid, but here goes :)
>
> I suggest that there are two possibilities:
>
> 1) Xerces on AIX is ignoring your request to use UTF-8, and
> is instead using the default 8859-1
> 2) Xerces, or the underlying transcoder it uses, is
> translating UTF-8, but is too lenient when it encounters the
> invalid escape sequence, and makes some ad-hoc (or buggy)
> attempt to convert the code anyway.
>
> I would suggest this experiment: feed the parser a document
> containing the valid sequence (C2 A3) and see if it is parsed
> correctly. If so, then the answer is most likely (2) else
> (1). Armed with that information you can seek the
> appropriate corrective action.
It won't be easy but I'll give it a go and report back.
In any case, that would be a bug in Xerces, wouldn't it? So the "approrpiate corrective action" would be to change my application to work around this bug, wouldn't it?
Giulio
RE: Invalid byte 1 (£) of a 1-byte sequence
Posted by John Lilley <jl...@datalever.com>.
I will also quite likely say some thing stupid, but here goes :)
I suggest that there are two possibilities:
1) Xerces on AIX is ignoring your request to use UTF-8, and is instead using the default 8859-1
2) Xerces, or the underlying transcoder it uses, is translating UTF-8, but is too lenient when it encounters the invalid escape sequence, and makes some ad-hoc (or buggy) attempt to convert the code anyway.
I would suggest this experiment: feed the parser a document containing the valid sequence (C2 A3) and see if it is parsed correctly. If so, then the answer is most likely (2) else (1). Armed with that information you can seek the appropriate corrective action.
john
-----Original Message-----
From: Giulio Troccoli [mailto:Giulio.Troccoli@uk.linedata.com]
Sent: Friday, August 14, 2009 2:29 AM
To: c-users@xerces.apache.org
Subject: RE: Invalid byte 1 (£) of a 1-byte sequence
>
Linedata Services (UK) Ltd
Registered Office: Bishopsgate Court, 4-12 Norton Folgate, London, E1 6DB
Registered in England and Wales No 3027851 VAT Reg No 778499447
-----Original Message-----
> From: David Bertoni [mailto:dbertoni@apache.org]
> Sent: 13 August 2009 18:38
> To: c-users@xerces.apache.org
> Subject: Re: Invalid byte 1 (£) of a 1-byte sequence
>
> Giulio Troccoli wrote:
> > Well, I configure the built as follows
> >
> > runConfigure -paix -cxlc -xxlC_r
> >
> > So it should have used the 'native' for transcoding. I'm
> afraid I don't know about this to be of more help.
> What could be happening is there's local code page
> transcoding somewhere in your processing stream, and on
> Windows, you're getting a Windows code page (probably 1252),
> not UTF-8.
>
> Note that Windows-1252 and ISO-8859-1 are not compatible, so
> don't assume you can interchange them. Rather than hack
> around your source files, you should make sure all of your
> processing is in UTF-8.
>
> You should also run the locale command on your AIX machine to
> verify what code page it's using. If it's UTF-8, that would
> be further evidence your application is doing inappropriate
> local code page transcoding.
I'm a bit out of depth here so forgive me if I say soemthing really stupid (which is quite likely).
My XML document is not in UTF-8. The pound sign is just A3, not C2 A3.
But I'm telling my application that the document IS in UTF-8 (using the encoding="UTF-8" option).
Windows correctly rejects it. AIX does not.
When you say "make sure all of your processing is in UTF-8", I can't do that. The XML is not in UTF-8 and I can not change that (it's created by a C programme and I have no idea how to do that).
I ran some locale commands on my AIX box and here's the result
ibuild@kylie% locale
LANG=en_US
LC_COLLATE="en_US"
LC_CTYPE="en_US"
LC_MONETARY="en_US"
LC_NUMERIC="en_US"
LC_TIME="en_US"
LC_MESSAGES="en_US"
LC_ALL=
ibuild@kylie% locale charmap
ISO8859-1
Is this why Xerces on AIX understands that A3 is in fact the pound sign?
Giulio
RE: Invalid byte 1 (£) of a 1-byte sequence
Posted by Giulio Troccoli <Gi...@uk.linedata.com>.
>
Linedata Services (UK) Ltd
Registered Office: Bishopsgate Court, 4-12 Norton Folgate, London, E1 6DB
Registered in England and Wales No 3027851 VAT Reg No 778499447
-----Original Message-----
> From: David Bertoni [mailto:dbertoni@apache.org]
> Sent: 13 August 2009 18:38
> To: c-users@xerces.apache.org
> Subject: Re: Invalid byte 1 (£) of a 1-byte sequence
>
> Giulio Troccoli wrote:
> > Well, I configure the built as follows
> >
> > runConfigure -paix -cxlc -xxlC_r
> >
> > So it should have used the 'native' for transcoding. I'm
> afraid I don't know about this to be of more help.
> What could be happening is there's local code page
> transcoding somewhere in your processing stream, and on
> Windows, you're getting a Windows code page (probably 1252),
> not UTF-8.
>
> Note that Windows-1252 and ISO-8859-1 are not compatible, so
> don't assume you can interchange them. Rather than hack
> around your source files, you should make sure all of your
> processing is in UTF-8.
>
> You should also run the locale command on your AIX machine to
> verify what code page it's using. If it's UTF-8, that would
> be further evidence your application is doing inappropriate
> local code page transcoding.
I'm a bit out of depth here so forgive me if I say soemthing really stupid (which is quite likely).
My XML document is not in UTF-8. The pound sign is just A3, not C2 A3.
But I'm telling my application that the document IS in UTF-8 (using the encoding="UTF-8" option).
Windows correctly rejects it. AIX does not.
When you say "make sure all of your processing is in UTF-8", I can't do that. The XML is not in UTF-8 and I can not change that (it's created by a C programme and I have no idea how to do that).
I ran some locale commands on my AIX box and here's the result
ibuild@kylie% locale
LANG=en_US
LC_COLLATE="en_US"
LC_CTYPE="en_US"
LC_MONETARY="en_US"
LC_NUMERIC="en_US"
LC_TIME="en_US"
LC_MESSAGES="en_US"
LC_ALL=
ibuild@kylie% locale charmap
ISO8859-1
Is this why Xerces on AIX understands that A3 is in fact the pound sign?
Giulio
Re: Invalid byte 1 (£) of a 1-byte sequence
Posted by David Bertoni <db...@apache.org>.
Giulio Troccoli wrote:
> Well, I configure the built as follows
>
> runConfigure -paix -cxlc -xxlC_r
>
> So it should have used the 'native' for transcoding. I'm afraid I don't know about this to be of more help.
What could be happening is there's local code page transcoding somewhere
in your processing stream, and on Windows, you're getting a Windows code
page (probably 1252), not UTF-8.
Note that Windows-1252 and ISO-8859-1 are not compatible, so don't
assume you can interchange them. Rather than hack around your source
files, you should make sure all of your processing is in UTF-8.
You should also run the locale command on your AIX machine to verify
what code page it's using. If it's UTF-8, that would be further evidence
your application is doing inappropriate local code page transcoding.
Dave
RE: Invalid byte 1 (£) of a 1-byte sequence
Posted by Giulio Troccoli <Gi...@uk.linedata.com>.
Well, I configure the built as follows
runConfigure -paix -cxlc -xxlC_r
So it should have used the 'native' for transcoding. I'm afraid I don't know about this to be of more help.
> -----Original Message-----
> From: John Lilley [mailto:jlilley@datalever.com]
> Sent: 13 August 2009 16:11
> To: c-users@xerces.apache.org
> Subject: RE: Invalid byte 1 (£) of a 1-byte sequence
>
> Could the difference be that on AIX Xerces is built to use
> ICU for transcoding, and ICU does not throw an exception for
> an invalid UTF8 sequence?
>
> If you give the AIX parser a valid UTF-8 sequence above ASCII
> range does it work correctly? The UTF8 for pound sign is 0xC2 0xA3.
>
> john
>
> -----Original Message-----
> From: Giulio Troccoli [mailto:Giulio.Troccoli@uk.linedata.com]
> Sent: Thursday, August 13, 2009 8:50 AM
> To: 'c-users@xerces.apache.org'
> Subject: RE: Invalid byte 1 (£) of a 1-byte sequence
>
> Thanks John, but my XML file is not in UTF-8, as I was trying
> to demonstrate with the hex dump extract (but if I'm wrong
> please let me know).
>
> And I tell Xerces that it is so it throws the error which is
> correct. All I wanted to know is why the same thing does not
> happen on AIX. Instead, on AIX, Xerces parses the file and
> even gets the pound sign. It seems it silently switch to
> ISO-8859-1 when it gets to the pound sign seeing that it's
> not in UTF-8.
>
> > -----Original Message-----
> > From: John Lilley [mailto:jlilley@datalever.com]
> > Sent: 13 August 2009 15:46
> > To: c-users@xerces.apache.org
> > Subject: RE: Invalid byte 1 (£) of a 1-byte sequence
> >
> > We've had no trouble reading and writing UTF-8. I set up
> an example
> > using the british pound symbol like you have, and Xerces
> 2.8 correctly
> > encodes and decodes it. We do not even supply XML header as it
> > defaults to UTF-8. Do you have a code snippet that produces this
> > output?
> >
> > john
> >
> > -----Original Message-----
> > From: Giulio Troccoli [mailto:Giulio.Troccoli@uk.linedata.com]
> > Sent: Thursday, August 13, 2009 4:19 AM
> > To: c-users@xerces.apache.org
> > Subject: Invalid byte 1 (£) of a 1-byte sequence
> >
> > Hello everybody.
> >
> > We have been using Xerces and Xalan for few years and
> recently I have
> > upgraded to Xerces 2.8 and Xalan 1.10. I have personally
> built both on
> > Windows and AIX.
> >
> > One of our applications produces an XML file that another
> application
> > processes. This second application throws the error in this email
> > subject.
> >
> > First of all I would like to make sure I have my facts right.
> >
> > The XML specifies an encoding of UTF-8 with <?xml version="1.0"
> > encoding="UTF-8"?> in the first line. However, I don't
> think it's been
> > saved in UTF-8 because if I open it as binary and go to where the £
> > is, I can see the following
> >
> > BEFCE0: 20 A3 30 2E 35 E8 2E 3C 2F 6C 69 6E 65 3E 0A 20
> > £0.58.</line>.
> >
> > I was expecting £ to be encoding with 2 bytes. Am I correct in
> > assuming this?
> >
> > If I am correct, then the error is correct too.
> >
> > My question is about AIX. I don't have the error in AIX and the XML
> > document is parsed correctly. Also I didn't have any
> problem with the
> > pound sign with the old versions of Xerces and Xalan, 2.1 and 1.4
> > respectively, but that doesn't matter now.
> >
> > Would any one be in a position to confirm that this is a
> bug in Xerces
> > 2.8 on AIX?
> >
> > If, of course, I change the encoding in the XML to
> ISO-8859-1 it works
> > on Windows too, and that's probably what we will do, as
> it's the right
> > thing to do. Still, I'd like to know whether there is a bug
> on AIX (so
> > that I can say "it's a bug"
> > when they ask me "why does it work on AIX then?")
> >
> > Thanks
> > Giulio
> >
> >
> > Linedata Services (UK) Ltd
> > Registered Office: Bishopsgate Court, 4-12 Norton Folgate,
> London, E1
> > 6DB
> > Registered in England and Wales No 3027851 VAT Reg No 778499447
> >
> >
> >
> >
RE: Invalid byte 1 (£) of a 1-byte sequence
Posted by John Lilley <jl...@datalever.com>.
Could the difference be that on AIX Xerces is built to use ICU for transcoding, and ICU does not throw an exception for an invalid UTF8 sequence?
If you give the AIX parser a valid UTF-8 sequence above ASCII range does it work correctly? The UTF8 for pound sign is 0xC2 0xA3.
john
-----Original Message-----
From: Giulio Troccoli [mailto:Giulio.Troccoli@uk.linedata.com]
Sent: Thursday, August 13, 2009 8:50 AM
To: 'c-users@xerces.apache.org'
Subject: RE: Invalid byte 1 (£) of a 1-byte sequence
Thanks John, but my XML file is not in UTF-8, as I was trying to demonstrate with the hex dump extract (but if I'm wrong please let me know).
And I tell Xerces that it is so it throws the error which is correct. All I wanted to know is why the same thing does not happen on AIX. Instead, on AIX, Xerces parses the file and even gets the pound sign. It seems it silently switch to ISO-8859-1 when it gets to the pound sign seeing that it's not in UTF-8.
> -----Original Message-----
> From: John Lilley [mailto:jlilley@datalever.com]
> Sent: 13 August 2009 15:46
> To: c-users@xerces.apache.org
> Subject: RE: Invalid byte 1 (£) of a 1-byte sequence
>
> We've had no trouble reading and writing UTF-8. I set up an
> example using the british pound symbol like you have, and
> Xerces 2.8 correctly encodes and decodes it. We do not even
> supply XML header as it defaults to UTF-8. Do you have a
> code snippet that produces this output?
>
> john
>
> -----Original Message-----
> From: Giulio Troccoli [mailto:Giulio.Troccoli@uk.linedata.com]
> Sent: Thursday, August 13, 2009 4:19 AM
> To: c-users@xerces.apache.org
> Subject: Invalid byte 1 (£) of a 1-byte sequence
>
> Hello everybody.
>
> We have been using Xerces and Xalan for few years and
> recently I have upgraded to Xerces 2.8 and Xalan 1.10. I have
> personally built both on Windows and AIX.
>
> One of our applications produces an XML file that another
> application processes. This second application throws the
> error in this email subject.
>
> First of all I would like to make sure I have my facts right.
>
> The XML specifies an encoding of UTF-8 with <?xml
> version="1.0" encoding="UTF-8"?> in the first line. However,
> I don't think it's been saved in UTF-8 because if I open it
> as binary and go to where the £ is, I can see the following
>
> BEFCE0: 20 A3 30 2E 35 E8 2E 3C 2F 6C 69 6E 65 3E 0A 20
> £0.58.</line>.
>
> I was expecting £ to be encoding with 2 bytes. Am I correct
> in assuming this?
>
> If I am correct, then the error is correct too.
>
> My question is about AIX. I don't have the error in AIX and
> the XML document is parsed correctly. Also I didn't have any
> problem with the pound sign with the old versions of Xerces
> and Xalan, 2.1 and 1.4 respectively, but that doesn't matter now.
>
> Would any one be in a position to confirm that this is a bug
> in Xerces 2.8 on AIX?
>
> If, of course, I change the encoding in the XML to ISO-8859-1
> it works on Windows too, and that's probably what we will do,
> as it's the right thing to do. Still, I'd like to know
> whether there is a bug on AIX (so that I can say "it's a bug"
> when they ask me "why does it work on AIX then?")
>
> Thanks
> Giulio
>
>
> Linedata Services (UK) Ltd
> Registered Office: Bishopsgate Court, 4-12 Norton Folgate,
> London, E1 6DB
> Registered in England and Wales No 3027851 VAT Reg No 778499447
>
>
>
>
RE: Invalid byte 1 (£) of a 1-byte sequence
Posted by Giulio Troccoli <Gi...@uk.linedata.com>.
Thanks John, but my XML file is not in UTF-8, as I was trying to demonstrate with the hex dump extract (but if I'm wrong please let me know).
And I tell Xerces that it is so it throws the error which is correct. All I wanted to know is why the same thing does not happen on AIX. Instead, on AIX, Xerces parses the file and even gets the pound sign. It seems it silently switch to ISO-8859-1 when it gets to the pound sign seeing that it's not in UTF-8.
> -----Original Message-----
> From: John Lilley [mailto:jlilley@datalever.com]
> Sent: 13 August 2009 15:46
> To: c-users@xerces.apache.org
> Subject: RE: Invalid byte 1 (£) of a 1-byte sequence
>
> We've had no trouble reading and writing UTF-8. I set up an
> example using the british pound symbol like you have, and
> Xerces 2.8 correctly encodes and decodes it. We do not even
> supply XML header as it defaults to UTF-8. Do you have a
> code snippet that produces this output?
>
> john
>
> -----Original Message-----
> From: Giulio Troccoli [mailto:Giulio.Troccoli@uk.linedata.com]
> Sent: Thursday, August 13, 2009 4:19 AM
> To: c-users@xerces.apache.org
> Subject: Invalid byte 1 (£) of a 1-byte sequence
>
> Hello everybody.
>
> We have been using Xerces and Xalan for few years and
> recently I have upgraded to Xerces 2.8 and Xalan 1.10. I have
> personally built both on Windows and AIX.
>
> One of our applications produces an XML file that another
> application processes. This second application throws the
> error in this email subject.
>
> First of all I would like to make sure I have my facts right.
>
> The XML specifies an encoding of UTF-8 with <?xml
> version="1.0" encoding="UTF-8"?> in the first line. However,
> I don't think it's been saved in UTF-8 because if I open it
> as binary and go to where the £ is, I can see the following
>
> BEFCE0: 20 A3 30 2E 35 E8 2E 3C 2F 6C 69 6E 65 3E 0A 20
> £0.58.</line>.
>
> I was expecting £ to be encoding with 2 bytes. Am I correct
> in assuming this?
>
> If I am correct, then the error is correct too.
>
> My question is about AIX. I don't have the error in AIX and
> the XML document is parsed correctly. Also I didn't have any
> problem with the pound sign with the old versions of Xerces
> and Xalan, 2.1 and 1.4 respectively, but that doesn't matter now.
>
> Would any one be in a position to confirm that this is a bug
> in Xerces 2.8 on AIX?
>
> If, of course, I change the encoding in the XML to ISO-8859-1
> it works on Windows too, and that's probably what we will do,
> as it's the right thing to do. Still, I'd like to know
> whether there is a bug on AIX (so that I can say "it's a bug"
> when they ask me "why does it work on AIX then?")
>
> Thanks
> Giulio
>
>
> Linedata Services (UK) Ltd
> Registered Office: Bishopsgate Court, 4-12 Norton Folgate,
> London, E1 6DB
> Registered in England and Wales No 3027851 VAT Reg No 778499447
>
>
>
>
RE: Invalid byte 1 (£) of a 1-byte sequence
Posted by John Lilley <jl...@datalever.com>.
We've had no trouble reading and writing UTF-8. I set up an example using the british pound symbol like you have, and Xerces 2.8 correctly encodes and decodes it. We do not even supply XML header as it defaults to UTF-8. Do you have a code snippet that produces this output?
john
-----Original Message-----
From: Giulio Troccoli [mailto:Giulio.Troccoli@uk.linedata.com]
Sent: Thursday, August 13, 2009 4:19 AM
To: c-users@xerces.apache.org
Subject: Invalid byte 1 (£) of a 1-byte sequence
Hello everybody.
We have been using Xerces and Xalan for few years and recently I have upgraded to Xerces 2.8 and Xalan 1.10. I have personally built both on Windows and AIX.
One of our applications produces an XML file that another application processes. This second application throws the error in this email subject.
First of all I would like to make sure I have my facts right.
The XML specifies an encoding of UTF-8 with <?xml version="1.0" encoding="UTF-8"?> in the first line. However, I don't think it's been saved in UTF-8 because if I open it as binary and go to where the £ is, I can see the following
BEFCE0: 20 A3 30 2E 35 E8 2E 3C 2F 6C 69 6E 65 3E 0A 20 £0.58.</line>.
I was expecting £ to be encoding with 2 bytes. Am I correct in assuming this?
If I am correct, then the error is correct too.
My question is about AIX. I don't have the error in AIX and the XML document is parsed correctly. Also I didn't have any problem with the pound sign with the old versions of Xerces and Xalan, 2.1 and 1.4 respectively, but that doesn't matter now.
Would any one be in a position to confirm that this is a bug in Xerces 2.8 on AIX?
If, of course, I change the encoding in the XML to ISO-8859-1 it works on Windows too, and that's probably what we will do, as it's the right thing to do. Still, I'd like to know whether there is a bug on AIX (so that I can say "it's a bug" when they ask me "why does it work on AIX then?")
Thanks
Giulio
Linedata Services (UK) Ltd
Registered Office: Bishopsgate Court, 4-12 Norton Folgate, London, E1 6DB
Registered in England and Wales No 3027851 VAT Reg No 778499447