You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by "mr.andersen" <xm...@bec.dk> on 2008/03/05 10:00:57 UTC

Nordic charaters problem in SOAP/MQ

I have at problem with sending nordic charaters such as æøå via SOAP over MQ.
I'm using cxf version 2.0.1 and SAAJ version 1.3.

I have tried to trace where is could go wrong and what I can see, the å
looks okay when it is added to W3CDOMXMLStreamWriter.writeCharaters(), but
when the SOAPMessage.writeTo() method is called with System.out or
OutputStream in SAAJOutInterceptor it is viewed as å.

Can anyone tell me if I can do anything to avoid this behaviour or have it
something to do with the SAAJ implementation?

-- 
View this message in context: http://www.nabble.com/Nordic-charaters-problem-in-SOAP-MQ-tp15845741p15845741.html
Sent from the cxf-user mailing list archive at Nabble.com.

unsubscribe

Posted by al...@orange-ftgroup.com.


*********************************
This message and any attachments (the "message") are confidential and intended solely for the addressees. 
Any unauthorised use or dissemination is prohibited.
Messages are susceptible to alteration. 
France Telecom Group shall not be liable for the message if altered, changed or falsified.
If you are not the intended addressee of this message, please cancel it immediately and inform the sender.
********************************

Re: Nordic charaters problem in SOAP/MQ

Posted by "mr.andersen" <xm...@bec.dk>.
Hmm - maybe I should look have the MQ is setup - but I don't think the
problems has come so far yet, since the å (0xC3 0xA5) is wrong before the
message is put on a queue.


Benson Margulies-4 wrote:
> 
> Could you please send along these characters in hex? I am not confident
> that
> we're seeing the same stuff.
> 
> I suspect that the CXF->MQ connector is failing to tell MQ about UTF-8
> somehow, but I need a little more traction.
> 
> On Wed, Mar 5, 2008 at 7:31 AM, mr.andersen <xm...@bec.dk> wrote:
> 
>>
>> I'm expecting UTF-8 since the reciever expects that I live up to WS-I
>> Basic
>> Profile.
>> And it's also my understanding that CXF uses UTF-8 all the way, but I
>> cannot
>> understand why my å is converted to Ã¥ when SOAPMessage.writeTo() is
>> used.
>>
>>
>> Benson Margulies-4 wrote:
>> >
>> > In every other area, CXF writes UTF-8. Are you expecting UTF-8, or
>> > something
>> > else?
>> >
>> > On Wed, Mar 5, 2008 at 4:00 AM, mr.andersen <xm...@bec.dk> wrote:
>> >
>> >>
>> >> I have at problem with sending nordic charaters such as æøå via SOAP
>> over
>> >> MQ.
>> >> I'm using cxf version 2.0.1 and SAAJ version 1.3.
>> >>
>> >> I have tried to trace where is could go wrong and what I can see, the
>> å
>> >> looks okay when it is added to W3CDOMXMLStreamWriter.writeCharaters(),
>> >> but
>> >> when the SOAPMessage.writeTo() method is called with System.out or
>> >> OutputStream in SAAJOutInterceptor it is viewed as å.
>> >>
>> >> Can anyone tell me if I can do anything to avoid this behaviour or
>> have
>> >> it
>> >> something to do with the SAAJ implementation?
>> >>
>> >> --
>> >> View this message in context:
>> >>
>> http://www.nabble.com/Nordic-charaters-problem-in-SOAP-MQ-tp15845741p15845741.html
>> >> Sent from the cxf-user mailing list archive at Nabble.com.
>> >>
>> >>
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Nordic-charaters-problem-in-SOAP-MQ-tp15845741p15848926.html
>> Sent from the cxf-user mailing list archive at Nabble.com.
>>
>>
> 
> 

-- 
View this message in context: http://www.nabble.com/Nordic-charaters-problem-in-SOAP-MQ-tp15845741p15867235.html
Sent from the cxf-user mailing list archive at Nabble.com.

Re: Nordic charaters problem in SOAP/MQ

Posted by Benson Margulies <bi...@gmail.com>.
Could you please send along these characters in hex? I am not confident that
we're seeing the same stuff.

I suspect that the CXF->MQ connector is failing to tell MQ about UTF-8
somehow, but I need a little more traction.

On Wed, Mar 5, 2008 at 7:31 AM, mr.andersen <xm...@bec.dk> wrote:

>
> I'm expecting UTF-8 since the reciever expects that I live up to WS-I
> Basic
> Profile.
> And it's also my understanding that CXF uses UTF-8 all the way, but I
> cannot
> understand why my å is converted to Ã¥ when SOAPMessage.writeTo() is used.
>
>
> Benson Margulies-4 wrote:
> >
> > In every other area, CXF writes UTF-8. Are you expecting UTF-8, or
> > something
> > else?
> >
> > On Wed, Mar 5, 2008 at 4:00 AM, mr.andersen <xm...@bec.dk> wrote:
> >
> >>
> >> I have at problem with sending nordic charaters such as æøå via SOAP
> over
> >> MQ.
> >> I'm using cxf version 2.0.1 and SAAJ version 1.3.
> >>
> >> I have tried to trace where is could go wrong and what I can see, the å
> >> looks okay when it is added to W3CDOMXMLStreamWriter.writeCharaters(),
> >> but
> >> when the SOAPMessage.writeTo() method is called with System.out or
> >> OutputStream in SAAJOutInterceptor it is viewed as å.
> >>
> >> Can anyone tell me if I can do anything to avoid this behaviour or have
> >> it
> >> something to do with the SAAJ implementation?
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Nordic-charaters-problem-in-SOAP-MQ-tp15845741p15845741.html
> >> Sent from the cxf-user mailing list archive at Nabble.com.
> >>
> >>
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Nordic-charaters-problem-in-SOAP-MQ-tp15845741p15848926.html
> Sent from the cxf-user mailing list archive at Nabble.com.
>
>

Re: Nordic charaters problem in SOAP/MQ

Posted by "mr.andersen" <xm...@bec.dk>.
I'm expecting UTF-8 since the reciever expects that I live up to WS-I Basic
Profile.
And it's also my understanding that CXF uses UTF-8 all the way, but I cannot
understand why my å is converted to Ã¥ when SOAPMessage.writeTo() is used.


Benson Margulies-4 wrote:
> 
> In every other area, CXF writes UTF-8. Are you expecting UTF-8, or
> something
> else?
> 
> On Wed, Mar 5, 2008 at 4:00 AM, mr.andersen <xm...@bec.dk> wrote:
> 
>>
>> I have at problem with sending nordic charaters such as æøå via SOAP over
>> MQ.
>> I'm using cxf version 2.0.1 and SAAJ version 1.3.
>>
>> I have tried to trace where is could go wrong and what I can see, the å
>> looks okay when it is added to W3CDOMXMLStreamWriter.writeCharaters(),
>> but
>> when the SOAPMessage.writeTo() method is called with System.out or
>> OutputStream in SAAJOutInterceptor it is viewed as å.
>>
>> Can anyone tell me if I can do anything to avoid this behaviour or have
>> it
>> something to do with the SAAJ implementation?
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Nordic-charaters-problem-in-SOAP-MQ-tp15845741p15845741.html
>> Sent from the cxf-user mailing list archive at Nabble.com.
>>
>>
> 
> 

-- 
View this message in context: http://www.nabble.com/Nordic-charaters-problem-in-SOAP-MQ-tp15845741p15848926.html
Sent from the cxf-user mailing list archive at Nabble.com.

Re: Nordic charaters problem in SOAP/MQ

Posted by Benson Margulies <bi...@gmail.com>.
In every other area, CXF writes UTF-8. Are you expecting UTF-8, or something
else?

On Wed, Mar 5, 2008 at 4:00 AM, mr.andersen <xm...@bec.dk> wrote:

>
> I have at problem with sending nordic charaters such as æøå via SOAP over
> MQ.
> I'm using cxf version 2.0.1 and SAAJ version 1.3.
>
> I have tried to trace where is could go wrong and what I can see, the å
> looks okay when it is added to W3CDOMXMLStreamWriter.writeCharaters(), but
> when the SOAPMessage.writeTo() method is called with System.out or
> OutputStream in SAAJOutInterceptor it is viewed as å.
>
> Can anyone tell me if I can do anything to avoid this behaviour or have it
> something to do with the SAAJ implementation?
>
> --
> View this message in context:
> http://www.nabble.com/Nordic-charaters-problem-in-SOAP-MQ-tp15845741p15845741.html
> Sent from the cxf-user mailing list archive at Nabble.com.
>
>

Re: Nordic charaters problem in SOAP/MQ

Posted by Benson Margulies <bi...@gmail.com>.
WIndows does not have a code page for UTF-8. If you want to display UTF-8,
you have either send UTF-16 to the 'W' API or convert to some local code
page.

On Fri, Mar 7, 2008 at 2:54 AM, mr.andersen <xm...@bec.dk> wrote:

>
>
> Problem found and solved!
>
> First of all - thanks for the quick responses.
>
> I have looked more into kode and the bytes written to my log.
> And I can conclude that there is no problem with SAAJ (which would wonder
> me) or CXF.
> The problem lays in the way bytes are presented onscreen, when nordic
> characters are displayed.
>
> Maybe I should start to tell more about the environment i'm working in:
>
> Windows (danish XP version)
> RAD 7, which by the way uses cp1252 as encoding in the projects)
> IBM MQ
>
>
> As you proberly know nordic characters is presented as 2 bytes in UTF-8 -
> for example is &aring; -> C3 A5.
> When the that character is written onscreen or I use Wordpad, the cp1252
> is
> taking over and the "translation" is &Atilde;&yen;. But if you use
> ultraedit
> to see which bytes (short cut: ctrl+h) it is, it's okay.
>
> So where is the problem? Well the problem lays in the way IBM have
> implemented javax.jms.TextMesage. It seams like when a TextMessage is
> created the dobbelt bytes are converted into dobbelt bytes. So &Atilde; is
> converted to C3 A0 and &yen; C2 A5.
>
> So the way to solve this is using ByteMessage and luckily CXF gives a way
> to
> set the runtime policy in the WSDL (&lt;jms:runtimePolicy
> messageType="binary"/&gt;)
> The problem here is that doesn't work in CXF 2.0.1, so I had to take out
> the
> conduit and set the messageType by myself.
>
>
> JMSConduit conduit = (JMSConduit)client.getConduit();
> conduit.getRuntimePolicy().setMessageType(MessageFormatType.BINARY);
>
>
>
> --
> View this message in context:
> http://www.nabble.com/Nordic-charaters-problem-in-SOAP-MQ-tp15845741p15891076.html
> Sent from the cxf-user mailing list archive at Nabble.com.
>

Re: Nordic charaters problem in SOAP/MQ

Posted by "mr.andersen" <xm...@bec.dk>.

Problem found and solved!

First of all - thanks for the quick responses.

I have looked more into kode and the bytes written to my log.
And I can conclude that there is no problem with SAAJ (which would wonder
me) or CXF.
The problem lays in the way bytes are presented onscreen, when nordic
characters are displayed.

Maybe I should start to tell more about the environment i'm working in:

Windows (danish XP version)
RAD 7, which by the way uses cp1252 as encoding in the projects)
IBM MQ


As you proberly know nordic characters is presented as 2 bytes in UTF-8 -
for example is &aring; -> C3 A5.
When the that character is written onscreen or I use Wordpad, the cp1252 is
taking over and the "translation" is &Atilde;&yen;. But if you use ultraedit
to see which bytes (short cut: ctrl+h) it is, it's okay.

So where is the problem? Well the problem lays in the way IBM have
implemented javax.jms.TextMesage. It seams like when a TextMessage is
created the dobbelt bytes are converted into dobbelt bytes. So &Atilde; is
converted to C3 A0 and &yen; C2 A5.

So the way to solve this is using ByteMessage and luckily CXF gives a way to
set the runtime policy in the WSDL (&lt;jms:runtimePolicy
messageType="binary"/&gt;)
The problem here is that doesn't work in CXF 2.0.1, so I had to take out the
conduit and set the messageType by myself.


JMSConduit conduit = (JMSConduit)client.getConduit();
conduit.getRuntimePolicy().setMessageType(MessageFormatType.BINARY);



-- 
View this message in context: http://www.nabble.com/Nordic-charaters-problem-in-SOAP-MQ-tp15845741p15891076.html
Sent from the cxf-user mailing list archive at Nabble.com.

Re: Nordic charaters problem in SOAP/MQ

Posted by Nino Saturnino Martinez Vazquez Wael <ni...@jayway.dk>.

Nino Saturnino Martinez Vazquez Wael wrote:
>
>
> Nino Saturnino Martinez Vazquez Wael wrote:
>>
>>
>> mr.andersen wrote:
>>> The idea of escaping the characters is not possible, since the reciever
>>> doesn't expect that.
>>>
>>>   
>> I guess both sender and reciver should expect it.. Okay no go for 
>> this then...
> Just thought of something though, you could try to send the char as an 
> escaped unicode. like this :  stringIwanttosend+= \u00C6 ;
>
> That should append æ...
With that I mean that you dont have to worry about your file encoding... 
If it's something system wise it'll produce the exact same error...
>>> The text I'm trying to send throw cxf is hardcoded at this point.
>>>
>>> About the encoding for my files - do you mean java files or du you 
>>> assume my
>>> text was being read from a file?
>>>
>>>   
>> I actually assumed, that you were testing this directly from your 
>> java source.. I've had some trouble in other projects where encoding 
>> were set by eclipse, forexample macroman, or windows cpXXXX... This 
>> could probably cause trouble aswell..
>>> Nino.Martinez wrote:
>>>  
>>>> Hmm, could'nt you just escape it(that would be soft fix)?
>>>>
>>>>
>>>> this might be a dumb question, but which kind of encoding do you 
>>>> use on your files? Or are the chars fetched/passed dynamically via 
>>>> db etc?
>>>>
>>>> regards Nino
>>>>
>>>> mr.andersen wrote:
>>>>   
>>>>> I have at problem with sending nordic charaters such as æøå via 
>>>>> SOAP over
>>>>> MQ.
>>>>> I'm using cxf version 2.0.1 and SAAJ version 1.3.
>>>>>
>>>>> I have tried to trace where is could go wrong and what I can see, 
>>>>> the å
>>>>> looks okay when it is added to 
>>>>> W3CDOMXMLStreamWriter.writeCharaters(),
>>>>> but
>>>>> when the SOAPMessage.writeTo() method is called with System.out or
>>>>> OutputStream in SAAJOutInterceptor it is viewed as å.
>>>>>
>>>>> Can anyone tell me if I can do anything to avoid this behaviour or 
>>>>> have
>>>>> it
>>>>> something to do with the SAAJ implementation?
>>>>>
>>>>>         
>>>> -- 
>>>> -Wicket for love
>>>> -Jme for fun
>>>>
>>>> Nino Martinez Wael
>>>> Java Specialist @ Jayway DK
>>>> http://www.jayway.dk
>>>> +45 2936 7684
>>>>
>>>>
>>>>
>>>>     
>>>
>>>   
>>
>

-- 
-Wicket for love
-Jme for fun

Nino Martinez Wael
Java Specialist @ Jayway DK
http://www.jayway.dk
+45 2936 7684

Re: Nordic charaters problem in SOAP/MQ

Posted by Nino Saturnino Martinez Vazquez Wael <ni...@jayway.dk>.

Nino Saturnino Martinez Vazquez Wael wrote:
>
>
> mr.andersen wrote:
>> The idea of escaping the characters is not possible, since the reciever
>> doesn't expect that.
>>
>>   
> I guess both sender and reciver should expect it.. Okay no go for this 
> then...
Just thought of something though, you could try to send the char as an 
escaped unicode. like this :  stringIwanttosend+= \u00C6 ;

That should append æ...
>> The text I'm trying to send throw cxf is hardcoded at this point.
>>
>> About the encoding for my files - do you mean java files or du you 
>> assume my
>> text was being read from a file?
>>
>>   
> I actually assumed, that you were testing this directly from your java 
> source.. I've had some trouble in other projects where encoding were 
> set by eclipse, forexample macroman, or windows cpXXXX... This could 
> probably cause trouble aswell..
>> Nino.Martinez wrote:
>>  
>>> Hmm, could'nt you just escape it(that would be soft fix)?
>>>
>>>
>>> this might be a dumb question, but which kind of encoding do you use 
>>> on your files? Or are the chars fetched/passed dynamically via db etc?
>>>
>>> regards Nino
>>>
>>> mr.andersen wrote:
>>>    
>>>> I have at problem with sending nordic charaters such as æøå via 
>>>> SOAP over
>>>> MQ.
>>>> I'm using cxf version 2.0.1 and SAAJ version 1.3.
>>>>
>>>> I have tried to trace where is could go wrong and what I can see, 
>>>> the å
>>>> looks okay when it is added to W3CDOMXMLStreamWriter.writeCharaters(),
>>>> but
>>>> when the SOAPMessage.writeTo() method is called with System.out or
>>>> OutputStream in SAAJOutInterceptor it is viewed as å.
>>>>
>>>> Can anyone tell me if I can do anything to avoid this behaviour or 
>>>> have
>>>> it
>>>> something to do with the SAAJ implementation?
>>>>
>>>>         
>>> -- 
>>> -Wicket for love
>>> -Jme for fun
>>>
>>> Nino Martinez Wael
>>> Java Specialist @ Jayway DK
>>> http://www.jayway.dk
>>> +45 2936 7684
>>>
>>>
>>>
>>>     
>>
>>   
>

-- 
-Wicket for love
-Jme for fun

Nino Martinez Wael
Java Specialist @ Jayway DK
http://www.jayway.dk
+45 2936 7684

Re: Nordic charaters problem in SOAP/MQ

Posted by Nino Saturnino Martinez Vazquez Wael <ni...@jayway.dk>.

mr.andersen wrote:
> The idea of escaping the characters is not possible, since the reciever
> doesn't expect that.
>
>   
I guess both sender and reciver should expect it.. Okay no go for this 
then...
> The text I'm trying to send throw cxf is hardcoded at this point.
>
> About the encoding for my files - do you mean java files or du you assume my
> text was being read from a file?
>
>   
I actually assumed, that you were testing this directly from your java 
source.. I've had some trouble in other projects where encoding were set 
by eclipse, forexample macroman, or windows cpXXXX... This could 
probably cause trouble aswell..
> Nino.Martinez wrote:
>   
>> Hmm, could'nt you just escape it(that would be soft fix)?
>>
>>
>> this might be a dumb question, but which kind of encoding do you use on 
>> your files? Or are the chars fetched/passed dynamically via db etc?
>>
>> regards Nino
>>
>> mr.andersen wrote:
>>     
>>> I have at problem with sending nordic charaters such as æøå via SOAP over
>>> MQ.
>>> I'm using cxf version 2.0.1 and SAAJ version 1.3.
>>>
>>> I have tried to trace where is could go wrong and what I can see, the å
>>> looks okay when it is added to W3CDOMXMLStreamWriter.writeCharaters(),
>>> but
>>> when the SOAPMessage.writeTo() method is called with System.out or
>>> OutputStream in SAAJOutInterceptor it is viewed as å.
>>>
>>> Can anyone tell me if I can do anything to avoid this behaviour or have
>>> it
>>> something to do with the SAAJ implementation?
>>>
>>>   
>>>       
>> -- 
>> -Wicket for love
>> -Jme for fun
>>
>> Nino Martinez Wael
>> Java Specialist @ Jayway DK
>> http://www.jayway.dk
>> +45 2936 7684
>>
>>
>>
>>     
>
>   

-- 
-Wicket for love
-Jme for fun

Nino Martinez Wael
Java Specialist @ Jayway DK
http://www.jayway.dk
+45 2936 7684

Re: Nordic charaters problem in SOAP/MQ

Posted by "mr.andersen" <xm...@bec.dk>.
The idea of escaping the characters is not possible, since the reciever
doesn't expect that.

The text I'm trying to send throw cxf is hardcoded at this point.

About the encoding for my files - do you mean java files or du you assume my
text was being read from a file?


Nino.Martinez wrote:
> 
> Hmm, could'nt you just escape it(that would be soft fix)?
> 
> 
> this might be a dumb question, but which kind of encoding do you use on 
> your files? Or are the chars fetched/passed dynamically via db etc?
> 
> regards Nino
> 
> mr.andersen wrote:
>> I have at problem with sending nordic charaters such as æøå via SOAP over
>> MQ.
>> I'm using cxf version 2.0.1 and SAAJ version 1.3.
>>
>> I have tried to trace where is could go wrong and what I can see, the å
>> looks okay when it is added to W3CDOMXMLStreamWriter.writeCharaters(),
>> but
>> when the SOAPMessage.writeTo() method is called with System.out or
>> OutputStream in SAAJOutInterceptor it is viewed as å.
>>
>> Can anyone tell me if I can do anything to avoid this behaviour or have
>> it
>> something to do with the SAAJ implementation?
>>
>>   
> 
> -- 
> -Wicket for love
> -Jme for fun
> 
> Nino Martinez Wael
> Java Specialist @ Jayway DK
> http://www.jayway.dk
> +45 2936 7684
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Nordic-charaters-problem-in-SOAP-MQ-tp15845741p15848433.html
Sent from the cxf-user mailing list archive at Nabble.com.

Re: Nordic charaters problem in SOAP/MQ

Posted by Nino Saturnino Martinez Vazquez Wael <ni...@jayway.dk>.
Hmm, could'nt you just escape it(that would be soft fix)?


this might be a dumb question, but which kind of encoding do you use on 
your files? Or are the chars fetched/passed dynamically via db etc?

regards Nino

mr.andersen wrote:
> I have at problem with sending nordic charaters such as æøå via SOAP over MQ.
> I'm using cxf version 2.0.1 and SAAJ version 1.3.
>
> I have tried to trace where is could go wrong and what I can see, the å
> looks okay when it is added to W3CDOMXMLStreamWriter.writeCharaters(), but
> when the SOAPMessage.writeTo() method is called with System.out or
> OutputStream in SAAJOutInterceptor it is viewed as å.
>
> Can anyone tell me if I can do anything to avoid this behaviour or have it
> something to do with the SAAJ implementation?
>
>   

-- 
-Wicket for love
-Jme for fun

Nino Martinez Wael
Java Specialist @ Jayway DK
http://www.jayway.dk
+45 2936 7684

Re: Nordic charaters problem in SOAP/MQ

Posted by Daniel Kulp <dk...@apache.org>.
Can you try just using the SAAJ API's in a quick standalone program to 
create a soap message then write it out to a file, then read it into a 
new SAAJ message and see if the problem remains?   If so, it's 
definitely a bug in the SAAJ implemenetation.   If not, it MAY be an 
issue with the JMS transport.  

Dan


On Wednesday 05 March 2008, mr.andersen wrote:
> I have at problem with sending nordic charaters such as æøå via SOAP
> over MQ. I'm using cxf version 2.0.1 and SAAJ version 1.3.
>
> I have tried to trace where is could go wrong and what I can see, the
> å looks okay when it is added to
> W3CDOMXMLStreamWriter.writeCharaters(), but when the
> SOAPMessage.writeTo() method is called with System.out or OutputStream
> in SAAJOutInterceptor it is viewed as å.
>
> Can anyone tell me if I can do anything to avoid this behaviour or
> have it something to do with the SAAJ implementation?



-- 
J. Daniel Kulp
Principal Engineer, IONA
dkulp@apache.org
http://www.dankulp.com/blog