You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by Loh Kok Jeng <ko...@gmail.com> on 2010/06/21 19:00:35 UTC
UTF-8 Characters in Request Params for REST Service
Dear all,
I'm struggling with non-ASCII characters passed in request params of a
REST service. The characters become ? when received by my app
developed using CXF. The encoding is set to UTF-8 by the REST client.
Why do I get "?"?
Thanks in advance.
Re: UTF-8 Characters in Request Params for REST Service
Posted by Rakesh Rai <ra...@gmail.com>.
Before you do log.info("Message: " + message);
Simply copy paste the lines between ====== and ==========
and as shown in the code below try to get the log info for the same.. you
will get correct code without special characters ? &??
=========================================================
String newMessage = utfEightConvert(message);
log.info("Message: " + newMeessage);
/**
*
* @param utfEightString
* @return String converted to USF-8 format and send it to the caller .
* @throws java.io.UnsupportedEncodingException
*/
public static final String utfEightConvert(String utfEightString)
throws java.io.UnsupportedEncodingException {
byte[] bytes = new byte[utfEightString.length()];
for (int i = 0; i < utfEightString.length(); i++) {
bytes[i] = (byte) utfEightString.charAt(i);
}
return new String(bytes, "UTF-8");
}
=================================================================
On Thu, Jun 24, 2010 at 1:38 AM, Loh Kok Jeng <ko...@gmail.com> wrote:
> Hi,
>
> I develop on Windows and deploy on AIX. On both OS, I get the same result.
>
> Thanks for the tips on Eclipse. I have made the change but the
> characters are still not displayed correctly.
>
> BTW, I use SoapUI to submit the request, and I set the Encoding to UTF-8.
>
> regards,
> Loh Kok Jeng
>
>
>
> On 24 June 2010 13:22, Ron Grimes <rg...@sinclairoil.com> wrote:
> > Have you considered that it is not the conversion to UTF-8 that is
> incorrect, but the software you're using to display the log? For example,
> when viewing logged results to the Eclipse console, it will do a similar
> thing if you do not go into Preferences and change the "text file encoding"
> to UTF-8. I don't recall you mentioning your development box O/S or the
> software used to show the log. But, it is possible that is the culprit. Or
> not, but something to check.
> >
> > Ron
> >
> >
> > ________________________________________
> > From: Loh Kok Jeng [kokjeng.loh@gmail.com]
> > Sent: Wednesday, June 23, 2010 7:24 PM
> > To: users@cxf.apache.org
> > Subject: Re: UTF-8 Characters in Request Params for REST Service
> >
> > On 24 June 2010 09:11, Loh Kok Jeng <ko...@gmail.com> wrote:
> >> Hi Sergey,
> >>
> >> This is for form based submission.
> >>
> >> Below is a snippet of my code. I want to be able to accept non-ASCII
> >> characters in "message" parameter below. However, when I tested with
> >> Chinese characters, they are converted to "?" as shown in my logs.
> >>
> >> Any help will be very much appreciated.
> >>
> >> @POST
> >> // @Consumes("application/x-www-form-urlencoded")
> >> public SMSSendSmsResponse sendSms(@HeaderParam("Authorization")
> >> String authorization,
> >> @Context MessageContext mc,
> >> @FormParam("address") List<String> address,
> >> @FormParam("message") String
> message,
> >> @FormParam("notifyURL") String
> notifyURL,
> >> @FormParam("correlator") String
> >> correlator,
> >> @FormParam("senderName") String
> >> senderName,
> >> @FormParam("Charging") String
> Charging)
> >> throws MalformedURLException, PolicyException, ServiceException
> {
> >>
> >> if (address != null) {
> >> for (Iterator<String> i = address.iterator(); i.hasNext();) {
> >> log.info("Address: " + i.next());
> >> }
> >> }
> >> log.info("Message: " + message);
> >> log.info("Notify URL: " + notifyURL);
> >> log.info("Correlator: " + correlator);
> >> log.info("Sender Name: " + senderName);
> >> log.info("Charging: " + Charging);
> >>
> >> ....
> >>
> >> }
> >> regards,
> >> Loh Kok Jeng
> >>
> >>
> >>
> >> On 24 June 2010 02:00, Sergey Beryozkin <sb...@gmail.com> wrote:
> >>> I need to know the details of the request and how a resource method
> >>> expecting the values looks like
> >>>
> >>> thanks, Sergey
> >>>
> >>> On Tue, Jun 22, 2010 at 7:10 PM, Rakesh Rai <ra...@gmail.com>
> wrote:
> >>>
> >>>> This is the method that does the trick.... converts to UTF-8 and
> transforms
> >>>> to UTF-8 format string and sends it back to the caller
> >>>> Wherever in your service class / action class add this method and
> convert
> >>>> the existing string to return the UTF-8 transformed string / text
> >>>> /**
> >>>> *
> >>>> * @param utfEightString
> >>>> * @return String converted to USF-8 format and send it to the
> caller .
> >>>> * @throws java.io.
> >>>> UnsupportedEncodingException
> >>>> */
> >>>> public static final String utfEightConvert(String utfEightString)
> >>>> throws java.io.UnsupportedEncodingException {
> >>>>
> >>>> byte[] bytes = new byte[utfEightString.length()];
> >>>> for (int i = 0; i < utfEightString.length(); i++) {
> >>>> bytes[i] = (byte) utfEightString.charAt(i);
> >>>> }
> >>>> return new String(bytes, "UTF-8");
> >>>>
> >>>> }
> >>>>
> >>>> Hope it helps.
> >>>> Rakesh
> >>>>
> >>>>
> >>>> On Tue, Jun 22, 2010 at 1:47 PM, Sergey Beryozkin <
> sberyozkin@gmail.com
> >>>> >wrote:
> >>>>
> >>>> > Is it a form based submission ? or XML is posted in the body ?
> >>>> >
> >>>> >
> >>>> > cheers, Sergey
> >>>> >
> >>>> > On Mon, Jun 21, 2010 at 6:00 PM, Loh Kok Jeng <
> kokjeng.loh@gmail.com>
> >>>> > wrote:
> >>>> >
> >>>> > > Dear all,
> >>>> > >
> >>>> > > I'm struggling with non-ASCII characters passed in request params
> of a
> >>>> > > REST service. The characters become ? when received by my app
> >>>> > > developed using CXF. The encoding is set to UTF-8 by the REST
> client.
> >>>> > > Why do I get "?"?
> >>>> > >
> >>>> > > Thanks in advance.
> >>>> > >
> >>>> >
> >>>>
> >>>
> >>
>
--
Email : rairakesh@gmail.com
Addr: 116 Fringetree Drive, West Chester, PA, 19380
Cell Ph : 551 998 4457
Fax : 270-573-1611
Renaissance of Darwinism : It is not the strongest of the species that
survive, nor the most intelligent, but the ones most responsive to change.
Re: UTF-8 Characters in Request Params for REST Service
Posted by Loh Kok Jeng <ko...@gmail.com>.
Hi,
I develop on Windows and deploy on AIX. On both OS, I get the same result.
Thanks for the tips on Eclipse. I have made the change but the
characters are still not displayed correctly.
BTW, I use SoapUI to submit the request, and I set the Encoding to UTF-8.
regards,
Loh Kok Jeng
On 24 June 2010 13:22, Ron Grimes <rg...@sinclairoil.com> wrote:
> Have you considered that it is not the conversion to UTF-8 that is incorrect, but the software you're using to display the log? For example, when viewing logged results to the Eclipse console, it will do a similar thing if you do not go into Preferences and change the "text file encoding" to UTF-8. I don't recall you mentioning your development box O/S or the software used to show the log. But, it is possible that is the culprit. Or not, but something to check.
>
> Ron
>
>
> ________________________________________
> From: Loh Kok Jeng [kokjeng.loh@gmail.com]
> Sent: Wednesday, June 23, 2010 7:24 PM
> To: users@cxf.apache.org
> Subject: Re: UTF-8 Characters in Request Params for REST Service
>
> On 24 June 2010 09:11, Loh Kok Jeng <ko...@gmail.com> wrote:
>> Hi Sergey,
>>
>> This is for form based submission.
>>
>> Below is a snippet of my code. I want to be able to accept non-ASCII
>> characters in "message" parameter below. However, when I tested with
>> Chinese characters, they are converted to "?" as shown in my logs.
>>
>> Any help will be very much appreciated.
>>
>> @POST
>> // @Consumes("application/x-www-form-urlencoded")
>> public SMSSendSmsResponse sendSms(@HeaderParam("Authorization")
>> String authorization,
>> @Context MessageContext mc,
>> @FormParam("address") List<String> address,
>> @FormParam("message") String message,
>> @FormParam("notifyURL") String notifyURL,
>> @FormParam("correlator") String
>> correlator,
>> @FormParam("senderName") String
>> senderName,
>> @FormParam("Charging") String Charging)
>> throws MalformedURLException, PolicyException, ServiceException {
>>
>> if (address != null) {
>> for (Iterator<String> i = address.iterator(); i.hasNext();) {
>> log.info("Address: " + i.next());
>> }
>> }
>> log.info("Message: " + message);
>> log.info("Notify URL: " + notifyURL);
>> log.info("Correlator: " + correlator);
>> log.info("Sender Name: " + senderName);
>> log.info("Charging: " + Charging);
>>
>> ....
>>
>> }
>> regards,
>> Loh Kok Jeng
>>
>>
>>
>> On 24 June 2010 02:00, Sergey Beryozkin <sb...@gmail.com> wrote:
>>> I need to know the details of the request and how a resource method
>>> expecting the values looks like
>>>
>>> thanks, Sergey
>>>
>>> On Tue, Jun 22, 2010 at 7:10 PM, Rakesh Rai <ra...@gmail.com> wrote:
>>>
>>>> This is the method that does the trick.... converts to UTF-8 and transforms
>>>> to UTF-8 format string and sends it back to the caller
>>>> Wherever in your service class / action class add this method and convert
>>>> the existing string to return the UTF-8 transformed string / text
>>>> /**
>>>> *
>>>> * @param utfEightString
>>>> * @return String converted to USF-8 format and send it to the caller .
>>>> * @throws java.io.
>>>> UnsupportedEncodingException
>>>> */
>>>> public static final String utfEightConvert(String utfEightString)
>>>> throws java.io.UnsupportedEncodingException {
>>>>
>>>> byte[] bytes = new byte[utfEightString.length()];
>>>> for (int i = 0; i < utfEightString.length(); i++) {
>>>> bytes[i] = (byte) utfEightString.charAt(i);
>>>> }
>>>> return new String(bytes, "UTF-8");
>>>>
>>>> }
>>>>
>>>> Hope it helps.
>>>> Rakesh
>>>>
>>>>
>>>> On Tue, Jun 22, 2010 at 1:47 PM, Sergey Beryozkin <sberyozkin@gmail.com
>>>> >wrote:
>>>>
>>>> > Is it a form based submission ? or XML is posted in the body ?
>>>> >
>>>> >
>>>> > cheers, Sergey
>>>> >
>>>> > On Mon, Jun 21, 2010 at 6:00 PM, Loh Kok Jeng <ko...@gmail.com>
>>>> > wrote:
>>>> >
>>>> > > Dear all,
>>>> > >
>>>> > > I'm struggling with non-ASCII characters passed in request params of a
>>>> > > REST service. The characters become ? when received by my app
>>>> > > developed using CXF. The encoding is set to UTF-8 by the REST client.
>>>> > > Why do I get "?"?
>>>> > >
>>>> > > Thanks in advance.
>>>> > >
>>>> >
>>>>
>>>
>>
RE: UTF-8 Characters in Request Params for REST Service
Posted by Ron Grimes <rg...@sinclairoil.com>.
Have you considered that it is not the conversion to UTF-8 that is incorrect, but the software you're using to display the log? For example, when viewing logged results to the Eclipse console, it will do a similar thing if you do not go into Preferences and change the "text file encoding" to UTF-8. I don't recall you mentioning your development box O/S or the software used to show the log. But, it is possible that is the culprit. Or not, but something to check.
Ron
________________________________________
From: Loh Kok Jeng [kokjeng.loh@gmail.com]
Sent: Wednesday, June 23, 2010 7:24 PM
To: users@cxf.apache.org
Subject: Re: UTF-8 Characters in Request Params for REST Service
On 24 June 2010 09:11, Loh Kok Jeng <ko...@gmail.com> wrote:
> Hi Sergey,
>
> This is for form based submission.
>
> Below is a snippet of my code. I want to be able to accept non-ASCII
> characters in "message" parameter below. However, when I tested with
> Chinese characters, they are converted to "?" as shown in my logs.
>
> Any help will be very much appreciated.
>
> @POST
> // @Consumes("application/x-www-form-urlencoded")
> public SMSSendSmsResponse sendSms(@HeaderParam("Authorization")
> String authorization,
> @Context MessageContext mc,
> @FormParam("address") List<String> address,
> @FormParam("message") String message,
> @FormParam("notifyURL") String notifyURL,
> @FormParam("correlator") String
> correlator,
> @FormParam("senderName") String
> senderName,
> @FormParam("Charging") String Charging)
> throws MalformedURLException, PolicyException, ServiceException {
>
> if (address != null) {
> for (Iterator<String> i = address.iterator(); i.hasNext();) {
> log.info("Address: " + i.next());
> }
> }
> log.info("Message: " + message);
> log.info("Notify URL: " + notifyURL);
> log.info("Correlator: " + correlator);
> log.info("Sender Name: " + senderName);
> log.info("Charging: " + Charging);
>
> ....
>
> }
> regards,
> Loh Kok Jeng
>
>
>
> On 24 June 2010 02:00, Sergey Beryozkin <sb...@gmail.com> wrote:
>> I need to know the details of the request and how a resource method
>> expecting the values looks like
>>
>> thanks, Sergey
>>
>> On Tue, Jun 22, 2010 at 7:10 PM, Rakesh Rai <ra...@gmail.com> wrote:
>>
>>> This is the method that does the trick.... converts to UTF-8 and transforms
>>> to UTF-8 format string and sends it back to the caller
>>> Wherever in your service class / action class add this method and convert
>>> the existing string to return the UTF-8 transformed string / text
>>> /**
>>> *
>>> * @param utfEightString
>>> * @return String converted to USF-8 format and send it to the caller .
>>> * @throws java.io.
>>> UnsupportedEncodingException
>>> */
>>> public static final String utfEightConvert(String utfEightString)
>>> throws java.io.UnsupportedEncodingException {
>>>
>>> byte[] bytes = new byte[utfEightString.length()];
>>> for (int i = 0; i < utfEightString.length(); i++) {
>>> bytes[i] = (byte) utfEightString.charAt(i);
>>> }
>>> return new String(bytes, "UTF-8");
>>>
>>> }
>>>
>>> Hope it helps.
>>> Rakesh
>>>
>>>
>>> On Tue, Jun 22, 2010 at 1:47 PM, Sergey Beryozkin <sberyozkin@gmail.com
>>> >wrote:
>>>
>>> > Is it a form based submission ? or XML is posted in the body ?
>>> >
>>> >
>>> > cheers, Sergey
>>> >
>>> > On Mon, Jun 21, 2010 at 6:00 PM, Loh Kok Jeng <ko...@gmail.com>
>>> > wrote:
>>> >
>>> > > Dear all,
>>> > >
>>> > > I'm struggling with non-ASCII characters passed in request params of a
>>> > > REST service. The characters become ? when received by my app
>>> > > developed using CXF. The encoding is set to UTF-8 by the REST client.
>>> > > Why do I get "?"?
>>> > >
>>> > > Thanks in advance.
>>> > >
>>> >
>>>
>>
>
Re: UTF-8 Characters in Request Params for REST Service
Posted by Loh Kok Jeng <ko...@gmail.com>.
BTW, I'm using CXF 2.2.5 on Tomcat 5.5.
Thanks.
regards,
Loh Kok Jeng
On 24 June 2010 09:11, Loh Kok Jeng <ko...@gmail.com> wrote:
> Hi Sergey,
>
> This is for form based submission.
>
> Below is a snippet of my code. I want to be able to accept non-ASCII
> characters in "message" parameter below. However, when I tested with
> Chinese characters, they are converted to "?" as shown in my logs.
>
> Any help will be very much appreciated.
>
> @POST
> // @Consumes("application/x-www-form-urlencoded")
> public SMSSendSmsResponse sendSms(@HeaderParam("Authorization")
> String authorization,
> @Context MessageContext mc,
> @FormParam("address") List<String> address,
> @FormParam("message") String message,
> @FormParam("notifyURL") String notifyURL,
> @FormParam("correlator") String
> correlator,
> @FormParam("senderName") String
> senderName,
> @FormParam("Charging") String Charging)
> throws MalformedURLException, PolicyException, ServiceException {
>
> if (address != null) {
> for (Iterator<String> i = address.iterator(); i.hasNext();) {
> log.info("Address: " + i.next());
> }
> }
> log.info("Message: " + message);
> log.info("Notify URL: " + notifyURL);
> log.info("Correlator: " + correlator);
> log.info("Sender Name: " + senderName);
> log.info("Charging: " + Charging);
>
> ....
>
> }
> regards,
> Loh Kok Jeng
>
>
>
> On 24 June 2010 02:00, Sergey Beryozkin <sb...@gmail.com> wrote:
>> I need to know the details of the request and how a resource method
>> expecting the values looks like
>>
>> thanks, Sergey
>>
>> On Tue, Jun 22, 2010 at 7:10 PM, Rakesh Rai <ra...@gmail.com> wrote:
>>
>>> This is the method that does the trick.... converts to UTF-8 and transforms
>>> to UTF-8 format string and sends it back to the caller
>>> Wherever in your service class / action class add this method and convert
>>> the existing string to return the UTF-8 transformed string / text
>>> /**
>>> *
>>> * @param utfEightString
>>> * @return String converted to USF-8 format and send it to the caller .
>>> * @throws java.io.
>>> UnsupportedEncodingException
>>> */
>>> public static final String utfEightConvert(String utfEightString)
>>> throws java.io.UnsupportedEncodingException {
>>>
>>> byte[] bytes = new byte[utfEightString.length()];
>>> for (int i = 0; i < utfEightString.length(); i++) {
>>> bytes[i] = (byte) utfEightString.charAt(i);
>>> }
>>> return new String(bytes, "UTF-8");
>>>
>>> }
>>>
>>> Hope it helps.
>>> Rakesh
>>>
>>>
>>> On Tue, Jun 22, 2010 at 1:47 PM, Sergey Beryozkin <sberyozkin@gmail.com
>>> >wrote:
>>>
>>> > Is it a form based submission ? or XML is posted in the body ?
>>> >
>>> >
>>> > cheers, Sergey
>>> >
>>> > On Mon, Jun 21, 2010 at 6:00 PM, Loh Kok Jeng <ko...@gmail.com>
>>> > wrote:
>>> >
>>> > > Dear all,
>>> > >
>>> > > I'm struggling with non-ASCII characters passed in request params of a
>>> > > REST service. The characters become ? when received by my app
>>> > > developed using CXF. The encoding is set to UTF-8 by the REST client.
>>> > > Why do I get "?"?
>>> > >
>>> > > Thanks in advance.
>>> > >
>>> >
>>>
>>
>
Re: UTF-8 Characters in Request Params for REST Service
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi
Sorry for a delay. Just create a custom JAXRS MessageBodyReader (have a look
at the existing FormEncodingProvider) which implements readFrom (reads the
body from InputStream and splits the name=value pairs). At the moment
"UTF-8" is used to read the stream but I suspect that the sequence which
SoapUI sends is not UTF-8 encoded. So you need to check the "charset"
parameter of the provided MediaType and use UTF-8 by default...
cheers, Sergey
On Thu, Jun 24, 2010 at 5:14 PM, Loh Kok Jeng <ko...@gmail.com> wrote:
> Hi Sergey,
>
> Can u point me to where to get info on creating a custom FormProvider?
> What is the workaround?
>
> Thanks.
>
> regards,
> Loh Kok Jeng
>
>
>
> On 24 June 2010 23:16, Sergey Beryozkin <sb...@gmail.com> wrote:
> > Hi - indeed, there's a bug in the CXF JAXRS FormProvider in that it
> ignores
> > the encoding of the incoming ContentType - I'll try to fix it these
> > weekends; a custom FormProvider can be easily created as well as a
> > workaround
> >
> > cheers, Sergey
> >
> > On Thu, Jun 24, 2010 at 4:14 PM, Ron Grimes <rg...@sinclairoil.com>
> wrote:
> >
> >> I think I may have found the answer. See
> >> http://java.sun.com/docs/books/tutorial/i18n/text/string.html ).
> >>
> >> Here, they talk about how it is incorrect to load a string value into a
> >> bytes array without specifying the encoding. Notice, in their example,
> that
> >> the difference in
> >>
> >> byte[] bytes = utfEightString.getBytes(); \\ bad
> >> byte[] bytes = utfEightString.getBytes("UTF8"); \\ good
> >>
> >> will produce a different number of byte elements. Essentially, your loop
> is
> >> doing the same thing. For a five characters string, you will always end
> up
> >> with a 5 element byte array, when it might have given you an 8 byte
> array,
> >> had you specified the encoding.
> >>
> >> Instead, you should do this:
> >>
> >> byte[] bytes = utfEightString.getBytes("UTF8");
> >> return new String(bytes, "UTF8");
> >>
> >>
> >> Ron Grimes
> >>
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: Loh Kok Jeng [mailto:kokjeng.loh@gmail.com]
> >> Sent: Wednesday, June 23, 2010 7:11 PM
> >> To: users@cxf.apache.org
> >> Subject: Re: UTF-8 Characters in Request Params for REST Service
> >>
> >> Hi Sergey,
> >>
> >> This is for form based submission.
> >>
> >> Below is a snippet of my code. I want to be able to accept non-ASCII
> >> characters in "message" parameter below. However, when I tested with
> >> Chinese characters, they are converted to "?" as shown in my logs.
> >>
> >> Any help will be very much appreciated.
> >>
> >> @POST
> >> // @Consumes("application/x-www-form-urlencoded")
> >> public SMSSendSmsResponse sendSms(@HeaderParam("Authorization")
> >> String authorization,
> >> @Context MessageContext mc,
> >> @FormParam("address") List<String> address,
> >> @FormParam("message") String
> message,
> >> @FormParam("notifyURL") String
> >> notifyURL,
> >> @FormParam("correlator") String
> >> correlator,
> >> @FormParam("senderName") String
> >> senderName,
> >> @FormParam("Charging") String
> >> Charging)
> >> throws MalformedURLException, PolicyException, ServiceException
> {
> >>
> >> if (address != null) {
> >> for (Iterator<String> i = address.iterator(); i.hasNext();) {
> >> log.info("Address: " + i.next());
> >> }
> >> }
> >> log.info("Message: " + message);
> >> log.info("Notify URL: " + notifyURL);
> >> log.info("Correlator: " + correlator);
> >> log.info("Sender Name: " + senderName);
> >> log.info("Charging: " + Charging);
> >>
> >> ....
> >>
> >> }
> >> regards,
> >> Loh Kok Jeng
> >>
> >>
> >>
> >> On 24 June 2010 02:00, Sergey Beryozkin <sb...@gmail.com> wrote:
> >> > I need to know the details of the request and how a resource method
> >> > expecting the values looks like
> >> >
> >> > thanks, Sergey
> >> >
> >> > On Tue, Jun 22, 2010 at 7:10 PM, Rakesh Rai <ra...@gmail.com>
> wrote:
> >> >
> >> >> This is the method that does the trick.... converts to UTF-8 and
> >> transforms
> >> >> to UTF-8 format string and sends it back to the caller
> >> >> Wherever in your service class / action class add this method and
> >> convert
> >> >> the existing string to return the UTF-8 transformed string / text
> >> >> /**
> >> >> *
> >> >> * @param utfEightString
> >> >> * @return String converted to USF-8 format and send it to the
> caller
> >> .
> >> >> * @throws java.io.
> >> >> UnsupportedEncodingException
> >> >> */
> >> >> public static final String utfEightConvert(String utfEightString)
> >> >> throws java.io.UnsupportedEncodingException {
> >> >>
> >> >> byte[] bytes = new byte[utfEightString.length()];
> >> >> for (int i = 0; i < utfEightString.length(); i++) {
> >> >> bytes[i] = (byte) utfEightString.charAt(i);
> >> >> }
> >> >> return new String(bytes, "UTF-8");
> >> >>
> >> >> }
> >> >>
> >> >> Hope it helps.
> >> >> Rakesh
> >> >>
> >> >>
> >> >> On Tue, Jun 22, 2010 at 1:47 PM, Sergey Beryozkin <
> sberyozkin@gmail.com
> >> >> >wrote:
> >> >>
> >> >> > Is it a form based submission ? or XML is posted in the body ?
> >> >> >
> >> >> >
> >> >> > cheers, Sergey
> >> >> >
> >> >> > On Mon, Jun 21, 2010 at 6:00 PM, Loh Kok Jeng <
> kokjeng.loh@gmail.com>
> >> >> > wrote:
> >> >> >
> >> >> > > Dear all,
> >> >> > >
> >> >> > > I'm struggling with non-ASCII characters passed in request params
> of
> >> a
> >> >> > > REST service. The characters become ? when received by my app
> >> >> > > developed using CXF. The encoding is set to UTF-8 by the REST
> >> client.
> >> >> > > Why do I get "?"?
> >> >> > >
> >> >> > > Thanks in advance.
> >> >> > >
> >> >> >
> >> >>
> >> >
> >>
> >
>
Re: UTF-8 Characters in Request Params for REST Service
Posted by Loh Kok Jeng <ko...@gmail.com>.
Hi Sergey,
Can u point me to where to get info on creating a custom FormProvider?
What is the workaround?
Thanks.
regards,
Loh Kok Jeng
On 24 June 2010 23:16, Sergey Beryozkin <sb...@gmail.com> wrote:
> Hi - indeed, there's a bug in the CXF JAXRS FormProvider in that it ignores
> the encoding of the incoming ContentType - I'll try to fix it these
> weekends; a custom FormProvider can be easily created as well as a
> workaround
>
> cheers, Sergey
>
> On Thu, Jun 24, 2010 at 4:14 PM, Ron Grimes <rg...@sinclairoil.com> wrote:
>
>> I think I may have found the answer. See
>> http://java.sun.com/docs/books/tutorial/i18n/text/string.html ).
>>
>> Here, they talk about how it is incorrect to load a string value into a
>> bytes array without specifying the encoding. Notice, in their example, that
>> the difference in
>>
>> byte[] bytes = utfEightString.getBytes(); \\ bad
>> byte[] bytes = utfEightString.getBytes("UTF8"); \\ good
>>
>> will produce a different number of byte elements. Essentially, your loop is
>> doing the same thing. For a five characters string, you will always end up
>> with a 5 element byte array, when it might have given you an 8 byte array,
>> had you specified the encoding.
>>
>> Instead, you should do this:
>>
>> byte[] bytes = utfEightString.getBytes("UTF8");
>> return new String(bytes, "UTF8");
>>
>>
>> Ron Grimes
>>
>>
>>
>>
>> -----Original Message-----
>> From: Loh Kok Jeng [mailto:kokjeng.loh@gmail.com]
>> Sent: Wednesday, June 23, 2010 7:11 PM
>> To: users@cxf.apache.org
>> Subject: Re: UTF-8 Characters in Request Params for REST Service
>>
>> Hi Sergey,
>>
>> This is for form based submission.
>>
>> Below is a snippet of my code. I want to be able to accept non-ASCII
>> characters in "message" parameter below. However, when I tested with
>> Chinese characters, they are converted to "?" as shown in my logs.
>>
>> Any help will be very much appreciated.
>>
>> @POST
>> // @Consumes("application/x-www-form-urlencoded")
>> public SMSSendSmsResponse sendSms(@HeaderParam("Authorization")
>> String authorization,
>> @Context MessageContext mc,
>> @FormParam("address") List<String> address,
>> @FormParam("message") String message,
>> @FormParam("notifyURL") String
>> notifyURL,
>> @FormParam("correlator") String
>> correlator,
>> @FormParam("senderName") String
>> senderName,
>> @FormParam("Charging") String
>> Charging)
>> throws MalformedURLException, PolicyException, ServiceException {
>>
>> if (address != null) {
>> for (Iterator<String> i = address.iterator(); i.hasNext();) {
>> log.info("Address: " + i.next());
>> }
>> }
>> log.info("Message: " + message);
>> log.info("Notify URL: " + notifyURL);
>> log.info("Correlator: " + correlator);
>> log.info("Sender Name: " + senderName);
>> log.info("Charging: " + Charging);
>>
>> ....
>>
>> }
>> regards,
>> Loh Kok Jeng
>>
>>
>>
>> On 24 June 2010 02:00, Sergey Beryozkin <sb...@gmail.com> wrote:
>> > I need to know the details of the request and how a resource method
>> > expecting the values looks like
>> >
>> > thanks, Sergey
>> >
>> > On Tue, Jun 22, 2010 at 7:10 PM, Rakesh Rai <ra...@gmail.com> wrote:
>> >
>> >> This is the method that does the trick.... converts to UTF-8 and
>> transforms
>> >> to UTF-8 format string and sends it back to the caller
>> >> Wherever in your service class / action class add this method and
>> convert
>> >> the existing string to return the UTF-8 transformed string / text
>> >> /**
>> >> *
>> >> * @param utfEightString
>> >> * @return String converted to USF-8 format and send it to the caller
>> .
>> >> * @throws java.io.
>> >> UnsupportedEncodingException
>> >> */
>> >> public static final String utfEightConvert(String utfEightString)
>> >> throws java.io.UnsupportedEncodingException {
>> >>
>> >> byte[] bytes = new byte[utfEightString.length()];
>> >> for (int i = 0; i < utfEightString.length(); i++) {
>> >> bytes[i] = (byte) utfEightString.charAt(i);
>> >> }
>> >> return new String(bytes, "UTF-8");
>> >>
>> >> }
>> >>
>> >> Hope it helps.
>> >> Rakesh
>> >>
>> >>
>> >> On Tue, Jun 22, 2010 at 1:47 PM, Sergey Beryozkin <sberyozkin@gmail.com
>> >> >wrote:
>> >>
>> >> > Is it a form based submission ? or XML is posted in the body ?
>> >> >
>> >> >
>> >> > cheers, Sergey
>> >> >
>> >> > On Mon, Jun 21, 2010 at 6:00 PM, Loh Kok Jeng <ko...@gmail.com>
>> >> > wrote:
>> >> >
>> >> > > Dear all,
>> >> > >
>> >> > > I'm struggling with non-ASCII characters passed in request params of
>> a
>> >> > > REST service. The characters become ? when received by my app
>> >> > > developed using CXF. The encoding is set to UTF-8 by the REST
>> client.
>> >> > > Why do I get "?"?
>> >> > >
>> >> > > Thanks in advance.
>> >> > >
>> >> >
>> >>
>> >
>>
>
Re: UTF-8 Characters in Request Params for REST Service
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi - indeed, there's a bug in the CXF JAXRS FormProvider in that it ignores
the encoding of the incoming ContentType - I'll try to fix it these
weekends; a custom FormProvider can be easily created as well as a
workaround
cheers, Sergey
On Thu, Jun 24, 2010 at 4:14 PM, Ron Grimes <rg...@sinclairoil.com> wrote:
> I think I may have found the answer. See
> http://java.sun.com/docs/books/tutorial/i18n/text/string.html ).
>
> Here, they talk about how it is incorrect to load a string value into a
> bytes array without specifying the encoding. Notice, in their example, that
> the difference in
>
> byte[] bytes = utfEightString.getBytes(); \\ bad
> byte[] bytes = utfEightString.getBytes("UTF8"); \\ good
>
> will produce a different number of byte elements. Essentially, your loop is
> doing the same thing. For a five characters string, you will always end up
> with a 5 element byte array, when it might have given you an 8 byte array,
> had you specified the encoding.
>
> Instead, you should do this:
>
> byte[] bytes = utfEightString.getBytes("UTF8");
> return new String(bytes, "UTF8");
>
>
> Ron Grimes
>
>
>
>
> -----Original Message-----
> From: Loh Kok Jeng [mailto:kokjeng.loh@gmail.com]
> Sent: Wednesday, June 23, 2010 7:11 PM
> To: users@cxf.apache.org
> Subject: Re: UTF-8 Characters in Request Params for REST Service
>
> Hi Sergey,
>
> This is for form based submission.
>
> Below is a snippet of my code. I want to be able to accept non-ASCII
> characters in "message" parameter below. However, when I tested with
> Chinese characters, they are converted to "?" as shown in my logs.
>
> Any help will be very much appreciated.
>
> @POST
> // @Consumes("application/x-www-form-urlencoded")
> public SMSSendSmsResponse sendSms(@HeaderParam("Authorization")
> String authorization,
> @Context MessageContext mc,
> @FormParam("address") List<String> address,
> @FormParam("message") String message,
> @FormParam("notifyURL") String
> notifyURL,
> @FormParam("correlator") String
> correlator,
> @FormParam("senderName") String
> senderName,
> @FormParam("Charging") String
> Charging)
> throws MalformedURLException, PolicyException, ServiceException {
>
> if (address != null) {
> for (Iterator<String> i = address.iterator(); i.hasNext();) {
> log.info("Address: " + i.next());
> }
> }
> log.info("Message: " + message);
> log.info("Notify URL: " + notifyURL);
> log.info("Correlator: " + correlator);
> log.info("Sender Name: " + senderName);
> log.info("Charging: " + Charging);
>
> ....
>
> }
> regards,
> Loh Kok Jeng
>
>
>
> On 24 June 2010 02:00, Sergey Beryozkin <sb...@gmail.com> wrote:
> > I need to know the details of the request and how a resource method
> > expecting the values looks like
> >
> > thanks, Sergey
> >
> > On Tue, Jun 22, 2010 at 7:10 PM, Rakesh Rai <ra...@gmail.com> wrote:
> >
> >> This is the method that does the trick.... converts to UTF-8 and
> transforms
> >> to UTF-8 format string and sends it back to the caller
> >> Wherever in your service class / action class add this method and
> convert
> >> the existing string to return the UTF-8 transformed string / text
> >> /**
> >> *
> >> * @param utfEightString
> >> * @return String converted to USF-8 format and send it to the caller
> .
> >> * @throws java.io.
> >> UnsupportedEncodingException
> >> */
> >> public static final String utfEightConvert(String utfEightString)
> >> throws java.io.UnsupportedEncodingException {
> >>
> >> byte[] bytes = new byte[utfEightString.length()];
> >> for (int i = 0; i < utfEightString.length(); i++) {
> >> bytes[i] = (byte) utfEightString.charAt(i);
> >> }
> >> return new String(bytes, "UTF-8");
> >>
> >> }
> >>
> >> Hope it helps.
> >> Rakesh
> >>
> >>
> >> On Tue, Jun 22, 2010 at 1:47 PM, Sergey Beryozkin <sberyozkin@gmail.com
> >> >wrote:
> >>
> >> > Is it a form based submission ? or XML is posted in the body ?
> >> >
> >> >
> >> > cheers, Sergey
> >> >
> >> > On Mon, Jun 21, 2010 at 6:00 PM, Loh Kok Jeng <ko...@gmail.com>
> >> > wrote:
> >> >
> >> > > Dear all,
> >> > >
> >> > > I'm struggling with non-ASCII characters passed in request params of
> a
> >> > > REST service. The characters become ? when received by my app
> >> > > developed using CXF. The encoding is set to UTF-8 by the REST
> client.
> >> > > Why do I get "?"?
> >> > >
> >> > > Thanks in advance.
> >> > >
> >> >
> >>
> >
>
RE: UTF-8 Characters in Request Params for REST Service
Posted by Ron Grimes <rg...@sinclairoil.com>.
I think I may have found the answer. See http://java.sun.com/docs/books/tutorial/i18n/text/string.html ).
Here, they talk about how it is incorrect to load a string value into a bytes array without specifying the encoding. Notice, in their example, that the difference in
byte[] bytes = utfEightString.getBytes(); \\ bad
byte[] bytes = utfEightString.getBytes("UTF8"); \\ good
will produce a different number of byte elements. Essentially, your loop is doing the same thing. For a five characters string, you will always end up with a 5 element byte array, when it might have given you an 8 byte array, had you specified the encoding.
Instead, you should do this:
byte[] bytes = utfEightString.getBytes("UTF8");
return new String(bytes, "UTF8");
Ron Grimes
-----Original Message-----
From: Loh Kok Jeng [mailto:kokjeng.loh@gmail.com]
Sent: Wednesday, June 23, 2010 7:11 PM
To: users@cxf.apache.org
Subject: Re: UTF-8 Characters in Request Params for REST Service
Hi Sergey,
This is for form based submission.
Below is a snippet of my code. I want to be able to accept non-ASCII
characters in "message" parameter below. However, when I tested with
Chinese characters, they are converted to "?" as shown in my logs.
Any help will be very much appreciated.
@POST
// @Consumes("application/x-www-form-urlencoded")
public SMSSendSmsResponse sendSms(@HeaderParam("Authorization")
String authorization,
@Context MessageContext mc,
@FormParam("address") List<String> address,
@FormParam("message") String message,
@FormParam("notifyURL") String notifyURL,
@FormParam("correlator") String
correlator,
@FormParam("senderName") String
senderName,
@FormParam("Charging") String Charging)
throws MalformedURLException, PolicyException, ServiceException {
if (address != null) {
for (Iterator<String> i = address.iterator(); i.hasNext();) {
log.info("Address: " + i.next());
}
}
log.info("Message: " + message);
log.info("Notify URL: " + notifyURL);
log.info("Correlator: " + correlator);
log.info("Sender Name: " + senderName);
log.info("Charging: " + Charging);
....
}
regards,
Loh Kok Jeng
On 24 June 2010 02:00, Sergey Beryozkin <sb...@gmail.com> wrote:
> I need to know the details of the request and how a resource method
> expecting the values looks like
>
> thanks, Sergey
>
> On Tue, Jun 22, 2010 at 7:10 PM, Rakesh Rai <ra...@gmail.com> wrote:
>
>> This is the method that does the trick.... converts to UTF-8 and transforms
>> to UTF-8 format string and sends it back to the caller
>> Wherever in your service class / action class add this method and convert
>> the existing string to return the UTF-8 transformed string / text
>> /**
>> *
>> * @param utfEightString
>> * @return String converted to USF-8 format and send it to the caller .
>> * @throws java.io.
>> UnsupportedEncodingException
>> */
>> public static final String utfEightConvert(String utfEightString)
>> throws java.io.UnsupportedEncodingException {
>>
>> byte[] bytes = new byte[utfEightString.length()];
>> for (int i = 0; i < utfEightString.length(); i++) {
>> bytes[i] = (byte) utfEightString.charAt(i);
>> }
>> return new String(bytes, "UTF-8");
>>
>> }
>>
>> Hope it helps.
>> Rakesh
>>
>>
>> On Tue, Jun 22, 2010 at 1:47 PM, Sergey Beryozkin <sberyozkin@gmail.com
>> >wrote:
>>
>> > Is it a form based submission ? or XML is posted in the body ?
>> >
>> >
>> > cheers, Sergey
>> >
>> > On Mon, Jun 21, 2010 at 6:00 PM, Loh Kok Jeng <ko...@gmail.com>
>> > wrote:
>> >
>> > > Dear all,
>> > >
>> > > I'm struggling with non-ASCII characters passed in request params of a
>> > > REST service. The characters become ? when received by my app
>> > > developed using CXF. The encoding is set to UTF-8 by the REST client.
>> > > Why do I get "?"?
>> > >
>> > > Thanks in advance.
>> > >
>> >
>>
>
Re: UTF-8 Characters in Request Params for REST Service
Posted by Loh Kok Jeng <ko...@gmail.com>.
Hi Sergey,
This is for form based submission.
Below is a snippet of my code. I want to be able to accept non-ASCII
characters in "message" parameter below. However, when I tested with
Chinese characters, they are converted to "?" as shown in my logs.
Any help will be very much appreciated.
@POST
// @Consumes("application/x-www-form-urlencoded")
public SMSSendSmsResponse sendSms(@HeaderParam("Authorization")
String authorization,
@Context MessageContext mc,
@FormParam("address") List<String> address,
@FormParam("message") String message,
@FormParam("notifyURL") String notifyURL,
@FormParam("correlator") String
correlator,
@FormParam("senderName") String
senderName,
@FormParam("Charging") String Charging)
throws MalformedURLException, PolicyException, ServiceException {
if (address != null) {
for (Iterator<String> i = address.iterator(); i.hasNext();) {
log.info("Address: " + i.next());
}
}
log.info("Message: " + message);
log.info("Notify URL: " + notifyURL);
log.info("Correlator: " + correlator);
log.info("Sender Name: " + senderName);
log.info("Charging: " + Charging);
....
}
regards,
Loh Kok Jeng
On 24 June 2010 02:00, Sergey Beryozkin <sb...@gmail.com> wrote:
> I need to know the details of the request and how a resource method
> expecting the values looks like
>
> thanks, Sergey
>
> On Tue, Jun 22, 2010 at 7:10 PM, Rakesh Rai <ra...@gmail.com> wrote:
>
>> This is the method that does the trick.... converts to UTF-8 and transforms
>> to UTF-8 format string and sends it back to the caller
>> Wherever in your service class / action class add this method and convert
>> the existing string to return the UTF-8 transformed string / text
>> /**
>> *
>> * @param utfEightString
>> * @return String converted to USF-8 format and send it to the caller .
>> * @throws java.io.
>> UnsupportedEncodingException
>> */
>> public static final String utfEightConvert(String utfEightString)
>> throws java.io.UnsupportedEncodingException {
>>
>> byte[] bytes = new byte[utfEightString.length()];
>> for (int i = 0; i < utfEightString.length(); i++) {
>> bytes[i] = (byte) utfEightString.charAt(i);
>> }
>> return new String(bytes, "UTF-8");
>>
>> }
>>
>> Hope it helps.
>> Rakesh
>>
>>
>> On Tue, Jun 22, 2010 at 1:47 PM, Sergey Beryozkin <sberyozkin@gmail.com
>> >wrote:
>>
>> > Is it a form based submission ? or XML is posted in the body ?
>> >
>> >
>> > cheers, Sergey
>> >
>> > On Mon, Jun 21, 2010 at 6:00 PM, Loh Kok Jeng <ko...@gmail.com>
>> > wrote:
>> >
>> > > Dear all,
>> > >
>> > > I'm struggling with non-ASCII characters passed in request params of a
>> > > REST service. The characters become ? when received by my app
>> > > developed using CXF. The encoding is set to UTF-8 by the REST client.
>> > > Why do I get "?"?
>> > >
>> > > Thanks in advance.
>> > >
>> >
>>
>
Re: UTF-8 Characters in Request Params for REST Service
Posted by Sergey Beryozkin <sb...@gmail.com>.
I need to know the details of the request and how a resource method
expecting the values looks like
thanks, Sergey
On Tue, Jun 22, 2010 at 7:10 PM, Rakesh Rai <ra...@gmail.com> wrote:
> This is the method that does the trick.... converts to UTF-8 and transforms
> to UTF-8 format string and sends it back to the caller
> Wherever in your service class / action class add this method and convert
> the existing string to return the UTF-8 transformed string / text
> /**
> *
> * @param utfEightString
> * @return String converted to USF-8 format and send it to the caller .
> * @throws java.io.
> UnsupportedEncodingException
> */
> public static final String utfEightConvert(String utfEightString)
> throws java.io.UnsupportedEncodingException {
>
> byte[] bytes = new byte[utfEightString.length()];
> for (int i = 0; i < utfEightString.length(); i++) {
> bytes[i] = (byte) utfEightString.charAt(i);
> }
> return new String(bytes, "UTF-8");
>
> }
>
> Hope it helps.
> Rakesh
>
>
> On Tue, Jun 22, 2010 at 1:47 PM, Sergey Beryozkin <sberyozkin@gmail.com
> >wrote:
>
> > Is it a form based submission ? or XML is posted in the body ?
> >
> >
> > cheers, Sergey
> >
> > On Mon, Jun 21, 2010 at 6:00 PM, Loh Kok Jeng <ko...@gmail.com>
> > wrote:
> >
> > > Dear all,
> > >
> > > I'm struggling with non-ASCII characters passed in request params of a
> > > REST service. The characters become ? when received by my app
> > > developed using CXF. The encoding is set to UTF-8 by the REST client.
> > > Why do I get "?"?
> > >
> > > Thanks in advance.
> > >
> >
>
Re: UTF-8 Characters in Request Params for REST Service
Posted by Rakesh Rai <ra...@gmail.com>.
This is the method that does the trick.... converts to UTF-8 and transforms
to UTF-8 format string and sends it back to the caller
Wherever in your service class / action class add this method and convert
the existing string to return the UTF-8 transformed string / text
/**
*
* @param utfEightString
* @return String converted to USF-8 format and send it to the caller .
* @throws java.io.
UnsupportedEncodingException
*/
public static final String utfEightConvert(String utfEightString)
throws java.io.UnsupportedEncodingException {
byte[] bytes = new byte[utfEightString.length()];
for (int i = 0; i < utfEightString.length(); i++) {
bytes[i] = (byte) utfEightString.charAt(i);
}
return new String(bytes, "UTF-8");
}
Hope it helps.
Rakesh
On Tue, Jun 22, 2010 at 1:47 PM, Sergey Beryozkin <sb...@gmail.com>wrote:
> Is it a form based submission ? or XML is posted in the body ?
>
>
> cheers, Sergey
>
> On Mon, Jun 21, 2010 at 6:00 PM, Loh Kok Jeng <ko...@gmail.com>
> wrote:
>
> > Dear all,
> >
> > I'm struggling with non-ASCII characters passed in request params of a
> > REST service. The characters become ? when received by my app
> > developed using CXF. The encoding is set to UTF-8 by the REST client.
> > Why do I get "?"?
> >
> > Thanks in advance.
> >
>
Re: UTF-8 Characters in Request Params for REST Service
Posted by Sergey Beryozkin <sb...@gmail.com>.
Is it a form based submission ? or XML is posted in the body ?
cheers, Sergey
On Mon, Jun 21, 2010 at 6:00 PM, Loh Kok Jeng <ko...@gmail.com> wrote:
> Dear all,
>
> I'm struggling with non-ASCII characters passed in request params of a
> REST service. The characters become ? when received by my app
> developed using CXF. The encoding is set to UTF-8 by the REST client.
> Why do I get "?"?
>
> Thanks in advance.
>