Encoding of form parameters in proxies

[Sergey Beryozkin <sb...@gmail.com>]

You also got @FormParams 'involved' in the thread about encoding '%' so 
I'm starting a new thread too.

So as far as the client API is concerned, the only 'standard' form 
parameter support is captured here:
https://jax-rs-spec.java.net/nonav/2.0-rev-a/apidocs/javax/ws/rs/core/Form.html

That is it. The way the payloads are processed is up to the 
implementations - in our case you can register an instance of 
FormEncodingProvider with the proxy and set its 'expectEncoded' property 
to true - in this case it will not double encode.

Sergey

On 08/03/16 16:58, Sergey Beryozkin wrote:
> I started another thread on why '+' is encoded because this thread has
> been all about encoding a '%' symbol.
> On 08/03/16 16:50, Sergey Beryozkin wrote:
>> I'm not going to repeat most of what I've already said few times in this
>> thread but only state again that CXF proxies depend on JAX-RS UriBuilder
>> and therefore UriBuilder dictates how %-encoded query (and other)
>> parameters get processed.
>>
>> This is a won't fix 'issue'. However I also mentioned that if you
>> validate that Jersey's (i.e RI's) UriBuilder behaves differently then I
>> will act accordingly.
>>
>> Sergey
>>
>> On 08/03/16 16:44, Volkov Sergey wrote:
>>> I agree that UriBuilder implements what is states.
>>>
>>> Such values are allowed in the query representations. The UriBuilder
>>>> does not know what the intention of the CXF proxy user is, it may have
>>>> an encoded representation of a given character stored in the DB and
>>>> wants to pass it as is via the proxy.
>>>>
>>>
>>> "+" value also allowed in query representation, but is encoded for some
>>> reason by UriBuilder. So there would be problems with stored in db
>>> values
>>> if they contains "+"
>>>
>>> I'm not using jerecy, only
>>> org.apache.cxf.jaxrs.client.JAXRSClientFactoryBean to create proxies.
>>>
>>> I found https://issues.apache.org/jira/browse/CXF-6015 and
>>> "url.encode.client.parameters" property, but because
>>> org.apache.cxf.jaxrs.client.UrlEncodingParamConverter encodes space as
>>> "+",
>>> and "+" is encoded as "%2B" by UriBuilder it's also not very useful
>>> (Sequence like "+ +" would be converted to "%2B%2B%2B" and decoded as
>>> "+++") Also it double encodes @FormParam values.
>>>
>>>
>>>
>>>> Please forget about CXF proxies. It is down to UriBuilder.
>>>> http://docs.oracle.com/javaee/6/api/javax/ws/rs/core/UriBuilder.html
>>>> states:
>>>>
>>>>
>>>> 'Percent encoded values are also recognized where allowed and will not
>>>> be double encoded.'
>>>>
>>>> Such values are allowed in the query representations. The UriBuilder
>>>> does not know what the intention of the CXF proxy user is, it may have
>>>> an encoded representation of a given character stored in the DB and
>>>> wants to pass it as is via the proxy.
>>>>
>>>> Experiment with Jersey and if they do double encode (I doubt but...)
>>>> then we will continue, I think I've given you my best explanations so
>>>> far, lets continue later on when you have the Jersey test results
>>>> ready.
>>>>
>>>> Sergey
>>>>
>>>> On 04/03/16 11:16, Volkov Sergey wrote:
>>>>
>>>>> i.e CXF proxy do not urlencode parameters? (Instead of it use
>>>>> org.apache.cxf.jaxrs.utils.HttpUtils#encodePartiallyEncoded encoding
>>>> which
>>>>> is not reversible)
>>>>>
>>>>> And the only reason of this encoding is to avoid reports about
>>>>> correctly
>>>>> encoded parameters?
>>>>>
>>>>> As I said, CXF proxy runtime uses JAX-RS UriBuilder.
>>>>>>
>>>>>> And UriBuilder does not touch valid percent-encoded representations.
>>>>>>
>>>>>> And this is how the RS client runtime will continue operating.
>>>>>>
>>>>>> Sergey
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 04/03/16 10:45, Volkov Sergey wrote:
>>>>>>
>>>>>>>>
>>>>>>>> "100%25BETTER VALID STRING"
>>>>>>>>
>>>>>>>> instead of
>>>>>>>>
>>>>>>>> "100%BETTER VALID STRING"
>>>>>>>>
>>>>>>>
>>>>>>> "100%25BETTER VALID STRING" is correct way to urlencode "100%BETTER
>>>>>> VALID
>>>>>>> STRING" - so actually CXF is over-smart now, when it skips all %xx
>>>>>> sequence
>>>>>>> instead of encode % as %25
>>>>>>>
>>>>>>> "100%VALID STRING" at the moment encodes by cxf as "100%25VALID
>>>> STRING"
>>>>>>>
>>>>>>>
>>>>>>>> In the former case it it has no characters that can be used to form
>>>> the
>>>>>>>> hex %-encoded representations.
>>>>>>>>
>>>>>>>> The bottom line is, you should not expect the runtime auto-encode
>>>>>>>> '%'
>>>>>>>> given that this symbol plays a special role in having correct URIs
>>>>>> being
>>>>>>>> created.
>>>>>>>>
>>>>>>>> If CXF will try to me over-smart then the next day someone will
>>>> report
>>>>>>>> that it started producing
>>>>>>>>
>>>>>>>> "100%25BETTER VALID STRING"
>>>>>>>>
>>>>>>>> instead of
>>>>>>>>
>>>>>>>> "100%BETTER VALID STRING"
>>>>>>>>
>>>>>>>> Sergey
>>>>>>>> On 04/03/16 07:43, Volkov Sergey wrote:
>>>>>>>>
>>>>>>>>> Looks like POST request and @FormParam could be used, but I think
>>>> this
>>>>>>>>> behavior of @QueryParam may lead to tricky bugs in many places
>>>>>>>>> where
>>>>>> it
>>>>>>>> used
>>>>>>>>>
>>>>>>>>> 2016-03-04 10:37 GMT+03:00 Volkov Sergey <[hidden email]
>>>>>>>> <
>>>>>>
>>>> http://cxf.547215.n5.nabble.com/user/SendEmail.jtp?type=node&node=5766616&i=0>>:
>>>>
>>>>
>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Yes, I don't understand why it does not touch this sequences.
>>>>>>>>>>
>>>>>>>>>> It's a bit strange because I can successfully transfer "100%VALID
>>>>>>>> STRING"
>>>>>>>>>> but not "100%BETTER VALID STRING"
>>>>>>>>>>
>>>>>>>>>> Both this string not percent encoded, but second contains %BE
>>>>>>>>>> which
>>>>>>>> just
>>>>>>>>>> looks like percent encoded.
>>>>>>>>>>
>>>>>>>>>> If instead of
>>>>>>>> org.apache.cxf.jaxrs.utils.HttpUtils#encodePartiallyEncoded
>>>>>>>>>> cxf just urlencode parameter value when building query string
>>>>>>>>>> both
>>>>>>>> examples
>>>>>>>>>> would be transferred correctly.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> If you are saying that
>>>>>>>>>>>
>>>>>>>>>>> when you pass something like "%D1%82", or to make it clearer,
>>>>>>>>>>> "%D1" and expect CXF do "%25D1", then no, it won't work.
>>>>>>>>>>> UriBuilder does not touch percent-encoded sequences %XX.
>>>>>>>>>>>
>>>>>>>>>>> May be try building the query with UriBuilder before passing
>>>>>>>>>>> it to
>>>>>> the
>>>>>>>>>>> proxy, something like
>>>>>>>>>>>
>>>>>>>>>>> UriBuilder.fromUri("{a}%D21").build("%").toString()
>>>>>>>>>>>
>>>>>>>>>>> Though not sure it will be cheaper.
>>>>>>>>>>>
>>>>>>>>>>> Sergey
>>>>>>>>>>>
>>>>>>>>>>> On 03/03/16 21:27, Sergey Beryozkin wrote:
>>>>>>>>>>>
>>>>>>>>>>>>      From the original post in this thread:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>>>> When I pass string "т" (Cyrillic symbol) to generated
>>>>>>>>>>>>>>>> client
>>>> in
>>>>>>>>>>>>> encodes
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> "%D1%82" in url and decodes as "т" on server what is fine,
>>>> but
>>>>>>>> when
>>>>>>>>>>> i
>>>>>>>>>>>>>>> pass
>>>>>>>>>>>>>>>> "%D1%82" rather then "%25D1%2582" in encodes also to
>>>>>>>>>>>>>>>> "%D1%82"
>>>>>> and
>>>>>>>>>>>>>>> decodes
>>>>>>>>>>>>>>>> also to "т".
>>>>>>>>>>>>
>>>>>>>>>>>> I honestly don't get what the problem is. You pass a Cyrillic
>>>>>> symbol
>>>>>>>>>>>> either as is or already percent encoded and in both case, as
>>>>>>>>>>>> you
>>>>>> say
>>>>>>>>>>>> above, you get this Cyrillic symbol.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>
>


-- 
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/