You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@groovy.apache.org by David Clark <pl...@gmail.com> on 2016/11/20 14:39:00 UTC
Re: HTTPBuilder & URIBuilder - plus sign will not encoded
Plus sign is legal in URI's so they don't need to get encoded to be legal:
http://stackoverflow.com/questions/1547899/which-characters-make-a-url-invalid/1547940#1547940
This is complicated by the fact that they are url encoded when part of
the query string, but that's not what you are doing here. It sounds to
me like the problem is on the server or there some other part of your
request that is incorrect.
If there is a problem inside HTTPBuilder, you are probably out of luck,
that project has been dead for a while.
However, I have created a successor to HTTPBuilder called
http-builder-ng that is active here:
https://github.com/dwclark/http-builder-ng
It is NOT a backwards compatible upgrade. When I was examining the
original source code I found many threading issues, code duplication, it
was imposible to use @TypeChecked/@CompileStatic, and found it hard to
integrated different clients inside HTTPBuilder. The result of these
frustrations was a ground up re-write. Bug reports/pull requests are
encouraged.
On 11/20/2016 07:25 AM, Christian Lotz wrote:
> Hi all,
> I try to upload files via HTTPBuilder (Groovy Version: 2.4.7 JVM:
> 1.8.0_40 Vendor: Oracle Corporation OS: Mac OS X). After a couple of
> test I realized, that filenames containing a plus sign ("+") can't be
> uploaded. The REST Service responds with HTTP/1.1 400 Bad Request.
> After some more debugging I think these filenames will not get encoded
> correctly:
> uri.path = "/rest/" + objectNamespace + "/" + objectFile.name
>
> groovyx.net.http.HTTPBuilder doRequest
> FINE: PUT http://172.16.29.10/rest/TEST/s*pace%20space.doc* -
> Filename: space\ space.doc - OK
>
> groovyx.net.http.HTTPBuilder doRequest
> FINE: PUT http://172.16.29.10/rest/TEST/*plus+plus.doc* -Filename
> plus+plus.doc - ERROR
>
> All other "special" characters like spaces, #, &, % or ? will
> get encoded correctly ...
> Does anybody know how to resolve this issue?
> Thanks in advance
> Paolo
Re: HTTPBuilder & URIBuilder - plus sign will not encoded
Posted by Guillaume Laforge <gl...@gmail.com>.
Thanks a lot for these explanations!
On Sun, Nov 20, 2016 at 8:20 PM, David Clark <pl...@gmail.com>
wrote:
> I haven't used groovy-wslite, so any comparisons may be ignorant on my
> part. The main design goals of http-builder-ng are:
>
> 1) Use modern groovy DSL design (https://www.infoq.com/
> presentations/groovy-dsl-2015). The original HTTPBuilder was nice but
> depended on propertyMissing/methodMissing magic to work.
>
> 2) Support full use of @CompileStatic/@TypeChecked.
>
> 3) Make IDE support natural through using @DelegatesTo everywhere (this
> also makes #2 possible).
>
> 4) Support multiple http clients. Right now both Apache Http Client and
> the native Java HttpURLConnection are fully supported. Adding new http
> clients should hopefully be simple.
>
> 5) Support threading and asynchronous calls safely and with no additional
> work on the part of users of http-builder-ng. Just give it an Executor and
> then use the async variants of the http verbs: getAsync() instead of get(),
> putAsync() instead of put() etc.
>
> 6) Adding new encoders and decoders is very easy.
>
> 7) Configuration should be cummulative and natural. This one is hard to
> describe, but it allows for applications to configure a single instance of
> HttpBuilder with default options and then every request can override or
> inherit those settings without affecting any other request or the single
> HttpBuilder instance.
>
> 8) Support interception of requests and responses to allow for logging,
> metrics, debugging etc.
>
> 9) High performance. We are using http-builder-ng for production
> applications at my job. We have seen cases where http-builder-ng + Apache
> Http Client will overwhelm a server with the load it can generate.
>
> The README and User Guide go into this in more detail.
>
> Taking a quick peek at groovy-wslite:
>
> 1) SOAP and multipart are supported natively by wslite. Adding SOAP
> support to http-builder-ng would mainly involve either using the native xml
> support directly or writing a custom encoder/decoder (should be pretty
> easy) that uses a SOAP library. Mulitpart support is on our list of things
> to do.
>
> 2) wslite allows for using Java 5. http-builder-ng requires Java 8.
>
> 3) wslite uses different classes to support different use cases
> (RESTClient, SOAPClient, etc.). http-builder-ng provides a single class
> that handles everything (HttpBuilder). Extending HttpBuilder is a matter of
> adding new encoders/decoders at runtime.
>
> 4) wslite handles network configuration at the top level. http-builder-ng
> depends upon http clients being configured correctly. This is nice because
> all features of the native clients (HttpURLConnection, Apache Http Client)
> are available. This is bad because you have to know about the features of
> the native clients, some of which are confusing.
>
> Hope this helps.
> On 11/20/2016 09:02 AM, Guillaume Laforge wrote:
>
> I didn't know about NG, David!
>
> These days, I tend to use groovy wslite:
> https://github.com/jwagenleitner/groovy-wslite
>
> What are the pros & cons of http-builder-ng and wslite?
>
> On Sun, Nov 20, 2016 at 3:39 PM, David Clark <pl...@gmail.com>
> wrote:
>
>> Plus sign is legal in URI's so they don't need to get encoded to be legal:
>>
>> http://stackoverflow.com/questions/1547899/which-characters-
>> make-a-url-invalid/1547940#1547940
>> This is complicated by the fact that they are url encoded when part of
>> the query string, but that's not what you are doing here. It sounds to me
>> like the problem is on the server or there some other part of your request
>> that is incorrect.
>>
>> If there is a problem inside HTTPBuilder, you are probably out of luck,
>> that project has been dead for a while.
>>
>> However, I have created a successor to HTTPBuilder called http-builder-ng
>> that is active here:
>>
>> https://github.com/dwclark/http-builder-ng
>>
>> It is NOT a backwards compatible upgrade. When I was examining the
>> original source code I found many threading issues, code duplication, it
>> was imposible to use @TypeChecked/@CompileStatic, and found it hard to
>> integrated different clients inside HTTPBuilder. The result of these
>> frustrations was a ground up re-write. Bug reports/pull requests are
>> encouraged.
>>
>>
>>
>> On 11/20/2016 07:25 AM, Christian Lotz wrote:
>>
>> Hi all,
>> I try to upload files via HTTPBuilder (Groovy Version: 2.4.7 JVM:
>> 1.8.0_40 Vendor: Oracle Corporation OS: Mac OS X). After a couple of test I
>> realized, that filenames containing a plus sign ("+") can't be uploaded.
>> The REST Service responds with HTTP/1.1 400 Bad Request. After some more
>> debugging I think these filenames will not get encoded correctly:
>>
>> uri.path = "/rest/" + objectNamespace + "/" + objectFile.name
>>
>> groovyx.net.http.HTTPBuilder doRequest
>> FINE: PUT http://172.16.29.10/rest/TEST/s*pace%20space.doc* - Filename:
>> space\ space.doc - OK
>>
>> groovyx.net.http.HTTPBuilder doRequest
>> FINE: PUT http://172.16.29.10/rest/TEST/*plus+plus.doc* -Filename
>> plus+plus.doc - ERROR
>>
>> All other "special" characters like spaces, #, &, % or ? will get encoded
>> correctly ...
>>
>> Does anybody know how to resolve this issue?
>>
>> Thanks in advance
>> Paolo
>>
>>
>>
>
>
> --
> Guillaume Laforge
> Apache Groovy committer & PMC Vice-President
> Developer Advocate @ Google Cloud Platform
>
> Blog: http://glaforge.appspot.com/
> Social: @glaforge <http://twitter.com/glaforge> / Google+
> <https://plus.google.com/u/0/114130972232398734985/posts>
>
>
>
--
Guillaume Laforge
Apache Groovy committer & PMC Vice-President
Developer Advocate @ Google Cloud Platform
Blog: http://glaforge.appspot.com/
Social: @glaforge <http://twitter.com/glaforge> / Google+
<https://plus.google.com/u/0/114130972232398734985/posts>
Re: HTTPBuilder & URIBuilder - plus sign will not encoded
Posted by David Clark <pl...@gmail.com>.
I haven't used groovy-wslite, so any comparisons may be ignorant on my
part. The main design goals of http-builder-ng are:
1) Use modern groovy DSL design
(https://www.infoq.com/presentations/groovy-dsl-2015). The original
HTTPBuilder was nice but depended on propertyMissing/methodMissing magic
to work.
2) Support full use of @CompileStatic/@TypeChecked.
3) Make IDE support natural through using @DelegatesTo everywhere (this
also makes #2 possible).
4) Support multiple http clients. Right now both Apache Http Client and
the native Java HttpURLConnection are fully supported. Adding new http
clients should hopefully be simple.
5) Support threading and asynchronous calls safely and with no
additional work on the part of users of http-builder-ng. Just give it an
Executor and then use the async variants of the http verbs: getAsync()
instead of get(), putAsync() instead of put() etc.
6) Adding new encoders and decoders is very easy.
7) Configuration should be cummulative and natural. This one is hard to
describe, but it allows for applications to configure a single instance
of HttpBuilder with default options and then every request can override
or inherit those settings without affecting any other request or the
single HttpBuilder instance.
8) Support interception of requests and responses to allow for logging,
metrics, debugging etc.
9) High performance. We are using http-builder-ng for production
applications at my job. We have seen cases where http-builder-ng +
Apache Http Client will overwhelm a server with the load it can generate.
The README and User Guide go into this in more detail.
Taking a quick peek at groovy-wslite:
1) SOAP and multipart are supported natively by wslite. Adding SOAP
support to http-builder-ng would mainly involve either using the native
xml support directly or writing a custom encoder/decoder (should be
pretty easy) that uses a SOAP library. Mulitpart support is on our list
of things to do.
2) wslite allows for using Java 5. http-builder-ng requires Java 8.
3) wslite uses different classes to support different use cases
(RESTClient, SOAPClient, etc.). http-builder-ng provides a single class
that handles everything (HttpBuilder). Extending HttpBuilder is a matter
of adding new encoders/decoders at runtime.
4) wslite handles network configuration at the top level.
http-builder-ng depends upon http clients being configured correctly.
This is nice because all features of the native clients
(HttpURLConnection, Apache Http Client) are available. This is bad
because you have to know about the features of the native clients, some
of which are confusing.
Hope this helps.
On 11/20/2016 09:02 AM, Guillaume Laforge wrote:
> I didn't know about NG, David!
>
> These days, I tend to use groovy wslite:
> https://github.com/jwagenleitner/groovy-wslite
>
> What are the pros & cons of http-builder-ng and wslite?
>
> On Sun, Nov 20, 2016 at 3:39 PM, David Clark <plotinussmith@gmail.com
> <ma...@gmail.com>> wrote:
>
> Plus sign is legal in URI's so they don't need to get encoded to
> be legal:
>
> http://stackoverflow.com/questions/1547899/which-characters-make-a-url-invalid/1547940#1547940
> <http://stackoverflow.com/questions/1547899/which-characters-make-a-url-invalid/1547940#1547940>
>
> This is complicated by the fact that they are url encoded when
> part of the query string, but that's not what you are doing here.
> It sounds to me like the problem is on the server or there some
> other part of your request that is incorrect.
>
> If there is a problem inside HTTPBuilder, you are probably out of
> luck, that project has been dead for a while.
>
> However, I have created a successor to HTTPBuilder called
> http-builder-ng that is active here:
>
> https://github.com/dwclark/http-builder-ng
> <https://github.com/dwclark/http-builder-ng>
>
> It is NOT a backwards compatible upgrade. When I was examining the
> original source code I found many threading issues, code
> duplication, it was imposible to use @TypeChecked/@CompileStatic,
> and found it hard to integrated different clients inside
> HTTPBuilder. The result of these frustrations was a ground up
> re-write. Bug reports/pull requests are encouraged.
>
>
>
> On 11/20/2016 07:25 AM, Christian Lotz wrote:
>> Hi all,
>> I try to upload files via HTTPBuilder (Groovy Version: 2.4.7 JVM:
>> 1.8.0_40 Vendor: Oracle Corporation OS: Mac OS X). After a couple
>> of test I realized, that filenames containing a plus sign ("+")
>> can't be uploaded. The REST Service responds with HTTP/1.1 400
>> Bad Request. After some more debugging I think these filenames
>> will not get encoded correctly:
>> uri.path = "/rest/" + objectNamespace + "/" + objectFile.name
>>
>> groovyx.net.http.HTTPBuilder doRequest
>> FINE: PUT http://172.16.29.10/rest/TEST/s
>> <http://172.16.29.10/rest/TEST/s>*pace%20space.doc* - Filename:
>> space\ space.doc - OK
>>
>> groovyx.net.http.HTTPBuilder doRequest
>> FINE: PUT http://172.16.29.10/rest/TEST/*plus+plus.doc* -Filename
>> plus+plus.doc - ERROR
>>
>> All other "special" characters like spaces, #, &, % or ? will
>> get encoded correctly ...
>> Does anybody know how to resolve this issue?
>> Thanks in advance
>> Paolo
>
>
>
>
> --
> Guillaume Laforge
> Apache Groovy committer & PMC Vice-President
> Developer Advocate @ Google Cloud Platform
>
> Blog: http://glaforge.appspot.com/
> Social: @glaforge <http://twitter.com/glaforge> / Google+
> <https://plus.google.com/u/0/114130972232398734985/posts>
Re: HTTPBuilder & URIBuilder - plus sign will not encoded
Posted by Guillaume Laforge <gl...@gmail.com>.
I didn't know about NG, David!
These days, I tend to use groovy wslite:
https://github.com/jwagenleitner/groovy-wslite
What are the pros & cons of http-builder-ng and wslite?
On Sun, Nov 20, 2016 at 3:39 PM, David Clark <pl...@gmail.com>
wrote:
> Plus sign is legal in URI's so they don't need to get encoded to be legal:
>
> http://stackoverflow.com/questions/1547899/which-
> characters-make-a-url-invalid/1547940#1547940
> This is complicated by the fact that they are url encoded when part of the
> query string, but that's not what you are doing here. It sounds to me like
> the problem is on the server or there some other part of your request that
> is incorrect.
>
> If there is a problem inside HTTPBuilder, you are probably out of luck,
> that project has been dead for a while.
>
> However, I have created a successor to HTTPBuilder called http-builder-ng
> that is active here:
>
> https://github.com/dwclark/http-builder-ng
>
> It is NOT a backwards compatible upgrade. When I was examining the
> original source code I found many threading issues, code duplication, it
> was imposible to use @TypeChecked/@CompileStatic, and found it hard to
> integrated different clients inside HTTPBuilder. The result of these
> frustrations was a ground up re-write. Bug reports/pull requests are
> encouraged.
>
>
>
> On 11/20/2016 07:25 AM, Christian Lotz wrote:
>
> Hi all,
> I try to upload files via HTTPBuilder (Groovy Version: 2.4.7 JVM: 1.8.0_40
> Vendor: Oracle Corporation OS: Mac OS X). After a couple of test I
> realized, that filenames containing a plus sign ("+") can't be uploaded.
> The REST Service responds with HTTP/1.1 400 Bad Request. After some more
> debugging I think these filenames will not get encoded correctly:
>
> uri.path = "/rest/" + objectNamespace + "/" + objectFile.name
>
> groovyx.net.http.HTTPBuilder doRequest
> FINE: PUT http://172.16.29.10/rest/TEST/s*pace%20space.doc* - Filename:
> space\ space.doc - OK
>
> groovyx.net.http.HTTPBuilder doRequest
> FINE: PUT http://172.16.29.10/rest/TEST/*plus+plus.doc* -Filename
> plus+plus.doc - ERROR
>
> All other "special" characters like spaces, #, &, % or ? will get encoded
> correctly ...
>
> Does anybody know how to resolve this issue?
>
> Thanks in advance
> Paolo
>
>
>
--
Guillaume Laforge
Apache Groovy committer & PMC Vice-President
Developer Advocate @ Google Cloud Platform
Blog: http://glaforge.appspot.com/
Social: @glaforge <http://twitter.com/glaforge> / Google+
<https://plus.google.com/u/0/114130972232398734985/posts>
Re: Aw: Re: HTTPBuilder & URIBuilder - plus sign will not encoded
Posted by David Clark <pl...@gmail.com>.
Both HTTBuilder and http-builder-ng use the java.net.URI class to store
and manipulate URI information. java.net.URI has the following behavior:
1) If you set the full path using the single string constructor, URI
keeps the %2B in the URI.
2) If you set the path using the constructors that take the uri parts,
the path is url encoded
3) Extracting the path will always result in url decoding
Both libraries also always use the constructor that takes uri parts,
meaning you will always get the encoding and decoding that may be
causing you problems. The following groovy code shows the behavior:
def full = new URI('http://172.16.29.10/rest/TEST/c%2B%2B.doc')
def parts = new URI('http', '172.16.29.10', '/rest/TEST/c%2B%2B.doc', null)
println(full.toString())
println(full.path)
println(parts.toString())
println(parts.path)
On 11/20/2016 09:12 AM, Christian Lotz wrote:
> OK ... then I guess that the REST service expects "+" to be enoded
> (%2B). Is there a way to "enforce" the encoding of the plus sign to
> "%2B" using the URIBuilder for uri.path? If I replace the plus
> sign and assign the string to uri.path it gets encoded again (%252B)
> by replacing the "%" with "%25" ...
> Thanks in advance
> *Gesendet:* Sonntag, 20. November 2016 um 15:39 Uhr
> *Von:* "David Clark" <pl...@gmail.com>
> *An:* users@groovy.apache.org
> *Betreff:* Re: HTTPBuilder & URIBuilder - plus sign will not encoded
>
> Plus sign is legal in URI's so they don't need to get encoded to be legal:
>
> http://stackoverflow.com/questions/1547899/which-characters-make-a-url-invalid/1547940#1547940
>
> This is complicated by the fact that they are url encoded when part of
> the query string, but that's not what you are doing here. It sounds to
> me like the problem is on the server or there some other part of your
> request that is incorrect.
>
> If there is a problem inside HTTPBuilder, you are probably out of
> luck, that project has been dead for a while.
>
> However, I have created a successor to HTTPBuilder called
> http-builder-ng that is active here:
>
> https://github.com/dwclark/http-builder-ng
>
> It is NOT a backwards compatible upgrade. When I was examining the
> original source code I found many threading issues, code duplication,
> it was imposible to use @TypeChecked/@CompileStatic, and found it hard
> to integrated different clients inside HTTPBuilder. The result of
> these frustrations was a ground up re-write. Bug reports/pull requests
> are encouraged.
>
> On 11/20/2016 07:25 AM, Christian Lotz wrote:
>
> Hi all,
> I try to upload files via HTTPBuilder (Groovy Version: 2.4.7 JVM:
> 1.8.0_40 Vendor: Oracle Corporation OS: Mac OS X). After a couple
> of test I realized, that filenames containing a plus sign ("+")
> can't be uploaded. The REST Service responds with HTTP/1.1 400 Bad
> Request. After some more debugging I think these filenames will
> not get encoded correctly:
> uri.path = "/rest/" + objectNamespace + "/" + objectFile.name
>
> groovyx.net.http.HTTPBuilder doRequest
> FINE: PUT http://172.16.29.10/rest/TEST/s*pace%20space.doc* -
> Filename: space\ space.doc - OK
>
> groovyx.net.http.HTTPBuilder doRequest
> FINE: PUT http://172.16.29.10/rest/TEST/*plus+plus.doc* -Filename
> plus+plus.doc - ERROR
>
> All other "special" characters like spaces, #, &, % or ? will
> get encoded correctly ...
> Does anybody know how to resolve this issue?
> Thanks in advance
> Paolo
>