You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Ortwin Glück <or...@nose.ch> on 2002/06/26 17:13:05 UTC
[httpclient] Volunteering to add support for PROXY AUTH and STREAMING
INPUT for POST requests
Hello
Our company would like to help in the HttpClient project.
We wish to add the following features:
______________________________________
1. Support for authenticating proxies
The HttpClient currently only supports authentication against the web
server but not against the proxy.
We hereby suggest to change the API as follows:
HttpState:
- add field: proxyCredentials
- add setter method for proxyCredentials
We could have used the existing credential map. But we prefer to
create a separate field for two reasons:
- proxy auth uses different HTTP Headers than WWW Auth. So we want
to avoid mixing up www auth and proxy auth.
- A connection can only have one proxy. Its realm is not relevant
and in most cases not known to the programmer anyway.
- change HttpMethodBase.execute
Include a case for the SC_PROXY_AUTHENTICATION_REQUIRED status code
which performs proxy authentication
- change Authenticator.authenticate
Include proxy auth
The changes leave the HttpClient compatible with previous versions.
_________________________________________________
2. Support for streaming input for POST requests
The HttpClient currently only accepts Strings as the body of a POST
request. This is not suitable for applications like upload of huge files
or binary data. Moreover buffering the whole request in memory is
problematic when dealing with large amounts of data.
We hereby suggest to change the API as follows:
PostMethod:
- add setContentLength(int)
A convenience method to add the content length header directly or in
an indirect fashion. See below.
- add int constants CONTENT_LENGTH_AUTO = 0, CONTENT_LENGTH_CHUNKED = -1
They are used with setContentLength
- add setRequestBody(InputStream)
The stream is handled depending on the argument of setContentLength.
If the content length was set to CONTENT_LENGTH_AUTO or the content
length was not set at all, the input is buffered and the correct content
length is calculated automatically. If content length was set to
CONTENT_LENGTH_CHUNKED the stream is not buffered and sent with chunked
transfer encoding (note that chunked POST is not widely supported by web
servers, but it works okay with Tomcat and actually defined in rfc
2068). If content length was set to anything else the stream is not
buffered and the content length will be used as set by the user. If
however the stream ends before that length, an IOException is thrown (in
HttpClient.executeMethod).
This enables POSTing of large files without the need to buffer the
data in memory thus beeing more conservative on memory. More complex
data structures (like Multipart MIME, Base64 encoded data) can be
wrapped in a user defined input stream which is then passed to the HTTP
client.
- deprecate setRequestBody(String) because it is dangerous
The platform's default character encoding is used here. This is bad
for portability. Have a Mac OS-X and a Windows client talk to a Linux
Server and you will be messing up data completely. When writing to a
network connection only byte data should be used and not strings that
are implicitely encoded.
We need these features for one of our projects. Please let us know if
you will incorporate these feature into the current code base if we
submit a patch.
Best regards
Ortwin Glueck
--
_________________________________________________________________
NOSE applied intelligence ag [perspectix-nose digital b.i]
[www] http://www.nose.ch
ortwin glück [email] ortwin.glueck@nose.ch
hardturmstrasse 171 [office] +41-1-277 57 35
8005 zurich [fax] +41-1-277 57 12
switzerland
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [httpclient] Volunteering to add support for PROXY AUTH
andSTREAMING INPUT for POST requests
Posted by Evert Hoff <ev...@pixie.co.za>.
Hi,
I have the same requirement. The patch that I sent yesterday includes a
setter for SSLSocketFactory in HttpMultiClient. I have preferred not to
make it a static, because someone somewhere someday might have a
requirement to run different instances HttpMultiClient that each uses a
different factory.
The setter in HttpMultiClient just calls the same setter in
HttpConnectionManager which in turn creates HttpConnections with this
factory.
Evert
On Wed, 2002-06-26 at 23:19, Simon Roberts wrote:
> Hi guys,
>
> I've got a patch that I've been using myself, but it might be of some use to
> others. Or (better!) perhaps someone could tell me a way around my need for
> it! If they do, I'll document it for USING_HTTPS.txt :)
>
> The (fugly) patch makes HttpConnection use a static to find a
> SSLSocketFactory, rather than using SSLSocketFactory.getDefault. The reason
> is that the US-export version of JSSE doesn't allow the users to override
> the implementation created by SSLSocketFactory.getDefault().
>
> The default implementation (of SSLSocketFactory) can be configured by
> setting a few system/security properties, such as "javax.net.ssl.keyStore"
> etc, but we need to be able to use a different trustmanager (our acceptance
> of server certs is more involved than certificate chains). There is a
> security property "sun.ssl.keymanager.type" which can be used to set the
> default keymanager algorithm, but I can't figure out how to register my own
> KeyManager against that name. Help!
>
> See
> http://java.sun.com/products/jsse/doc/guide/API_users_guide.html#Installatio
> nAndCustomization
>
> I'd love to hear if anyone knows about registering KeyManager algorithms, or
> wants my fugly patch.
>
> Cheers, Simon
>
> ----- Original Message -----
> From: "Evert Hoff" <ev...@pixie.co.za>
> Newsgroups: gmane.comp.jakarta.commons.devel
> Sent: Thursday, June 27, 2002 4:21 AM
> Subject: Re: [httpclient] Volunteering to add support for PROXY AUTH
> andSTREAMING INPUT for POST requests
>
>
> Hi Ortwin,
>
> Please have a look at the patch that I sent on 13 June. It includes
> changes sent by Sachin on June 12. My part of the code is still
> untested, because I am still trying to get access to an authenticated
> proxy server that supports HTTPS that I can test with.
>
> Your proposed changes sound great to me.
>
> I am not a committer on this project, so I can't give you the assurance
> that your changes will be incorporated. I don't think anyone would give
> you such an assurance before seeing the final product. But, on the
> bright side, I doubt if your changes will be turned down, because they
> are useful and well thought through.
>
> Regards,
>
> Evert
>
> On Wed, 2002-06-26 at 17:13, Ortwin Glück wrote:
> > Hello
> >
> > Our company would like to help in the HttpClient project.
> >
> > We wish to add the following features:
> >
> > ______________________________________
> > 1. Support for authenticating proxies
> >
> > The HttpClient currently only supports authentication against the web
> > server but not against the proxy.
> >
> > We hereby suggest to change the API as follows:
> >
> > HttpState:
> > - add field: proxyCredentials
> > - add setter method for proxyCredentials
> > We could have used the existing credential map. But we prefer to
> > create a separate field for two reasons:
> > - proxy auth uses different HTTP Headers than WWW Auth. So we want
> > to avoid mixing up www auth and proxy auth.
> > - A connection can only have one proxy. Its realm is not relevant
> > and in most cases not known to the programmer anyway.
> >
> > - change HttpMethodBase.execute
> > Include a case for the SC_PROXY_AUTHENTICATION_REQUIRED status code
> > which performs proxy authentication
> >
> > - change Authenticator.authenticate
> > Include proxy auth
> >
> > The changes leave the HttpClient compatible with previous versions.
> >
> > _________________________________________________
> > 2. Support for streaming input for POST requests
> >
> > The HttpClient currently only accepts Strings as the body of a POST
> > request. This is not suitable for applications like upload of huge files
> > or binary data. Moreover buffering the whole request in memory is
> > problematic when dealing with large amounts of data.
> >
> > We hereby suggest to change the API as follows:
> >
> > PostMethod:
> > - add setContentLength(int)
> > A convenience method to add the content length header directly or in
> > an indirect fashion. See below.
> >
> > - add int constants CONTENT_LENGTH_AUTO = 0, CONTENT_LENGTH_CHUNKED = -1
> > They are used with setContentLength
> >
> > - add setRequestBody(InputStream)
> > The stream is handled depending on the argument of setContentLength.
> > If the content length was set to CONTENT_LENGTH_AUTO or the content
> > length was not set at all, the input is buffered and the correct content
> > length is calculated automatically. If content length was set to
> > CONTENT_LENGTH_CHUNKED the stream is not buffered and sent with chunked
> > transfer encoding (note that chunked POST is not widely supported by web
> > servers, but it works okay with Tomcat and actually defined in rfc
> > 2068). If content length was set to anything else the stream is not
> > buffered and the content length will be used as set by the user. If
> > however the stream ends before that length, an IOException is thrown (in
> > HttpClient.executeMethod).
> > This enables POSTing of large files without the need to buffer the
> > data in memory thus beeing more conservative on memory. More complex
> > data structures (like Multipart MIME, Base64 encoded data) can be
> > wrapped in a user defined input stream which is then passed to the HTTP
> > client.
> >
> > - deprecate setRequestBody(String) because it is dangerous
> > The platform's default character encoding is used here. This is bad
> > for portability. Have a Mac OS-X and a Windows client talk to a Linux
> > Server and you will be messing up data completely. When writing to a
> > network connection only byte data should be used and not strings that
> > are implicitely encoded.
> >
> >
> > We need these features for one of our projects. Please let us know if
> > you will incorporate these feature into the current code base if we
> > submit a patch.
> >
> > Best regards
> >
> > Ortwin Glueck
> >
> > --
> > _________________________________________________________________
> > NOSE applied intelligence ag [perspectix-nose digital b.i]
> > [www] http://www.nose.ch
> > ortwin glück [email] ortwin.glueck@nose.ch
> > hardturmstrasse 171 [office] +41-1-277 57 35
> > 8005 zurich [fax] +41-1-277 57 12
> > switzerland
> >
> >
> >
> > --
> > To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> > For additional commands, e-mail:
> <ma...@jakarta.apache.org>
> >
>
>
> ---
> Outgoing mail is certified Virus Free.
> Checked by AVG anti-virus system (http://www.grisoft.com).
> Version: 6.0.371 / Virus Database: 206 - Release Date: 16/06/2002
>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [httpclient] Volunteering to add support for PROXY AUTH
andSTREAMING INPUT for POST requests
Posted by Simon Roberts <si...@fifthweb.net>.
Hi guys,
I've got a patch that I've been using myself, but it might be of some use to
others. Or (better!) perhaps someone could tell me a way around my need for
it! If they do, I'll document it for USING_HTTPS.txt :)
The (fugly) patch makes HttpConnection use a static to find a
SSLSocketFactory, rather than using SSLSocketFactory.getDefault. The reason
is that the US-export version of JSSE doesn't allow the users to override
the implementation created by SSLSocketFactory.getDefault().
The default implementation (of SSLSocketFactory) can be configured by
setting a few system/security properties, such as "javax.net.ssl.keyStore"
etc, but we need to be able to use a different trustmanager (our acceptance
of server certs is more involved than certificate chains). There is a
security property "sun.ssl.keymanager.type" which can be used to set the
default keymanager algorithm, but I can't figure out how to register my own
KeyManager against that name. Help!
See
http://java.sun.com/products/jsse/doc/guide/API_users_guide.html#Installatio
nAndCustomization
I'd love to hear if anyone knows about registering KeyManager algorithms, or
wants my fugly patch.
Cheers, Simon
----- Original Message -----
From: "Evert Hoff" <ev...@pixie.co.za>
Newsgroups: gmane.comp.jakarta.commons.devel
Sent: Thursday, June 27, 2002 4:21 AM
Subject: Re: [httpclient] Volunteering to add support for PROXY AUTH
andSTREAMING INPUT for POST requests
Hi Ortwin,
Please have a look at the patch that I sent on 13 June. It includes
changes sent by Sachin on June 12. My part of the code is still
untested, because I am still trying to get access to an authenticated
proxy server that supports HTTPS that I can test with.
Your proposed changes sound great to me.
I am not a committer on this project, so I can't give you the assurance
that your changes will be incorporated. I don't think anyone would give
you such an assurance before seeing the final product. But, on the
bright side, I doubt if your changes will be turned down, because they
are useful and well thought through.
Regards,
Evert
On Wed, 2002-06-26 at 17:13, Ortwin Glück wrote:
> Hello
>
> Our company would like to help in the HttpClient project.
>
> We wish to add the following features:
>
> ______________________________________
> 1. Support for authenticating proxies
>
> The HttpClient currently only supports authentication against the web
> server but not against the proxy.
>
> We hereby suggest to change the API as follows:
>
> HttpState:
> - add field: proxyCredentials
> - add setter method for proxyCredentials
> We could have used the existing credential map. But we prefer to
> create a separate field for two reasons:
> - proxy auth uses different HTTP Headers than WWW Auth. So we want
> to avoid mixing up www auth and proxy auth.
> - A connection can only have one proxy. Its realm is not relevant
> and in most cases not known to the programmer anyway.
>
> - change HttpMethodBase.execute
> Include a case for the SC_PROXY_AUTHENTICATION_REQUIRED status code
> which performs proxy authentication
>
> - change Authenticator.authenticate
> Include proxy auth
>
> The changes leave the HttpClient compatible with previous versions.
>
> _________________________________________________
> 2. Support for streaming input for POST requests
>
> The HttpClient currently only accepts Strings as the body of a POST
> request. This is not suitable for applications like upload of huge files
> or binary data. Moreover buffering the whole request in memory is
> problematic when dealing with large amounts of data.
>
> We hereby suggest to change the API as follows:
>
> PostMethod:
> - add setContentLength(int)
> A convenience method to add the content length header directly or in
> an indirect fashion. See below.
>
> - add int constants CONTENT_LENGTH_AUTO = 0, CONTENT_LENGTH_CHUNKED = -1
> They are used with setContentLength
>
> - add setRequestBody(InputStream)
> The stream is handled depending on the argument of setContentLength.
> If the content length was set to CONTENT_LENGTH_AUTO or the content
> length was not set at all, the input is buffered and the correct content
> length is calculated automatically. If content length was set to
> CONTENT_LENGTH_CHUNKED the stream is not buffered and sent with chunked
> transfer encoding (note that chunked POST is not widely supported by web
> servers, but it works okay with Tomcat and actually defined in rfc
> 2068). If content length was set to anything else the stream is not
> buffered and the content length will be used as set by the user. If
> however the stream ends before that length, an IOException is thrown (in
> HttpClient.executeMethod).
> This enables POSTing of large files without the need to buffer the
> data in memory thus beeing more conservative on memory. More complex
> data structures (like Multipart MIME, Base64 encoded data) can be
> wrapped in a user defined input stream which is then passed to the HTTP
> client.
>
> - deprecate setRequestBody(String) because it is dangerous
> The platform's default character encoding is used here. This is bad
> for portability. Have a Mac OS-X and a Windows client talk to a Linux
> Server and you will be messing up data completely. When writing to a
> network connection only byte data should be used and not strings that
> are implicitely encoded.
>
>
> We need these features for one of our projects. Please let us know if
> you will incorporate these feature into the current code base if we
> submit a patch.
>
> Best regards
>
> Ortwin Glueck
>
> --
> _________________________________________________________________
> NOSE applied intelligence ag [perspectix-nose digital b.i]
> [www] http://www.nose.ch
> ortwin glück [email] ortwin.glueck@nose.ch
> hardturmstrasse 171 [office] +41-1-277 57 35
> 8005 zurich [fax] +41-1-277 57 12
> switzerland
>
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
>
---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.371 / Virus Database: 206 - Release Date: 16/06/2002
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [httpclient] Volunteering to add support for PROXY AUTH and
STREAMING INPUT for POST requests
Posted by Evert Hoff <ev...@pixie.co.za>.
Hi Ortwin,
Please have a look at the patch that I sent on 13 June. It includes
changes sent by Sachin on June 12. My part of the code is still
untested, because I am still trying to get access to an authenticated
proxy server that supports HTTPS that I can test with.
Your proposed changes sound great to me.
I am not a committer on this project, so I can't give you the assurance
that your changes will be incorporated. I don't think anyone would give
you such an assurance before seeing the final product. But, on the
bright side, I doubt if your changes will be turned down, because they
are useful and well thought through.
Regards,
Evert
On Wed, 2002-06-26 at 17:13, Ortwin Glück wrote:
> Hello
>
> Our company would like to help in the HttpClient project.
>
> We wish to add the following features:
>
> ______________________________________
> 1. Support for authenticating proxies
>
> The HttpClient currently only supports authentication against the web
> server but not against the proxy.
>
> We hereby suggest to change the API as follows:
>
> HttpState:
> - add field: proxyCredentials
> - add setter method for proxyCredentials
> We could have used the existing credential map. But we prefer to
> create a separate field for two reasons:
> - proxy auth uses different HTTP Headers than WWW Auth. So we want
> to avoid mixing up www auth and proxy auth.
> - A connection can only have one proxy. Its realm is not relevant
> and in most cases not known to the programmer anyway.
>
> - change HttpMethodBase.execute
> Include a case for the SC_PROXY_AUTHENTICATION_REQUIRED status code
> which performs proxy authentication
>
> - change Authenticator.authenticate
> Include proxy auth
>
> The changes leave the HttpClient compatible with previous versions.
>
> _________________________________________________
> 2. Support for streaming input for POST requests
>
> The HttpClient currently only accepts Strings as the body of a POST
> request. This is not suitable for applications like upload of huge files
> or binary data. Moreover buffering the whole request in memory is
> problematic when dealing with large amounts of data.
>
> We hereby suggest to change the API as follows:
>
> PostMethod:
> - add setContentLength(int)
> A convenience method to add the content length header directly or in
> an indirect fashion. See below.
>
> - add int constants CONTENT_LENGTH_AUTO = 0, CONTENT_LENGTH_CHUNKED = -1
> They are used with setContentLength
>
> - add setRequestBody(InputStream)
> The stream is handled depending on the argument of setContentLength.
> If the content length was set to CONTENT_LENGTH_AUTO or the content
> length was not set at all, the input is buffered and the correct content
> length is calculated automatically. If content length was set to
> CONTENT_LENGTH_CHUNKED the stream is not buffered and sent with chunked
> transfer encoding (note that chunked POST is not widely supported by web
> servers, but it works okay with Tomcat and actually defined in rfc
> 2068). If content length was set to anything else the stream is not
> buffered and the content length will be used as set by the user. If
> however the stream ends before that length, an IOException is thrown (in
> HttpClient.executeMethod).
> This enables POSTing of large files without the need to buffer the
> data in memory thus beeing more conservative on memory. More complex
> data structures (like Multipart MIME, Base64 encoded data) can be
> wrapped in a user defined input stream which is then passed to the HTTP
> client.
>
> - deprecate setRequestBody(String) because it is dangerous
> The platform's default character encoding is used here. This is bad
> for portability. Have a Mac OS-X and a Windows client talk to a Linux
> Server and you will be messing up data completely. When writing to a
> network connection only byte data should be used and not strings that
> are implicitely encoded.
>
>
> We need these features for one of our projects. Please let us know if
> you will incorporate these feature into the current code base if we
> submit a patch.
>
> Best regards
>
> Ortwin Glueck
>
> --
> _________________________________________________________________
> NOSE applied intelligence ag [perspectix-nose digital b.i]
> [www] http://www.nose.ch
> ortwin glück [email] ortwin.glueck@nose.ch
> hardturmstrasse 171 [office] +41-1-277 57 35
> 8005 zurich [fax] +41-1-277 57 12
> switzerland
>
>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>