You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by Wynand <wo...@gmail.com> on 2005/10/18 14:30:24 UTC
HttpClient 3.0 - Tunneled HTTPS connections through HTTP proxies
Hi All,
I've just started using the commons httpclient 3.0 rc4. It works just as
expected, except for connecting to a HTTPS site through a HTTP proxy
(Squid/2.4.STABLE6).
I have tried the the example as per the SSL guide, but to no avail; I get
the following error :
2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
www.verisign.com <http://www.verisign.com/> failed to respond <
org.apache.commons.httpclient.NoHttpResponseException: The server
www.verisign.com <http://www.verisign.com/> failed to respond>
org.apache.commons.httpclient.NoHttpResponseException: The server
www.verisign.com <http://www.verisign.com/> failed to respond
at org.apache.commons.httpclient.HttpMethodBase.readStatusLine (
HttpMethodBase.java:1835)
at org.apache.commons.httpclient.HttpMethodBase.readResponse(
HttpMethodBase.java:1590)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java
:995)
at org.apache.commons.httpclient.ConnectMethod.execute (ConnectMethod.java
:144)
at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
HttpMethodDirector.java:487)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
HttpMethodDirector.java :388)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
HttpMethodDirector.java:170)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java
:396)
at org.apache.commons.httpclient.HttpClient.executeMethod (HttpClient.java
:324)
at com.wolman.webagent.TestCase.simplestTest(TestCase.java:43)
at com.wolman.webagent.TestCase.main(TestCase.java:21)
Just to clarify here is the example i used :
HttpClient httpclient = new HttpClient();
httpclient.getHostConfiguration().setProxy("myproxyhost", 8080);
httpclient.getState().setProxyCredentials("my-proxy-realm", " myproxyhost",
new UsernamePasswordCredentials("my-proxy-username", "my-proxy-password"));
GetMethod httpget = new GetMethod("*https://www.verisign.com/*");
httpclient.executeMethod(httpget);
System.out.println(httpget.getStatusLine().toString());
It works fine if I change the *https* to *http* in the url. Can someone
please confirm that this example is indeed working or if I'm missing
something
Thanks alot
Re: [Collections] MultiKeyMap vs concatenating keys that are strings
Posted by Stephen Colebourne <sc...@btopenworld.com>.
The short answer is to say, this is open source, so look at the source ;-)
The longer answer is that MultiKeyMap uses specially optimised data
structures internally to store the key parts. This avoids any hacky (and
slow) string concatenation. Given a choice, I would always use a
MultiKeyMap for a multiply-keyed map.
Stephen
Mark Shifman wrote:
> Hi:
>
> If I am building a map with a compound key composed of say 3 strings ,
> is there any advantage to using MultiKeyMap over a regular Hash(ed)Map
> with the key = key1+"_"+key2+"_"+key3;
>
> speed?..elegance??
>
> thanks in advance
> mas
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org
[Collections] MultiKeyMap vs concatenating keys that are strings
Posted by Mark Shifman <ma...@yale.edu>.
Hi:
If I am building a map with a compound key composed of say 3 strings ,
is there any advantage to using MultiKeyMap over a regular Hash(ed)Map
with the key = key1+"_"+key2+"_"+key3;
speed?..elegance??
thanks in advance
mas
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org
Re: HttpClient 3.0 - Tunneled HTTPS connections through HTTP proxies
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Fri, Oct 21, 2005 at 01:25:12PM +0200, Wynand wrote:
> Hi Oleg,
> I've decided this morning that I'm going to solve this little problem if
> it's the last thing I do. And the problem was not hard to solve once I got
> it tracked. It seems that with a Squid proxy at least, preemptive
> authorization is needed when opening the ssl tunnel The problem was that the
> preemptive Proxy-Authorization header was not being sent when the client
> tries to open the ssl tunnel, no matter what I did. I set up all the
> preemptive authorization flags that I could set, but those flags didn't
> propagate down to the code where the client opens up the ssl tunnel to the
> host. All that was needed was for the preemptive auth flags to be set in the
> executeConnect method by doing the following copied piece of code :
> In HttpMethodDirector's executeConnect() function line 470
> *if (this.params.isAuthenticationPreemptive()) {
> LOG.debug("Preemptively sending default basic credentials");
> this.connectMethod.getHostAuthState().setPreemptive();
> this.connectMethod.getHostAuthState().setAuthAttempted(true);
> if (this.conn.isProxied()) {
> this.connectMethod.getProxyAuthState().setPreemptive();
> this.connectMethod.getProxyAuthState().setAuthAttempted(true);
> }
> } *
> **
> * *
> SSL through a squid proxy now works like a charm.
> *
> *
Please file a bug report in Bugzilla and I'll get the bug fixed
Oleg
> On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
> >
> > On Tue, Oct 18, 2005 at 06:40:06PM +0200, Wynand wrote:
> > > Oleg
> > > I've been thinking (and trying various things), how come a browser like
> > IE
> > > can do ssl through this proxy server of mine to https://www.verisign.com
> > .
> > > Can you think of what is it doing differently? Any suggestion will help,
> > > this seems to me like a solvable problem that may not need a proxy
> > server
> > > re-configuration.
> > > Thanks
> >
> > Try adding 'Proxy-Connection: close' header to the request or using HTTP
> > version 1.0 and see if that makes any difference
> >
> > Oleg
> >
> >
> > > On 10/18/05, Wynand <wo...@gmail.com> wrote:
> > > >
> > > > Yes, thanks alot.
> > > > This isn't good news for me but at least it narrows my options down to
> > > > just one ;-)
> > > >
> > > > On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
> > > > >
> > > > > On Tue, Oct 18, 2005 at 04:39:44PM +0200, Wynand wrote:
> > > > > > Oleg,
> > > > > > I may have stumbled on the cause of this problem by accident. I
> > read
> > > > > that
> > > > > > there is such an option as "http_access deny CONNECT" in the squid
> > > > > > configuration, and that's exactly what httpclient tries to do when
> > it
> > > > > tries
> > > > > > to make a ssl connection though a proxy. I don't have access to
> > the
> > > > > squid
> > > > > > configuration, but that's what I'm guessing the problem is. Your
> > > > > comments
> > > > > > are appreciated.
> > > > >
> > > > > This is precisely the cause of the problem. To make matters worse
> > the
> > > > > version of Squid you are using appears to have an issue with
> > connection
> > > > > management. The first time it returns status 407, Proxy-Authenticate
> > and
> > > > > Connection: keep-alive headers, which is perfectly ok. However, when
> > > > > HttpClient attempts to authenticate using given credentials, the
> > proxy
> > > > > simply drops the connection on unsuspected HttpClient:
> > > > >
> > > > > - >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443> <
> > http://www.verisign.com:443/> <http://www.verisign.com:443/
> > > > > > HTTP/1.1"
> > > > > - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
> > > > > - >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n
> > > > > <http://www.verisign.com%5b/r%5D%5B/n>>]"
> > > > > - >> "Proxy-Connection: Keep-Alive[\r][\n]"
> > > > > - >> "[\r][\n]"
> > > > > - << "HTTP/1.0 407 Proxy Authentication Required[\r][\n]"
> > > > > - << "Server: Squid/2.4.STABLE6[\r][\n]"
> > > > > - << "Mime-Version: 1.0[\r][\n]"
> > > > > - << "Date: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
> > > > > - << "Content-Type: text/html[\r][\n]"
> > > > > - << "Content-Length: 984[\r][\n]"
> > > > > - << "Expires: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
> > > > > - << "X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> > > > > - << "Proxy-Authenticate: Basic realm="Squid proxy-caching web
> > > > > server"[\r][\n]"
> > > > > - << "X-Cache: MISS from neutrino.XXXXXXXXX.co.za[\r][\n]"
> > > > > - << "Proxy-Connection: keep-alive[\r][\n]"
> > > > > - >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443> <
> > http://www.verisign.com:443/><
> > > > > http://www.verisign.com:443/> HTTP/1.0"
> > > > > - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
> > > > > - >> "Proxy-Connection: Keep-Alive[\r][\n]"
> > > > > - >> "Proxy-Authorization: Basic d29sbWFydzp0eXRlbndv[\r][\n]"
> > > > > - >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n<
> > http://www.verisign.com%5b/r%5D%5B/n>>]"
> > > > >
> > > > > - >> "[\r][\n]"
> > > > >
> > > > > Your only option is to get someone to reconfigure that proxy server
> > (and
> > > > > optionally upgrade it from version 2.4.STABLE6 to something slightly
> > > > > more modern)
> > > > >
> > > > > Hope this helps
> > > > >
> > > > > Oleg
> > > > >
> > > > >
> > > > > > I'm not sure what a wire log is, but here is all the debug info
> > ;-)
> > > > > > 2005/10/18 13:28:18:828 CAT [DEBUG] HttpClient - Java version:
> > > > > 1.4.2_08
> > > > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java vendor: Sun
> > > > > > Microsystems Inc.
> > > > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java class path:
> > > > > >
> > C:\eclipse\workspace\SimpleWebAgent\bin;C:\Projects\java\lib\jericho-
> > > > > > html-1.5-dev1.jar;C:\Projects\java\lib\commons- logging-1.0.4.jar
> > > > > > ;C:\Projects\java\lib\commons-codec-1.3.jar
> > ;C:\Projects\java\commons-
> > > > > > httpclient-3.0-rc4\commons-httpclient-3.0-rc4.jar
> > > > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > > > > name:
> > > > > > Windows 2000
> > > > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > > > > > architecture: x86
> > > > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > > > > version:
> > > > > > 5.0
> > > > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SUN 1.42: SUN
> > (DSA
> > > > > > key/parameter generation; DSA signing; SHA-1, MD5 digests;
> > > > > SecureRandom;
> > > > > > X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
> > > > > > CertPathBuilder; LDAP, Collection CertStores)
> > > > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJSSE 1.42: Sun
> > > > > JSSE
> > > > > > provider(implements RSA Signatures, PKCS12, SunX509 key/trust
> > > > > factories,
> > > > > > SSLv3, TLSv1)
> > > > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunRsaSign 1.42:
> > > > > SUN's
> > > > > > provider for RSA signatures
> > > > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJCE 1.42:
> > SunJCE
> > > > > > Provider (implements DES, Triple DES, AES, Blowfish, PBE,
> > > > > Diffie-Hellman,
> > > > > > HMAC-MD5, HMAC-SHA1)
> > > > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJGSS 1.0: Sun
> > > > > (Kerberos
> > > > > > v5)
> > > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> > parameter
> > > > > > http.useragent = Jakarta Commons-HttpClient/3.0-rc4
> > > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> > parameter
> > > > > > http.protocol.version = HTTP/1.1
> > > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> > parameter
> > > > > > http.connection-manager.class = class
> > > > > > org.apache.commons.httpclient.SimpleHttpConnectionManager
> > > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> > parameter
> > > > > > http.protocol.cookie-policy = rfc2109
> > > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> > parameter
> > > > > > http.protocol.element-charset = US-ASCII
> > > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> > parameter
> > > > > > http.protocol.content-charset = ISO-8859-1
> > > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> > parameter
> > > > > > http.method.retry-handler =
> > > > > >
> > org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@197a37c
> > > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> > parameter
> > > > > > http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE,
> > > > > dd-MMM-yy
> > > > > > HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy HH:mm:ss
> > z,
> > > > > EEE,
> > > > > > dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy
> > > > > HH:mm:ss
> > > > > > z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE
> > > > > dd-MMM-yy
> > > > > > HH:mm:ss z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z,
> > > > > > EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy HH:mm:ss z]
> > > > > > 2005/10/18 13:28:19:109 CAT [DEBUG] HttpConnection - Open
> > connection
> > > > > to
> > > > > > proxy.XXXXXXXXX.co.za:3128 <http://proxy.XXXXXXXXX.co.za:3128> <
> > http://proxy.xxxxxxxxx.co.za:3128/> <
> > > > > http://proxy.XXXXXXXXX.co.za:3128 <
> > http://proxy.xxxxxxxxx.co.za:3128/>>
> > > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "CONNECT
> > > > > > www.verisign.com:443 <http://www.verisign.com:443> <
> > http://www.verisign.com:443/> <
> > > > > http://www.verisign.com:443/> HTTP/1.1"
> > > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] HttpMethodBase - Adding Host
> > > > > request
> > > > > > header
> > > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "User-Agent:
> > Jakarta
> > > > > > Commons-HttpClient/3.0-rc4[\r][\n]"
> > > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Host:
> > > > > > www.verisign.com[\r][\n <http://www.verisign.com[/r][/n<
> > http://www.verisign.com%5b/r%5D%5B/n>
> > > > > >]"
> > > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Proxy-Connection:
> > > > > > Keep-Alive[\r][\n]"
> > > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "[\r][\n]"
> > > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "HTTP/1.0 407
> > Proxy
> > > > > > Authentication Required[\r][\n]"
> > > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Server:
> > > > > > Squid/2.4.STABLE6[\r][\n]"
> > > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Mime-Version: 1.0
> > > > > [\r][\n]"
> > > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Date: Tue, 18 Oct
> > > > > 2005
> > > > > > 11:27:51 GMT[\r][\n]"
> > > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Type:
> > > > > > text/html[\r][\n]"
> > > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Length:
> > > > > > 984[\r][\n]"
> > > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Expires: Tue, 18
> > Oct
> > > > > 2005
> > > > > > 11:27:51 GMT[\r][\n]"
> > > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Squid-Error:
> > > > > > ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> > > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - <<
> > "Proxy-Authenticate:
> > > > > Basic
> > > > > > realm="Squid proxy-caching web server"[\r][\n]"
> > > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Cache: MISS
> > from
> > > > > > neutrino.XXXXXXXXX.co.za[\r][\n]"
> > > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Connection:
> > > > > > keep-alive[\r][\n]"
> > > > > > 2005/10/18 13:28:19:203 CAT [DEBUG] ConnectMethod - CONNECT status
> > > > > code 407
> > > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor -
> > Supported
> > > > > > authentication schemes in the order of preference: [ntlm, digest,
> > > > > basic]
> > > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor -
> > Challenge
> > > > > for
> > > > > > ntlm authentication scheme not available
> > > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor -
> > Challenge
> > > > > for
> > > > > > digest authentication scheme not available
> > > > > > 2005/10/18 13:28:19:218 CAT [INFO] AuthChallengeProcessor - basic
> > > > > > authentication scheme selected
> > > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Using
> > > > > > authentication scheme: basic
> > > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor -
> > > > > Authorization
> > > > > > challenge processed
> > > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Proxy
> > > > > > authentication scope: BASIC 'Squid proxy-caching web
> > > > > > server'@proxy.XXXXXXXXX.co.za:3128<http://server'@proxy.XXXXXXXXX.co.za:3128>
> > <http://server'@proxy.xxxxxxxxx.co.za:3128/>
> > > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodBase - Should NOT
> > close
> > > > > > connection in response to directive: keep-alive
> > > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpConnection - Connection is
> > > > > locked.
> > > > > > Call to releaseConnection() ignored.
> > > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector -
> > > > > Authenticating with
> > > > > > BASIC 'Squid proxy-caching web server'@proxy.XXXXXXXXX.co.za:3128<http://server'@proxy.XXXXXXXXX.co.za:3128>
> > <http://server'@proxy.xxxxxxxxx.co.za:3128/>
> > > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodParams - Credential
> > > > > charset
> > > > > > not configured, using HTTP element charset
> > > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "CONNECT
> > > > > > www.verisign.com:443 <http://www.verisign.com:443> <
> > http://www.verisign.com:443/> <
> > > > > http://www.verisign.com:443/> HTTP/1.0"
> > > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodBase - Adding Host
> > > > > request
> > > > > > header
> > > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "User-Agent:
> > Jakarta
> > > > > > Commons-HttpClient/3.0-rc4[\r][\n]"
> > > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Connection:
> > > > > > Keep-Alive[\r][\n]"
> > > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >>
> > "Proxy-Authorization:
> > > > > Basic
> > > > > > d29sbWFydzp0eXRlbndv[\r][\n]"
> > > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Host:
> > > > > > www.verisign.com[\r][\n <http://www.verisign.com[/r][/n<
> > http://www.verisign.com%5b/r%5D%5B/n>
> > > > > >]"
> > > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "[\r][\n]"
> > > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - Closing
> > the
> > > > > > connection.
> > > > > > 2005/10/18 13:28:19:234 CAT [INFO] HttpMethodDirector - I/O
> > exception
> > > > > (
> > > > > > org.apache.commons.httpclient.NoHttpResponseException) caught when
> > > > > > processing request: The server www.verisign.com<http://www.verisign.com>
> > <http://www.verisign.com/>
> > > > > > <http://www.verisign.com/>failed to respond
> > > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The
> > server
> > > > > > www.verisign.com <http://www.verisign.com> <
> > http://www.verisign.com/> <http://www.verisign.com/>
> > > > > failed to respond <
> > > > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > > > www.verisign.com <http://www.verisign.com> <
> > http://www.verisign.com/> <http://www.verisign.com/>
> > > > > failed to respond>
> > > > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > > > www.verisign.com <http://www.verisign.com> <
> > http://www.verisign.com/> <http://www.verisign.com/>
> > > > > failed to respond
> > > > > > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(
> > > > > > HttpMethodBase.java:1835)
> > > > > > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > > > > > HttpMethodBase.java:1590)
> > > > > > at org.apache.commons.httpclient.HttpMethodBase.execute(
> > > > > HttpMethodBase.java
> > > > > > :995)
> > > > > > at org.apache.commons.httpclient.ConnectMethod.execute (
> > > > > ConnectMethod.java
> > > > > > :144)
> > > > > > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect
> > (
> > > > > > HttpMethodDirector.java:487)
> > > > > > at
> > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry (
> > > > > > HttpMethodDirector.java:388)
> > > > > > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > > > > > HttpMethodDirector.java:170)
> > > > > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > > > > HttpClient.java
> > > > > > :396)
> > > > > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > > > > HttpClient.java
> > > > > > :324)
> > > > > > at com.XXXXXXXXX.webagent.TestCase.simplestTest(TestCase.java:43)
> > > > > > at com.XXXXXXXXX.webagent.TestCase.main (TestCase.java:21)
> > > > > > On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
> > > > > >
> > > > > > > On Tue, Oct 18, 2005 at 02:30:24PM +0200, Wynand wrote:
> > > > > > > > Hi All,
> > > > > > > > I've just started using the commons httpclient 3.0 rc4. It
> > works
> > > > > just as
> > > > > > > > expected, except for connecting to a HTTPS site through a HTTP
> > > > > proxy
> > > > > > > > (Squid/2.4.STABLE6).
> > > > > > > > I have tried the the example as per the SSL guide, but to no
> > > > > avail; I
> > > > > > > get
> > > > > > > > the following error :
> > > > > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The
> > > > > server
> > > > > > > > www.verisign.com <http://www.verisign.com> <
> > http://www.verisign.com/> <
> > > > > http://www.verisign.com> <http://www.verisign.com/>
> > > > > > > failed to respond <
> > > > > > > > org.apache.commons.httpclient.NoHttpResponseException: The
> > server
> > > > > > > > www.verisign.com <http://www.verisign.com> <
> > http://www.verisign.com/> <
> > > > > http://www.verisign.com > <http://www.verisign.com/>
> > > > > > > failed to respond>
> > > > > > > > org.apache.commons.httpclient.NoHttpResponseException: The
> > server
> > > > > > > > www.verisign.com <http://www.verisign.com> <
> > http://www.verisign.com/> <
> > > > > http://www.verisign.com> <http://www.verisign.com/>
> > > > > > > failed to respond
> > > > > > > > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(
> > > > > > > > HttpMethodBase.java:1835)
> > > > > > > > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > > > > > > > HttpMethodBase.java:1590)
> > > > > > > > at org.apache.commons.httpclient.HttpMethodBase.execute (
> > > > > > > HttpMethodBase.java
> > > > > > > > :995)
> > > > > > > > at org.apache.commons.httpclient.ConnectMethod.execute (
> > > > > > > ConnectMethod.java
> > > > > > > > :144)
> > > > > > > > at
> > org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> > > > > > > > HttpMethodDirector.java:487)
> > > > > > > > at
> > > > > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> > > > > > > > HttpMethodDirector.java :388)
> > > > > > > > at
> > org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > > > > > > > HttpMethodDirector.java:170)
> > > > > > > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > > > > > > HttpClient.java
> > > > > > > > :396)
> > > > > > > > at org.apache.commons.httpclient.HttpClient.executeMethod (
> > > > > > > HttpClient.java
> > > > > > > > :324)
> > > > > > > > at com.wolman.webagent.TestCase.simplestTest(TestCase.java:43)
> > > > > > > > at com.wolman.webagent.TestCase.main(TestCase.java:21)
> > > > > > > > Just to clarify here is the example i used :
> > > > > > > > HttpClient httpclient = new HttpClient();
> > > > > > > > httpclient.getHostConfiguration().setProxy("myproxyhost",
> > 8080);
> > > > > > > > httpclient.getState().setProxyCredentials("my-proxy-realm", "
> > > > > > > myproxyhost",
> > > > > > > > new UsernamePasswordCredentials("my-proxy-username",
> > > > > > > "my-proxy-password"));
> > > > > > > > GetMethod httpget = new GetMethod("*
> > https://www.verisign.com/*")<https://www.verisign.com/*%22)>
> > > > > ;
> > > > > > > > httpclient.executeMethod(httpget);
> > > > > > > > System.out.println(httpget.getStatusLine().toString());
> > > > > > > > It works fine if I change the *https* to *http* in the url.
> > Can
> > > > > someone
> > > > > > > > please confirm that this example is indeed working or if I'm
> > > > > missing
> > > > > > > > something
> > > > > > > > Thanks alot
> > > > > > >
> > > > > > > Wynand,
> > > > > > >
> > > > > > > Please send the complete wire log.
> > > > > > >
> > > > > > > Oleg
> > > > > > >
> > > > > > >
> > > > >
> > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail:
> > commons-user-unsubscribe@jakarta.apache.org
> > > > > > > For additional commands, e-mail:
> > > > > commons-user-help@jakarta.apache.org
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> > > > > For additional commands, e-mail:
> > commons-user-help@jakarta.apache.org
> > > > >
> > > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: commons-user-help@jakarta.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org
Re: HttpClient 3.0 - Tunneled HTTPS connections through HTTP proxies
Posted by Wynand <wo...@gmail.com>.
Hi Oleg,
I've decided this morning that I'm going to solve this little problem if
it's the last thing I do. And the problem was not hard to solve once I got
it tracked. It seems that with a Squid proxy at least, preemptive
authorization is needed when opening the ssl tunnel The problem was that the
preemptive Proxy-Authorization header was not being sent when the client
tries to open the ssl tunnel, no matter what I did. I set up all the
preemptive authorization flags that I could set, but those flags didn't
propagate down to the code where the client opens up the ssl tunnel to the
host. All that was needed was for the preemptive auth flags to be set in the
executeConnect method by doing the following copied piece of code :
In HttpMethodDirector's executeConnect() function line 470
*if (this.params.isAuthenticationPreemptive()) {
LOG.debug("Preemptively sending default basic credentials");
this.connectMethod.getHostAuthState().setPreemptive();
this.connectMethod.getHostAuthState().setAuthAttempted(true);
if (this.conn.isProxied()) {
this.connectMethod.getProxyAuthState().setPreemptive();
this.connectMethod.getProxyAuthState().setAuthAttempted(true);
}
} *
**
* *
SSL through a squid proxy now works like a charm.
*
*
On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
>
> On Tue, Oct 18, 2005 at 06:40:06PM +0200, Wynand wrote:
> > Oleg
> > I've been thinking (and trying various things), how come a browser like
> IE
> > can do ssl through this proxy server of mine to https://www.verisign.com
> .
> > Can you think of what is it doing differently? Any suggestion will help,
> > this seems to me like a solvable problem that may not need a proxy
> server
> > re-configuration.
> > Thanks
>
> Try adding 'Proxy-Connection: close' header to the request or using HTTP
> version 1.0 and see if that makes any difference
>
> Oleg
>
>
> > On 10/18/05, Wynand <wo...@gmail.com> wrote:
> > >
> > > Yes, thanks alot.
> > > This isn't good news for me but at least it narrows my options down to
> > > just one ;-)
> > >
> > > On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
> > > >
> > > > On Tue, Oct 18, 2005 at 04:39:44PM +0200, Wynand wrote:
> > > > > Oleg,
> > > > > I may have stumbled on the cause of this problem by accident. I
> read
> > > > that
> > > > > there is such an option as "http_access deny CONNECT" in the squid
> > > > > configuration, and that's exactly what httpclient tries to do when
> it
> > > > tries
> > > > > to make a ssl connection though a proxy. I don't have access to
> the
> > > > squid
> > > > > configuration, but that's what I'm guessing the problem is. Your
> > > > comments
> > > > > are appreciated.
> > > >
> > > > This is precisely the cause of the problem. To make matters worse
> the
> > > > version of Squid you are using appears to have an issue with
> connection
> > > > management. The first time it returns status 407, Proxy-Authenticate
> and
> > > > Connection: keep-alive headers, which is perfectly ok. However, when
> > > > HttpClient attempts to authenticate using given credentials, the
> proxy
> > > > simply drops the connection on unsuspected HttpClient:
> > > >
> > > > - >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443> <
> http://www.verisign.com:443/> <http://www.verisign.com:443/
> > > > > HTTP/1.1"
> > > > - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
> > > > - >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n
> > > > <http://www.verisign.com%5b/r%5D%5B/n>>]"
> > > > - >> "Proxy-Connection: Keep-Alive[\r][\n]"
> > > > - >> "[\r][\n]"
> > > > - << "HTTP/1.0 407 Proxy Authentication Required[\r][\n]"
> > > > - << "Server: Squid/2.4.STABLE6[\r][\n]"
> > > > - << "Mime-Version: 1.0[\r][\n]"
> > > > - << "Date: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
> > > > - << "Content-Type: text/html[\r][\n]"
> > > > - << "Content-Length: 984[\r][\n]"
> > > > - << "Expires: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
> > > > - << "X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> > > > - << "Proxy-Authenticate: Basic realm="Squid proxy-caching web
> > > > server"[\r][\n]"
> > > > - << "X-Cache: MISS from neutrino.XXXXXXXXX.co.za[\r][\n]"
> > > > - << "Proxy-Connection: keep-alive[\r][\n]"
> > > > - >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443> <
> http://www.verisign.com:443/><
> > > > http://www.verisign.com:443/> HTTP/1.0"
> > > > - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
> > > > - >> "Proxy-Connection: Keep-Alive[\r][\n]"
> > > > - >> "Proxy-Authorization: Basic d29sbWFydzp0eXRlbndv[\r][\n]"
> > > > - >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n<
> http://www.verisign.com%5b/r%5D%5B/n>>]"
> > > >
> > > > - >> "[\r][\n]"
> > > >
> > > > Your only option is to get someone to reconfigure that proxy server
> (and
> > > > optionally upgrade it from version 2.4.STABLE6 to something slightly
> > > > more modern)
> > > >
> > > > Hope this helps
> > > >
> > > > Oleg
> > > >
> > > >
> > > > > I'm not sure what a wire log is, but here is all the debug info
> ;-)
> > > > > 2005/10/18 13:28:18:828 CAT [DEBUG] HttpClient - Java version:
> > > > 1.4.2_08
> > > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java vendor: Sun
> > > > > Microsystems Inc.
> > > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java class path:
> > > > >
> C:\eclipse\workspace\SimpleWebAgent\bin;C:\Projects\java\lib\jericho-
> > > > > html-1.5-dev1.jar;C:\Projects\java\lib\commons- logging-1.0.4.jar
> > > > > ;C:\Projects\java\lib\commons-codec-1.3.jar
> ;C:\Projects\java\commons-
> > > > > httpclient-3.0-rc4\commons-httpclient-3.0-rc4.jar
> > > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > > > name:
> > > > > Windows 2000
> > > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > > > > architecture: x86
> > > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > > > version:
> > > > > 5.0
> > > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SUN 1.42: SUN
> (DSA
> > > > > key/parameter generation; DSA signing; SHA-1, MD5 digests;
> > > > SecureRandom;
> > > > > X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
> > > > > CertPathBuilder; LDAP, Collection CertStores)
> > > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJSSE 1.42: Sun
> > > > JSSE
> > > > > provider(implements RSA Signatures, PKCS12, SunX509 key/trust
> > > > factories,
> > > > > SSLv3, TLSv1)
> > > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunRsaSign 1.42:
> > > > SUN's
> > > > > provider for RSA signatures
> > > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJCE 1.42:
> SunJCE
> > > > > Provider (implements DES, Triple DES, AES, Blowfish, PBE,
> > > > Diffie-Hellman,
> > > > > HMAC-MD5, HMAC-SHA1)
> > > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJGSS 1.0: Sun
> > > > (Kerberos
> > > > > v5)
> > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> parameter
> > > > > http.useragent = Jakarta Commons-HttpClient/3.0-rc4
> > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> parameter
> > > > > http.protocol.version = HTTP/1.1
> > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> parameter
> > > > > http.connection-manager.class = class
> > > > > org.apache.commons.httpclient.SimpleHttpConnectionManager
> > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> parameter
> > > > > http.protocol.cookie-policy = rfc2109
> > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> parameter
> > > > > http.protocol.element-charset = US-ASCII
> > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> parameter
> > > > > http.protocol.content-charset = ISO-8859-1
> > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> parameter
> > > > > http.method.retry-handler =
> > > > >
> org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@197a37c
> > > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set
> parameter
> > > > > http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE,
> > > > dd-MMM-yy
> > > > > HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy HH:mm:ss
> z,
> > > > EEE,
> > > > > dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy
> > > > HH:mm:ss
> > > > > z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE
> > > > dd-MMM-yy
> > > > > HH:mm:ss z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z,
> > > > > EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy HH:mm:ss z]
> > > > > 2005/10/18 13:28:19:109 CAT [DEBUG] HttpConnection - Open
> connection
> > > > to
> > > > > proxy.XXXXXXXXX.co.za:3128 <http://proxy.XXXXXXXXX.co.za:3128> <
> http://proxy.xxxxxxxxx.co.za:3128/> <
> > > > http://proxy.XXXXXXXXX.co.za:3128 <
> http://proxy.xxxxxxxxx.co.za:3128/>>
> > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "CONNECT
> > > > > www.verisign.com:443 <http://www.verisign.com:443> <
> http://www.verisign.com:443/> <
> > > > http://www.verisign.com:443/> HTTP/1.1"
> > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] HttpMethodBase - Adding Host
> > > > request
> > > > > header
> > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "User-Agent:
> Jakarta
> > > > > Commons-HttpClient/3.0-rc4[\r][\n]"
> > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Host:
> > > > > www.verisign.com[\r][\n <http://www.verisign.com[/r][/n<
> http://www.verisign.com%5b/r%5D%5B/n>
> > > > >]"
> > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Proxy-Connection:
> > > > > Keep-Alive[\r][\n]"
> > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "[\r][\n]"
> > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "HTTP/1.0 407
> Proxy
> > > > > Authentication Required[\r][\n]"
> > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Server:
> > > > > Squid/2.4.STABLE6[\r][\n]"
> > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Mime-Version: 1.0
> > > > [\r][\n]"
> > > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Date: Tue, 18 Oct
> > > > 2005
> > > > > 11:27:51 GMT[\r][\n]"
> > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Type:
> > > > > text/html[\r][\n]"
> > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Length:
> > > > > 984[\r][\n]"
> > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Expires: Tue, 18
> Oct
> > > > 2005
> > > > > 11:27:51 GMT[\r][\n]"
> > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Squid-Error:
> > > > > ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - <<
> "Proxy-Authenticate:
> > > > Basic
> > > > > realm="Squid proxy-caching web server"[\r][\n]"
> > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Cache: MISS
> from
> > > > > neutrino.XXXXXXXXX.co.za[\r][\n]"
> > > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Connection:
> > > > > keep-alive[\r][\n]"
> > > > > 2005/10/18 13:28:19:203 CAT [DEBUG] ConnectMethod - CONNECT status
> > > > code 407
> > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor -
> Supported
> > > > > authentication schemes in the order of preference: [ntlm, digest,
> > > > basic]
> > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor -
> Challenge
> > > > for
> > > > > ntlm authentication scheme not available
> > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor -
> Challenge
> > > > for
> > > > > digest authentication scheme not available
> > > > > 2005/10/18 13:28:19:218 CAT [INFO] AuthChallengeProcessor - basic
> > > > > authentication scheme selected
> > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Using
> > > > > authentication scheme: basic
> > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor -
> > > > Authorization
> > > > > challenge processed
> > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Proxy
> > > > > authentication scope: BASIC 'Squid proxy-caching web
> > > > > server'@proxy.XXXXXXXXX.co.za:3128<http://server'@proxy.XXXXXXXXX.co.za:3128>
> <http://server'@proxy.xxxxxxxxx.co.za:3128/>
> > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodBase - Should NOT
> close
> > > > > connection in response to directive: keep-alive
> > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpConnection - Connection is
> > > > locked.
> > > > > Call to releaseConnection() ignored.
> > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector -
> > > > Authenticating with
> > > > > BASIC 'Squid proxy-caching web server'@proxy.XXXXXXXXX.co.za:3128<http://server'@proxy.XXXXXXXXX.co.za:3128>
> <http://server'@proxy.xxxxxxxxx.co.za:3128/>
> > > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodParams - Credential
> > > > charset
> > > > > not configured, using HTTP element charset
> > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "CONNECT
> > > > > www.verisign.com:443 <http://www.verisign.com:443> <
> http://www.verisign.com:443/> <
> > > > http://www.verisign.com:443/> HTTP/1.0"
> > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodBase - Adding Host
> > > > request
> > > > > header
> > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "User-Agent:
> Jakarta
> > > > > Commons-HttpClient/3.0-rc4[\r][\n]"
> > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Connection:
> > > > > Keep-Alive[\r][\n]"
> > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >>
> "Proxy-Authorization:
> > > > Basic
> > > > > d29sbWFydzp0eXRlbndv[\r][\n]"
> > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Host:
> > > > > www.verisign.com[\r][\n <http://www.verisign.com[/r][/n<
> http://www.verisign.com%5b/r%5D%5B/n>
> > > > >]"
> > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "[\r][\n]"
> > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - Closing
> the
> > > > > connection.
> > > > > 2005/10/18 13:28:19:234 CAT [INFO] HttpMethodDirector - I/O
> exception
> > > > (
> > > > > org.apache.commons.httpclient.NoHttpResponseException) caught when
> > > > > processing request: The server www.verisign.com<http://www.verisign.com>
> <http://www.verisign.com/>
> > > > > <http://www.verisign.com/>failed to respond
> > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The
> server
> > > > > www.verisign.com <http://www.verisign.com> <
> http://www.verisign.com/> <http://www.verisign.com/>
> > > > failed to respond <
> > > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > > www.verisign.com <http://www.verisign.com> <
> http://www.verisign.com/> <http://www.verisign.com/>
> > > > failed to respond>
> > > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > > www.verisign.com <http://www.verisign.com> <
> http://www.verisign.com/> <http://www.verisign.com/>
> > > > failed to respond
> > > > > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(
> > > > > HttpMethodBase.java:1835)
> > > > > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > > > > HttpMethodBase.java:1590)
> > > > > at org.apache.commons.httpclient.HttpMethodBase.execute(
> > > > HttpMethodBase.java
> > > > > :995)
> > > > > at org.apache.commons.httpclient.ConnectMethod.execute (
> > > > ConnectMethod.java
> > > > > :144)
> > > > > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect
> (
> > > > > HttpMethodDirector.java:487)
> > > > > at
> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry (
> > > > > HttpMethodDirector.java:388)
> > > > > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > > > > HttpMethodDirector.java:170)
> > > > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > > > HttpClient.java
> > > > > :396)
> > > > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > > > HttpClient.java
> > > > > :324)
> > > > > at com.XXXXXXXXX.webagent.TestCase.simplestTest(TestCase.java:43)
> > > > > at com.XXXXXXXXX.webagent.TestCase.main (TestCase.java:21)
> > > > > On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
> > > > >
> > > > > > On Tue, Oct 18, 2005 at 02:30:24PM +0200, Wynand wrote:
> > > > > > > Hi All,
> > > > > > > I've just started using the commons httpclient 3.0 rc4. It
> works
> > > > just as
> > > > > > > expected, except for connecting to a HTTPS site through a HTTP
> > > > proxy
> > > > > > > (Squid/2.4.STABLE6).
> > > > > > > I have tried the the example as per the SSL guide, but to no
> > > > avail; I
> > > > > > get
> > > > > > > the following error :
> > > > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The
> > > > server
> > > > > > > www.verisign.com <http://www.verisign.com> <
> http://www.verisign.com/> <
> > > > http://www.verisign.com> <http://www.verisign.com/>
> > > > > > failed to respond <
> > > > > > > org.apache.commons.httpclient.NoHttpResponseException: The
> server
> > > > > > > www.verisign.com <http://www.verisign.com> <
> http://www.verisign.com/> <
> > > > http://www.verisign.com > <http://www.verisign.com/>
> > > > > > failed to respond>
> > > > > > > org.apache.commons.httpclient.NoHttpResponseException: The
> server
> > > > > > > www.verisign.com <http://www.verisign.com> <
> http://www.verisign.com/> <
> > > > http://www.verisign.com> <http://www.verisign.com/>
> > > > > > failed to respond
> > > > > > > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(
> > > > > > > HttpMethodBase.java:1835)
> > > > > > > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > > > > > > HttpMethodBase.java:1590)
> > > > > > > at org.apache.commons.httpclient.HttpMethodBase.execute (
> > > > > > HttpMethodBase.java
> > > > > > > :995)
> > > > > > > at org.apache.commons.httpclient.ConnectMethod.execute (
> > > > > > ConnectMethod.java
> > > > > > > :144)
> > > > > > > at
> org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> > > > > > > HttpMethodDirector.java:487)
> > > > > > > at
> > > > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> > > > > > > HttpMethodDirector.java :388)
> > > > > > > at
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > > > > > > HttpMethodDirector.java:170)
> > > > > > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > > > > > HttpClient.java
> > > > > > > :396)
> > > > > > > at org.apache.commons.httpclient.HttpClient.executeMethod (
> > > > > > HttpClient.java
> > > > > > > :324)
> > > > > > > at com.wolman.webagent.TestCase.simplestTest(TestCase.java:43)
> > > > > > > at com.wolman.webagent.TestCase.main(TestCase.java:21)
> > > > > > > Just to clarify here is the example i used :
> > > > > > > HttpClient httpclient = new HttpClient();
> > > > > > > httpclient.getHostConfiguration().setProxy("myproxyhost",
> 8080);
> > > > > > > httpclient.getState().setProxyCredentials("my-proxy-realm", "
> > > > > > myproxyhost",
> > > > > > > new UsernamePasswordCredentials("my-proxy-username",
> > > > > > "my-proxy-password"));
> > > > > > > GetMethod httpget = new GetMethod("*
> https://www.verisign.com/*")<https://www.verisign.com/*%22)>
> > > > ;
> > > > > > > httpclient.executeMethod(httpget);
> > > > > > > System.out.println(httpget.getStatusLine().toString());
> > > > > > > It works fine if I change the *https* to *http* in the url.
> Can
> > > > someone
> > > > > > > please confirm that this example is indeed working or if I'm
> > > > missing
> > > > > > > something
> > > > > > > Thanks alot
> > > > > >
> > > > > > Wynand,
> > > > > >
> > > > > > Please send the complete wire log.
> > > > > >
> > > > > > Oleg
> > > > > >
> > > > > >
> > > >
> ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail:
> commons-user-unsubscribe@jakarta.apache.org
> > > > > > For additional commands, e-mail:
> > > > commons-user-help@jakarta.apache.org
> > > > > >
> > > > > >
> > > >
> > > >
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail:
> commons-user-help@jakarta.apache.org
> > > >
> > > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
>
Re: HttpClient 3.0 - Tunneled HTTPS connections through HTTP proxies
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, Oct 18, 2005 at 06:40:06PM +0200, Wynand wrote:
> Oleg
> I've been thinking (and trying various things), how come a browser like IE
> can do ssl through this proxy server of mine to https://www.verisign.com.
> Can you think of what is it doing differently? Any suggestion will help,
> this seems to me like a solvable problem that may not need a proxy server
> re-configuration.
> Thanks
Try adding 'Proxy-Connection: close' header to the request or using HTTP
version 1.0 and see if that makes any difference
Oleg
> On 10/18/05, Wynand <wo...@gmail.com> wrote:
> >
> > Yes, thanks alot.
> > This isn't good news for me but at least it narrows my options down to
> > just one ;-)
> >
> > On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
> > >
> > > On Tue, Oct 18, 2005 at 04:39:44PM +0200, Wynand wrote:
> > > > Oleg,
> > > > I may have stumbled on the cause of this problem by accident. I read
> > > that
> > > > there is such an option as "http_access deny CONNECT" in the squid
> > > > configuration, and that's exactly what httpclient tries to do when it
> > > tries
> > > > to make a ssl connection though a proxy. I don't have access to the
> > > squid
> > > > configuration, but that's what I'm guessing the problem is. Your
> > > comments
> > > > are appreciated.
> > >
> > > This is precisely the cause of the problem. To make matters worse the
> > > version of Squid you are using appears to have an issue with connection
> > > management. The first time it returns status 407, Proxy-Authenticate and
> > > Connection: keep-alive headers, which is perfectly ok. However, when
> > > HttpClient attempts to authenticate using given credentials, the proxy
> > > simply drops the connection on unsuspected HttpClient:
> > >
> > > - >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443/> <http://www.verisign.com:443/
> > > > HTTP/1.1"
> > > - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
> > > - >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n
> > > <http://www.verisign.com%5b/r%5D%5B/n>>]"
> > > - >> "Proxy-Connection: Keep-Alive[\r][\n]"
> > > - >> "[\r][\n]"
> > > - << "HTTP/1.0 407 Proxy Authentication Required[\r][\n]"
> > > - << "Server: Squid/2.4.STABLE6[\r][\n]"
> > > - << "Mime-Version: 1.0[\r][\n]"
> > > - << "Date: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
> > > - << "Content-Type: text/html[\r][\n]"
> > > - << "Content-Length: 984[\r][\n]"
> > > - << "Expires: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
> > > - << "X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> > > - << "Proxy-Authenticate: Basic realm="Squid proxy-caching web
> > > server"[\r][\n]"
> > > - << "X-Cache: MISS from neutrino.XXXXXXXXX.co.za[\r][\n]"
> > > - << "Proxy-Connection: keep-alive[\r][\n]"
> > > - >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443/><
> > > http://www.verisign.com:443/> HTTP/1.0"
> > > - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
> > > - >> "Proxy-Connection: Keep-Alive[\r][\n]"
> > > - >> "Proxy-Authorization: Basic d29sbWFydzp0eXRlbndv[\r][\n]"
> > > - >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n<http://www.verisign.com%5b/r%5D%5B/n>>]"
> > >
> > > - >> "[\r][\n]"
> > >
> > > Your only option is to get someone to reconfigure that proxy server (and
> > > optionally upgrade it from version 2.4.STABLE6 to something slightly
> > > more modern)
> > >
> > > Hope this helps
> > >
> > > Oleg
> > >
> > >
> > > > I'm not sure what a wire log is, but here is all the debug info ;-)
> > > > 2005/10/18 13:28:18:828 CAT [DEBUG] HttpClient - Java version:
> > > 1.4.2_08
> > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java vendor: Sun
> > > > Microsystems Inc.
> > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java class path:
> > > > C:\eclipse\workspace\SimpleWebAgent\bin;C:\Projects\java\lib\jericho-
> > > > html-1.5-dev1.jar;C:\Projects\java\lib\commons- logging-1.0.4.jar
> > > > ;C:\Projects\java\lib\commons-codec-1.3.jar;C:\Projects\java\commons-
> > > > httpclient-3.0-rc4\commons-httpclient-3.0-rc4.jar
> > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > > name:
> > > > Windows 2000
> > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > > > architecture: x86
> > > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > > version:
> > > > 5.0
> > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SUN 1.42: SUN (DSA
> > > > key/parameter generation; DSA signing; SHA-1, MD5 digests;
> > > SecureRandom;
> > > > X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
> > > > CertPathBuilder; LDAP, Collection CertStores)
> > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJSSE 1.42: Sun
> > > JSSE
> > > > provider(implements RSA Signatures, PKCS12, SunX509 key/trust
> > > factories,
> > > > SSLv3, TLSv1)
> > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunRsaSign 1.42:
> > > SUN's
> > > > provider for RSA signatures
> > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJCE 1.42: SunJCE
> > > > Provider (implements DES, Triple DES, AES, Blowfish, PBE,
> > > Diffie-Hellman,
> > > > HMAC-MD5, HMAC-SHA1)
> > > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJGSS 1.0: Sun
> > > (Kerberos
> > > > v5)
> > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > > http.useragent = Jakarta Commons-HttpClient/3.0-rc4
> > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > > http.protocol.version = HTTP/1.1
> > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > > http.connection-manager.class = class
> > > > org.apache.commons.httpclient.SimpleHttpConnectionManager
> > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > > http.protocol.cookie-policy = rfc2109
> > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > > http.protocol.element-charset = US-ASCII
> > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > > http.protocol.content-charset = ISO-8859-1
> > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > > http.method.retry-handler =
> > > > org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@197a37c
> > > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > > http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE,
> > > dd-MMM-yy
> > > > HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy HH:mm:ss z,
> > > EEE,
> > > > dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy
> > > HH:mm:ss
> > > > z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE
> > > dd-MMM-yy
> > > > HH:mm:ss z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z,
> > > > EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy HH:mm:ss z]
> > > > 2005/10/18 13:28:19:109 CAT [DEBUG] HttpConnection - Open connection
> > > to
> > > > proxy.XXXXXXXXX.co.za:3128 <http://proxy.xxxxxxxxx.co.za:3128/> <
> > > http://proxy.XXXXXXXXX.co.za:3128 <http://proxy.xxxxxxxxx.co.za:3128/>>
> > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "CONNECT
> > > > www.verisign.com:443 <http://www.verisign.com:443/> <
> > > http://www.verisign.com:443/> HTTP/1.1"
> > > > 2005/10/18 13:28:19:156 CAT [DEBUG] HttpMethodBase - Adding Host
> > > request
> > > > header
> > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "User-Agent: Jakarta
> > > > Commons-HttpClient/3.0-rc4[\r][\n]"
> > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Host:
> > > > www.verisign.com[\r][\n <http://www.verisign.com[/r][/n<http://www.verisign.com%5b/r%5D%5B/n>
> > > >]"
> > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Proxy-Connection:
> > > > Keep-Alive[\r][\n]"
> > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "[\r][\n]"
> > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "HTTP/1.0 407 Proxy
> > > > Authentication Required[\r][\n]"
> > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Server:
> > > > Squid/2.4.STABLE6[\r][\n]"
> > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Mime-Version: 1.0
> > > [\r][\n]"
> > > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Date: Tue, 18 Oct
> > > 2005
> > > > 11:27:51 GMT[\r][\n]"
> > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Type:
> > > > text/html[\r][\n]"
> > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Length:
> > > > 984[\r][\n]"
> > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Expires: Tue, 18 Oct
> > > 2005
> > > > 11:27:51 GMT[\r][\n]"
> > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Squid-Error:
> > > > ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Authenticate:
> > > Basic
> > > > realm="Squid proxy-caching web server"[\r][\n]"
> > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Cache: MISS from
> > > > neutrino.XXXXXXXXX.co.za[\r][\n]"
> > > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Connection:
> > > > keep-alive[\r][\n]"
> > > > 2005/10/18 13:28:19:203 CAT [DEBUG] ConnectMethod - CONNECT status
> > > code 407
> > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Supported
> > > > authentication schemes in the order of preference: [ntlm, digest,
> > > basic]
> > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge
> > > for
> > > > ntlm authentication scheme not available
> > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge
> > > for
> > > > digest authentication scheme not available
> > > > 2005/10/18 13:28:19:218 CAT [INFO] AuthChallengeProcessor - basic
> > > > authentication scheme selected
> > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Using
> > > > authentication scheme: basic
> > > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor -
> > > Authorization
> > > > challenge processed
> > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Proxy
> > > > authentication scope: BASIC 'Squid proxy-caching web
> > > > server'@proxy.XXXXXXXXX.co.za:3128<http://server'@proxy.xxxxxxxxx.co.za:3128/>
> > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodBase - Should NOT close
> > > > connection in response to directive: keep-alive
> > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpConnection - Connection is
> > > locked.
> > > > Call to releaseConnection() ignored.
> > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector -
> > > Authenticating with
> > > > BASIC 'Squid proxy-caching web server'@proxy.XXXXXXXXX.co.za:3128<http://server'@proxy.xxxxxxxxx.co.za:3128/>
> > > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodParams - Credential
> > > charset
> > > > not configured, using HTTP element charset
> > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "CONNECT
> > > > www.verisign.com:443 <http://www.verisign.com:443/> <
> > > http://www.verisign.com:443/> HTTP/1.0"
> > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodBase - Adding Host
> > > request
> > > > header
> > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "User-Agent: Jakarta
> > > > Commons-HttpClient/3.0-rc4[\r][\n]"
> > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Connection:
> > > > Keep-Alive[\r][\n]"
> > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Authorization:
> > > Basic
> > > > d29sbWFydzp0eXRlbndv[\r][\n]"
> > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Host:
> > > > www.verisign.com[\r][\n <http://www.verisign.com[/r][/n<http://www.verisign.com%5b/r%5D%5B/n>
> > > >]"
> > > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "[\r][\n]"
> > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - Closing the
> > > > connection.
> > > > 2005/10/18 13:28:19:234 CAT [INFO] HttpMethodDirector - I/O exception
> > > (
> > > > org.apache.commons.httpclient.NoHttpResponseException) caught when
> > > > processing request: The server www.verisign.com<http://www.verisign.com/>
> > > > <http://www.verisign.com/>failed to respond
> > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> > > > www.verisign.com <http://www.verisign.com/> <http://www.verisign.com/>
> > > failed to respond <
> > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > www.verisign.com <http://www.verisign.com/> <http://www.verisign.com/>
> > > failed to respond>
> > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > www.verisign.com <http://www.verisign.com/> <http://www.verisign.com/>
> > > failed to respond
> > > > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(
> > > > HttpMethodBase.java:1835)
> > > > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > > > HttpMethodBase.java:1590)
> > > > at org.apache.commons.httpclient.HttpMethodBase.execute(
> > > HttpMethodBase.java
> > > > :995)
> > > > at org.apache.commons.httpclient.ConnectMethod.execute (
> > > ConnectMethod.java
> > > > :144)
> > > > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> > > > HttpMethodDirector.java:487)
> > > > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry (
> > > > HttpMethodDirector.java:388)
> > > > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > > > HttpMethodDirector.java:170)
> > > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > > HttpClient.java
> > > > :396)
> > > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > > HttpClient.java
> > > > :324)
> > > > at com.XXXXXXXXX.webagent.TestCase.simplestTest(TestCase.java:43)
> > > > at com.XXXXXXXXX.webagent.TestCase.main (TestCase.java:21)
> > > > On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
> > > >
> > > > > On Tue, Oct 18, 2005 at 02:30:24PM +0200, Wynand wrote:
> > > > > > Hi All,
> > > > > > I've just started using the commons httpclient 3.0 rc4. It works
> > > just as
> > > > > > expected, except for connecting to a HTTPS site through a HTTP
> > > proxy
> > > > > > (Squid/2.4.STABLE6).
> > > > > > I have tried the the example as per the SSL guide, but to no
> > > avail; I
> > > > > get
> > > > > > the following error :
> > > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The
> > > server
> > > > > > www.verisign.com <http://www.verisign.com/> <
> > > http://www.verisign.com> <http://www.verisign.com/>
> > > > > failed to respond <
> > > > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > > > www.verisign.com <http://www.verisign.com/> <
> > > http://www.verisign.com > <http://www.verisign.com/>
> > > > > failed to respond>
> > > > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > > > www.verisign.com <http://www.verisign.com/> <
> > > http://www.verisign.com> <http://www.verisign.com/>
> > > > > failed to respond
> > > > > > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine (
> > > > > > HttpMethodBase.java:1835)
> > > > > > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > > > > > HttpMethodBase.java:1590)
> > > > > > at org.apache.commons.httpclient.HttpMethodBase.execute (
> > > > > HttpMethodBase.java
> > > > > > :995)
> > > > > > at org.apache.commons.httpclient.ConnectMethod.execute (
> > > > > ConnectMethod.java
> > > > > > :144)
> > > > > > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> > > > > > HttpMethodDirector.java:487)
> > > > > > at
> > > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> > > > > > HttpMethodDirector.java :388)
> > > > > > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > > > > > HttpMethodDirector.java:170)
> > > > > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > > > > HttpClient.java
> > > > > > :396)
> > > > > > at org.apache.commons.httpclient.HttpClient.executeMethod (
> > > > > HttpClient.java
> > > > > > :324)
> > > > > > at com.wolman.webagent.TestCase.simplestTest(TestCase.java:43)
> > > > > > at com.wolman.webagent.TestCase.main(TestCase.java:21)
> > > > > > Just to clarify here is the example i used :
> > > > > > HttpClient httpclient = new HttpClient();
> > > > > > httpclient.getHostConfiguration().setProxy("myproxyhost", 8080);
> > > > > > httpclient.getState().setProxyCredentials("my-proxy-realm", "
> > > > > myproxyhost",
> > > > > > new UsernamePasswordCredentials("my-proxy-username",
> > > > > "my-proxy-password"));
> > > > > > GetMethod httpget = new GetMethod("* https://www.verisign.com/*")<https://www.verisign.com/*%22)>
> > > ;
> > > > > > httpclient.executeMethod(httpget);
> > > > > > System.out.println(httpget.getStatusLine().toString());
> > > > > > It works fine if I change the *https* to *http* in the url. Can
> > > someone
> > > > > > please confirm that this example is indeed working or if I'm
> > > missing
> > > > > > something
> > > > > > Thanks alot
> > > > >
> > > > > Wynand,
> > > > >
> > > > > Please send the complete wire log.
> > > > >
> > > > > Oleg
> > > > >
> > > > >
> > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> > > > > For additional commands, e-mail:
> > > commons-user-help@jakarta.apache.org
> > > > >
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: commons-user-help@jakarta.apache.org
> > >
> > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org
Re: HttpClient 3.0 - Tunneled HTTPS connections through HTTP proxies
Posted by Wynand <wo...@gmail.com>.
Oleg
I've been thinking (and trying various things), how come a browser like IE
can do ssl through this proxy server of mine to https://www.verisign.com.
Can you think of what is it doing differently? Any suggestion will help,
this seems to me like a solvable problem that may not need a proxy server
re-configuration.
Thanks
On 10/18/05, Wynand <wo...@gmail.com> wrote:
>
> Yes, thanks alot.
> This isn't good news for me but at least it narrows my options down to
> just one ;-)
>
> On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
> >
> > On Tue, Oct 18, 2005 at 04:39:44PM +0200, Wynand wrote:
> > > Oleg,
> > > I may have stumbled on the cause of this problem by accident. I read
> > that
> > > there is such an option as "http_access deny CONNECT" in the squid
> > > configuration, and that's exactly what httpclient tries to do when it
> > tries
> > > to make a ssl connection though a proxy. I don't have access to the
> > squid
> > > configuration, but that's what I'm guessing the problem is. Your
> > comments
> > > are appreciated.
> >
> > This is precisely the cause of the problem. To make matters worse the
> > version of Squid you are using appears to have an issue with connection
> > management. The first time it returns status 407, Proxy-Authenticate and
> > Connection: keep-alive headers, which is perfectly ok. However, when
> > HttpClient attempts to authenticate using given credentials, the proxy
> > simply drops the connection on unsuspected HttpClient:
> >
> > - >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443/> <http://www.verisign.com:443/
> > > HTTP/1.1"
> > - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
> > - >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n
> > <http://www.verisign.com%5b/r%5D%5B/n>>]"
> > - >> "Proxy-Connection: Keep-Alive[\r][\n]"
> > - >> "[\r][\n]"
> > - << "HTTP/1.0 407 Proxy Authentication Required[\r][\n]"
> > - << "Server: Squid/2.4.STABLE6[\r][\n]"
> > - << "Mime-Version: 1.0[\r][\n]"
> > - << "Date: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
> > - << "Content-Type: text/html[\r][\n]"
> > - << "Content-Length: 984[\r][\n]"
> > - << "Expires: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
> > - << "X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> > - << "Proxy-Authenticate: Basic realm="Squid proxy-caching web
> > server"[\r][\n]"
> > - << "X-Cache: MISS from neutrino.XXXXXXXXX.co.za[\r][\n]"
> > - << "Proxy-Connection: keep-alive[\r][\n]"
> > - >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443/><
> > http://www.verisign.com:443/> HTTP/1.0"
> > - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
> > - >> "Proxy-Connection: Keep-Alive[\r][\n]"
> > - >> "Proxy-Authorization: Basic d29sbWFydzp0eXRlbndv[\r][\n]"
> > - >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n<http://www.verisign.com%5b/r%5D%5B/n>>]"
> >
> > - >> "[\r][\n]"
> >
> > Your only option is to get someone to reconfigure that proxy server (and
> > optionally upgrade it from version 2.4.STABLE6 to something slightly
> > more modern)
> >
> > Hope this helps
> >
> > Oleg
> >
> >
> > > I'm not sure what a wire log is, but here is all the debug info ;-)
> > > 2005/10/18 13:28:18:828 CAT [DEBUG] HttpClient - Java version:
> > 1.4.2_08
> > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java vendor: Sun
> > > Microsystems Inc.
> > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java class path:
> > > C:\eclipse\workspace\SimpleWebAgent\bin;C:\Projects\java\lib\jericho-
> > > html-1.5-dev1.jar;C:\Projects\java\lib\commons- logging-1.0.4.jar
> > > ;C:\Projects\java\lib\commons-codec-1.3.jar;C:\Projects\java\commons-
> > > httpclient-3.0-rc4\commons-httpclient-3.0-rc4.jar
> > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > name:
> > > Windows 2000
> > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > > architecture: x86
> > > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > version:
> > > 5.0
> > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SUN 1.42: SUN (DSA
> > > key/parameter generation; DSA signing; SHA-1, MD5 digests;
> > SecureRandom;
> > > X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
> > > CertPathBuilder; LDAP, Collection CertStores)
> > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJSSE 1.42: Sun
> > JSSE
> > > provider(implements RSA Signatures, PKCS12, SunX509 key/trust
> > factories,
> > > SSLv3, TLSv1)
> > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunRsaSign 1.42:
> > SUN's
> > > provider for RSA signatures
> > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJCE 1.42: SunJCE
> > > Provider (implements DES, Triple DES, AES, Blowfish, PBE,
> > Diffie-Hellman,
> > > HMAC-MD5, HMAC-SHA1)
> > > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJGSS 1.0: Sun
> > (Kerberos
> > > v5)
> > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > http.useragent = Jakarta Commons-HttpClient/3.0-rc4
> > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > http.protocol.version = HTTP/1.1
> > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > http.connection-manager.class = class
> > > org.apache.commons.httpclient.SimpleHttpConnectionManager
> > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > http.protocol.cookie-policy = rfc2109
> > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > http.protocol.element-charset = US-ASCII
> > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > http.protocol.content-charset = ISO-8859-1
> > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > http.method.retry-handler =
> > > org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@197a37c
> > > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > > http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE,
> > dd-MMM-yy
> > > HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy HH:mm:ss z,
> > EEE,
> > > dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy
> > HH:mm:ss
> > > z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE
> > dd-MMM-yy
> > > HH:mm:ss z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z,
> > > EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy HH:mm:ss z]
> > > 2005/10/18 13:28:19:109 CAT [DEBUG] HttpConnection - Open connection
> > to
> > > proxy.XXXXXXXXX.co.za:3128 <http://proxy.xxxxxxxxx.co.za:3128/> <
> > http://proxy.XXXXXXXXX.co.za:3128 <http://proxy.xxxxxxxxx.co.za:3128/>>
> > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "CONNECT
> > > www.verisign.com:443 <http://www.verisign.com:443/> <
> > http://www.verisign.com:443/> HTTP/1.1"
> > > 2005/10/18 13:28:19:156 CAT [DEBUG] HttpMethodBase - Adding Host
> > request
> > > header
> > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "User-Agent: Jakarta
> > > Commons-HttpClient/3.0-rc4[\r][\n]"
> > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Host:
> > > www.verisign.com[\r][\n <http://www.verisign.com[/r][/n<http://www.verisign.com%5b/r%5D%5B/n>
> > >]"
> > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Proxy-Connection:
> > > Keep-Alive[\r][\n]"
> > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "[\r][\n]"
> > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "HTTP/1.0 407 Proxy
> > > Authentication Required[\r][\n]"
> > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Server:
> > > Squid/2.4.STABLE6[\r][\n]"
> > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Mime-Version: 1.0
> > [\r][\n]"
> > > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Date: Tue, 18 Oct
> > 2005
> > > 11:27:51 GMT[\r][\n]"
> > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Type:
> > > text/html[\r][\n]"
> > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Length:
> > > 984[\r][\n]"
> > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Expires: Tue, 18 Oct
> > 2005
> > > 11:27:51 GMT[\r][\n]"
> > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Squid-Error:
> > > ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Authenticate:
> > Basic
> > > realm="Squid proxy-caching web server"[\r][\n]"
> > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Cache: MISS from
> > > neutrino.XXXXXXXXX.co.za[\r][\n]"
> > > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Connection:
> > > keep-alive[\r][\n]"
> > > 2005/10/18 13:28:19:203 CAT [DEBUG] ConnectMethod - CONNECT status
> > code 407
> > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Supported
> > > authentication schemes in the order of preference: [ntlm, digest,
> > basic]
> > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge
> > for
> > > ntlm authentication scheme not available
> > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge
> > for
> > > digest authentication scheme not available
> > > 2005/10/18 13:28:19:218 CAT [INFO] AuthChallengeProcessor - basic
> > > authentication scheme selected
> > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Using
> > > authentication scheme: basic
> > > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor -
> > Authorization
> > > challenge processed
> > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Proxy
> > > authentication scope: BASIC 'Squid proxy-caching web
> > > server'@proxy.XXXXXXXXX.co.za:3128<http://server'@proxy.xxxxxxxxx.co.za:3128/>
> > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodBase - Should NOT close
> > > connection in response to directive: keep-alive
> > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpConnection - Connection is
> > locked.
> > > Call to releaseConnection() ignored.
> > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector -
> > Authenticating with
> > > BASIC 'Squid proxy-caching web server'@proxy.XXXXXXXXX.co.za:3128<http://server'@proxy.xxxxxxxxx.co.za:3128/>
> > > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodParams - Credential
> > charset
> > > not configured, using HTTP element charset
> > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "CONNECT
> > > www.verisign.com:443 <http://www.verisign.com:443/> <
> > http://www.verisign.com:443/> HTTP/1.0"
> > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodBase - Adding Host
> > request
> > > header
> > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "User-Agent: Jakarta
> > > Commons-HttpClient/3.0-rc4[\r][\n]"
> > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Connection:
> > > Keep-Alive[\r][\n]"
> > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Authorization:
> > Basic
> > > d29sbWFydzp0eXRlbndv[\r][\n]"
> > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Host:
> > > www.verisign.com[\r][\n <http://www.verisign.com[/r][/n<http://www.verisign.com%5b/r%5D%5B/n>
> > >]"
> > > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "[\r][\n]"
> > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - Closing the
> > > connection.
> > > 2005/10/18 13:28:19:234 CAT [INFO] HttpMethodDirector - I/O exception
> > (
> > > org.apache.commons.httpclient.NoHttpResponseException) caught when
> > > processing request: The server www.verisign.com<http://www.verisign.com/>
> > > <http://www.verisign.com/>failed to respond
> > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> > > www.verisign.com <http://www.verisign.com/> <http://www.verisign.com/>
> > failed to respond <
> > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > www.verisign.com <http://www.verisign.com/> <http://www.verisign.com/>
> > failed to respond>
> > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > www.verisign.com <http://www.verisign.com/> <http://www.verisign.com/>
> > failed to respond
> > > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(
> > > HttpMethodBase.java:1835)
> > > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > > HttpMethodBase.java:1590)
> > > at org.apache.commons.httpclient.HttpMethodBase.execute(
> > HttpMethodBase.java
> > > :995)
> > > at org.apache.commons.httpclient.ConnectMethod.execute (
> > ConnectMethod.java
> > > :144)
> > > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> > > HttpMethodDirector.java:487)
> > > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry (
> > > HttpMethodDirector.java:388)
> > > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > > HttpMethodDirector.java:170)
> > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > HttpClient.java
> > > :396)
> > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > HttpClient.java
> > > :324)
> > > at com.XXXXXXXXX.webagent.TestCase.simplestTest(TestCase.java:43)
> > > at com.XXXXXXXXX.webagent.TestCase.main (TestCase.java:21)
> > > On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
> > >
> > > > On Tue, Oct 18, 2005 at 02:30:24PM +0200, Wynand wrote:
> > > > > Hi All,
> > > > > I've just started using the commons httpclient 3.0 rc4. It works
> > just as
> > > > > expected, except for connecting to a HTTPS site through a HTTP
> > proxy
> > > > > (Squid/2.4.STABLE6).
> > > > > I have tried the the example as per the SSL guide, but to no
> > avail; I
> > > > get
> > > > > the following error :
> > > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The
> > server
> > > > > www.verisign.com <http://www.verisign.com/> <
> > http://www.verisign.com> <http://www.verisign.com/>
> > > > failed to respond <
> > > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > > www.verisign.com <http://www.verisign.com/> <
> > http://www.verisign.com > <http://www.verisign.com/>
> > > > failed to respond>
> > > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > > www.verisign.com <http://www.verisign.com/> <
> > http://www.verisign.com> <http://www.verisign.com/>
> > > > failed to respond
> > > > > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine (
> > > > > HttpMethodBase.java:1835)
> > > > > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > > > > HttpMethodBase.java:1590)
> > > > > at org.apache.commons.httpclient.HttpMethodBase.execute (
> > > > HttpMethodBase.java
> > > > > :995)
> > > > > at org.apache.commons.httpclient.ConnectMethod.execute (
> > > > ConnectMethod.java
> > > > > :144)
> > > > > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> > > > > HttpMethodDirector.java:487)
> > > > > at
> > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> > > > > HttpMethodDirector.java :388)
> > > > > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > > > > HttpMethodDirector.java:170)
> > > > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > > > HttpClient.java
> > > > > :396)
> > > > > at org.apache.commons.httpclient.HttpClient.executeMethod (
> > > > HttpClient.java
> > > > > :324)
> > > > > at com.wolman.webagent.TestCase.simplestTest(TestCase.java:43)
> > > > > at com.wolman.webagent.TestCase.main(TestCase.java:21)
> > > > > Just to clarify here is the example i used :
> > > > > HttpClient httpclient = new HttpClient();
> > > > > httpclient.getHostConfiguration().setProxy("myproxyhost", 8080);
> > > > > httpclient.getState().setProxyCredentials("my-proxy-realm", "
> > > > myproxyhost",
> > > > > new UsernamePasswordCredentials("my-proxy-username",
> > > > "my-proxy-password"));
> > > > > GetMethod httpget = new GetMethod("* https://www.verisign.com/*")<https://www.verisign.com/*%22)>
> > ;
> > > > > httpclient.executeMethod(httpget);
> > > > > System.out.println(httpget.getStatusLine().toString());
> > > > > It works fine if I change the *https* to *http* in the url. Can
> > someone
> > > > > please confirm that this example is indeed working or if I'm
> > missing
> > > > > something
> > > > > Thanks alot
> > > >
> > > > Wynand,
> > > >
> > > > Please send the complete wire log.
> > > >
> > > > Oleg
> > > >
> > > >
> > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail:
> > commons-user-help@jakarta.apache.org
> > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: commons-user-help@jakarta.apache.org
> >
> >
>
Re: HttpClient 3.0 - Tunneled HTTPS connections through HTTP proxies
Posted by Wynand <wo...@gmail.com>.
Yes, thanks alot.
This isn't good news for me but at least it narrows my options down to just
one ;-)
On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
>
> On Tue, Oct 18, 2005 at 04:39:44PM +0200, Wynand wrote:
> > Oleg,
> > I may have stumbled on the cause of this problem by accident. I read
> that
> > there is such an option as "http_access deny CONNECT" in the squid
> > configuration, and that's exactly what httpclient tries to do when it
> tries
> > to make a ssl connection though a proxy. I don't have access to the
> squid
> > configuration, but that's what I'm guessing the problem is. Your
> comments
> > are appreciated.
>
> This is precisely the cause of the problem. To make matters worse the
> version of Squid you are using appears to have an issue with connection
> management. The first time it returns status 407, Proxy-Authenticate and
> Connection: keep-alive headers, which is perfectly ok. However, when
> HttpClient attempts to authenticate using given credentials, the proxy
> simply drops the connection on unsuspected HttpClient:
>
> - >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443> <
> http://www.verisign.com:443/> HTTP/1.1"
> - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
> - >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
> - >> "Proxy-Connection: Keep-Alive[\r][\n]"
> - >> "[\r][\n]"
> - << "HTTP/1.0 407 Proxy Authentication Required[\r][\n]"
> - << "Server: Squid/2.4.STABLE6[\r][\n]"
> - << "Mime-Version: 1.0[\r][\n]"
> - << "Date: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
> - << "Content-Type: text/html[\r][\n]"
> - << "Content-Length: 984[\r][\n]"
> - << "Expires: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
> - << "X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> - << "Proxy-Authenticate: Basic realm="Squid proxy-caching web
> server"[\r][\n]"
> - << "X-Cache: MISS from neutrino.XXXXXXXXX.co.za[\r][\n]"
> - << "Proxy-Connection: keep-alive[\r][\n]"
> - >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443> <
> http://www.verisign.com:443/> HTTP/1.0"
> - >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
> - >> "Proxy-Connection: Keep-Alive[\r][\n]"
> - >> "Proxy-Authorization: Basic d29sbWFydzp0eXRlbndv[\r][\n]"
> - >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
> - >> "[\r][\n]"
>
> Your only option is to get someone to reconfigure that proxy server (and
> optionally upgrade it from version 2.4.STABLE6 to something slightly
> more modern)
>
> Hope this helps
>
> Oleg
>
>
> > I'm not sure what a wire log is, but here is all the debug info ;-)
> > 2005/10/18 13:28:18:828 CAT [DEBUG] HttpClient - Java version: 1.4.2_08
> > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java vendor: Sun
> > Microsystems Inc.
> > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java class path:
> > C:\eclipse\workspace\SimpleWebAgent\bin;C:\Projects\java\lib\jericho-
> > html-1.5-dev1.jar;C:\Projects\java\lib\commons-logging-1.0.4.jar
> > ;C:\Projects\java\lib\commons-codec-1.3.jar;C:\Projects\java\commons-
> > httpclient-3.0-rc4\commons-httpclient-3.0-rc4.jar
> > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system name:
> > Windows 2000
> > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> > architecture: x86
> > 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> version:
> > 5.0
> > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SUN 1.42: SUN (DSA
> > key/parameter generation; DSA signing; SHA-1, MD5 digests; SecureRandom;
> > X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
> > CertPathBuilder; LDAP, Collection CertStores)
> > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJSSE 1.42: Sun JSSE
> > provider(implements RSA Signatures, PKCS12, SunX509 key/trust factories,
> > SSLv3, TLSv1)
> > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunRsaSign 1.42: SUN's
> > provider for RSA signatures
> > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJCE 1.42: SunJCE
> > Provider (implements DES, Triple DES, AES, Blowfish, PBE,
> Diffie-Hellman,
> > HMAC-MD5, HMAC-SHA1)
> > 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJGSS 1.0: Sun
> (Kerberos
> > v5)
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.useragent = Jakarta Commons-HttpClient/3.0-rc4
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.protocol.version = HTTP/1.1
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.connection-manager.class = class
> > org.apache.commons.httpclient.SimpleHttpConnectionManager
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.protocol.cookie-policy = rfc2109
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.protocol.element-charset = US-ASCII
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.protocol.content-charset = ISO-8859-1
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.method.retry-handler =
> > org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@197a37c
> > 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> > http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE,
> dd-MMM-yy
> > HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy HH:mm:ss z, EEE,
> > dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy
> HH:mm:ss
> > z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE dd-MMM-yy
> > HH:mm:ss z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z,
> > EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy HH:mm:ss z]
> > 2005/10/18 13:28:19:109 CAT [DEBUG] HttpConnection - Open connection to
> > proxy.XXXXXXXXX.co.za:3128 <http://proxy.XXXXXXXXX.co.za:3128> <
> http://proxy.XXXXXXXXX.co.za:3128>
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "CONNECT
> > www.verisign.com:443 <http://www.verisign.com:443> <
> http://www.verisign.com:443/> HTTP/1.1"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] HttpMethodBase - Adding Host request
> > header
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "User-Agent: Jakarta
> > Commons-HttpClient/3.0-rc4[\r][\n]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Host:
> > www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Proxy-Connection:
> > Keep-Alive[\r][\n]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "[\r][\n]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "HTTP/1.0 407 Proxy
> > Authentication Required[\r][\n]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Server:
> > Squid/2.4.STABLE6[\r][\n]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Mime-Version: 1.0
> [\r][\n]"
> > 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Date: Tue, 18 Oct 2005
> > 11:27:51 GMT[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Type:
> > text/html[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Length:
> > 984[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Expires: Tue, 18 Oct
> 2005
> > 11:27:51 GMT[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Squid-Error:
> > ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Authenticate:
> Basic
> > realm="Squid proxy-caching web server"[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Cache: MISS from
> > neutrino.XXXXXXXXX.co.za[\r][\n]"
> > 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Connection:
> > keep-alive[\r][\n]"
> > 2005/10/18 13:28:19:203 CAT [DEBUG] ConnectMethod - CONNECT status code
> 407
> > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Supported
> > authentication schemes in the order of preference: [ntlm, digest, basic]
> > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge
> for
> > ntlm authentication scheme not available
> > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge
> for
> > digest authentication scheme not available
> > 2005/10/18 13:28:19:218 CAT [INFO] AuthChallengeProcessor - basic
> > authentication scheme selected
> > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Using
> > authentication scheme: basic
> > 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor -
> Authorization
> > challenge processed
> > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Proxy
> > authentication scope: BASIC 'Squid proxy-caching web
> > server'@proxy.XXXXXXXXX.co.za:3128<http://server'@proxy.XXXXXXXXX.co.za:3128>
> > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodBase - Should NOT close
> > connection in response to directive: keep-alive
> > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpConnection - Connection is
> locked.
> > Call to releaseConnection() ignored.
> > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Authenticating
> with
> > BASIC 'Squid proxy-caching web server'@proxy.XXXXXXXXX.co.za:3128<http://server'@proxy.XXXXXXXXX.co.za:3128>
> > 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodParams - Credential
> charset
> > not configured, using HTTP element charset
> > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "CONNECT
> > www.verisign.com:443 <http://www.verisign.com:443> <
> http://www.verisign.com:443/> HTTP/1.0"
> > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodBase - Adding Host request
> > header
> > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "User-Agent: Jakarta
> > Commons-HttpClient/3.0-rc4[\r][\n]"
> > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Connection:
> > Keep-Alive[\r][\n]"
> > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Authorization:
> Basic
> > d29sbWFydzp0eXRlbndv[\r][\n]"
> > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Host:
> > www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
> > 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "[\r][\n]"
> > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - Closing the
> > connection.
> > 2005/10/18 13:28:19:234 CAT [INFO] HttpMethodDirector - I/O exception (
> > org.apache.commons.httpclient.NoHttpResponseException) caught when
> > processing request: The server www.verisign.com<http://www.verisign.com>
> > <http://www.verisign.com/>failed to respond
> > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> failed to respond <
> > org.apache.commons.httpclient.NoHttpResponseException: The server
> > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> failed to respond>
> > org.apache.commons.httpclient.NoHttpResponseException: The server
> > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> failed to respond
> > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(
> > HttpMethodBase.java:1835)
> > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > HttpMethodBase.java:1590)
> > at org.apache.commons.httpclient.HttpMethodBase.execute(
> HttpMethodBase.java
> > :995)
> > at org.apache.commons.httpclient.ConnectMethod.execute(
> ConnectMethod.java
> > :144)
> > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> > HttpMethodDirector.java:487)
> > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> > HttpMethodDirector.java:388)
> > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > HttpMethodDirector.java:170)
> > at org.apache.commons.httpclient.HttpClient.executeMethod(
> HttpClient.java
> > :396)
> > at org.apache.commons.httpclient.HttpClient.executeMethod(
> HttpClient.java
> > :324)
> > at com.XXXXXXXXX.webagent.TestCase.simplestTest(TestCase.java:43)
> > at com.XXXXXXXXX.webagent.TestCase.main(TestCase.java:21)
> > On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
> >
> > > On Tue, Oct 18, 2005 at 02:30:24PM +0200, Wynand wrote:
> > > > Hi All,
> > > > I've just started using the commons httpclient 3.0 rc4. It works
> just as
> > > > expected, except for connecting to a HTTPS site through a HTTP proxy
> > > > (Squid/2.4.STABLE6).
> > > > I have tried the the example as per the SSL guide, but to no avail;
> I
> > > get
> > > > the following error :
> > > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> > > > www.verisign.com <http://www.verisign.com> <http://www.verisign.com>
> <http://www.verisign.com/>
> > > failed to respond <
> > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > www.verisign.com <http://www.verisign.com> <http://www.verisign.com>
> <http://www.verisign.com/>
> > > failed to respond>
> > > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > > www.verisign.com <http://www.verisign.com> <http://www.verisign.com>
> <http://www.verisign.com/>
> > > failed to respond
> > > > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine (
> > > > HttpMethodBase.java:1835)
> > > > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > > > HttpMethodBase.java:1590)
> > > > at org.apache.commons.httpclient.HttpMethodBase.execute(
> > > HttpMethodBase.java
> > > > :995)
> > > > at org.apache.commons.httpclient.ConnectMethod.execute (
> > > ConnectMethod.java
> > > > :144)
> > > > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> > > > HttpMethodDirector.java:487)
> > > > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry
> (
> > > > HttpMethodDirector.java :388)
> > > > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > > > HttpMethodDirector.java:170)
> > > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > > HttpClient.java
> > > > :396)
> > > > at org.apache.commons.httpclient.HttpClient.executeMethod (
> > > HttpClient.java
> > > > :324)
> > > > at com.wolman.webagent.TestCase.simplestTest(TestCase.java:43)
> > > > at com.wolman.webagent.TestCase.main(TestCase.java:21)
> > > > Just to clarify here is the example i used :
> > > > HttpClient httpclient = new HttpClient();
> > > > httpclient.getHostConfiguration().setProxy("myproxyhost", 8080);
> > > > httpclient.getState().setProxyCredentials("my-proxy-realm", "
> > > myproxyhost",
> > > > new UsernamePasswordCredentials("my-proxy-username",
> > > "my-proxy-password"));
> > > > GetMethod httpget = new GetMethod("*https://www.verisign.com/*");
> > > > httpclient.executeMethod(httpget);
> > > > System.out.println(httpget.getStatusLine().toString());
> > > > It works fine if I change the *https* to *http* in the url. Can
> someone
> > > > please confirm that this example is indeed working or if I'm missing
> > > > something
> > > > Thanks alot
> > >
> > > Wynand,
> > >
> > > Please send the complete wire log.
> > >
> > > Oleg
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: commons-user-help@jakarta.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
>
Re: HttpClient 3.0 - Tunneled HTTPS connections through HTTP proxies
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, Oct 18, 2005 at 04:39:44PM +0200, Wynand wrote:
> Oleg,
> I may have stumbled on the cause of this problem by accident. I read that
> there is such an option as "http_access deny CONNECT" in the squid
> configuration, and that's exactly what httpclient tries to do when it tries
> to make a ssl connection though a proxy. I don't have access to the squid
> configuration, but that's what I'm guessing the problem is. Your comments
> are appreciated.
This is precisely the cause of the problem. To make matters worse the
version of Squid you are using appears to have an issue with connection
management. The first time it returns status 407, Proxy-Authenticate and
Connection: keep-alive headers, which is perfectly ok. However, when
HttpClient attempts to authenticate using given credentials, the proxy
simply drops the connection on unsuspected HttpClient:
- >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443/> HTTP/1.1"
- >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
- >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
- >> "Proxy-Connection: Keep-Alive[\r][\n]"
- >> "[\r][\n]"
- << "HTTP/1.0 407 Proxy Authentication Required[\r][\n]"
- << "Server: Squid/2.4.STABLE6[\r][\n]"
- << "Mime-Version: 1.0[\r][\n]"
- << "Date: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
- << "Content-Type: text/html[\r][\n]"
- << "Content-Length: 984[\r][\n]"
- << "Expires: Tue, 18 Oct 2005 11:27:51 GMT[\r][\n]"
- << "X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
- << "Proxy-Authenticate: Basic realm="Squid proxy-caching web server"[\r][\n]"
- << "X-Cache: MISS from neutrino.XXXXXXXXX.co.za[\r][\n]"
- << "Proxy-Connection: keep-alive[\r][\n]"
- >> "CONNECT www.verisign.com:443 <http://www.verisign.com:443/> HTTP/1.0"
- >> "User-Agent: Jakarta Commons-HttpClient/3.0-rc4[\r][\n]"
- >> "Proxy-Connection: Keep-Alive[\r][\n]"
- >> "Proxy-Authorization: Basic d29sbWFydzp0eXRlbndv[\r][\n]"
- >> "Host: www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
- >> "[\r][\n]"
Your only option is to get someone to reconfigure that proxy server (and
optionally upgrade it from version 2.4.STABLE6 to something slightly
more modern)
Hope this helps
Oleg
> I'm not sure what a wire log is, but here is all the debug info ;-)
> 2005/10/18 13:28:18:828 CAT [DEBUG] HttpClient - Java version: 1.4.2_08
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java vendor: Sun
> Microsystems Inc.
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java class path:
> C:\eclipse\workspace\SimpleWebAgent\bin;C:\Projects\java\lib\jericho-
> html-1.5-dev1.jar;C:\Projects\java\lib\commons-logging-1.0.4.jar
> ;C:\Projects\java\lib\commons-codec-1.3.jar;C:\Projects\java\commons-
> httpclient-3.0-rc4\commons-httpclient-3.0-rc4.jar
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system name:
> Windows 2000
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> architecture: x86
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system version:
> 5.0
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SUN 1.42: SUN (DSA
> key/parameter generation; DSA signing; SHA-1, MD5 digests; SecureRandom;
> X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
> CertPathBuilder; LDAP, Collection CertStores)
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJSSE 1.42: Sun JSSE
> provider(implements RSA Signatures, PKCS12, SunX509 key/trust factories,
> SSLv3, TLSv1)
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunRsaSign 1.42: SUN's
> provider for RSA signatures
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJCE 1.42: SunJCE
> Provider (implements DES, Triple DES, AES, Blowfish, PBE, Diffie-Hellman,
> HMAC-MD5, HMAC-SHA1)
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJGSS 1.0: Sun (Kerberos
> v5)
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.useragent = Jakarta Commons-HttpClient/3.0-rc4
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.version = HTTP/1.1
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.connection-manager.class = class
> org.apache.commons.httpclient.SimpleHttpConnectionManager
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.cookie-policy = rfc2109
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.element-charset = US-ASCII
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.content-charset = ISO-8859-1
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.method.retry-handler =
> org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@197a37c
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE, dd-MMM-yy
> HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy HH:mm:ss z, EEE,
> dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy HH:mm:ss
> z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE dd-MMM-yy
> HH:mm:ss z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z,
> EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy HH:mm:ss z]
> 2005/10/18 13:28:19:109 CAT [DEBUG] HttpConnection - Open connection to
> proxy.XXXXXXXXX.co.za:3128 <http://proxy.XXXXXXXXX.co.za:3128>
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "CONNECT
> www.verisign.com:443 <http://www.verisign.com:443/> HTTP/1.1"
> 2005/10/18 13:28:19:156 CAT [DEBUG] HttpMethodBase - Adding Host request
> header
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "User-Agent: Jakarta
> Commons-HttpClient/3.0-rc4[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Host:
> www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Proxy-Connection:
> Keep-Alive[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "HTTP/1.0 407 Proxy
> Authentication Required[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Server:
> Squid/2.4.STABLE6[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Mime-Version: 1.0[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Date: Tue, 18 Oct 2005
> 11:27:51 GMT[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Type:
> text/html[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Length:
> 984[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Expires: Tue, 18 Oct 2005
> 11:27:51 GMT[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Squid-Error:
> ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Authenticate: Basic
> realm="Squid proxy-caching web server"[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Cache: MISS from
> neutrino.XXXXXXXXX.co.za[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Connection:
> keep-alive[\r][\n]"
> 2005/10/18 13:28:19:203 CAT [DEBUG] ConnectMethod - CONNECT status code 407
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Supported
> authentication schemes in the order of preference: [ntlm, digest, basic]
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge for
> ntlm authentication scheme not available
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge for
> digest authentication scheme not available
> 2005/10/18 13:28:19:218 CAT [INFO] AuthChallengeProcessor - basic
> authentication scheme selected
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Using
> authentication scheme: basic
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Authorization
> challenge processed
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Proxy
> authentication scope: BASIC 'Squid proxy-caching web
> server'@proxy.XXXXXXXXX.co.za:3128
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodBase - Should NOT close
> connection in response to directive: keep-alive
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpConnection - Connection is locked.
> Call to releaseConnection() ignored.
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Authenticating with
> BASIC 'Squid proxy-caching web server'@proxy.XXXXXXXXX.co.za:3128
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodParams - Credential charset
> not configured, using HTTP element charset
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "CONNECT
> www.verisign.com:443 <http://www.verisign.com:443/> HTTP/1.0"
> 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodBase - Adding Host request
> header
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "User-Agent: Jakarta
> Commons-HttpClient/3.0-rc4[\r][\n]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Connection:
> Keep-Alive[\r][\n]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Authorization: Basic
> d29sbWFydzp0eXRlbndv[\r][\n]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Host:
> www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "[\r][\n]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - Closing the
> connection.
> 2005/10/18 13:28:19:234 CAT [INFO] HttpMethodDirector - I/O exception (
> org.apache.commons.httpclient.NoHttpResponseException) caught when
> processing request: The server www.verisign.com
> <http://www.verisign.com/>failed to respond
> 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> www.verisign.com <http://www.verisign.com/> failed to respond <
> org.apache.commons.httpclient.NoHttpResponseException: The server
> www.verisign.com <http://www.verisign.com/> failed to respond>
> org.apache.commons.httpclient.NoHttpResponseException: The server
> www.verisign.com <http://www.verisign.com/> failed to respond
> at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(
> HttpMethodBase.java:1835)
> at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> HttpMethodBase.java:1590)
> at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java
> :995)
> at org.apache.commons.httpclient.ConnectMethod.execute(ConnectMethod.java
> :144)
> at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> HttpMethodDirector.java:487)
> at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> HttpMethodDirector.java:388)
> at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> HttpMethodDirector.java:170)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java
> :396)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java
> :324)
> at com.XXXXXXXXX.webagent.TestCase.simplestTest(TestCase.java:43)
> at com.XXXXXXXXX.webagent.TestCase.main(TestCase.java:21)
> On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
>
> > On Tue, Oct 18, 2005 at 02:30:24PM +0200, Wynand wrote:
> > > Hi All,
> > > I've just started using the commons httpclient 3.0 rc4. It works just as
> > > expected, except for connecting to a HTTPS site through a HTTP proxy
> > > (Squid/2.4.STABLE6).
> > > I have tried the the example as per the SSL guide, but to no avail; I
> > get
> > > the following error :
> > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> > > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> > failed to respond <
> > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> > failed to respond>
> > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> > failed to respond
> > > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine (
> > > HttpMethodBase.java:1835)
> > > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > > HttpMethodBase.java:1590)
> > > at org.apache.commons.httpclient.HttpMethodBase.execute(
> > HttpMethodBase.java
> > > :995)
> > > at org.apache.commons.httpclient.ConnectMethod.execute (
> > ConnectMethod.java
> > > :144)
> > > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> > > HttpMethodDirector.java:487)
> > > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> > > HttpMethodDirector.java :388)
> > > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > > HttpMethodDirector.java:170)
> > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > HttpClient.java
> > > :396)
> > > at org.apache.commons.httpclient.HttpClient.executeMethod (
> > HttpClient.java
> > > :324)
> > > at com.wolman.webagent.TestCase.simplestTest(TestCase.java:43)
> > > at com.wolman.webagent.TestCase.main(TestCase.java:21)
> > > Just to clarify here is the example i used :
> > > HttpClient httpclient = new HttpClient();
> > > httpclient.getHostConfiguration().setProxy("myproxyhost", 8080);
> > > httpclient.getState().setProxyCredentials("my-proxy-realm", "
> > myproxyhost",
> > > new UsernamePasswordCredentials("my-proxy-username",
> > "my-proxy-password"));
> > > GetMethod httpget = new GetMethod("*https://www.verisign.com/*");
> > > httpclient.executeMethod(httpget);
> > > System.out.println(httpget.getStatusLine().toString());
> > > It works fine if I change the *https* to *http* in the url. Can someone
> > > please confirm that this example is indeed working or if I'm missing
> > > something
> > > Thanks alot
> >
> > Wynand,
> >
> > Please send the complete wire log.
> >
> > Oleg
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: commons-user-help@jakarta.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org
Re: HttpClient 3.0 - Tunneled HTTPS connections through HTTP proxies
Posted by Wynand <wo...@gmail.com>.
Just a small change is made to the example is :
*httpclient.getState().setProxyCredentials(AuthScope.ANY, new
UsernamePasswordCredentials("MyUser", "xxx"));*
maybe it's important, dunno...
On 10/18/05, Wynand <wo...@gmail.com> wrote:
>
> Oleg,
> I may have stumbled on the cause of this problem by accident. I read that
> there is such an option as "http_access deny CONNECT" in the squid
> configuration, and that's exactly what httpclient tries to do when it tries
> to make a ssl connection though a proxy. I don't have access to the squid
> configuration, but that's what I'm guessing the problem is. Your comments
> are appreciated.
> I'm not sure what a wire log is, but here is all the debug info ;-)
> 2005/10/18 13:28:18:828 CAT [DEBUG] HttpClient - Java version: 1.4.2_08
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java vendor: Sun
> Microsystems Inc.
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java class path:
> C:\eclipse\workspace\SimpleWebAgent\bin;C:\Projects\java\lib\jericho-
> html-1.5-dev1.jar;C:\Projects\java\lib\commons-logging-1.0.4.jar
> ;C:\Projects\java\lib\commons-codec-1.3.jar;C:\Projects\java\commons-
> httpclient-3.0-rc4\commons-httpclient-3.0-rc4.jar
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system name:
> Windows 2000
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
> architecture: x86
> 2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system version:
> 5.0
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SUN 1.42: SUN (DSA
> key/parameter generation; DSA signing; SHA-1, MD5 digests; SecureRandom;
> X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
> CertPathBuilder; LDAP, Collection CertStores)
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJSSE 1.42: Sun JSSE
> provider(implements RSA Signatures, PKCS12, SunX509 key/trust factories,
> SSLv3, TLSv1)
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunRsaSign 1.42: SUN's
> provider for RSA signatures
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJCE 1.42: SunJCE
> Provider (implements DES, Triple DES, AES, Blowfish, PBE, Diffie-Hellman,
> HMAC-MD5, HMAC-SHA1)
> 2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJGSS 1.0: Sun
> (Kerberos v5)
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.useragent = Jakarta Commons-HttpClient/3.0-rc4
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.version = HTTP/1.1
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.connection-manager.class = class
> org.apache.commons.httpclient.SimpleHttpConnectionManager
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.cookie-policy = rfc2109
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.element-charset = US-ASCII
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.protocol.content-charset = ISO-8859-1
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.method.retry-handler =
> org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@197a37c
> 2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
> http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE, dd-MMM-yy
> HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy HH:mm:ss z, EEE,
> dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy HH:mm:ss
> z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE dd-MMM-yy
> HH:mm:ss z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z,
> EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy HH:mm:ss z]
> 2005/10/18 13:28:19:109 CAT [DEBUG] HttpConnection - Open connection to
> proxy.XXXXXXXXX.co.za:3128 <http://proxy.xxxxxxxxx.co.za:3128/>
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "CONNECT
> www.verisign.com:443 <http://www.verisign.com:443/> HTTP/1.1"
> 2005/10/18 13:28:19:156 CAT [DEBUG] HttpMethodBase - Adding Host request
> header
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "User-Agent: Jakarta
> Commons-HttpClient/3.0-rc4[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Host:
> www.verisign.com[\r][\n <http://www.verisign.com%5b/r%5D%5B/n>]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Proxy-Connection:
> Keep-Alive[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "HTTP/1.0 407 Proxy
> Authentication Required[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Server:
> Squid/2.4.STABLE6[\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Mime-Version: 1.0
> [\r][\n]"
> 2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Date: Tue, 18 Oct 2005
> 11:27:51 GMT[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Type:
> text/html[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Length:
> 984[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Expires: Tue, 18 Oct 2005
> 11:27:51 GMT[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Squid-Error:
> ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Authenticate: Basic
> realm="Squid proxy-caching web server"[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Cache: MISS from
> neutrino.XXXXXXXXX.co.za[\r][\n]"
> 2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Connection:
> keep-alive[\r][\n]"
> 2005/10/18 13:28:19:203 CAT [DEBUG] ConnectMethod - CONNECT status code
> 407
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Supported
> authentication schemes in the order of preference: [ntlm, digest, basic]
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge for
> ntlm authentication scheme not available
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge for
> digest authentication scheme not available
> 2005/10/18 13:28:19:218 CAT [INFO] AuthChallengeProcessor - basic
> authentication scheme selected
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Using
> authentication scheme: basic
> 2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Authorization
> challenge processed
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Proxy
> authentication scope: BASIC 'Squid proxy-caching web
> server'@proxy.XXXXXXXXX.co.za:3128
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodBase - Should NOT close
> connection in response to directive: keep-alive
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpConnection - Connection is locked.
> Call to releaseConnection() ignored.
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Authenticating
> with BASIC 'Squid proxy-caching web server'@proxy.XXXXXXXXX.co.za:3128
> 2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodParams - Credential charset
> not configured, using HTTP element charset
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "CONNECT
> www.verisign.com:443 <http://www.verisign.com:443/> HTTP/1.0"
> 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodBase - Adding Host request
> header
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "User-Agent: Jakarta
> Commons-HttpClient/3.0-rc4[\r][\n]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Connection:
> Keep-Alive[\r][\n]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Authorization:
> Basic d29sbWFydzp0eXRlbndv[\r][\n]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Host:
> www.verisign.com[\r][\n <http://www.verisign.com%5b/r%5D%5B/n>]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "[\r][\n]"
> 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - Closing the
> connection.
> 2005/10/18 13:28:19:234 CAT [INFO] HttpMethodDirector - I/O exception (
> org.apache.commons.httpclient.NoHttpResponseException) caught when
> processing request: The server www.verisign.com <http://www.verisign.com/>failed to respond
> 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> www.verisign.com <http://www.verisign.com/> failed to respond <
> org.apache.commons.httpclient.NoHttpResponseException: The server
> www.verisign.com <http://www.verisign.com/> failed to respond>
> org.apache.commons.httpclient.NoHttpResponseException: The server
> www.verisign.com <http://www.verisign.com/> failed to respond
> at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(
> HttpMethodBase.java :1835)
> at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> HttpMethodBase.java:1590)
> at org.apache.commons.httpclient.HttpMethodBase.execute(
> HttpMethodBase.java:995)
> at org.apache.commons.httpclient.ConnectMethod.execute (ConnectMethod.java
> :144)
> at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> HttpMethodDirector.java:487)
> at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> HttpMethodDirector.java :388)
> at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> HttpMethodDirector.java:170)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java
> :396)
> at org.apache.commons.httpclient.HttpClient.executeMethod (HttpClient.java
> :324)
> at com.XXXXXXXXX.webagent.TestCase.simplestTest(TestCase.java:43)
> at com.XXXXXXXXX.webagent.TestCase.main(TestCase.java:21)
> On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
>
> > On Tue, Oct 18, 2005 at 02:30:24PM +0200, Wynand wrote:
> > > Hi All,
> > > I've just started using the commons httpclient 3.0 rc4. It works just
> > as
> > > expected, except for connecting to a HTTPS site through a HTTP proxy
> > > (Squid/2.4.STABLE6).
> > > I have tried the the example as per the SSL guide, but to no avail; I
> > get
> > > the following error :
> > > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> > > www.verisign.com <http://www.verisign.com/> <http://www.verisign.com/>
> > failed to respond <
> > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > www.verisign.com <http://www.verisign.com/> <http://www.verisign.com/>
> > failed to respond>
> > > org.apache.commons.httpclient.NoHttpResponseException: The server
> > > www.verisign.com <http://www.verisign.com/> <http://www.verisign.com/>
> > failed to respond
> > > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine (
> > > HttpMethodBase.java:1835)
> > > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > > HttpMethodBase.java:1590)
> > > at org.apache.commons.httpclient.HttpMethodBase.execute(
> > HttpMethodBase.java
> > > :995)
> > > at org.apache.commons.httpclient.ConnectMethod.execute (
> > ConnectMethod.java
> > > :144)
> > > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect (
> > > HttpMethodDirector.java:487)
> > > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> > > HttpMethodDirector.java :388)
> > > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod (
> > > HttpMethodDirector.java:170)
> > > at org.apache.commons.httpclient.HttpClient.executeMethod(
> > HttpClient.java
> > > :396)
> > > at org.apache.commons.httpclient.HttpClient.executeMethod (
> > HttpClient.java
> > > :324)
> > > at com.wolman.webagent.TestCase.simplestTest(TestCase.java:43)
> > > at com.wolman.webagent.TestCase.main(TestCase.java:21)
> > > Just to clarify here is the example i used :
> > > HttpClient httpclient = new HttpClient();
> > > httpclient.getHostConfiguration().setProxy("myproxyhost", 8080);
> > > httpclient.getState().setProxyCredentials("my-proxy-realm", "
> > myproxyhost",
> > > new UsernamePasswordCredentials("my-proxy-username",
> > "my-proxy-password"));
> > > GetMethod httpget = new GetMethod("*https://www.verisign.com/*")<https://www.verisign.com/*%22)>
> > ;
> > > httpclient.executeMethod(httpget);
> > > System.out.println(httpget.getStatusLine ().toString());
> > > It works fine if I change the *https* to *http* in the url. Can
> > someone
> > > please confirm that this example is indeed working or if I'm missing
> > > something
> > > Thanks alot
> >
> > Wynand,
> >
> > Please send the complete wire log.
> >
> > Oleg
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: commons-user-help@jakarta.apache.org
> >
> >
>
Re: HttpClient 3.0 - Tunneled HTTPS connections through HTTP proxies
Posted by Wynand <wo...@gmail.com>.
Oleg,
I may have stumbled on the cause of this problem by accident. I read that
there is such an option as "http_access deny CONNECT" in the squid
configuration, and that's exactly what httpclient tries to do when it tries
to make a ssl connection though a proxy. I don't have access to the squid
configuration, but that's what I'm guessing the problem is. Your comments
are appreciated.
I'm not sure what a wire log is, but here is all the debug info ;-)
2005/10/18 13:28:18:828 CAT [DEBUG] HttpClient - Java version: 1.4.2_08
2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java vendor: Sun
Microsystems Inc.
2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Java class path:
C:\eclipse\workspace\SimpleWebAgent\bin;C:\Projects\java\lib\jericho-
html-1.5-dev1.jar;C:\Projects\java\lib\commons-logging-1.0.4.jar
;C:\Projects\java\lib\commons-codec-1.3.jar;C:\Projects\java\commons-
httpclient-3.0-rc4\commons-httpclient-3.0-rc4.jar
2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system name:
Windows 2000
2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system
architecture: x86
2005/10/18 13:28:18:843 CAT [DEBUG] HttpClient - Operating system version:
5.0
2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SUN 1.42: SUN (DSA
key/parameter generation; DSA signing; SHA-1, MD5 digests; SecureRandom;
X.509 certificates; JKS keystore; PKIX CertPathValidator; PKIX
CertPathBuilder; LDAP, Collection CertStores)
2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJSSE 1.42: Sun JSSE
provider(implements RSA Signatures, PKCS12, SunX509 key/trust factories,
SSLv3, TLSv1)
2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunRsaSign 1.42: SUN's
provider for RSA signatures
2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJCE 1.42: SunJCE
Provider (implements DES, Triple DES, AES, Blowfish, PBE, Diffie-Hellman,
HMAC-MD5, HMAC-SHA1)
2005/10/18 13:28:19:000 CAT [DEBUG] HttpClient - SunJGSS 1.0: Sun (Kerberos
v5)
2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
http.useragent = Jakarta Commons-HttpClient/3.0-rc4
2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
http.protocol.version = HTTP/1.1
2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
http.connection-manager.class = class
org.apache.commons.httpclient.SimpleHttpConnectionManager
2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
http.protocol.cookie-policy = rfc2109
2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
http.protocol.element-charset = US-ASCII
2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
http.protocol.content-charset = ISO-8859-1
2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
http.method.retry-handler =
org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@197a37c
2005/10/18 13:28:19:015 CAT [DEBUG] DefaultHttpParams - Set parameter
http.dateparser.patterns = [EEE, dd MMM yyyy HH:mm:ss zzz, EEEE, dd-MMM-yy
HH:mm:ss zzz, EEE MMM d HH:mm:ss yyyy, EEE, dd-MMM-yyyy HH:mm:ss z, EEE,
dd-MMM-yyyy HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM-yyyy HH:mm:ss
z, EEE dd MMM yyyy HH:mm:ss z, EEE dd-MMM-yyyy HH-mm-ss z, EEE dd-MMM-yy
HH:mm:ss z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z,
EEE,dd-MMM-yyyy HH:mm:ss z, EEE, dd-MM-yyyy HH:mm:ss z]
2005/10/18 13:28:19:109 CAT [DEBUG] HttpConnection - Open connection to
proxy.XXXXXXXXX.co.za:3128 <http://proxy.XXXXXXXXX.co.za:3128>
2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "CONNECT
www.verisign.com:443 <http://www.verisign.com:443/> HTTP/1.1"
2005/10/18 13:28:19:156 CAT [DEBUG] HttpMethodBase - Adding Host request
header
2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "User-Agent: Jakarta
Commons-HttpClient/3.0-rc4[\r][\n]"
2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Host:
www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "Proxy-Connection:
Keep-Alive[\r][\n]"
2005/10/18 13:28:19:156 CAT [DEBUG] header - >> "[\r][\n]"
2005/10/18 13:28:19:156 CAT [DEBUG] header - << "HTTP/1.0 407 Proxy
Authentication Required[\r][\n]"
2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Server:
Squid/2.4.STABLE6[\r][\n]"
2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Mime-Version: 1.0[\r][\n]"
2005/10/18 13:28:19:156 CAT [DEBUG] header - << "Date: Tue, 18 Oct 2005
11:27:51 GMT[\r][\n]"
2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Type:
text/html[\r][\n]"
2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Content-Length:
984[\r][\n]"
2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Expires: Tue, 18 Oct 2005
11:27:51 GMT[\r][\n]"
2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Squid-Error:
ERR_CACHE_ACCESS_DENIED 0[\r][\n]"
2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Authenticate: Basic
realm="Squid proxy-caching web server"[\r][\n]"
2005/10/18 13:28:19:187 CAT [DEBUG] header - << "X-Cache: MISS from
neutrino.XXXXXXXXX.co.za[\r][\n]"
2005/10/18 13:28:19:187 CAT [DEBUG] header - << "Proxy-Connection:
keep-alive[\r][\n]"
2005/10/18 13:28:19:203 CAT [DEBUG] ConnectMethod - CONNECT status code 407
2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Supported
authentication schemes in the order of preference: [ntlm, digest, basic]
2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge for
ntlm authentication scheme not available
2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Challenge for
digest authentication scheme not available
2005/10/18 13:28:19:218 CAT [INFO] AuthChallengeProcessor - basic
authentication scheme selected
2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Using
authentication scheme: basic
2005/10/18 13:28:19:218 CAT [DEBUG] AuthChallengeProcessor - Authorization
challenge processed
2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Proxy
authentication scope: BASIC 'Squid proxy-caching web
server'@proxy.XXXXXXXXX.co.za:3128
2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodBase - Should NOT close
connection in response to directive: keep-alive
2005/10/18 13:28:19:218 CAT [DEBUG] HttpConnection - Connection is locked.
Call to releaseConnection() ignored.
2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodDirector - Authenticating with
BASIC 'Squid proxy-caching web server'@proxy.XXXXXXXXX.co.za:3128
2005/10/18 13:28:19:218 CAT [DEBUG] HttpMethodParams - Credential charset
not configured, using HTTP element charset
2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "CONNECT
www.verisign.com:443 <http://www.verisign.com:443/> HTTP/1.0"
2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodBase - Adding Host request
header
2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "User-Agent: Jakarta
Commons-HttpClient/3.0-rc4[\r][\n]"
2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Connection:
Keep-Alive[\r][\n]"
2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Proxy-Authorization: Basic
d29sbWFydzp0eXRlbndv[\r][\n]"
2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "Host:
www.verisign.com[\r][\n <http://www.verisign.com[/r][/n>]"
2005/10/18 13:28:19:234 CAT [DEBUG] header - >> "[\r][\n]"
2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - Closing the
connection.
2005/10/18 13:28:19:234 CAT [INFO] HttpMethodDirector - I/O exception (
org.apache.commons.httpclient.NoHttpResponseException) caught when
processing request: The server www.verisign.com
<http://www.verisign.com/>failed to respond
2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
www.verisign.com <http://www.verisign.com/> failed to respond <
org.apache.commons.httpclient.NoHttpResponseException: The server
www.verisign.com <http://www.verisign.com/> failed to respond>
org.apache.commons.httpclient.NoHttpResponseException: The server
www.verisign.com <http://www.verisign.com/> failed to respond
at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(
HttpMethodBase.java:1835)
at org.apache.commons.httpclient.HttpMethodBase.readResponse(
HttpMethodBase.java:1590)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java
:995)
at org.apache.commons.httpclient.ConnectMethod.execute(ConnectMethod.java
:144)
at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
HttpMethodDirector.java:487)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
HttpMethodDirector.java:388)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
HttpMethodDirector.java:170)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java
:396)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java
:324)
at com.XXXXXXXXX.webagent.TestCase.simplestTest(TestCase.java:43)
at com.XXXXXXXXX.webagent.TestCase.main(TestCase.java:21)
On 10/18/05, Oleg Kalnichevski <ol...@apache.org> wrote:
> On Tue, Oct 18, 2005 at 02:30:24PM +0200, Wynand wrote:
> > Hi All,
> > I've just started using the commons httpclient 3.0 rc4. It works just as
> > expected, except for connecting to a HTTPS site through a HTTP proxy
> > (Squid/2.4.STABLE6).
> > I have tried the the example as per the SSL guide, but to no avail; I
> get
> > the following error :
> > 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> failed to respond <
> > org.apache.commons.httpclient.NoHttpResponseException: The server
> > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> failed to respond>
> > org.apache.commons.httpclient.NoHttpResponseException: The server
> > www.verisign.com <http://www.verisign.com> <http://www.verisign.com/>
> failed to respond
> > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine (
> > HttpMethodBase.java:1835)
> > at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> > HttpMethodBase.java:1590)
> > at org.apache.commons.httpclient.HttpMethodBase.execute(
> HttpMethodBase.java
> > :995)
> > at org.apache.commons.httpclient.ConnectMethod.execute (
> ConnectMethod.java
> > :144)
> > at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> > HttpMethodDirector.java:487)
> > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> > HttpMethodDirector.java :388)
> > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> > HttpMethodDirector.java:170)
> > at org.apache.commons.httpclient.HttpClient.executeMethod(
> HttpClient.java
> > :396)
> > at org.apache.commons.httpclient.HttpClient.executeMethod (
> HttpClient.java
> > :324)
> > at com.wolman.webagent.TestCase.simplestTest(TestCase.java:43)
> > at com.wolman.webagent.TestCase.main(TestCase.java:21)
> > Just to clarify here is the example i used :
> > HttpClient httpclient = new HttpClient();
> > httpclient.getHostConfiguration().setProxy("myproxyhost", 8080);
> > httpclient.getState().setProxyCredentials("my-proxy-realm", "
> myproxyhost",
> > new UsernamePasswordCredentials("my-proxy-username",
> "my-proxy-password"));
> > GetMethod httpget = new GetMethod("*https://www.verisign.com/*");
> > httpclient.executeMethod(httpget);
> > System.out.println(httpget.getStatusLine().toString());
> > It works fine if I change the *https* to *http* in the url. Can someone
> > please confirm that this example is indeed working or if I'm missing
> > something
> > Thanks alot
>
> Wynand,
>
> Please send the complete wire log.
>
> Oleg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
>
Re: HttpClient 3.0 - Tunneled HTTPS connections through HTTP proxies
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, Oct 18, 2005 at 02:30:24PM +0200, Wynand wrote:
> Hi All,
> I've just started using the commons httpclient 3.0 rc4. It works just as
> expected, except for connecting to a HTTPS site through a HTTP proxy
> (Squid/2.4.STABLE6).
> I have tried the the example as per the SSL guide, but to no avail; I get
> the following error :
> 2005/10/18 13:28:19:234 CAT [DEBUG] HttpMethodDirector - The server
> www.verisign.com <http://www.verisign.com/> failed to respond <
> org.apache.commons.httpclient.NoHttpResponseException: The server
> www.verisign.com <http://www.verisign.com/> failed to respond>
> org.apache.commons.httpclient.NoHttpResponseException: The server
> www.verisign.com <http://www.verisign.com/> failed to respond
> at org.apache.commons.httpclient.HttpMethodBase.readStatusLine (
> HttpMethodBase.java:1835)
> at org.apache.commons.httpclient.HttpMethodBase.readResponse(
> HttpMethodBase.java:1590)
> at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java
> :995)
> at org.apache.commons.httpclient.ConnectMethod.execute (ConnectMethod.java
> :144)
> at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(
> HttpMethodDirector.java:487)
> at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(
> HttpMethodDirector.java :388)
> at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(
> HttpMethodDirector.java:170)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java
> :396)
> at org.apache.commons.httpclient.HttpClient.executeMethod (HttpClient.java
> :324)
> at com.wolman.webagent.TestCase.simplestTest(TestCase.java:43)
> at com.wolman.webagent.TestCase.main(TestCase.java:21)
> Just to clarify here is the example i used :
> HttpClient httpclient = new HttpClient();
> httpclient.getHostConfiguration().setProxy("myproxyhost", 8080);
> httpclient.getState().setProxyCredentials("my-proxy-realm", " myproxyhost",
> new UsernamePasswordCredentials("my-proxy-username", "my-proxy-password"));
> GetMethod httpget = new GetMethod("*https://www.verisign.com/*");
> httpclient.executeMethod(httpget);
> System.out.println(httpget.getStatusLine().toString());
> It works fine if I change the *https* to *http* in the url. Can someone
> please confirm that this example is indeed working or if I'm missing
> something
> Thanks alot
Wynand,
Please send the complete wire log.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org