You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by David Motes <da...@gmail.com> on 2011/01/11 00:00:08 UTC
Need help with ProxySelectorRoutePlanner
I seem to be missing something. I am trying to use this sample code
from the doc to access a proxy without
configuring the proxy parms.
One can also instruct HttpClient to use standard JRE proxy selector to
obtain proxy information:
DefaultHttpClient httpclient = new DefaultHttpClient();
ProxySelectorRoutePlanner routePlanner = new ProxySelectorRoutePlanner(
httpclient.getConnectionManager().getSchemeRegistry(),
ProxySelector.getDefault());
httpclient.setRoutePlanner(routePlanner);
My program is not an applet, it will eventually be a web start application.
The url is https.
I set up the default browser to use a proxy, then set the Java config
to use a proxy and my application will
not use the proxy.
Is there some other configuration or some other code that I need? Is
it the SSL?
Do I not understand what ProxySelectorRoutePlanner does?
Thanks.
Here is the code:
DefaultHttpClient cli = new DefaultHttpClient();
HttpGet method = new HttpGet(url);
HttpResponse resp = null;
HttpEntity ent = null;
method.addHeader("User-Agent", ServerComm.TheUserAgent);
ProxySelectorRoutePlanner routePlanner = new ProxySelectorRoutePlanner(
cli.getConnectionManager().getSchemeRegistry(),
ProxySelector.getDefault());
cli.setRoutePlanner(routePlanner);
//execute the method
String responseBody = null;
try{
resp = cli.execute(method);
ent = resp.getEntity();
}
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Need help with ProxySelectorRoutePlanner
Posted by David Motes <da...@gmail.com>.
Here is my final report, after doing some research.
The problem with using the system proxy configuration as been
discussed since 2005 with no resolution.
You cannot get the JVM sockets code to use the system
proxy config. It either ignores it or tries to use a socks proxy
instead of the http proxy.
I used a test program which used the
System.setProperty("java.net.useSystemProxies",
"true"). It opened a socket, sent a get to a web server, read the
response, and closed the socket (not using httpclient).
I tested on XP, Vista, windows 7, and a Mac. I set the proxy config in
the Java control panel and then set it to use the browser,system proxy
settings.
The Mac ignores the setting and does not use the proxy no matter how
you configure it. The windows systems either ignore it or try to use a
socks proxy and get hung up.
The Java sockets code does not understand system proxies which is why
the HttpClient does not work when java.net.useSystemProxies is true.
If anyone says it works for them they are using java.net.URLConnection.
URLConnection works the way it is supposed to on windows and Mac.
Changed the test program to get a URL using URLConnection. It used
the proxy in the system config and worked the way it should.
So apparently URLConnection does not end up in the usual Sockets code.
Hopefully this will help someone. Don't waste your time trying to use
java.net.useSystemProxies with Sockets or HttpClient.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Need help with ProxySelectorRoutePlanner
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2011-01-13 at 16:57 -0500, David Motes wrote:
> I probably was not clear about the ProxySelectorRoutePlanner, I took
> that code out.
>
> With ProxySelectorRoutePlanner and
> System.setProperty("java.net.useSystemProxies",
> "true") the routing code looks right based on the trace:
> 2011/01/11 10:58:34:296 EST [DEBUG] SingleClientConnManager - Get
> connection for route
> HttpRoute[{}->http://192.168.1.144:8000->http://www2.mytest.com]
>
> But the java tcp code uses ProxySelector to select a proxy so it ends
> up trying to proxy->proxy->destination.
>
> Without ProxySelectorRoutePlanner I get the following stack trace when
> configured to use proxy:
> java.net.SocksSocketImpl.connect(SocksSocketImpl.java:423)
> <-----NOTE it is in the socks code
> java.net.Socket.connect(Socket.java:518)
> org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:123)
> <-----NOTE connect
> org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:133)
> org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
> org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:108)
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
>
> Note it is in SocksSocketImpl waiting for a response from the socks
> server, will not pull out.
>
> Not configured to use proxy results in this:
> java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:340)
> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
> java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
> java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> <-----NOTE it is in the socks code
Which still seems to suggest a configuration issue.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Need help with ProxySelectorRoutePlanner
Posted by David Motes <da...@gmail.com>.
I probably was not clear about the ProxySelectorRoutePlanner, I took
that code out.
With ProxySelectorRoutePlanner and
System.setProperty("java.net.useSystemProxies",
"true") the routing code looks right based on the trace:
2011/01/11 10:58:34:296 EST [DEBUG] SingleClientConnManager - Get
connection for route
HttpRoute[{}->http://192.168.1.144:8000->http://www2.mytest.com]
But the java tcp code uses ProxySelector to select a proxy so it ends
up trying to proxy->proxy->destination.
Without ProxySelectorRoutePlanner I get the following stack trace when
configured to use proxy:
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:423)
<-----NOTE it is in the socks code
java.net.Socket.connect(Socket.java:518)
org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:123)
<-----NOTE connect
org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:133)
org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:108)
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
Note it is in SocksSocketImpl waiting for a response from the socks
server, will not pull out.
Not configured to use proxy results in this:
java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:340)
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
<-----NOTE it is in the socks code
java.net.Socket.connect(Socket.java:518)
org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:123)
<-----NOTE connect same place as above
org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:133)
org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:108)
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
This is after the connection is made and the request will work ok
This seems to be a problem with Java. I don't understand why other
things work through the proxy as they are supposed to though.
Kind of in a hurry, let me know what is not clear....
Thanks.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Need help with ProxySelectorRoutePlanner
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2011-01-12 at 14:07 -0500, David Motes wrote:
> After spending too much time on ProxySelectorRoutePlanner, here is
> what I have discovered.
>
> Note this is for a Java application not an applet.
>
> If you do not use System.setProperty("java.net.useSystemProxies",
> "true"); in your program Java will NOT use a proxy no matter how it is
> configured.
> I used the Java control panel to set it to use browser settings and
> when that did not work I set the proxy config in the Java control
> panel.
> This code > DefaultHttpClient httpclient = new DefaultHttpClient();
> > ProxySelectorRoutePlanner routePlanner = new ProxySelectorRoutePlanner(
> > httpclient.getConnectionManager().getSchemeRegistry(),
> > ProxySelector.getDefault());
> > httpclient.setRoutePlanner(routePlanner);
> makes no difference either.
>
> If you use System.setProperty("java.net.useSystemProxies", "true"); in
> your program here is what happens.
> If the Java control panel is set to use browser settings it gets the
> proxy settings from the browser. Yeah!
> If the Java control panel is set to use proxy config in the control
> panel is uses that.
>
> The code that loads icons from a server or uses HTMLEditorKit USES the
> proxy and works fine.
>
> Code that uses HTTPClient ends up getting hung up because the low
> level tcp code tries to use a SOCKS proxy not an HTTP proxy.
This might be a configuration issue.
> It connects to the HTTP proxy, sends some kind of request, and waits
> FOREVER for the reply from what it thinks is a socks proxy.
>
> Why does the HTMLEditorKit work the way it is supposed to and the
> HTTPClient does not?
> I would think that eventually they would both get down to the same
> Java tcp code.
>
I double-checked ProxySelectorRoutePlanner code to make sure it
explicitly checked the type of proxy entries and selected HTTP proxies
only.
> I would be willing to look at this some more if anyone has any suggestion.
>
Make a copy of the ProxySelectorRoutePlanner class and add additional
debug logs to capture detailed proxy entries returned by the
ProxySelector
Hope this helps
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Need help with ProxySelectorRoutePlanner
Posted by David Motes <da...@gmail.com>.
After spending too much time on ProxySelectorRoutePlanner, here is
what I have discovered.
Note this is for a Java application not an applet.
If you do not use System.setProperty("java.net.useSystemProxies",
"true"); in your program Java will NOT use a proxy no matter how it is
configured.
I used the Java control panel to set it to use browser settings and
when that did not work I set the proxy config in the Java control
panel.
This code > DefaultHttpClient httpclient = new DefaultHttpClient();
> ProxySelectorRoutePlanner routePlanner = new ProxySelectorRoutePlanner(
> httpclient.getConnectionManager().getSchemeRegistry(),
> ProxySelector.getDefault());
> httpclient.setRoutePlanner(routePlanner);
makes no difference either.
If you use System.setProperty("java.net.useSystemProxies", "true"); in
your program here is what happens.
If the Java control panel is set to use browser settings it gets the
proxy settings from the browser. Yeah!
If the Java control panel is set to use proxy config in the control
panel is uses that.
The code that loads icons from a server or uses HTMLEditorKit USES the
proxy and works fine.
Code that uses HTTPClient ends up getting hung up because the low
level tcp code tries to use a SOCKS proxy not an HTTP proxy.
It connects to the HTTP proxy, sends some kind of request, and waits
FOREVER for the reply from what it thinks is a socks proxy.
Why does the HTMLEditorKit work the way it is supposed to and the
HTTPClient does not?
I would think that eventually they would both get down to the same
Java tcp code.
I would be willing to look at this some more if anyone has any suggestion.
I would really like to be able to get proxy info from the browser as I
would rather not have an end user try to figure out how to config the
proxy parms.
THanks.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Need help with ProxySelectorRoutePlanner
Posted by David Motes <da...@gmail.com>.
Some progress, you need to add this to use the system proxies.
System.setProperty("java.net.useSystemProxies", "true");
I have a jlabel that contains an icon loaded from a web server and it
now goes through the proxy.
It gets the proxy info from the Internet Options configured in browser.
HttpClient seems to hang up when I do a Get. Getting this line only
in the wire trace.
Which looks correct as 192.168.1.144:8000 is the proxy.
2011/01/11 10:58:34:296 EST [DEBUG] SingleClientConnManager - Get
connection for route
HttpRoute[{}->http://192.168.1.144:8000->http://www2.mytest.com]
Am going to mess with this later today, will let you know if I
discover anything...
Thanks
On Tue, Jan 11, 2011 at 8:21 AM, Oleg Kalnichevski <ol...@apache.org> wrote:
> On Mon, 2011-01-10 at 18:00 -0500, David Motes wrote:
>> I seem to be missing something. I am trying to use this sample code
>> from the doc to access a proxy without
>> configuring the proxy parms.
>>
>> One can also instruct HttpClient to use standard JRE proxy selector to
>> obtain proxy information:
>> DefaultHttpClient httpclient = new DefaultHttpClient();
>> ProxySelectorRoutePlanner routePlanner = new ProxySelectorRoutePlanner(
>> httpclient.getConnectionManager().getSchemeRegistry(),
>> ProxySelector.getDefault());
>> httpclient.setRoutePlanner(routePlanner);
>>
>> My program is not an applet, it will eventually be a web start application.
>> The url is https.
>> I set up the default browser to use a proxy, then set the Java config
>> to use a proxy and my application will
>> not use the proxy.
>> Is there some other configuration or some other code that I need? Is
>> it the SSL?
>> Do I not understand what ProxySelectorRoutePlanner does?
>>
>> Thanks.
>>
>
> David
>
> My understanding is that ProxySelector gets pre-configured by the Applet
> container based on the settings of the host browser, but one still needs
> to configure ProxySelector using system properties when running in a
> normal JSE environment.
>
> http://download.oracle.com/javase/6/docs/technotes/guides/net/proxies.html
>
> Hope this helps
>
> Oleg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Need help with ProxySelectorRoutePlanner
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2011-01-10 at 18:00 -0500, David Motes wrote:
> I seem to be missing something. I am trying to use this sample code
> from the doc to access a proxy without
> configuring the proxy parms.
>
> One can also instruct HttpClient to use standard JRE proxy selector to
> obtain proxy information:
> DefaultHttpClient httpclient = new DefaultHttpClient();
> ProxySelectorRoutePlanner routePlanner = new ProxySelectorRoutePlanner(
> httpclient.getConnectionManager().getSchemeRegistry(),
> ProxySelector.getDefault());
> httpclient.setRoutePlanner(routePlanner);
>
> My program is not an applet, it will eventually be a web start application.
> The url is https.
> I set up the default browser to use a proxy, then set the Java config
> to use a proxy and my application will
> not use the proxy.
> Is there some other configuration or some other code that I need? Is
> it the SSL?
> Do I not understand what ProxySelectorRoutePlanner does?
>
> Thanks.
>
David
My understanding is that ProxySelector gets pre-configured by the Applet
container based on the settings of the host browser, but one still needs
to configure ProxySelector using system properties when running in a
normal JSE environment.
http://download.oracle.com/javase/6/docs/technotes/guides/net/proxies.html
Hope this helps
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org