You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Josh Gordineer <jo...@yahoo.com> on 2010/04/20 19:53:24 UTC
Fw: SOCKS proxy example??
I've been trying to implement a SOCKS proxy using Apache HttpClient 4.0.1 and haven't found a straightforward example, most proxy examples focus on http proxies.
I want to access a protected resource that is accessible behind a socks proxy. I can curl the resource like so:
curl --socks4 socks.somewhere.com:1080 http://www.somewhere.com/hidden
However with the following code:
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpHost proxy = new HttpHost("socks.somewhere.com", 1080);
httpclient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
HttpGet httpget = new HttpGet("http://www.somewhere.com/hidden");
HttpResponse response = httpclient.execute(httpget);
System.out.println(response.getStatusLine());
HttpEntity entity = response.getEntity();
if (entity != null) {
entity.consumeContent();
}
httpclient.getConnectionManager().shutdown();
I get an exception:
Exception in thread "main" org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:101)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:210)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:271)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:227)
at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:209)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:292)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:126)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:483)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
I have been able to do something with the JDK Proxy and HttpUrlConnection but haven't gotten anything working in HttpClient.
Any input would be appreciated. I've done a few web searches, but haven't come up with anything definitive.
Thanks in advance..
Josh
Re: Fw: SOCKS proxy example??
Posted by John Smith <de...@gmail.com>.
Last example throw exception. I looked through PlainSocketFactory
source and mixed it with your example. See below:
public class Main {
public static void main(String[] args) throws Exception {
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params, "UTF-8");
HttpProtocolParams.setUseExpectContinue(params, false);
HttpConnectionParams.setConnectionTimeout(params, 3000);
DefaultHttpClient httpclient = new DefaultHttpClient(params);
httpclient.getConnectionManager().getSchemeRegistry().register(
new Scheme("http", new SocksSocketFactory("127.0.0.1",
9050), 80));
HttpHost target = new HttpHost("www.apache.org", 80, "http");
HttpGet req = new HttpGet("/");
System.out.println("executing request to " + target + " via
SOCKS proxy");
HttpResponse rsp = httpclient.execute(target, req);
HttpEntity entity = rsp.getEntity();
System.out.println("----------------------------------------");
System.out.println(rsp.getStatusLine());
Header[] headers = rsp.getAllHeaders();
for (int i = 0; i < headers.length; i++) {
System.out.println(headers[i]);
}
System.out.println("----------------------------------------");
if (entity != null) {
System.out.println(EntityUtils.toString(entity));
}
// When HttpClient instance is no longer needed,
// shut down the connection manager to ensure
// immediate deallocation of all system resources
httpclient.getConnectionManager().shutdown();
}
public static class SocksSocketFactory implements SocketFactory {
String proxyHost;
int proxyPort;
public SocksSocketFactory(String socksHost, int socksPort) {
proxyHost = socksHost;
proxyPort = socksPort;
}
public Socket createSocket() throws IOException {
InetSocketAddress socksaddr = new InetSocketAddress(proxyHost,
proxyPort);
Proxy proxy = new Proxy(Proxy.Type.SOCKS, socksaddr);
Socket sock = new Socket(proxy);
return sock;
}
public Socket connectSocket(Socket sock,
String host,
int port,
InetAddress localAddress,
int localPort,
HttpParams params) throws IOException,
UnknownHostException, ConnectTimeoutException {
if (host == null) {
throw new IllegalArgumentException("Target host may
not be null.");
}
if (params == null) {
throw new IllegalArgumentException("Parameters may not
be null.");
}
if (sock == null) {
sock = createSocket();
}
if ((localAddress != null) || (localPort > 0)) {
// we need to bind explicitly
if (localPort < 0) {
localPort = 0; // indicates "any"
}
InetSocketAddress isa =
new InetSocketAddress(localAddress, localPort);
sock.bind(isa);
}
int timeout = HttpConnectionParams.getConnectionTimeout(params);
InetSocketAddress remoteAddress = new InetSocketAddress(host, port);
try {
sock.connect(remoteAddress, timeout);
} catch (SocketTimeoutException ex) {
throw new ConnectTimeoutException("Connect to " +
remoteAddress + " timed out");
}
return sock;
}
public boolean isSecure(Socket sock) throws IllegalArgumentException {
if (sock == null) {
throw new IllegalArgumentException("Socket may not be null.");
}
// This check is performed last since it calls a method implemented
// by the argument object. getClass() is final in java.lang.Object.
if (sock.isClosed()) {
throw new IllegalArgumentException("Socket is closed.");
}
return false;
}
}
}
I hope it'll be useful.
Thank for your help!
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Fw: SOCKS proxy example??
Posted by John Smith <de...@gmail.com>.
Many thanks, Oleg!
Bow to you. Sorry, that I gave you a lot of bother.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Fw: SOCKS proxy example??
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Fri, 2010-04-23 at 09:52 +0200, John Smith wrote:
> So, I've used Google and done some attempt to write, but wothout any results.
> So, I realized that the best decision is
> set up environment before connection:
>
> System.setProperty("socksProxySet", "true");
> System.setProperty("socksProxyHost", hostname);
> System.setProperty("socksProxyPort", Integer.toString(port));
>
> and clear environment when you don't need to use SOCKS proxy:
>
> System.out.println("Get down socks!");
> System.setProperty("socksProxySet", "false");
> System.setProperty("socksProxyHost", "");
> System.setProperty("socksProxyPort", "");
>
> But it's useful for one thread tasks only.
>
> So, what did you mean in that sentence:
> "Yes, you can use HttpParams to pass some custom configuration parameters
> to your custom SocketFactory and create a Socket instance bound to a
> SOCKS server."
>
> I regret that I don't understand. How can it look in code?
> I think it's rhetorical question.
>
---
public static void main(String[] args)throws Exception {
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter("socks.host", "mysockshost");
httpclient.getParams().setParameter("socks.port", 1234);
httpclient.getConnectionManager().getSchemeRegistry().register(
new Scheme("http", new MySchemeSocketFactory(), 80));
HttpHost target = new HttpHost("www.apache.org", 80, "http");
HttpGet req = new HttpGet("/");
System.out.println("executing request to " + target + " via SOCKS
proxy");
HttpResponse rsp = httpclient.execute(target, req);
HttpEntity entity = rsp.getEntity();
System.out.println("----------------------------------------");
System.out.println(rsp.getStatusLine());
Header[] headers = rsp.getAllHeaders();
for (int i = 0; i<headers.length; i++) {
System.out.println(headers[i]);
}
System.out.println("----------------------------------------");
if (entity != null) {
System.out.println(EntityUtils.toString(entity));
}
// When HttpClient instance is no longer needed,
// shut down the connection manager to ensure
// immediate deallocation of all system resources
httpclient.getConnectionManager().shutdown();
}
static class MySchemeSocketFactory implements SocketFactory {
public Socket createSocket() throws IOException {
return new Socket();
}
public Socket connectSocket(
final Socket socket,
final String host, int port,
final InetAddress localAddress, int localPort,
final HttpParams params)
throws IOException, UnknownHostException,
ConnectTimeoutException {
if (host == null) {
throw new IllegalArgumentException("Host name may not be
null");
}
if (params == null) {
throw new IllegalArgumentException("HTTP parameters may not
be null");
}
String proxyHost = (String) params.getParameter("socks.host");
Integer proxyPort = (Integer) params.getParameter("socks.port");
InetSocketAddress socksaddr = new InetSocketAddress(proxyHost,
proxyPort);
Proxy proxy = new Proxy(Proxy.Type.SOCKS, socksaddr);
Socket sock = new Socket(proxy);
if (localAddress != null) {
InetSocketAddress local = new
InetSocketAddress(localAddress, localPort);
sock.bind(local);
}
int timeout = HttpConnectionParams.getConnectionTimeout(params);
InetSocketAddress remote = new InetSocketAddress(host, port);
try {
sock.connect(remote, timeout);
} catch (SocketTimeoutException ex) {
throw new ConnectTimeoutException("Connect to " +
remote.getHostName() + "/"
+ remote.getAddress() + " timed out");
}
return sock;
}
public boolean isSecure(final Socket sock) throws
IllegalArgumentException {
return false;
}
}
---
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Fw: SOCKS proxy example??
Posted by John Smith <de...@gmail.com>.
So, I've used Google and done some attempt to write, but wothout any results.
So, I realized that the best decision is
set up environment before connection:
System.setProperty("socksProxySet", "true");
System.setProperty("socksProxyHost", hostname);
System.setProperty("socksProxyPort", Integer.toString(port));
and clear environment when you don't need to use SOCKS proxy:
System.out.println("Get down socks!");
System.setProperty("socksProxySet", "false");
System.setProperty("socksProxyHost", "");
System.setProperty("socksProxyPort", "");
But it's useful for one thread tasks only.
So, what did you mean in that sentence:
"Yes, you can use HttpParams to pass some custom configuration parameters
to your custom SocketFactory and create a Socket instance bound to a
SOCKS server."
I regret that I don't understand. How can it look in code?
I think it's rhetorical question.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Fw: SOCKS proxy example??
Posted by John Smith <de...@gmail.com>.
Could you explain some more about using SocketFactory, please?
I try to modify ClientExecuteDirect for using socks proxy.
.........
SchemeRegistry supportedSchemes = new SchemeRegistry();
PlainSocketFactory sf = PlainSocketFactory.getSocketFactory();
// prepare parameters
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params, "UTF-8");
HttpProtocolParams.setUseExpectContinue(params, false);
supportedSchemes.register(new Scheme("http", sf, 80));
String proxyHost = "localhost";
int proxyPort = 1080;
SocketAddress addr = new InetSocketAddress(proxyHost, proxyPort);
Proxy proxy = new Proxy(Proxy.Type.SOCKS, addr);
Socket socket = new Socket(proxy);
try {
sf.connectSocket(socket,proxyHost, proxyPort, null, -1, params);
} catch (IOException ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
ClientConnectionManager connMgr = new
ThreadSafeClientConnManager(params,
supportedSchemes);
DefaultHttpClient httpclient = new DefaultHttpClient(connMgr, params);
HttpGet req = new HttpGet("http://192.168.1.1/");
HttpResponse rsp = null;
try {
rsp = httpclient.execute(req);
} catch (IOException ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
HttpEntity entity = rsp.getEntity();
....
So, I've got 2 different connections. First is connection to socks
proxy and second to http server, but not through proxy. Where is my
mistake?
Sorry, but I couldn't found any examples with using SocketFactory with
custom socket.
Any help will be appreciated!
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Fw: SOCKS proxy example??
Posted by Josh Gordineer <jo...@yahoo.com>.
Thanks you very much Oleg. That works like a charm.
________________________________
From: Oleg Kalnichevski <ol...@apache.org>
To: HttpClient User Discussion <ht...@hc.apache.org>
Sent: Tue, April 20, 2010 3:10:35 PM
Subject: Re: Fw: SOCKS proxy example??
On Tue, 2010-04-20 at 14:52 -0700, Josh Gordineer wrote:
> Thanks Oleg!! Just to finish the thought, would I pass the JDK system parameters 'socksProxyHost' and 'socksProxyPort' as parameters in the HttpParams object in the connectSocket() operation on the SocketFactory?
>
>
Yes, you can use HttpParams to pass some custom configuration parameters
to your custom SocketFactory and create a Socket instance bound to a
SOCKS server.
---
String proxyHost = (String) params.getParameter("my.socks.proxy.host")
Integer portPort = (Integer) params.getParameter("my.socks.proxy.port")
SocketAddress addr = new InetSocketAddress(proxyHost, proxyPort);
Proxy proxy = new Proxy(Proxy.Type.SOCKS, addr);
Socket socket = new Socket(proxy);
InetSocketAddress dest = new InetSocketAddress(host, port);
socket.connect(dest);
---
That is it. Hope this helps.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Fw: SOCKS proxy example??
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2010-04-20 at 14:52 -0700, Josh Gordineer wrote:
> Thanks Oleg!! Just to finish the thought, would I pass the JDK system parameters 'socksProxyHost' and 'socksProxyPort' as parameters in the HttpParams object in the connectSocket() operation on the SocketFactory?
>
>
Yes, you can use HttpParams to pass some custom configuration parameters
to your custom SocketFactory and create a Socket instance bound to a
SOCKS server.
---
String proxyHost = (String) params.getParameter("my.socks.proxy.host")
Integer portPort = (Integer) params.getParameter("my.socks.proxy.port")
SocketAddress addr = new InetSocketAddress(proxyHost, proxyPort);
Proxy proxy = new Proxy(Proxy.Type.SOCKS, addr);
Socket socket = new Socket(proxy);
InetSocketAddress dest = new InetSocketAddress(host, port);
socket.connect(dest);
---
That is it. Hope this helps.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Fw: SOCKS proxy example??
Posted by Josh Gordineer <jo...@yahoo.com>.
Thanks Oleg!! Just to finish the thought, would I pass the JDK system parameters 'socksProxyHost' and 'socksProxyPort' as parameters in the HttpParams object in the connectSocket() operation on the SocketFactory?
________________________________
From: Oleg Kalnichevski <ol...@apache.org>
To: HttpClient User Discussion <ht...@hc.apache.org>
Sent: Tue, April 20, 2010 2:06:00 PM
Subject: Re: Fw: SOCKS proxy example??
You can full control over network socket initialization by using a
custom SocketFactory
http://hc.apache.org/httpcomponents-client-4.0.1/tutorial/html/connmgmt.html#d4e484
http://hc.apache.org/httpcomponents-client-4.0.1/httpclient/apidocs/org/apache/http/conn/scheme/SocketFactory.html
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Fw: SOCKS proxy example??
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2010-04-20 at 13:48 -0700, Josh Gordineer wrote:
> Thanks John! So from this configuration it looks like there is no way to control socks proxy setting on a per request basis through HttpClient? We have a scenario where we let users give proxy configuration information to our system, and we in turn make an HTTP request on their behalf, therefore we can't make a system-wide property setting since we don't know what proxies the user will provide.
>
> Previously in the JDK implementation we were doing:
>
> Proxy.Type type;
> String scheme = # User input;
> if (scheme != null && scheme.equalsIgnoreCase("socks")) {
> type = Proxy.Type.SOCKS;
> } else {
> type = Proxy.Type.HTTP;
> }
>
> String host = # User input
> int port = # User input
>
> InetSocketAddress address = new InetSocketAddress(host, port);
> return new Proxy(type, address);
>
> Then pass that proxy into the httpurlconnection for URI:
>
> URL u = new URL("http://www.somewhere.com/hidden");
> HttpURLConnection ucon = (HttpURLConnection) u.openConnection(p);
>
>
> Now it appears we don't have that option in HttpClient?
>
You can full control over network socket initialization by using a
custom SocketFactory
http://hc.apache.org/httpcomponents-client-4.0.1/tutorial/html/connmgmt.html#d4e484
http://hc.apache.org/httpcomponents-client-4.0.1/httpclient/apidocs/org/apache/http/conn/scheme/SocketFactory.html
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Fw: SOCKS proxy example??
Posted by Josh Gordineer <jo...@yahoo.com>.
Thanks John! So from this configuration it looks like there is no way to control socks proxy setting on a per request basis through HttpClient? We have a scenario where we let users give proxy configuration information to our system, and we in turn make an HTTP request on their behalf, therefore we can't make a system-wide property setting since we don't know what proxies the user will provide.
Previously in the JDK implementation we were doing:
Proxy.Type type;
String scheme = # User input;
if (scheme != null && scheme.equalsIgnoreCase("socks")) {
type = Proxy.Type.SOCKS;
} else {
type = Proxy.Type.HTTP;
}
String host = # User input
int port = # User input
InetSocketAddress address = new InetSocketAddress(host, port);
return new Proxy(type, address);
Then pass that proxy into the httpurlconnection for URI:
URL u = new URL("http://www.somewhere.com/hidden");
HttpURLConnection ucon = (HttpURLConnection) u.openConnection(p);
Now it appears we don't have that option in HttpClient?
Thanks again for the help!
--Josh
________________________________
From: John Smith <de...@gmail.com>
To: HttpClient User Discussion <ht...@hc.apache.org>
Sent: Tue, April 20, 2010 11:44:23 AM
Subject: Re: Fw: SOCKS proxy example??
"Transparent connections through SOCKS proxies (version 4 & 5) using
native Java socket support"
Please look at this - http://hc.apache.org/httpclient-3.x/features.html
You should use something lake that -
System.setProperty("socksProxySet", "false");
System.setProperty("socksProxyHost", "localhost");
System.setProperty("socksProxyPort", "1080");
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Fw: SOCKS proxy example??
Posted by John Smith <de...@gmail.com>.
"Transparent connections through SOCKS proxies (version 4 & 5) using
native Java socket support"
Please look at this - http://hc.apache.org/httpclient-3.x/features.html
You should use something lake that -
System.setProperty("socksProxySet", "false");
System.setProperty("socksProxyHost", "localhost");
System.setProperty("socksProxyPort", "1080");
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org