You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by "H. Brunsting" <he...@zxcvbnm.nl> on 2010/12/14 12:32:00 UTC
Trying to get a safe multi-threaded implementation, getting I/O
exception (java.net.SocketException) caught when processing request:
Connection reset
Hi,
I'm trying to get a multi-threaded implementation of HttpClient.
Basically i have a number of threads that call getHttpData()
concurrently. It seems to work, but upon starting my threads I get a
bunch of exceptions:
Dec 14, 2010 11:32:17 AM
org.apache.http.impl.client.DefaultRequestDirector execute
INFO: I/O exception (java.net.SocketException) caught when processing
request: Connection reset
Dec 14, 2010 11:32:17 AM
org.apache.http.impl.client.DefaultRequestDirector execute
INFO: Retrying request
After this the threads seem to actually retry and succeed in
establishing a connection, but I'd like to find out what's going wrong
here.
It doesn't happen on every run, that's what makes this difficult to
track down. The program is a server, sometimes I can make dozens of
calls without hitting that exception, then suddenly it will display
errors for a large number of requests -- all on one server instance.
My program works like this:
1 call has about 40 sources.
for(String source : sourceList){
SearchThread searchThread = new SearchThread(name, source, id++);
Thread thread = new Thread(searchThread);
thread.start();
}
SearchThread is a simple Runnable that does data =
HttpClientWrapper.getHttpData(url); in its run() body.
Can anyone shed a light on this? Thanks in advance.
Below is the full code for HttpClientWrapper:
=============================
import java.io.BufferedReader;
public class HttpClientWrapper {
private static HttpClient httpClient;
private static HttpClient createHttpClient(){
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params,
HTTP.DEFAULT_CONTENT_CHARSET);
HttpProtocolParams.setUseExpectContinue(params, true);
HttpConnectionParams.setStaleCheckingEnabled(params, false);
HttpClientParams.setCookiePolicy(params, CookiePolicy.NETSCAPE);
HttpConnectionParams.setConnectionTimeout(params, 15 * 1000);
HttpConnectionParams.setSoTimeout(params, 15 * 1000);
HttpConnectionParams.setSocketBufferSize(params, 8192);
params.setParameter(ConnManagerPNames.MAX_TOTAL_CONNECTIONS, 100);
params.setParameter(ConnManagerPNames.MAX_CONNECTIONS_PER_ROUTE, new
ConnPerRouteBean(100));
SchemeRegistry schReg = new SchemeRegistry();
schReg.register(new Scheme("http",
PlainSocketFactory.getSocketFactory(), 80));
schReg.register(new Scheme("https",
SSLSocketFactory.getSocketFactory(), 443));
ClientConnectionManager conMgr = new
ThreadSafeClientConnManager(params, schReg);
return new DefaultHttpClient(conMgr, params);
}
public static HttpClient getHttpClient(){
if(httpClient == null){
httpClient = createHttpClient();
}
return httpClient;
}
public void shutdown(){
if(httpClient != null &&
httpClient.getConnectionManager() !=null){
httpClient.getConnectionManager().shutdown();
}
}
public static String getHttpData(String url){
HttpGet request;
try {
HttpClient httpClient = getHttpClient();
request = new HttpGet();
request.setURI(new URI(url));
HttpResponse response = httpClient.execute(request);
InputStream input = response.getEntity().getContent();
BufferedReader reader = new BufferedReader(new
InputStreamReader(input), 8192);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
request.abort();
return sb.toString();
} catch (Exception e){
e.printStackTrace();
}
return null;
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Trying to get a safe multi-threaded implementation, getting I/O
exception (java.net.SocketException) caught when processing request:
Connection reset
Posted by "H. Brunsting" <he...@zxcvbnm.nl>.
> You are likely overloading the server with too many concurrent
> connection requests.
>
> Oleg
>
>
Good point, hadn't even thought about that. Will have to investigate, thanks!
Henk
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Trying to get a safe multi-threaded implementation, getting
I/O exception (java.net.SocketException) caught when processing request:
Connection reset
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2010-12-14 at 12:32 +0100, H. Brunsting wrote:
> Hi,
>
> I'm trying to get a multi-threaded implementation of HttpClient.
> Basically i have a number of threads that call getHttpData()
> concurrently. It seems to work, but upon starting my threads I get a
> bunch of exceptions:
>
> Dec 14, 2010 11:32:17 AM
> org.apache.http.impl.client.DefaultRequestDirector execute
> INFO: I/O exception (java.net.SocketException) caught when processing
> request: Connection reset
> Dec 14, 2010 11:32:17 AM
> org.apache.http.impl.client.DefaultRequestDirector execute
> INFO: Retrying request
>
You are likely overloading the server with too many concurrent
connection requests.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org