You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Ken DeLong <ke...@babycenter.com> on 2008/06/27 00:17:11 UTC
Too Many Connections
We have a method in our app that calls httpclient like this:
private HttpClient client = new HttpClient();
public String send(String urlString, String xml)
{
PostMethod postMethod = new PostMethod(urlString);
// Put the XML in the request
try
{
RequestEntity entity = new
StringRequestEntity(xml, "text/xml", "UTF-8");
postMethod.setRequestEntity(entity);
}
catch(UnsupportedEncodingException e1)
{
logger.warn("Couldn't encode the xml?", e1);
return "" +
IContentPublishingService.ACTION_ERROR;
}
// Send the data and get the response
String response = null;
try
{
client.executeMethod(postMethod);
byte[] responseBody =
postMethod.getResponseBody();
response = new String(responseBody);
}
catch(HttpException e)
{
logger.warn("Fatal protocol violation: " +
e.getMessage());
return "" +
IContentPublishingService.ACTION_ERROR;
}
catch(IOException e)
{
logger.warn("Fatal transport error: " +
e.getMessage());
return "" +
IContentPublishingService.ACTION_ERROR;
}
finally
{
// Release the connection.
postMethod.releaseConnection();
}
return response;
}
This method is called rapidly by a single thread in our application.
After 1500 calls or so, we start to see java.net.SocketException, too
many open files. Netstat reveals that there are many sockets in
CLOSE_WAIT and TIME_WAIT states. It seems that httpclient is not
reusing the connection managed by the HttpClient instance, and instead
is closing the connection and creating a new one. This appears to be in
conflict with all the documentation that we could find.
An interesting wrinkle is that the sending and receiving side of the
connection are both on the same machine. The URL is constructed using
the machine's dns name, not "localhost".
------------------------------------------------------------------------
--
Kenneth DeLong
Software Architect
Engineering
BabyCenter, LLC
kdelong@babycenter.com
p: 415.344.7616
AIM: kenwdelong
http://www.babycenter.com
This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
RE: Too Many Connections
Posted by Pete Keyes <PK...@starbucks.com>.
The lock is most likely a threading issue...for that you must use the
threaded connection manager rather than the simple connection manager.
But, again - you may still exhaust all file handles because of o/s
settings.
...Pete
Starbucks Coffee Co.
2601 Utah Ave S.
Seattle, WA. 98134
(w) 206-318-5933
(c) 206-914-4134
-----Original Message-----
From: Ken DeLong [mailto:ken.delong@babycenter.com]
Sent: Friday, June 27, 2008 9:09 AM
To: HttpClient User Discussion
Cc: Stephen Hiley
Subject: RE: Too Many Connections
That didn't help. We changed to
private HttpClient client = new HttpClient(new
SimpleHttpConnectionManager());
But we still see the same behavior.
Turning on the debug logs shows messages like
$ fgrep 'Connection is locked. Call to releaseConnection() ignored.'
server.log | wc -l
176
$ fgrep 'Releasing connection back to connection manager.' server.log |
wc -l
264
What would cause the connection to be locked when releaseConnection() is
called?
-----Original Message-----
From: Pete Keyes [mailto:PKeyes@starbucks.com]
Sent: Thursday, June 26, 2008 3:27 PM
To: HttpClient User Discussion
Cc: Stephen Hiley
Subject: RE: Too Many Connections
You need to use either of these:
- org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
- org.apache.commons.httpclient.SimpleHttpConnectionManager
There are examples of both in the HttpClient site.
In addition to that you may need to tune the O/S to formally release the
CLOSE_WAIT files sooner. There is a delay between when an application
closes a socket and when the O/S can formally release that socket to
avoid orphan packets.
See the documentation for your O/S on setting that value.
...Pete
Starbucks Coffee Co.
2601 Utah Ave S.
Seattle, WA. 98134
(w) 206-318-5933
(c) 206-914-4134
-----Original Message-----
From: Ken DeLong [mailto:ken.delong@babycenter.com]
Sent: Thursday, June 26, 2008 3:17 PM
To: httpclient-users@hc.apache.org
Cc: Stephen Hiley
Subject: Too Many Connections
We have a method in our app that calls httpclient like this:
private HttpClient client = new HttpClient();
public String send(String urlString, String xml)
{
PostMethod postMethod = new PostMethod(urlString);
// Put the XML in the request
try
{
RequestEntity entity = new
StringRequestEntity(xml, "text/xml", "UTF-8");
postMethod.setRequestEntity(entity);
}
catch(UnsupportedEncodingException e1)
{
logger.warn("Couldn't encode the xml?", e1);
return "" +
IContentPublishingService.ACTION_ERROR;
}
// Send the data and get the response
String response = null;
try
{
client.executeMethod(postMethod);
byte[] responseBody =
postMethod.getResponseBody();
response = new String(responseBody);
}
catch(HttpException e)
{
logger.warn("Fatal protocol violation: " +
e.getMessage());
return "" +
IContentPublishingService.ACTION_ERROR;
}
catch(IOException e)
{
logger.warn("Fatal transport error: " +
e.getMessage());
return "" +
IContentPublishingService.ACTION_ERROR;
}
finally
{
// Release the connection.
postMethod.releaseConnection();
}
return response;
}
This method is called rapidly by a single thread in our application.
After 1500 calls or so, we start to see java.net.SocketException, too
many open files. Netstat reveals that there are many sockets in
CLOSE_WAIT and TIME_WAIT states. It seems that httpclient is not
reusing the connection managed by the HttpClient instance, and instead
is closing the connection and creating a new one. This appears to be in
conflict with all the documentation that we could find.
An interesting wrinkle is that the sending and receiving side of the
connection are both on the same machine. The URL is constructed using
the machine's dns name, not "localhost".
------------------------------------------------------------------------
--
Kenneth DeLong
Software Architect
Engineering
BabyCenter, LLC
kdelong@babycenter.com
p: 415.344.7616
AIM: kenwdelong
http://www.babycenter.com
This email message is for the sole use of the intended recipient(s) and
may contain confidential and privileged information. Any unauthorized
review, use, disclosure or distribution is prohibited. If you are not
the intended recipient, please contact the sender by reply email and
destroy all copies of the original message. If you are the intended
recipient, please be advised that the content of this message is subject
to access, review and disclosure by the sender's Email System
Administrator.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Too Many Connections
Posted by Ken DeLong <ke...@babycenter.com>.
I'm sorry. It turns out that the 3rd-party app we are using is also using myriads of HTTP connections and in fact it's THEIR code (non-httpclient) that is throwing the error. We found it by completely commenting out all of our (httpclient) code and the problem still manifests.
Sorry for the wild goose chase.
-----Original Message-----
From: Ken DeLong [mailto:ken.delong@babycenter.com]
Sent: Monday, June 30, 2008 5:51 PM
To: HttpClient User Discussion
Cc: Stephen Hiley
Subject: RE: Too Many Connections
- Switching to the MultiThreadedConnectionManager didn't fix the problem (curiously, the "connection locked" messages went away, tho, but the job still failed)
- turning on httpclient.wire=DEBUG caused it to slow down so much that the problem went away.
Why is it not reusing the connection?
-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]
Sent: Friday, June 27, 2008 1:25 PM
To: HttpClient User Discussion
Subject: RE: Too Many Connections
On Fri, 2008-06-27 at 11:25 -0700, Ken DeLong wrote:
> All of our connections are to the exact same host - in fact, the sending and receiving processes are on the same actual server. The same dns name is used for each request (we use the dns name of the machine, not the string "localhost").
>
Ken,
Try disabling the stale connection check. It can happen, especially with TLS/SSL, that perfectly good connections are reported as stale. You can get a better insight as to why connections get dropped by turning on the context logging. For details see
http://hc.apache.org/httpclient-3.x/logging.html
Oleg
> We'll try the multithreaded connection manager next. (I was pretty sure the calling app was single threaded but I'm not 100% sure).
>
> Thanks!
> Ken
>
>
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Friday, June 27, 2008 11:20 AM
> To: HttpClient User Discussion
> Subject: RE: Too Many Connections
>
> On Fri, 2008-06-27 at 09:08 -0700, Ken DeLong wrote:
> > That didn't help. We changed to
> >
> > private HttpClient client = new HttpClient(new
> > SimpleHttpConnectionManager());
> >
>
> Ken,
>
> SimpleHttpConnectionManager re-uses the underlying connection object
> but it cannot reuse the network socket when connecting to a different
> host. Ii needs to close the old socket and open a new one every time
> it connects to a different host. If you would like to keep alive
> connections to multiple hosts, consider using
> MultiThreadedHttpConnectionManager
>
> Regarding connections in TIME_WAIT state please see
>
> http://wiki.apache.org/HttpComponents/FrequentlyAskedConnectionManagem
> entQuestions
>
> Oleg
>
>
> > But we still see the same behavior.
> >
> > Turning on the debug logs shows messages like
> >
> > $ fgrep 'Connection is locked. Call to releaseConnection() ignored.'
> > server.log | wc -l
> > 176
> >
> > $ fgrep 'Releasing connection back to connection manager.'
> > server.log
> > | wc -l
> > 264
> >
> >
> > What would cause the connection to be locked when
> > releaseConnection() is called?
> >
> >
> >
> > -----Original Message-----
> > From: Pete Keyes [mailto:PKeyes@starbucks.com]
> > Sent: Thursday, June 26, 2008 3:27 PM
> > To: HttpClient User Discussion
> > Cc: Stephen Hiley
> > Subject: RE: Too Many Connections
> >
> > You need to use either of these:
> > - org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
> > - org.apache.commons.httpclient.SimpleHttpConnectionManager
> > There are examples of both in the HttpClient site.
> >
> > In addition to that you may need to tune the O/S to formally release
> > the CLOSE_WAIT files sooner. There is a delay between when an
> > application closes a socket and when the O/S can formally release
> > that socket to avoid orphan packets.
> >
> > See the documentation for your O/S on setting that value.
> >
> >
> > ...Pete
> > Starbucks Coffee Co.
> > 2601 Utah Ave S.
> > Seattle, WA. 98134
> > (w) 206-318-5933
> > (c) 206-914-4134
> >
> > -----Original Message-----
> > From: Ken DeLong [mailto:ken.delong@babycenter.com]
> > Sent: Thursday, June 26, 2008 3:17 PM
> > To: httpclient-users@hc.apache.org
> > Cc: Stephen Hiley
> > Subject: Too Many Connections
> >
> > We have a method in our app that calls httpclient like this:
> >
> > private HttpClient client = new HttpClient();
> >
> > public String send(String urlString, String xml)
> > {
> > PostMethod postMethod = new PostMethod(urlString);
> >
> > // Put the XML in the request
> > try
> > {
> > RequestEntity entity = new
> > StringRequestEntity(xml, "text/xml", "UTF-8");
> > postMethod.setRequestEntity(entity);
> > }
> > catch(UnsupportedEncodingException e1)
> > {
> > logger.warn("Couldn't encode the xml?", e1);
> > return "" +
> > IContentPublishingService.ACTION_ERROR;
> > }
> >
> > // Send the data and get the response
> > String response = null;
> > try
> > {
> > client.executeMethod(postMethod);
> > byte[] responseBody =
> > postMethod.getResponseBody();
> > response = new String(responseBody);
> > }
> > catch(HttpException e)
> > {
> > logger.warn("Fatal protocol violation: " + e.getMessage());
> > return "" +
> > IContentPublishingService.ACTION_ERROR;
> > }
> > catch(IOException e)
> > {
> > logger.warn("Fatal transport error: " + e.getMessage());
> > return "" +
> > IContentPublishingService.ACTION_ERROR;
> > }
> > finally
> > {
> > // Release the connection.
> > postMethod.releaseConnection();
> > }
> >
> > return response;
> > }
> >
> >
> > This method is called rapidly by a single thread in our application.
> > After 1500 calls or so, we start to see java.net.SocketException,
> > too many open files. Netstat reveals that there are many sockets in
> > CLOSE_WAIT and TIME_WAIT states. It seems that httpclient is not
> > reusing the connection managed by the HttpClient instance, and
> > instead is closing the connection and creating a new one. This
> > appears to be in conflict with all the documentation that we could find.
> >
> > An interesting wrinkle is that the sending and receiving side of the
> > connection are both on the same machine. The URL is constructed
> > using the machine's dns name, not "localhost".
> >
> > --------------------------------------------------------------------
> > --
> > --
> > --
> > Kenneth DeLong
> > Software Architect
> > Engineering
> > BabyCenter, LLC
> >
> > kdelong@babycenter.com
> > p: 415.344.7616
> > AIM: kenwdelong
> >
> > http://www.babycenter.com
> >
> >
> >
> > This email message is for the sole use of the intended recipient(s)
> > and may contain confidential and privileged information. Any
> > unauthorized review, use, disclosure or distribution is prohibited.
> > If you are not the intended recipient, please contact the sender by
> > reply email and destroy all copies of the original message. If you
> > are the intended recipient, please be advised that the content of
> > this message is subject to access, review and disclosure by the
> > sender's Email System Administrator.
> >
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Too Many Connections
Posted by Ken DeLong <ke...@babycenter.com>.
- Switching to the MultiThreadedConnectionManager didn't fix the problem (curiously, the "connection locked" messages went away, tho, but the job still failed)
- turning on httpclient.wire=DEBUG caused it to slow down so much that the problem went away.
Why is it not reusing the connection?
-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]
Sent: Friday, June 27, 2008 1:25 PM
To: HttpClient User Discussion
Subject: RE: Too Many Connections
On Fri, 2008-06-27 at 11:25 -0700, Ken DeLong wrote:
> All of our connections are to the exact same host - in fact, the sending and receiving processes are on the same actual server. The same dns name is used for each request (we use the dns name of the machine, not the string "localhost").
>
Ken,
Try disabling the stale connection check. It can happen, especially with TLS/SSL, that perfectly good connections are reported as stale. You can get a better insight as to why connections get dropped by turning on the context logging. For details see
http://hc.apache.org/httpclient-3.x/logging.html
Oleg
> We'll try the multithreaded connection manager next. (I was pretty sure the calling app was single threaded but I'm not 100% sure).
>
> Thanks!
> Ken
>
>
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Friday, June 27, 2008 11:20 AM
> To: HttpClient User Discussion
> Subject: RE: Too Many Connections
>
> On Fri, 2008-06-27 at 09:08 -0700, Ken DeLong wrote:
> > That didn't help. We changed to
> >
> > private HttpClient client = new HttpClient(new
> > SimpleHttpConnectionManager());
> >
>
> Ken,
>
> SimpleHttpConnectionManager re-uses the underlying connection object
> but it cannot reuse the network socket when connecting to a different
> host. Ii needs to close the old socket and open a new one every time
> it connects to a different host. If you would like to keep alive
> connections to multiple hosts, consider using
> MultiThreadedHttpConnectionManager
>
> Regarding connections in TIME_WAIT state please see
>
> http://wiki.apache.org/HttpComponents/FrequentlyAskedConnectionManagem
> entQuestions
>
> Oleg
>
>
> > But we still see the same behavior.
> >
> > Turning on the debug logs shows messages like
> >
> > $ fgrep 'Connection is locked. Call to releaseConnection() ignored.'
> > server.log | wc -l
> > 176
> >
> > $ fgrep 'Releasing connection back to connection manager.'
> > server.log
> > | wc -l
> > 264
> >
> >
> > What would cause the connection to be locked when
> > releaseConnection() is called?
> >
> >
> >
> > -----Original Message-----
> > From: Pete Keyes [mailto:PKeyes@starbucks.com]
> > Sent: Thursday, June 26, 2008 3:27 PM
> > To: HttpClient User Discussion
> > Cc: Stephen Hiley
> > Subject: RE: Too Many Connections
> >
> > You need to use either of these:
> > - org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
> > - org.apache.commons.httpclient.SimpleHttpConnectionManager
> > There are examples of both in the HttpClient site.
> >
> > In addition to that you may need to tune the O/S to formally release
> > the CLOSE_WAIT files sooner. There is a delay between when an
> > application closes a socket and when the O/S can formally release
> > that socket to avoid orphan packets.
> >
> > See the documentation for your O/S on setting that value.
> >
> >
> > ...Pete
> > Starbucks Coffee Co.
> > 2601 Utah Ave S.
> > Seattle, WA. 98134
> > (w) 206-318-5933
> > (c) 206-914-4134
> >
> > -----Original Message-----
> > From: Ken DeLong [mailto:ken.delong@babycenter.com]
> > Sent: Thursday, June 26, 2008 3:17 PM
> > To: httpclient-users@hc.apache.org
> > Cc: Stephen Hiley
> > Subject: Too Many Connections
> >
> > We have a method in our app that calls httpclient like this:
> >
> > private HttpClient client = new HttpClient();
> >
> > public String send(String urlString, String xml)
> > {
> > PostMethod postMethod = new PostMethod(urlString);
> >
> > // Put the XML in the request
> > try
> > {
> > RequestEntity entity = new
> > StringRequestEntity(xml, "text/xml", "UTF-8");
> > postMethod.setRequestEntity(entity);
> > }
> > catch(UnsupportedEncodingException e1)
> > {
> > logger.warn("Couldn't encode the xml?", e1);
> > return "" +
> > IContentPublishingService.ACTION_ERROR;
> > }
> >
> > // Send the data and get the response
> > String response = null;
> > try
> > {
> > client.executeMethod(postMethod);
> > byte[] responseBody =
> > postMethod.getResponseBody();
> > response = new String(responseBody);
> > }
> > catch(HttpException e)
> > {
> > logger.warn("Fatal protocol violation: " + e.getMessage());
> > return "" +
> > IContentPublishingService.ACTION_ERROR;
> > }
> > catch(IOException e)
> > {
> > logger.warn("Fatal transport error: " + e.getMessage());
> > return "" +
> > IContentPublishingService.ACTION_ERROR;
> > }
> > finally
> > {
> > // Release the connection.
> > postMethod.releaseConnection();
> > }
> >
> > return response;
> > }
> >
> >
> > This method is called rapidly by a single thread in our application.
> > After 1500 calls or so, we start to see java.net.SocketException,
> > too many open files. Netstat reveals that there are many sockets in
> > CLOSE_WAIT and TIME_WAIT states. It seems that httpclient is not
> > reusing the connection managed by the HttpClient instance, and
> > instead is closing the connection and creating a new one. This
> > appears to be in conflict with all the documentation that we could find.
> >
> > An interesting wrinkle is that the sending and receiving side of the
> > connection are both on the same machine. The URL is constructed
> > using the machine's dns name, not "localhost".
> >
> > --------------------------------------------------------------------
> > --
> > --
> > --
> > Kenneth DeLong
> > Software Architect
> > Engineering
> > BabyCenter, LLC
> >
> > kdelong@babycenter.com
> > p: 415.344.7616
> > AIM: kenwdelong
> >
> > http://www.babycenter.com
> >
> >
> >
> > This email message is for the sole use of the intended recipient(s)
> > and may contain confidential and privileged information. Any
> > unauthorized review, use, disclosure or distribution is prohibited.
> > If you are not the intended recipient, please contact the sender by
> > reply email and destroy all copies of the original message. If you
> > are the intended recipient, please be advised that the content of
> > this message is subject to access, review and disclosure by the
> > sender's Email System Administrator.
> >
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Too Many Connections
Posted by avin98 <av...@yahoo.com>.
I am also running into the same problem, and start seeing see
java.net.SocketException.
I've set the following properties on my HttpConnection
client = new HttpClient();
HttpClientParams clientParams = client.getParams();
HttpConnectionManagerParams connectionManagerParams =
httpClient.getHttpConnectionManager().getParams();
clientParams.setSoTimeout(_timeout);
connectionManagerParams.setConnectionTimeout(_timeout);
connectionManagerParams.setStaleCheckingEnabled(false);
However adding the Connection close to the header seems to be resolving the
number of open connections.
httppost.addRequestHeader("Connection", "close");
olegk wrote:
>
> On Fri, 2008-06-27 at 11:25 -0700, Ken DeLong wrote:
>> All of our connections are to the exact same host - in fact, the sending
>> and receiving processes are on the same actual server. The same dns name
>> is used for each request (we use the dns name of the machine, not the
>> string "localhost").
>>
>
> Ken,
>
> Try disabling the stale connection check. It can happen, especially with
> TLS/SSL, that perfectly good connections are reported as stale. You can
> get a better insight as to why connections get dropped by turning on the
> context logging. For details see
>
> http://hc.apache.org/httpclient-3.x/logging.html
>
> Oleg
>
>
>> We'll try the multithreaded connection manager next. (I was pretty sure
>> the calling app was single threaded but I'm not 100% sure).
>>
>> Thanks!
>> Ken
>>
>>
>> -----Original Message-----
>> From: Oleg Kalnichevski [mailto:olegk@apache.org]
>> Sent: Friday, June 27, 2008 11:20 AM
>> To: HttpClient User Discussion
>> Subject: RE: Too Many Connections
>>
>> On Fri, 2008-06-27 at 09:08 -0700, Ken DeLong wrote:
>> > That didn't help. We changed to
>> >
>> > private HttpClient client = new HttpClient(new
>> > SimpleHttpConnectionManager());
>> >
>>
>> Ken,
>>
>> SimpleHttpConnectionManager re-uses the underlying connection object but
>> it cannot reuse the network socket when connecting to a different host.
>> Ii needs to close the old socket and open a new one every time it
>> connects to a different host. If you would like to keep alive connections
>> to multiple hosts, consider using MultiThreadedHttpConnectionManager
>>
>> Regarding connections in TIME_WAIT state please see
>>
>> http://wiki.apache.org/HttpComponents/FrequentlyAskedConnectionManagementQuestions
>>
>> Oleg
>>
>>
>> > But we still see the same behavior.
>> >
>> > Turning on the debug logs shows messages like
>> >
>> > $ fgrep 'Connection is locked. Call to releaseConnection() ignored.'
>> > server.log | wc -l
>> > 176
>> >
>> > $ fgrep 'Releasing connection back to connection manager.' server.log
>> > | wc -l
>> > 264
>> >
>> >
>> > What would cause the connection to be locked when releaseConnection()
>> > is called?
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Pete Keyes [mailto:PKeyes@starbucks.com]
>> > Sent: Thursday, June 26, 2008 3:27 PM
>> > To: HttpClient User Discussion
>> > Cc: Stephen Hiley
>> > Subject: RE: Too Many Connections
>> >
>> > You need to use either of these:
>> > - org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
>> > - org.apache.commons.httpclient.SimpleHttpConnectionManager
>> > There are examples of both in the HttpClient site.
>> >
>> > In addition to that you may need to tune the O/S to formally release
>> > the CLOSE_WAIT files sooner. There is a delay between when an
>> > application closes a socket and when the O/S can formally release that
>> > socket to avoid orphan packets.
>> >
>> > See the documentation for your O/S on setting that value.
>> >
>> >
>> > ...Pete
>> > Starbucks Coffee Co.
>> > 2601 Utah Ave S.
>> > Seattle, WA. 98134
>> > (w) 206-318-5933
>> > (c) 206-914-4134
>> >
>> > -----Original Message-----
>> > From: Ken DeLong [mailto:ken.delong@babycenter.com]
>> > Sent: Thursday, June 26, 2008 3:17 PM
>> > To: httpclient-users@hc.apache.org
>> > Cc: Stephen Hiley
>> > Subject: Too Many Connections
>> >
>> > We have a method in our app that calls httpclient like this:
>> >
>> > private HttpClient client = new HttpClient();
>> >
>> > public String send(String urlString, String xml)
>> > {
>> > PostMethod postMethod = new PostMethod(urlString);
>> >
>> > // Put the XML in the request
>> > try
>> > {
>> > RequestEntity entity = new
>> > StringRequestEntity(xml, "text/xml", "UTF-8");
>> > postMethod.setRequestEntity(entity);
>> > }
>> > catch(UnsupportedEncodingException e1)
>> > {
>> > logger.warn("Couldn't encode the xml?", e1);
>> > return "" +
>> > IContentPublishingService.ACTION_ERROR;
>> > }
>> >
>> > // Send the data and get the response
>> > String response = null;
>> > try
>> > {
>> > client.executeMethod(postMethod);
>> > byte[] responseBody =
>> > postMethod.getResponseBody();
>> > response = new String(responseBody);
>> > }
>> > catch(HttpException e)
>> > {
>> > logger.warn("Fatal protocol violation: " + e.getMessage());
>> > return "" +
>> > IContentPublishingService.ACTION_ERROR;
>> > }
>> > catch(IOException e)
>> > {
>> > logger.warn("Fatal transport error: " + e.getMessage());
>> > return "" +
>> > IContentPublishingService.ACTION_ERROR;
>> > }
>> > finally
>> > {
>> > // Release the connection.
>> > postMethod.releaseConnection();
>> > }
>> >
>> > return response;
>> > }
>> >
>> >
>> > This method is called rapidly by a single thread in our application.
>> > After 1500 calls or so, we start to see java.net.SocketException, too
>> > many open files. Netstat reveals that there are many sockets in
>> > CLOSE_WAIT and TIME_WAIT states. It seems that httpclient is not
>> > reusing the connection managed by the HttpClient instance, and instead
>> > is closing the connection and creating a new one. This appears to be
>> > in conflict with all the documentation that we could find.
>> >
>> > An interesting wrinkle is that the sending and receiving side of the
>> > connection are both on the same machine. The URL is constructed using
>> > the machine's dns name, not "localhost".
>> >
>> > ----------------------------------------------------------------------
>> > --
>> > --
>> > Kenneth DeLong
>> > Software Architect
>> > Engineering
>> > BabyCenter, LLC
>> >
>> > kdelong@babycenter.com
>> > p: 415.344.7616
>> > AIM: kenwdelong
>> >
>> > http://www.babycenter.com
>> >
>> >
>> >
>> > This email message is for the sole use of the intended recipient(s)
>> > and may contain confidential and privileged information. Any
>> > unauthorized review, use, disclosure or distribution is prohibited. If
>> > you are not the intended recipient, please contact the sender by reply
>> > email and destroy all copies of the original message. If you are the
>> > intended recipient, please be advised that the content of this message
>> > is subject to access, review and disclosure by the sender's Email
>> > System Administrator.
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
>
--
View this message in context: http://www.nabble.com/Too-Many-Connections-tp18144578p18172951.html
Sent from the HttpClient-User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Too Many Connections
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Fri, 2008-06-27 at 11:25 -0700, Ken DeLong wrote:
> All of our connections are to the exact same host - in fact, the sending and receiving processes are on the same actual server. The same dns name is used for each request (we use the dns name of the machine, not the string "localhost").
>
Ken,
Try disabling the stale connection check. It can happen, especially with
TLS/SSL, that perfectly good connections are reported as stale. You can
get a better insight as to why connections get dropped by turning on the
context logging. For details see
http://hc.apache.org/httpclient-3.x/logging.html
Oleg
> We'll try the multithreaded connection manager next. (I was pretty sure the calling app was single threaded but I'm not 100% sure).
>
> Thanks!
> Ken
>
>
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Friday, June 27, 2008 11:20 AM
> To: HttpClient User Discussion
> Subject: RE: Too Many Connections
>
> On Fri, 2008-06-27 at 09:08 -0700, Ken DeLong wrote:
> > That didn't help. We changed to
> >
> > private HttpClient client = new HttpClient(new
> > SimpleHttpConnectionManager());
> >
>
> Ken,
>
> SimpleHttpConnectionManager re-uses the underlying connection object but it cannot reuse the network socket when connecting to a different host. Ii needs to close the old socket and open a new one every time it connects to a different host. If you would like to keep alive connections to multiple hosts, consider using MultiThreadedHttpConnectionManager
>
> Regarding connections in TIME_WAIT state please see
>
> http://wiki.apache.org/HttpComponents/FrequentlyAskedConnectionManagementQuestions
>
> Oleg
>
>
> > But we still see the same behavior.
> >
> > Turning on the debug logs shows messages like
> >
> > $ fgrep 'Connection is locked. Call to releaseConnection() ignored.'
> > server.log | wc -l
> > 176
> >
> > $ fgrep 'Releasing connection back to connection manager.' server.log
> > | wc -l
> > 264
> >
> >
> > What would cause the connection to be locked when releaseConnection()
> > is called?
> >
> >
> >
> > -----Original Message-----
> > From: Pete Keyes [mailto:PKeyes@starbucks.com]
> > Sent: Thursday, June 26, 2008 3:27 PM
> > To: HttpClient User Discussion
> > Cc: Stephen Hiley
> > Subject: RE: Too Many Connections
> >
> > You need to use either of these:
> > - org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
> > - org.apache.commons.httpclient.SimpleHttpConnectionManager
> > There are examples of both in the HttpClient site.
> >
> > In addition to that you may need to tune the O/S to formally release
> > the CLOSE_WAIT files sooner. There is a delay between when an
> > application closes a socket and when the O/S can formally release that
> > socket to avoid orphan packets.
> >
> > See the documentation for your O/S on setting that value.
> >
> >
> > ...Pete
> > Starbucks Coffee Co.
> > 2601 Utah Ave S.
> > Seattle, WA. 98134
> > (w) 206-318-5933
> > (c) 206-914-4134
> >
> > -----Original Message-----
> > From: Ken DeLong [mailto:ken.delong@babycenter.com]
> > Sent: Thursday, June 26, 2008 3:17 PM
> > To: httpclient-users@hc.apache.org
> > Cc: Stephen Hiley
> > Subject: Too Many Connections
> >
> > We have a method in our app that calls httpclient like this:
> >
> > private HttpClient client = new HttpClient();
> >
> > public String send(String urlString, String xml)
> > {
> > PostMethod postMethod = new PostMethod(urlString);
> >
> > // Put the XML in the request
> > try
> > {
> > RequestEntity entity = new
> > StringRequestEntity(xml, "text/xml", "UTF-8");
> > postMethod.setRequestEntity(entity);
> > }
> > catch(UnsupportedEncodingException e1)
> > {
> > logger.warn("Couldn't encode the xml?", e1);
> > return "" +
> > IContentPublishingService.ACTION_ERROR;
> > }
> >
> > // Send the data and get the response
> > String response = null;
> > try
> > {
> > client.executeMethod(postMethod);
> > byte[] responseBody =
> > postMethod.getResponseBody();
> > response = new String(responseBody);
> > }
> > catch(HttpException e)
> > {
> > logger.warn("Fatal protocol violation: " + e.getMessage());
> > return "" +
> > IContentPublishingService.ACTION_ERROR;
> > }
> > catch(IOException e)
> > {
> > logger.warn("Fatal transport error: " + e.getMessage());
> > return "" +
> > IContentPublishingService.ACTION_ERROR;
> > }
> > finally
> > {
> > // Release the connection.
> > postMethod.releaseConnection();
> > }
> >
> > return response;
> > }
> >
> >
> > This method is called rapidly by a single thread in our application.
> > After 1500 calls or so, we start to see java.net.SocketException, too
> > many open files. Netstat reveals that there are many sockets in
> > CLOSE_WAIT and TIME_WAIT states. It seems that httpclient is not
> > reusing the connection managed by the HttpClient instance, and instead
> > is closing the connection and creating a new one. This appears to be
> > in conflict with all the documentation that we could find.
> >
> > An interesting wrinkle is that the sending and receiving side of the
> > connection are both on the same machine. The URL is constructed using
> > the machine's dns name, not "localhost".
> >
> > ----------------------------------------------------------------------
> > --
> > --
> > Kenneth DeLong
> > Software Architect
> > Engineering
> > BabyCenter, LLC
> >
> > kdelong@babycenter.com
> > p: 415.344.7616
> > AIM: kenwdelong
> >
> > http://www.babycenter.com
> >
> >
> >
> > This email message is for the sole use of the intended recipient(s)
> > and may contain confidential and privileged information. Any
> > unauthorized review, use, disclosure or distribution is prohibited. If
> > you are not the intended recipient, please contact the sender by reply
> > email and destroy all copies of the original message. If you are the
> > intended recipient, please be advised that the content of this message
> > is subject to access, review and disclosure by the sender's Email
> > System Administrator.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Too Many Connections
Posted by Ken DeLong <ke...@babycenter.com>.
All of our connections are to the exact same host - in fact, the sending and receiving processes are on the same actual server. The same dns name is used for each request (we use the dns name of the machine, not the string "localhost").
We'll try the multithreaded connection manager next. (I was pretty sure the calling app was single threaded but I'm not 100% sure).
Thanks!
Ken
-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]
Sent: Friday, June 27, 2008 11:20 AM
To: HttpClient User Discussion
Subject: RE: Too Many Connections
On Fri, 2008-06-27 at 09:08 -0700, Ken DeLong wrote:
> That didn't help. We changed to
>
> private HttpClient client = new HttpClient(new
> SimpleHttpConnectionManager());
>
Ken,
SimpleHttpConnectionManager re-uses the underlying connection object but it cannot reuse the network socket when connecting to a different host. Ii needs to close the old socket and open a new one every time it connects to a different host. If you would like to keep alive connections to multiple hosts, consider using MultiThreadedHttpConnectionManager
Regarding connections in TIME_WAIT state please see
http://wiki.apache.org/HttpComponents/FrequentlyAskedConnectionManagementQuestions
Oleg
> But we still see the same behavior.
>
> Turning on the debug logs shows messages like
>
> $ fgrep 'Connection is locked. Call to releaseConnection() ignored.'
> server.log | wc -l
> 176
>
> $ fgrep 'Releasing connection back to connection manager.' server.log
> | wc -l
> 264
>
>
> What would cause the connection to be locked when releaseConnection()
> is called?
>
>
>
> -----Original Message-----
> From: Pete Keyes [mailto:PKeyes@starbucks.com]
> Sent: Thursday, June 26, 2008 3:27 PM
> To: HttpClient User Discussion
> Cc: Stephen Hiley
> Subject: RE: Too Many Connections
>
> You need to use either of these:
> - org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
> - org.apache.commons.httpclient.SimpleHttpConnectionManager
> There are examples of both in the HttpClient site.
>
> In addition to that you may need to tune the O/S to formally release
> the CLOSE_WAIT files sooner. There is a delay between when an
> application closes a socket and when the O/S can formally release that
> socket to avoid orphan packets.
>
> See the documentation for your O/S on setting that value.
>
>
> ...Pete
> Starbucks Coffee Co.
> 2601 Utah Ave S.
> Seattle, WA. 98134
> (w) 206-318-5933
> (c) 206-914-4134
>
> -----Original Message-----
> From: Ken DeLong [mailto:ken.delong@babycenter.com]
> Sent: Thursday, June 26, 2008 3:17 PM
> To: httpclient-users@hc.apache.org
> Cc: Stephen Hiley
> Subject: Too Many Connections
>
> We have a method in our app that calls httpclient like this:
>
> private HttpClient client = new HttpClient();
>
> public String send(String urlString, String xml)
> {
> PostMethod postMethod = new PostMethod(urlString);
>
> // Put the XML in the request
> try
> {
> RequestEntity entity = new
> StringRequestEntity(xml, "text/xml", "UTF-8");
> postMethod.setRequestEntity(entity);
> }
> catch(UnsupportedEncodingException e1)
> {
> logger.warn("Couldn't encode the xml?", e1);
> return "" +
> IContentPublishingService.ACTION_ERROR;
> }
>
> // Send the data and get the response
> String response = null;
> try
> {
> client.executeMethod(postMethod);
> byte[] responseBody =
> postMethod.getResponseBody();
> response = new String(responseBody);
> }
> catch(HttpException e)
> {
> logger.warn("Fatal protocol violation: " + e.getMessage());
> return "" +
> IContentPublishingService.ACTION_ERROR;
> }
> catch(IOException e)
> {
> logger.warn("Fatal transport error: " + e.getMessage());
> return "" +
> IContentPublishingService.ACTION_ERROR;
> }
> finally
> {
> // Release the connection.
> postMethod.releaseConnection();
> }
>
> return response;
> }
>
>
> This method is called rapidly by a single thread in our application.
> After 1500 calls or so, we start to see java.net.SocketException, too
> many open files. Netstat reveals that there are many sockets in
> CLOSE_WAIT and TIME_WAIT states. It seems that httpclient is not
> reusing the connection managed by the HttpClient instance, and instead
> is closing the connection and creating a new one. This appears to be
> in conflict with all the documentation that we could find.
>
> An interesting wrinkle is that the sending and receiving side of the
> connection are both on the same machine. The URL is constructed using
> the machine's dns name, not "localhost".
>
> ----------------------------------------------------------------------
> --
> --
> Kenneth DeLong
> Software Architect
> Engineering
> BabyCenter, LLC
>
> kdelong@babycenter.com
> p: 415.344.7616
> AIM: kenwdelong
>
> http://www.babycenter.com
>
>
>
> This email message is for the sole use of the intended recipient(s)
> and may contain confidential and privileged information. Any
> unauthorized review, use, disclosure or distribution is prohibited. If
> you are not the intended recipient, please contact the sender by reply
> email and destroy all copies of the original message. If you are the
> intended recipient, please be advised that the content of this message
> is subject to access, review and disclosure by the sender's Email
> System Administrator.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Too Many Connections
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Fri, 2008-06-27 at 09:08 -0700, Ken DeLong wrote:
> That didn't help. We changed to
>
> private HttpClient client = new HttpClient(new
> SimpleHttpConnectionManager());
>
Ken,
SimpleHttpConnectionManager re-uses the underlying connection object
but it cannot reuse the network socket when connecting to a different
host. Ii needs to close the old socket and open a new one every time it
connects to a different host. If you would like to keep alive
connections to multiple hosts, consider
using MultiThreadedHttpConnectionManager
Regarding connections in TIME_WAIT state please see
http://wiki.apache.org/HttpComponents/FrequentlyAskedConnectionManagementQuestions
Oleg
> But we still see the same behavior.
>
> Turning on the debug logs shows messages like
>
> $ fgrep 'Connection is locked. Call to releaseConnection() ignored.'
> server.log | wc -l
> 176
>
> $ fgrep 'Releasing connection back to connection manager.' server.log |
> wc -l
> 264
>
>
> What would cause the connection to be locked when releaseConnection() is
> called?
>
>
>
> -----Original Message-----
> From: Pete Keyes [mailto:PKeyes@starbucks.com]
> Sent: Thursday, June 26, 2008 3:27 PM
> To: HttpClient User Discussion
> Cc: Stephen Hiley
> Subject: RE: Too Many Connections
>
> You need to use either of these:
> - org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
> - org.apache.commons.httpclient.SimpleHttpConnectionManager
> There are examples of both in the HttpClient site.
>
> In addition to that you may need to tune the O/S to formally release the
> CLOSE_WAIT files sooner. There is a delay between when an application
> closes a socket and when the O/S can formally release that socket to
> avoid orphan packets.
>
> See the documentation for your O/S on setting that value.
>
>
> ...Pete
> Starbucks Coffee Co.
> 2601 Utah Ave S.
> Seattle, WA. 98134
> (w) 206-318-5933
> (c) 206-914-4134
>
> -----Original Message-----
> From: Ken DeLong [mailto:ken.delong@babycenter.com]
> Sent: Thursday, June 26, 2008 3:17 PM
> To: httpclient-users@hc.apache.org
> Cc: Stephen Hiley
> Subject: Too Many Connections
>
> We have a method in our app that calls httpclient like this:
>
> private HttpClient client = new HttpClient();
>
> public String send(String urlString, String xml)
> {
> PostMethod postMethod = new PostMethod(urlString);
>
> // Put the XML in the request
> try
> {
> RequestEntity entity = new
> StringRequestEntity(xml, "text/xml", "UTF-8");
> postMethod.setRequestEntity(entity);
> }
> catch(UnsupportedEncodingException e1)
> {
> logger.warn("Couldn't encode the xml?", e1);
> return "" +
> IContentPublishingService.ACTION_ERROR;
> }
>
> // Send the data and get the response
> String response = null;
> try
> {
> client.executeMethod(postMethod);
> byte[] responseBody =
> postMethod.getResponseBody();
> response = new String(responseBody);
> }
> catch(HttpException e)
> {
> logger.warn("Fatal protocol violation: " +
> e.getMessage());
> return "" +
> IContentPublishingService.ACTION_ERROR;
> }
> catch(IOException e)
> {
> logger.warn("Fatal transport error: " +
> e.getMessage());
> return "" +
> IContentPublishingService.ACTION_ERROR;
> }
> finally
> {
> // Release the connection.
> postMethod.releaseConnection();
> }
>
> return response;
> }
>
>
> This method is called rapidly by a single thread in our application.
> After 1500 calls or so, we start to see java.net.SocketException, too
> many open files. Netstat reveals that there are many sockets in
> CLOSE_WAIT and TIME_WAIT states. It seems that httpclient is not
> reusing the connection managed by the HttpClient instance, and instead
> is closing the connection and creating a new one. This appears to be in
> conflict with all the documentation that we could find.
>
> An interesting wrinkle is that the sending and receiving side of the
> connection are both on the same machine. The URL is constructed using
> the machine's dns name, not "localhost".
>
> ------------------------------------------------------------------------
> --
> Kenneth DeLong
> Software Architect
> Engineering
> BabyCenter, LLC
>
> kdelong@babycenter.com
> p: 415.344.7616
> AIM: kenwdelong
>
> http://www.babycenter.com
>
>
>
> This email message is for the sole use of the intended recipient(s) and
> may contain confidential and privileged information. Any unauthorized
> review, use, disclosure or distribution is prohibited. If you are not
> the intended recipient, please contact the sender by reply email and
> destroy all copies of the original message. If you are the intended
> recipient, please be advised that the content of this message is subject
> to access, review and disclosure by the sender's Email System
> Administrator.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Too Many Connections
Posted by Ken DeLong <ke...@babycenter.com>.
That didn't help. We changed to
private HttpClient client = new HttpClient(new
SimpleHttpConnectionManager());
But we still see the same behavior.
Turning on the debug logs shows messages like
$ fgrep 'Connection is locked. Call to releaseConnection() ignored.'
server.log | wc -l
176
$ fgrep 'Releasing connection back to connection manager.' server.log |
wc -l
264
What would cause the connection to be locked when releaseConnection() is
called?
-----Original Message-----
From: Pete Keyes [mailto:PKeyes@starbucks.com]
Sent: Thursday, June 26, 2008 3:27 PM
To: HttpClient User Discussion
Cc: Stephen Hiley
Subject: RE: Too Many Connections
You need to use either of these:
- org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
- org.apache.commons.httpclient.SimpleHttpConnectionManager
There are examples of both in the HttpClient site.
In addition to that you may need to tune the O/S to formally release the
CLOSE_WAIT files sooner. There is a delay between when an application
closes a socket and when the O/S can formally release that socket to
avoid orphan packets.
See the documentation for your O/S on setting that value.
...Pete
Starbucks Coffee Co.
2601 Utah Ave S.
Seattle, WA. 98134
(w) 206-318-5933
(c) 206-914-4134
-----Original Message-----
From: Ken DeLong [mailto:ken.delong@babycenter.com]
Sent: Thursday, June 26, 2008 3:17 PM
To: httpclient-users@hc.apache.org
Cc: Stephen Hiley
Subject: Too Many Connections
We have a method in our app that calls httpclient like this:
private HttpClient client = new HttpClient();
public String send(String urlString, String xml)
{
PostMethod postMethod = new PostMethod(urlString);
// Put the XML in the request
try
{
RequestEntity entity = new
StringRequestEntity(xml, "text/xml", "UTF-8");
postMethod.setRequestEntity(entity);
}
catch(UnsupportedEncodingException e1)
{
logger.warn("Couldn't encode the xml?", e1);
return "" +
IContentPublishingService.ACTION_ERROR;
}
// Send the data and get the response
String response = null;
try
{
client.executeMethod(postMethod);
byte[] responseBody =
postMethod.getResponseBody();
response = new String(responseBody);
}
catch(HttpException e)
{
logger.warn("Fatal protocol violation: " +
e.getMessage());
return "" +
IContentPublishingService.ACTION_ERROR;
}
catch(IOException e)
{
logger.warn("Fatal transport error: " +
e.getMessage());
return "" +
IContentPublishingService.ACTION_ERROR;
}
finally
{
// Release the connection.
postMethod.releaseConnection();
}
return response;
}
This method is called rapidly by a single thread in our application.
After 1500 calls or so, we start to see java.net.SocketException, too
many open files. Netstat reveals that there are many sockets in
CLOSE_WAIT and TIME_WAIT states. It seems that httpclient is not
reusing the connection managed by the HttpClient instance, and instead
is closing the connection and creating a new one. This appears to be in
conflict with all the documentation that we could find.
An interesting wrinkle is that the sending and receiving side of the
connection are both on the same machine. The URL is constructed using
the machine's dns name, not "localhost".
------------------------------------------------------------------------
--
Kenneth DeLong
Software Architect
Engineering
BabyCenter, LLC
kdelong@babycenter.com
p: 415.344.7616
AIM: kenwdelong
http://www.babycenter.com
This email message is for the sole use of the intended recipient(s) and
may contain confidential and privileged information. Any unauthorized
review, use, disclosure or distribution is prohibited. If you are not
the intended recipient, please contact the sender by reply email and
destroy all copies of the original message. If you are the intended
recipient, please be advised that the content of this message is subject
to access, review and disclosure by the sender's Email System
Administrator.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Too Many Connections
Posted by Pete Keyes <PK...@starbucks.com>.
You need to use either of these:
- org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
- org.apache.commons.httpclient.SimpleHttpConnectionManager
There are examples of both in the HttpClient site.
In addition to that you may need to tune the O/S to formally release the
CLOSE_WAIT files sooner. There is a delay between when an application
closes a socket and when the O/S can formally release that socket to
avoid orphan packets.
See the documentation for your O/S on setting that value.
...Pete
Starbucks Coffee Co.
2601 Utah Ave S.
Seattle, WA. 98134
(w) 206-318-5933
(c) 206-914-4134
-----Original Message-----
From: Ken DeLong [mailto:ken.delong@babycenter.com]
Sent: Thursday, June 26, 2008 3:17 PM
To: httpclient-users@hc.apache.org
Cc: Stephen Hiley
Subject: Too Many Connections
We have a method in our app that calls httpclient like this:
private HttpClient client = new HttpClient();
public String send(String urlString, String xml)
{
PostMethod postMethod = new PostMethod(urlString);
// Put the XML in the request
try
{
RequestEntity entity = new
StringRequestEntity(xml, "text/xml", "UTF-8");
postMethod.setRequestEntity(entity);
}
catch(UnsupportedEncodingException e1)
{
logger.warn("Couldn't encode the xml?", e1);
return "" +
IContentPublishingService.ACTION_ERROR;
}
// Send the data and get the response
String response = null;
try
{
client.executeMethod(postMethod);
byte[] responseBody =
postMethod.getResponseBody();
response = new String(responseBody);
}
catch(HttpException e)
{
logger.warn("Fatal protocol violation: " +
e.getMessage());
return "" +
IContentPublishingService.ACTION_ERROR;
}
catch(IOException e)
{
logger.warn("Fatal transport error: " +
e.getMessage());
return "" +
IContentPublishingService.ACTION_ERROR;
}
finally
{
// Release the connection.
postMethod.releaseConnection();
}
return response;
}
This method is called rapidly by a single thread in our application.
After 1500 calls or so, we start to see java.net.SocketException, too
many open files. Netstat reveals that there are many sockets in
CLOSE_WAIT and TIME_WAIT states. It seems that httpclient is not
reusing the connection managed by the HttpClient instance, and instead
is closing the connection and creating a new one. This appears to be in
conflict with all the documentation that we could find.
An interesting wrinkle is that the sending and receiving side of the
connection are both on the same machine. The URL is constructed using
the machine's dns name, not "localhost".
------------------------------------------------------------------------
--
Kenneth DeLong
Software Architect
Engineering
BabyCenter, LLC
kdelong@babycenter.com
p: 415.344.7616
AIM: kenwdelong
http://www.babycenter.com
This email message is for the sole use of the intended recipient(s) and
may contain confidential and privileged information. Any unauthorized
review, use, disclosure or distribution is prohibited. If you are not
the intended recipient, please contact the sender by reply email and
destroy all copies of the original message. If you are the intended
recipient, please be advised that the content of this message is subject
to access, review and disclosure by the sender's Email System
Administrator.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org