You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by George Peristeris <ts...@yahoo.gr> on 2004/12/16 16:26:44 UTC

Too many connections in FIN_WAIT_2 state

Hi all,

HttpLient 3.0 beta1
Java 1.4.1_06
Solaris 8 2/02 s28s_u7wos_08a SPARC

I am using the MultiThreadedHttpConnectionManager and albeit I set 
both DefaultMaxConnectionsPerHost and MaxTotalConnections, whenever 
the server I am hitting crashes I see too many TCP sockets in 
FIN_WAIT_2 state.
"Too many" meaning many more than the limits I have set.

I am using httpclient from within a thread to perform a simple GET on

a remote host.I instantiate MultiThreadedHttpConnectionManager and 
HttpClient once , I set the limits for my connections and the
timeouts
and from within its thread's run() I create a new GetMethod which I 
execute using the instance of HttpClient and then I always 
releaseConnection(). Currently , since I am still testing, I do read 
the response but when this goes live I wouldn't mind not reading it
at 
all (in fact I 'd rather not, since all the info I need is the 
response status code).


To sum it up:
1. Does the limit set by MaxTotalConnections(int) guarantee the 
maximum number  of sockets opened by the corresponding 
MultiThreadedHttpConnectionManager?
  -If so can you come up with an explanation of the issue I am
facing?
 (max=40 , netstat shows more than 250 sockets in FIN_WAIT_2 
 whenever the remote host crashes , that is, it is up and runing and
 due to the load I send at some point it crashes and obviously does 
 not send back  the FIN packet the socket on my side is waiting for ,
 so it stays there till it times out.)
 My problem is NOT that the socket is in FIN_WAIT_2, but the fact
that 
 the number of the sockets exceeds the limit I have set. If this is 
 the case my application will eventually run out of file descriptors.
 I don't mind If some threads cannot perform the GET but I do mind if

 the whole application crashes.

  -If not (in this case I have misunderstood what MaxTotalConnections

   is, and a brief explanation would be highly appreciated ) any 
   ideas/hints on how this functionality may be applied are welcome.
  I have thought of adding header "Connection : close", which from 
  what I have read in other threads may solve the problem, but I
think 
  this cancels the whole concept of connection reuse and propably 
  slows things down(?).


2. How would NOT reading the response body (but calling 
   releaseConnection() after each executeMethod() ) affect the whole 
   connection management thing? 

  This is the exception thrown when the remote server crashes and 
  FIN_WAIT_2 sockets start pilling up on my side

 
java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at
java.net.SocketInputStream.read(SocketInputStream.java:129)
        at
java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
        at
java.io.BufferedInputStream.read(BufferedInputStream.java:201)
        at
org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:76)
        at
org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:104)
        at
org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1112)
        at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1379)
        at
org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1825)
        at
org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1588)
        at
org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:999)
        at
org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:382)
        at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:168)
        at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:393)
        at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
        at
com.vodafone.sdp.global.access.authentication.spi.provider.GrCurAccessThread.run(GrCurAccessThread.java:138)
        
GrCurAccessThread.java:138 -> CurResponseStatusCode =
CurHttpClient.executeMethod(CurGetMethod);


Sample Code:

public class GrCurAccessThread extends Thread {
         .
         . 
         .
        static MultiThreadedHttpConnectionManager
multiThreadedHttpConnectionManager = new
MultiThreadedHttpConnectionManager();
        static HttpClient CurHttpClient = new
HttpClient(multiThreadedHttpConnectionManager);
        static {
          //HttpClient 3.0 API
          
multiThreadedHttpConnectionManager.getParams().setDefaultMaxConnectionsPerHost(40);
          
multiThreadedHttpConnectionManager.getParams().setMaxTotalConnections(40);

           //set all timeouts
          
multiThreadedHttpConnectionManager.getParams().setConnectionTimeout(HTTP_CONNECTION_TIMEOUT);
          
CurHttpClient.getParams().setConnectionManagerTimeout(HTTP_CONNECTION_FACTORY_TIMEOUT);
          
multiThreadedHttpConnectionManager.getParams().setSoTimeout(HTTP_SOCKET_TIMEOUT);
        }
        .
        .
        .
   public GrCurAccessThread(...){...}

	public void run(){
	.
	.
	.
	GetMethod CurGetMethod = new GetMethod(GR_CUR_HTTP_API_URL);

        int CurResponseStatusCode = -1 ;

        CurGetMethod.setRequestHeader("my_header1",my_header1_value);
        CurGetMethod.setRequestHeader("my_header2",my_header2_value);
        
        CurGetMethod.setFollowRedirects(false);

        try {
            CurResponseStatusCode =
CurHttpClient.executeMethod(CurGetMethod);
            
           if(logger.isDebugEnabled()){
              String tmp =
(CurGetMethod.getResponseHeader("Content-Length")).getValue();
             
logger.debug("getResponseHeader(\"Content-Length\").getValue() = " +
tmp);
              logger.debug("getResponseBody()=" + new
String(CurGetMethod.getResponseBody())); //}
              Header[] headers = CurGetMethod.getResponseHeaders();
              for (int i = 0; i < headers.length; i++) {
                logger.debug("Response header[" + i + "] :=="
+headers[i].toString());
              }
            }
        .
        .
        .logger.info(....);
        } catch (HttpException recex) { //we actually don't need to
catch this since we won't retry (HttpRecoverableException deprecated)
            .
            .
        } catch (IOException ioex) {...
        } catch (Exception ex){...
        } finally{
          CurGetMethod.releaseConnection();
        .
        .
        . }
        }
}

 thanx for your effort,
 
 george peristeris


P.S> I have seen that the httpClient 3.0 also needs
commons-codec-1.3, and I think it's not mentioned.

____________________________________________________________
Do You Yahoo!?
Αποκτήστε τη δωρεάν @yahoo.gr διεύθυνση σας στο http://www.otenet.gr

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org


Re: Too many connections in FIN_WAIT_2 state

Posted by Guillaume Cottenceau <gc...@mnc.ch>.
George Peristeris <tsinanski 'at' yahoo.gr> writes:

> Hi all,
> 
> HttpLient 3.0 beta1
> Java 1.4.1_06
> Solaris 8 2/02 s28s_u7wos_08a SPARC
> 
> I am using the MultiThreadedHttpConnectionManager and albeit I set 
> both DefaultMaxConnectionsPerHost and MaxTotalConnections, whenever 
> the server I am hitting crashes I see too many TCP sockets in 
> FIN_WAIT_2 state.
> "Too many" meaning many more than the limits I have set.

As far as I know, the FIN_WAIT_2 state really is something that
deals with the kernel of your operating system. Depending on the
TCP packets sent by the client and server programs, it may happen
that nothing but a kernel timeout can "close" a network
connection in the FIN_WAIT_2 state. If you think that this may
cause problems on your machine, you must blame the kernel, not
the programs that opened the network connections.

-- 
Guillaume Cottenceau

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org


Re: Too many connections in FIN_WAIT_2 state

Posted by Ortwin Glück <or...@nose.ch>.

George Peristeris wrote:
> Hi all,
> 
> HttpLient 3.0 beta1
> Java 1.4.1_06
> Solaris 8 2/02 s28s_u7wos_08a SPARC
> 
> I am using the MultiThreadedHttpConnectionManager and albeit I set 
> both DefaultMaxConnectionsPerHost and MaxTotalConnections, whenever 
> the server I am hitting crashes I see too many TCP sockets in 
> FIN_WAIT_2 state.
> "Too many" meaning many more than the limits I have set.

A socket that is in FIN_WAIT_2 state has been released by Java 
(Socket#close() has returned) and for the connection pool this 
connection is closed. So MultiThreadedHttpConnectionManager stays within 
it's limits, but the OS resources might not.

See http://apache.active-venture.com/fin_wait_2.html on what FIN_WAIT_2 
is and how to resolve problems with it.


> To sum it up:
> 1. Does the limit set by MaxTotalConnections(int) guarantee the 
> maximum number  of sockets opened by the corresponding 
> MultiThreadedHttpConnectionManager?

Yes, but only within the scope of what Java is able to see.

>   -If so can you come up with an explanation of the issue I am
> facing?

see above

> 
> 2. How would NOT reading the response body (but calling 
>    releaseConnection() after each executeMethod() ) affect the whole 
>    connection management thing? 

A connection that has it's body not fully consumed is not reusable. The 
body must be consumed before a new request can be issued over the same 
connection. There is no point in returning a connection to the pool 
which has stale data on its input stream. The connection pool checks for 
that either consumes the body behind the scenes or closes the connection.

If you do not read the whole response, please use the abort() on the method.

> 
> P.S> I have seen that the httpClient 3.0 also needs
> commons-codec-1.3, and I think it's not mentioned.

No worries. It is in the Maven Project Descriptor.

     <dependency>
       <id>commons-codec</id>
       <version>1.2</version>
       <url>http://jakarta.apache.org/commons/codec/</url>
     </dependency>

and as such appears here:

http://jakarta.apache.org/commons/httpclient/3.0/dependencies.html

HTH

Ortwin Glück
-- 
  _________________________________________________________________
  NOSE applied intelligence ag

  ortwin glück                      [www]      http://www.nose.ch
  software engineer
  hardturmstrasse 171               [pgp id]           0x81CF3416
  8005 zürich                       [office]      +41-1-277 57 35
  switzerland                       [fax]         +41-1-277 57 12

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org