You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-dev@axis.apache.org by "Alexis Midon (JIRA)" <ji...@apache.org> on 2009/02/26 02:37:02 UTC

[jira] Updated: (AXIS2-2931) CLONE -Web Service request loop causes many TIME_WAIT Connections & "BindException: Address already in use: connect"

     [ https://issues.apache.org/jira/browse/AXIS2-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alexis Midon updated AXIS2-2931:
--------------------------------

    Attachment: 0001-AXIS2-2931-AXIS2-3478-Release-the-HTTP-connection.patch

Here is attached a patch for AXIS2-2931 & AXIS2-3478.

## What's in there?
The patch consists in (always) releasing the HTTP connection to avoid connection leak (default behavior), or pool starvation if a connection mangaer is shared across multiple invocations.

The connection can be released safely because the response body is now fully read and wrapped in a ByteArrayInputStream. This avoids passing the connection stream directly and prevents reading from a close stream. (AXIS2-3478)
I don't think that loading the response in memory should be considered as a performance issue, because an xml parser will have to do it at some point anyway.

In that context, the option AUTO_RELEASE_CONNECTION does not male sense anymore.


## Where to apply the patch?

This patch can be seamlessly applied in axis2-1.3 and 1.4. But I guess you guys are working on 1.5 now.
So simply apply it:
 http://svn.eu.apache.org/repos/asf/webservices/commons/trunk/modules/transport/modules/http

## How to test it?

. activate the log category org.apache.commons.httpclient.HttpConnection to DEBUG.
. invoke some service with the ServiceClient
. you will get 2 consecutive log messages saying: 
         "enter HttpConnection.releaseConnection()"
         "Releasing connection back to connection manager."
. commit ;)

## Misc

The commit is also available at github [1]. 
A jar is ready for testing in this public Intalio repo [2].

[1] http://github.com/intalio/axis2/commit/c2a0b92b643713a5efaf18436616d465e2182a23

[2] http://www.intalio.org/public/maven2/org/apache/axis2/axis2-kernel/1.3i3/

> CLONE -Web Service request loop causes many TIME_WAIT Connections & "BindException: Address already in use: connect"
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: AXIS2-2931
>                 URL: https://issues.apache.org/jira/browse/AXIS2-2931
>             Project: Axis 2.0 (Axis2)
>          Issue Type: Bug
>    Affects Versions: 1.1.1, nightly
>         Environment: Windows XP, Tomcat, Axis2 nightly build (also encountered in 1.1.1), JiBX 1.1.3, Microsoft Windows XP [Version 5.1.2600], 126 GB free disk space, 1GB RAM.
>            Reporter: Gerry Holly
>            Priority: Critical
>         Attachments: 0001-AXIS2-2931-AXIS2-3478-Release-the-HTTP-connection.patch
>
>
> I am performing the following:
> - Executing a tight loop calling a singel web service
> - Using the same Stub over and over
> - Creating the Stub with a custom ConfigurationContext (see below)
> - Sending cleanup() to Stub after each message sent (although problem existed even before cleanup() was added)
> After approximately 4000 messages have been sent, an AxisFault is generated caused by the following exception (see below for full stack trace):
> 	java.net.BindException: Address already in use: connect
> It appears that connections from the client to Axis on Tomcat sometimes (but not always) remain in the TIME_WAIT state for several minutes.  There are 1200-1500 TIME_WAIT connections open when the AxisFault is generated.
> If I step through the web service messages in a debugger, the problem does not appear to occur, as the system eventually releases the connections after a minute or two.
> Also, if I insert a System.gc() in the client code each time it sends a message to the web service (the commented out line of code below), the AxisFault does not occur.  I have executed the loop over 60,000 times with the garbage collection included without generating an AxisFault.  There were only 400-700 connections in the TIME_WAIT state at a given time, but performance slowed down to a crawl.
> Here is the code I am using to create the Stub:
> 	HttpConnectionManagerParams connectionManagerParams = new HttpConnectionManagerParams();
> 	connectionManagerParams.setTcpNoDelay(true);
> 	connectionManagerParams.setStaleCheckingEnabled(true);
> 			
> 	MultiThreadedHttpConnectionManager connectionManager = new MultiThreadedHttpConnectionManager();
> 	connectionManager.setParams(connectionManagerParams);
> 	HttpClient httpClient = new HttpClient(connectionManager);
> 			
> 	ConfigurationContext configurationContext = ConfigurationContextFactory.createConfigurationContextFromFileSystem(null, null);
> 	configurationContext.setProperty(HTTPConstants.REUSE_HTTP_CLIENT, Boolean.TRUE);
> 	configurationContext.setProperty(HTTPConstants.CACHED_HTTP_CLIENT, httpClient);
> 			
> 	stub = new MyServiceStub(configurationContext, target);
> 	
> 	boolean success = true;
> 	while(success) {
> 		success = stub.performService(records);
> 		stub.cleanup();
> 		//  System.gc();
> 	}
> Here is the complete stack trace:
> 	Exception in thread "main" org.apache.axis2.AxisFault: Address already in use: connect
> 		at org.apache.axis2.AxisFault.makeFault(AxisFault.java:377)
> 		at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:179)
> 		at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:73)
> 		at org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:310)
> 		at org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:202)
> 		at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:446)
> 		at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:330)
> 		at org.apache.axis2.description.OutInAxisOperationClient.execute(OutInAxisOperation.java:294)
> 		at com.ws.service.MyServiceStub.performService(MyServiceStub.java:300)
> 		at com.ws.client.MyClient.main(MyClient.java:88)
> 	Caused by: java.net.BindException: Address already in use: connect
> 		at java.net.PlainSocketImpl.socketConnect(Native Method)
> 		at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
> 		at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
> 		at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
> 		at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> 		at java.net.Socket.connect(Socket.java:519)
> 		at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 		at java.lang.reflect.Method.invoke(Method.java:585)
> 		at org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:139)
> 		at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:124)
> 		at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706)
> 		at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386)
> 		at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
> 		at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
> 		at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
> 		at org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:558)
> 		at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:176)
> 		... 8 more

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.