You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-dev@axis.apache.org by "Vineeth Narayanan (JIRA)" <ji...@apache.org> on 2010/02/22 23:06:28 UTC

[jira] Commented: (AXIS2-2883) CLOSE_WAIT slowly building up over the period of time.

    [ https://issues.apache.org/jira/browse/AXIS2-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836938#action_12836938 ] 

Vineeth Narayanan commented on AXIS2-2883:
------------------------------------------

Hi,

We are trying to upgrade from Axis2 1.4 to Axis2 1.5.1. We are using autogenerated stub from the wsdl using the wsdl2java maven plugin. Our code was running fine using 1.4 version but since upgrading to 1.5.1 we are facing a couple of issues. the approach which we had taken with 1.4 was to create a context once for the object in the constructor using a custom axis2.xml(though this is the same as the axis2.xml which comes with the Axis2 distribution. 

                context = ConfigurationContextFactory.createBasicConfigurationContext("axis2.xml");

We create a stub once using this context and then reuse this stub throughout the life cycle. While initializing the stub, we create an instance of the multithreadedHttpConnectionmanager and set its properties using the params object. we create a httpclient using this manager instance and cache the client in the context with client reuse flag set to true.

                MultiThreadedHttpConnectionManager connectionManager = new MultiThreadedHttpConnectionManager();
                HttpConnectionManagerParams connectionManagerParams = new HttpConnectionManagerParams();
                // Maximum one socket connection to a specific host
                connectionManagerParams.setDefaultMaxConnectionsPerHost(1);
                connectionManagerParams.setTcpNoDelay(true);
                connectionManagerParams.setStaleCheckingEnabled(true);
                connectionManagerParams.setLinger(0);
                connectionManager.setParams(connectionManagerParams);
                HttpClient httpClient = new HttpClient(connectionManager);
                context.setProperty(HTTPConstants.REUSE_HTTP_CLIENT,
                    Boolean.TRUE);
                context.setProperty(HTTPConstants.CACHED_HTTP_CLIENT,
                    httpClient);

                stub = new SoapStub(context,endPoint);

The endPoint is passed as a parameter to the constructor. We then use the stub to call the actual service method to get the response and call the cleanup method of the stub in the finally block. the stub code uses the operationClinet.execute(true) method. All this was runnig fine in 1.4. since migrating to 1.5.1 we first faced the issue where our server just hung after a couple of request as it was uanble to get a connection from the webservice. To overcome this I have tried to create a stub for each request. as well as creating a context for each request. The problem which we now face is that after about 100 requests we get the following error

2010-02-22 13:55:19,734 INFO  org.apache.commons.httpclient.HttpMethodDirector - I/O exception (java.net.BindException) caught when processing request: Address already in use: connect
2010-02-22 13:55:19,734 INFO  org.apache.commons.httpclient.HttpMethodDirector - Retrying request
2010-02-22 13:55:19,734 INFO  org.apache.commons.httpclient.HttpMethodDirector - I/O exception (java.net.BindException) caught when processing request: Address already in use: connect
2010-02-22 13:55:19,734 INFO  org.apache.commons.httpclient.HttpMethodDirector - Retrying request
2010-02-22 13:55:19,734 INFO  org.apache.commons.httpclient.HttpMethodDirector - I/O exception (java.net.BindException) caught when processing request: Address already in use: connect
2010-02-22 13:55:19,734 INFO  org.apache.commons.httpclient.HttpMethodDirector - Retrying request
2010-02-22 13:55:19,734 INFO  org.apache.axis2.transport.http.HTTPSender - Unable to sendViaPost to url[http://uat.mds.gm.cib.intranet.db.com/mds_server_dapdev/MDSSoap]
java.net.BindException: Address already in use: connect
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
	at java.net.Socket.connect(Socket.java:525)
	at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:140)
	at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:125)
	at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
	at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
	at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
	at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
	at org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:542)
	at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:199)
	at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:76)
	at org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:400)
	at org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:225)
	at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:435)
	at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:402)
	at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:229)
	at org.apache.axis2.client.OperationClient.execute(OperationClient.java:165)

We have tried all possible combinations which  have been posted on different forums including setting the Options object but to no avail. We have tried calling the cleanuptransport method as well as calling the httpMehtod.releaseConnection & the Connectionmanager.closeIdleConnections in the finally block. But none of them resolve this issue. We cannot call Connectionmanager.shutdown as we then get the factory is shutdown error in subsequent calls. If we put a gc call in the finally block then all runs fine, but this slows up our application response time drastically. 

Is anybody else facing the same issue since migrating to 1.5.1? 

Since there is an inherent fix in 1.5.1 for the client reuse and CLOSE_WAIT issue, is there something that I need to change in my code? I am a newbie to using Axis2 for webservice.

I am sorry if this is not the correct forum to post this but I have exhausted all possible options to resolve this issue but am having no success. 

> CLOSE_WAIT slowly building up over the period of time.
> ------------------------------------------------------
>
>                 Key: AXIS2-2883
>                 URL: https://issues.apache.org/jira/browse/AXIS2-2883
>             Project: Axis2
>          Issue Type: Bug
>          Components: client-api
>    Affects Versions: 1.1
>         Environment: Operating System : Solaris 
> Axis2 Version : 1.1
> Application Server : weblogic 8.1 SP6
> Using with Cocoon and weblogic DSP
>            Reporter: Lakshmanan Venkatachalam
>            Assignee: Glen Daniels
>            Priority: Critical
>             Fix For: 1.5.1
>
>
> I am experiencing theconstant increase in close wait in the production environment over the period 7 days. 
> We are using Synchronous webservices and we are calling two webservices 24 times every day. We have allocated the maximum of 1.5 GB per application instance and we have two application instances. We are utilizing maximum of 250 - 300 MB in average. So Full GC never runs in our environment.
> It seems like the client API ServiceClient.java is not cleaning up the resources associated with this component. We are creating the new ServiceClient component on every call we have for webservices. Though we have called the cleanup() method at the end of every call to the webservices. At times its not getting executed.
> But when we force garabage collection from the application, it was able to clear all the CLOSE_WAIT components. Since we have similar cleanup() call on finalize() method, it is able to do proper clean up when GC is collecting these objects.
> Forcing GC cannot be a solution, I like to hear from axis2 experts on how we can resolve this problem properly and what could be the cause for this happening.
> Below is our client code for your reference.
> private WebServiceResponse invokeWebservice(OMElement inputElement,
> 			Options options) throws WebServiceInvokerException {
> 		ServiceClient serviceClient = null;
> 		try {
> 			serviceClient = new ServiceClient();
> 			serviceClient.setOptions(options);
> 			// This following line of code is used when we are using
> 			// WS-Addressing. User has to make sure the addressing MAR file in
> 			// class path before enable the following line of code
> 			//
> 			// serviceClient.engageModule(new QName(
> 			// org.apache.axis2.Constants.MODULE_ADDRESSING));
> 			// Invoking synchrounous webservice
> 			//
> 			OMElement result = serviceClient.sendReceive(inputElement);
> 			
> 			OMNode firstOMChild = result.getFirstOMChild();
> 			// Conver the OMelements to XML String
> 			//
> 			Writer stringWriter = new StringWriter();
> 			firstOMChild.serialize(stringWriter);
>             serviceClient.cleanup();
> 			stringWriter.flush();
> 			// Return the Axis2WebserviceResponse
> 			//
> 			return new Axis2WebServiceResponse(stringWriter.toString());
> 		} catch (AxisFault afe) {
> 			throw new WebServiceInvokerException(afe);
> 		} catch (XMLStreamException xse) {
> 			throw new WebServiceInvokerException(xse);
> 		} catch (IOException ioe) {
> 			throw new WebServiceInvokerException(ioe);
> 		} finally {
> 			try {
> 				serviceClient.cleanup();
>                 serviceClient=null;
> 			} catch (AxisFault axisFault) {
> 				//
> 			}
> 		}
> 	}
> }
> options are:
> Options options = new Options();
> 		options.setTo(targetEPR);
> 		options.setUseSeparateListener(false);
> 		options.setAction(wsRequest.getAction());
> 		options.setTimeOutInMilliSeconds(600000);
> 		options.setTransportInProtocol("http");
> 		options.setProperty(org.apache.axis2.context.MessageContextConstants.CHUNKED, org.apache.axis2.transport.http.HTTPConstants.HEADER_TRANSFER_ENCODING);
>  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.