You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@axis.apache.org by WS Researcher <ws...@gmail.com> on 2010/05/28 16:24:13 UTC
Timeout when using non-blocking client
Hi,
I'm using Apache Axis2 1.4 client libraries to send request to a cluster
setup which uses Apache Synapse's load balancing capabilities with Apache
Axis2 endpoints. The client I've written is a non-blocking client and it
continuously generates parallel requests to a web service using separate
threads. The requests are generated at intervals between 0.1 to 0.5 of a
second. The first couple of requests goes through successfully and the
results are received by the client. However, subsequent requests wait and
get timed out after about 30 seconds. The error I get on "OnError" of the
client call back is as follows,
org.apache.axis2.AxisFault: Timeout waiting for connection
at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430)
at
org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:203)
at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:76)
at
org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:400)
at
org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:225)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:435)
at
org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:402)
at
org.apache.axis2.description.OutInAxisOperationClient$NonBlockingInvocationWorker.run(OutInAxisOperation.java:442)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:713)
Caused by: org.apache.commons.httpclient.ConnectionPoolTimeoutException:
Timeout waiting for connection
at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.doGetConnection(MultiThreadedHttpConnectionManager.java:497)
at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.getConnectionWithTimeout(MultiThreadedHttpConnectionManager.java:416)
at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:153)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
at
org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:542)
at
org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:199)
... 9 more
29/05/2010 05:56:34 org.apache.axis2.transport.http.HTTPSender sendViaPost
INFO: Unable to sendViaPost to url[
http://192.168.0.251:8280/axis2/services/TestService]
org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout
waiting for connection
at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.doGetConnection(MultiThreadedHttpConnectionManager.java:497)
at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.getConnectionWithTimeout(MultiThreadedHttpConnectionManager.java:416)
at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:153)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
at
org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:542)
at
org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:199)
at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:76)
at
org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:400)
at
org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:225)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:435)
at
org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:402)
at
org.apache.axis2.description.OutInAxisOperationClient$NonBlockingInvocationWorker.run(OutInAxisOperation.java:442)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:713)
My guess was that this timeout happens at the client and not at the server
(in this case Synapse). Here's the code that I use at the client,
private ServiceClient serviceClient = new ServiceClient();
..
..
ServiceClient serviceClient = new ServiceClient();
Options opts = new Options();
opts.setTo(this.epr);
opts.setAction(this.action);
opts.setTimeOutInMilliSeconds(9000000);
serviceClient.setOptions(opts);
..
..
..
serviceClient.addHeader(createHeader(X, Y, Z));
long total = 0;
try {
ClientSideCallback callback = new ClientSideCallback(A, B, C, D, E);
serviceClient.sendReceiveNonBlocking(this.createPayload(P), callback);
serviceClient.cleanup();
serviceClient.cleanupTransport();
} catch (AxisFault e) {
e.printStackTrace();
} catch (RemoteException e) {
e.printStackTrace();
}
*) What causes the timeout?
*) Have the timeouts got to do with the version of Axis2 client I use?
*) Any problems in the way I've coded the client?
Thanks,
WSR
Re: Timeout when using non-blocking client
Posted by WS Researcher <ws...@gmail.com>.
I got over this problem as I found it I've been using Axis2 1.5.1 client and
not 1.4 as previously reported. I came across this blog post [1] by Amila
which solved the problem.
However, I'm faced with a different issue now. With the above fix I managed
to go past the 2 invocations per client limit and I've set it to 200. After
about 35-40 invocations I don't get any responses for the invocations the
clients make. It seems as if Synapse or the Axis2 endpoints get saturated
with requests (I'm using 5 clients in parallel to generate requests) and
stop responding to requests. After sometime I could see that Synapse and the
endpoints were having 100% idle CPU time, although the clients were
reporting sending requests. I don't get any exceptions or errors reported by
the client when this goes on or when I kill Synapse and Executors after them
being idle for sometime. Maybe the clients aren't sending the requests or
connecting to Synapse? Can anyone think of any reason why this could be
happening.
Thanks,
WSR
On Sat, May 29, 2010 at 12:24 AM, WS Researcher <ws...@gmail.com>wrote:
> Hi,
>
> I'm using Apache Axis2 1.4 client libraries to send request to a cluster
> setup which uses Apache Synapse's load balancing capabilities with Apache
> Axis2 endpoints. The client I've written is a non-blocking client and it
> continuously generates parallel requests to a web service using separate
> threads. The requests are generated at intervals between 0.1 to 0.5 of a
> second. The first couple of requests goes through successfully and the
> results are received by the client. However, subsequent requests wait and
> get timed out after about 30 seconds. The error I get on "OnError" of the
> client call back is as follows,
>
> org.apache.axis2.AxisFault: Timeout waiting for connection
> at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430)
> at
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:203)
> at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:76)
> at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:400)
> at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:225)
> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:435)
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:402)
> at
> org.apache.axis2.description.OutInAxisOperationClient$NonBlockingInvocationWorker.run(OutInAxisOperation.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:713)
> Caused by: org.apache.commons.httpclient.ConnectionPoolTimeoutException:
> Timeout waiting for connection
> at
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.doGetConnection(MultiThreadedHttpConnectionManager.java:497)
> at
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.getConnectionWithTimeout(MultiThreadedHttpConnectionManager.java:416)
> at
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:153)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
> at
> org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:542)
> at
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:199)
> ... 9 more
> 29/05/2010 05:56:34 org.apache.axis2.transport.http.HTTPSender sendViaPost
> INFO: Unable to sendViaPost to url[
> http://192.168.0.251:8280/axis2/services/TestService]
> org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout
> waiting for connection
> at
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.doGetConnection(MultiThreadedHttpConnectionManager.java:497)
> at
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.getConnectionWithTimeout(MultiThreadedHttpConnectionManager.java:416)
> at
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:153)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
> at
> org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:542)
> at
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:199)
> at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:76)
> at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:400)
> at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:225)
> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:435)
> at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:402)
> at
> org.apache.axis2.description.OutInAxisOperationClient$NonBlockingInvocationWorker.run(OutInAxisOperation.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:713)
>
>
> My guess was that this timeout happens at the client and not at the server
> (in this case Synapse). Here's the code that I use at the client,
>
> private ServiceClient serviceClient = new ServiceClient();
> ..
> ..
> ServiceClient serviceClient = new ServiceClient();
> Options opts = new Options();
>
> opts.setTo(this.epr);
> opts.setAction(this.action);
> opts.setTimeOutInMilliSeconds(9000000);
>
> serviceClient.setOptions(opts);
> ..
> ..
> ..
> serviceClient.addHeader(createHeader(X, Y, Z));
>
> long total = 0;
> try {
>
> ClientSideCallback callback = new ClientSideCallback(A, B, C, D, E);
> serviceClient.sendReceiveNonBlocking(this.createPayload(P), callback);
>
>
> serviceClient.cleanup();
> serviceClient.cleanupTransport();
>
>
> } catch (AxisFault e) {
> e.printStackTrace();
> } catch (RemoteException e) {
> e.printStackTrace();
> }
>
> *) What causes the timeout?
> *) Have the timeouts got to do with the version of Axis2 client I use?
> *) Any problems in the way I've coded the client?
>
> Thanks,
>
> WSR