You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@axis.apache.org by Vidura Gamini Abhaya <vi...@gmail.com> on 2010/03/01 06:31:02 UTC
Time out problem when using Axis2 and Synapse together
Hi All,
I'm using Axis2 and Synapse together for some experiments. Axis2 ver 1.4 is
used both as a client as well as to host a couple of web services. I use
Synapse (ver 1.2) as a load balancer/request redirector to 2 or more Axis2
instances. Axis2 servers run with a modified axis2-kernel where I've done
changes to AxisEngine and some of the transports. Synapse also uses this
modified kernel. I generate requests which gets load balanced according to
some algorithms I implement in Synapse. I change the arrival rates of the
requests (at Synapse) for various experiments.
I encounter some problems when there's a higher arrival of requests at
synapse (for lower arrival rates this seem to work fine). After some time
into the experiments, some of the requests time out on the client side
complaining there's no response from Synapse. When inspecting Synapse, I
could see that the following exceptions were thrown,
"[HttpClientWorker-4] ERROR ServerHandler IO error submiting response :
Connection is closed
org.apache.http.ConnectionClosedException: Connection is closed
at
org.apache.http.impl.nio.NHttpConnectionBase.assertNotClosed(NHttpConnectionBase.java:238)
at
org.apache.http.impl.nio.DefaultNHttpServerConnection.submitResponse(DefaultNHttpServerConnection.java:200)
at
org.apache.synapse.transport.nhttp.ServerHandler.commitResponse(ServerHandler.java:213)
at
org.apache.synapse.transport.nhttp.HttpCoreNIOSender.sendAsyncResponse(HttpCoreNIOSender.java:416)
at
org.apache.synapse.transport.nhttp.HttpCoreNIOSender.invoke(HttpCoreNIOSender.java:271)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:567)
at org.apache.synapse.core.axis2.Axis2Sender.sendBack(Axis2Sender.java:143)
at
org.apache.synapse.core.axis2.Axis2SynapseEnvironment.send(Axis2SynapseEnvironment.java:219)
at
org.apache.synapse.mediators.builtin.SendMediator.mediate(SendMediator.java:154)
at
org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:58)
at
org.apache.synapse.mediators.filters.OutMediator.mediate(OutMediator.java:60)
at
org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:58)
at
org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:125)
at
org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:176)
at
org.apache.synapse.core.axis2.SynapseCallbackReceiver.handleMessage(SynapseCallbackReceiver.java:344)
at
org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive(SynapseCallbackReceiver.java:139)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:274)
at
org.apache.synapse.transport.nhttp.ClientWorker.run(ClientWorker.java:200)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:713)"
The number of requests synapse handles are logged and strangely that equals
to more than the number of requests sent to Synapse. Investigating further
on this I could see that Synapse had processed the same request more than
once and sometimes, the result hasn't been sent to the client. I'm guessing
that this may be due to retransmission of requests from the client due to
not having any response from synapse.
The Axis2 client is used an asynchronous client and since the timing of the
requests is important, each request is created on a separate thread and use
a newly created serviceClient object inside each thread of execution. After
the non blocking call is done serviceClient.cleanup() and
serviceClient.cleanupTransport() is called. I'm wondering whether the
problems are connected with the version of the client I use (I've seen that
1.4 has some problems) and/or the way I'm using the client. Here's a part of
the code how it's done,
"ServiceClient serviceClient = new ServiceClient();
Options opts = new Options();
opts.setTo(EPR);
opts.setAction(Action);
opts.setTimeOutInMilliSeconds(5400000);
serviceClient.setOptions(opts);
serviceClient.sendReceiveNonBlocking(this.createPayload(parameter),
callback);
...
serviceClient.cleanup();
serviceClient.cleanupTransport();"
Migrating the Axis2 client libraries to 1.5.1 resulted in exceptions being
thrown and requests not reaching Synapse at all. I have a couple of
questions on the above,
1) Could this be due to bugs with the 1.4 version of Axis2?
2) Are there any changes required/suggested in the way the client code
works?
3) Could this be happening due to any problems with Synapse? (I posted this
to the Axis2 mailing list assuming that it might be to do with the client),
in which case please cross post this to the Synapse mailing list, thanks.
Appreciate any help and suggestions you could give in solving this issue.
Thanks,
Vidura
Re: Time out problem when using Axis2 and Synapse together
Posted by Crishantha Nanayakkara <c....@gmail.com>.
Dear Vidura,
Have you rectified this issue? I am having the same issue with Synapse with
Axis 1.4.
Regards
Crishantha
Vidura Gamini Abhaya wrote:
>
> Hi All,
>
> I'm using Axis2 and Synapse together for some experiments. Axis2 ver 1.4
> is
> used both as a client as well as to host a couple of web services. I use
> Synapse (ver 1.2) as a load balancer/request redirector to 2 or more Axis2
> instances. Axis2 servers run with a modified axis2-kernel where I've done
> changes to AxisEngine and some of the transports. Synapse also uses this
> modified kernel. I generate requests which gets load balanced according to
> some algorithms I implement in Synapse. I change the arrival rates of the
> requests (at Synapse) for various experiments.
>
> I encounter some problems when there's a higher arrival of requests at
> synapse (for lower arrival rates this seem to work fine). After some time
> into the experiments, some of the requests time out on the client side
> complaining there's no response from Synapse. When inspecting Synapse, I
> could see that the following exceptions were thrown,
>
> "[HttpClientWorker-4] ERROR ServerHandler IO error submiting response :
> Connection is closed
> org.apache.http.ConnectionClosedException: Connection is closed
> at
> org.apache.http.impl.nio.NHttpConnectionBase.assertNotClosed(NHttpConnectionBase.java:238)
> at
> org.apache.http.impl.nio.DefaultNHttpServerConnection.submitResponse(DefaultNHttpServerConnection.java:200)
> at
> org.apache.synapse.transport.nhttp.ServerHandler.commitResponse(ServerHandler.java:213)
> at
> org.apache.synapse.transport.nhttp.HttpCoreNIOSender.sendAsyncResponse(HttpCoreNIOSender.java:416)
> at
> org.apache.synapse.transport.nhttp.HttpCoreNIOSender.invoke(HttpCoreNIOSender.java:271)
> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:567)
> at
> org.apache.synapse.core.axis2.Axis2Sender.sendBack(Axis2Sender.java:143)
> at
> org.apache.synapse.core.axis2.Axis2SynapseEnvironment.send(Axis2SynapseEnvironment.java:219)
> at
> org.apache.synapse.mediators.builtin.SendMediator.mediate(SendMediator.java:154)
> at
> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:58)
> at
> org.apache.synapse.mediators.filters.OutMediator.mediate(OutMediator.java:60)
> at
> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:58)
> at
> org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:125)
> at
> org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:176)
> at
> org.apache.synapse.core.axis2.SynapseCallbackReceiver.handleMessage(SynapseCallbackReceiver.java:344)
> at
> org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive(SynapseCallbackReceiver.java:139)
> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:274)
> at
> org.apache.synapse.transport.nhttp.ClientWorker.run(ClientWorker.java:200)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:713)"
>
> The number of requests synapse handles are logged and strangely that
> equals
> to more than the number of requests sent to Synapse. Investigating further
> on this I could see that Synapse had processed the same request more than
> once and sometimes, the result hasn't been sent to the client. I'm
> guessing
> that this may be due to retransmission of requests from the client due to
> not having any response from synapse.
>
> The Axis2 client is used an asynchronous client and since the timing of
> the
> requests is important, each request is created on a separate thread and
> use
> a newly created serviceClient object inside each thread of execution.
> After
> the non blocking call is done serviceClient.cleanup() and
> serviceClient.cleanupTransport() is called. I'm wondering whether the
> problems are connected with the version of the client I use (I've seen
> that
> 1.4 has some problems) and/or the way I'm using the client. Here's a part
> of
> the code how it's done,
>
> "ServiceClient serviceClient = new ServiceClient();
> Options opts = new Options();
>
> opts.setTo(EPR);
> opts.setAction(Action);
>
> opts.setTimeOutInMilliSeconds(5400000);
> serviceClient.setOptions(opts);
>
> serviceClient.sendReceiveNonBlocking(this.createPayload(parameter),
> callback);
> ...
> serviceClient.cleanup();
> serviceClient.cleanupTransport();"
>
> Migrating the Axis2 client libraries to 1.5.1 resulted in exceptions being
> thrown and requests not reaching Synapse at all. I have a couple of
> questions on the above,
>
> 1) Could this be due to bugs with the 1.4 version of Axis2?
> 2) Are there any changes required/suggested in the way the client code
> works?
> 3) Could this be happening due to any problems with Synapse? (I posted
> this
> to the Axis2 mailing list assuming that it might be to do with the
> client),
> in which case please cross post this to the Synapse mailing list, thanks.
>
> Appreciate any help and suggestions you could give in solving this issue.
>
> Thanks,
>
> Vidura
>
>
--
View this message in context: http://old.nabble.com/Time-out-problem-when-using-Axis2-and-Synapse-together-tp27740325p30526784.html
Sent from the Axis - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@axis.apache.org
For additional commands, e-mail: java-user-help@axis.apache.org
Re: Time out problem when using Axis2 and Synapse together
Posted by Vidura Gamini Abhaya <vi...@gmail.com>.
Thanks for the suggestions Amila. I will try to make these changes and
see whether it makes a difference. I remember trying out increasing
the listeners on Axis2 and trying it out once (it was sometime ago)
but I will do it properly and see whether this works. When I conducted
the experiments back then I had to change some parameters to get over
the problem (i.e. not to create that much of load on the server).
However, I'd like to find out whether Crishantha had any improvements
to his problem and what changes he made.
Thanks,
Vidura
On Sun, Dec 26, 2010 at 12:33 PM, Amila Suriarachchi
<am...@gmail.com> wrote:
>
>
> On Mon, Mar 1, 2010 at 11:01 AM, Vidura Gamini Abhaya <vi...@gmail.com>
> wrote:
>>
>> Hi All,
>>
>> I'm using Axis2 and Synapse together for some experiments. Axis2 ver 1.4
>> is used both as a client as well as to host a couple of web services. I use
>> Synapse (ver 1.2) as a load balancer/request redirector to 2 or more Axis2
>> instances. Axis2 servers run with a modified axis2-kernel where I've done
>> changes to AxisEngine and some of the transports. Synapse also uses this
>> modified kernel. I generate requests which gets load balanced according to
>> some algorithms I implement in Synapse. I change the arrival rates of the
>> requests (at Synapse) for various experiments.
>>
>> I encounter some problems when there's a higher arrival of requests at
>> synapse (for lower arrival rates this seem to work fine). After some time
>> into the experiments, some of the requests time out on the client side
>> complaining there's no response from Synapse. When inspecting Synapse, I
>> could see that the following exceptions were thrown,
>>
>> "[HttpClientWorker-4] ERROR ServerHandler IO error submiting response :
>> Connection is closed
>> org.apache.http.ConnectionClosedException: Connection is closed
>> at
>> org.apache.http.impl.nio.NHttpConnectionBase.assertNotClosed(NHttpConnectionBase.java:238)
>> at
>> org.apache.http.impl.nio.DefaultNHttpServerConnection.submitResponse(DefaultNHttpServerConnection.java:200)
>> at
>> org.apache.synapse.transport.nhttp.ServerHandler.commitResponse(ServerHandler.java:213)
>> at
>> org.apache.synapse.transport.nhttp.HttpCoreNIOSender.sendAsyncResponse(HttpCoreNIOSender.java:416)
>> at
>> org.apache.synapse.transport.nhttp.HttpCoreNIOSender.invoke(HttpCoreNIOSender.java:271)
>> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:567)
>> at
>> org.apache.synapse.core.axis2.Axis2Sender.sendBack(Axis2Sender.java:143)
>> at
>> org.apache.synapse.core.axis2.Axis2SynapseEnvironment.send(Axis2SynapseEnvironment.java:219)
>> at
>> org.apache.synapse.mediators.builtin.SendMediator.mediate(SendMediator.java:154)
>> at
>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:58)
>> at
>> org.apache.synapse.mediators.filters.OutMediator.mediate(OutMediator.java:60)
>> at
>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:58)
>> at
>> org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:125)
>> at
>> org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:176)
>> at
>> org.apache.synapse.core.axis2.SynapseCallbackReceiver.handleMessage(SynapseCallbackReceiver.java:344)
>> at
>> org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive(SynapseCallbackReceiver.java:139)
>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:274)
>> at
>> org.apache.synapse.transport.nhttp.ClientWorker.run(ClientWorker.java:200)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
>> at java.lang.Thread.run(Thread.java:713)"
>>
>> The number of requests synapse handles are logged and strangely that
>> equals to more than the number of requests sent to Synapse. Investigating
>> further on this I could see that Synapse had processed the same request more
>> than once and sometimes, the result hasn't been sent to the client. I'm
>> guessing that this may be due to retransmission of requests from the client
>> due to not having any response from synapse.
>>
>> The Axis2 client is used an asynchronous client and since the timing of
>> the requests is important, each request is created on a separate thread and
>> use a newly created serviceClient object inside each thread of execution.
>> After the non blocking call is done serviceClient.cleanup() and
>> serviceClient.cleanupTransport() is called. I'm wondering whether the
>> problems are connected with the version of the client I use (I've seen that
>> 1.4 has some problems) and/or the way I'm using the client. Here's a part of
>> the code how it's done,
>
> For synapse issue try to increase the time out value in nhttp properties
> file.
>>
>> "ServiceClient serviceClient = new ServiceClient();
>> Options opts = new Options();
>>
>> opts.setTo(EPR);
>> opts.setAction(Action);
>>
>> opts.setTimeOutInMilliSeconds(5400000);
>> serviceClient.setOptions(opts);
>>
>> serviceClient.sendReceiveNonBlocking(this.createPayload(parameter),
>> callback);
>> ...
>> serviceClient.cleanup();
>> serviceClient.cleanupTransport();"
>
> you can not clean up the transports before getting the response.
>
>>
>> Migrating the Axis2 client libraries to 1.5.1 resulted in exceptions being
>> thrown and requests not reaching Synapse at all. I have a couple of
>> questions on the above,
>
> With axis2 1.5.1 by default you have only 2 connections. Unless you increase
> this value it may gives exceptions for a high load scenarios.
>
> thanks,
> Amila.
>>
>> 1) Could this be due to bugs with the 1.4 version of Axis2?
>> 2) Are there any changes required/suggested in the way the client code
>> works?
>> 3) Could this be happening due to any problems with Synapse? (I posted
>> this to the Axis2 mailing list assuming that it might be to do with the
>> client), in which case please cross post this to the Synapse mailing list,
>> thanks.
>>
>> Appreciate any help and suggestions you could give in solving this issue.
>>
>> Thanks,
>>
>> Vidura
>>
>
>
>
> --
> Amila Suriarachchi
> WSO2 Inc.
> blog: http://amilachinthaka.blogspot.com/
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@axis.apache.org
For additional commands, e-mail: java-user-help@axis.apache.org
Re: Time out problem when using Axis2 and Synapse together
Posted by Vidura Gamini Abhaya <vi...@gmail.com>.
Hi Amila,
> With axis2 1.5.1 by default you have only 2 connections. Unless you increase
> this value it may gives exceptions for a high load scenarios.
Could you please be a bit more specific on the suggested solution. How
should the number of connections be increased?
Furthermore, the initial runs I had the problems (timeouts) with were
using Axis2 version 1.4. Does the possible cause (and solution) apply
to 1.4 as well? or anything else needs to be done?
Thanks,
Vidura
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@axis.apache.org
For additional commands, e-mail: java-user-help@axis.apache.org
Re: Time out problem when using Axis2 and Synapse together
Posted by Thamizh <tc...@yahoo.co.in>.
Hi,
kindly remove my email id from this mailing list.
Regards,
Thamizhannal
--- On Sun, 26/12/10, Amila Suriarachchi <am...@gmail.com> wrote:
From: Amila Suriarachchi <am...@gmail.com>
Subject: Re: Time out problem when using Axis2 and Synapse together
To: java-user@axis.apache.org
Cc: "axis-user" <ax...@ws.apache.org>
Date: Sunday, 26 December, 2010, 7:03 AM
On Mon, Mar 1, 2010 at 11:01 AM, Vidura Gamini Abhaya <vi...@gmail.com> wrote:
Hi All,
I'm using Axis2 and Synapse together for some experiments. Axis2 ver 1.4 is used both as a client as well as to host a couple of web services. I use Synapse (ver 1.2) as a load balancer/request redirector to 2 or more Axis2 instances. Axis2 servers run with a modified axis2-kernel where I've done changes to AxisEngine and some of the transports. Synapse also uses this modified kernel. I generate requests which gets load balanced according to some algorithms I implement in Synapse. I change the arrival rates of the requests (at Synapse) for various experiments.
I encounter some problems when there's a higher arrival of requests at synapse (for lower arrival rates this seem to work fine). After some time into the experiments, some of the requests time out on the client side complaining there's no response from Synapse. When inspecting Synapse, I could see that the following exceptions were thrown,
"[HttpClientWorker-4] ERROR ServerHandler IO error submiting response : Connection is closed
org.apache.http.ConnectionClosedException: Connection is closed
at org.apache.http.impl.nio.NHttpConnectionBase.assertNotClosed(NHttpConnectionBase.java:238)
at org.apache.http.impl.nio.DefaultNHttpServerConnection.submitResponse(DefaultNHttpServerConnection.java:200)
at org.apache.synapse.transport.nhttp.ServerHandler.commitResponse(ServerHandler.java:213)
at org.apache.synapse.transport.nhttp.HttpCoreNIOSender.sendAsyncResponse(HttpCoreNIOSender.java:416)
at org.apache.synapse.transport.nhttp.HttpCoreNIOSender.invoke(HttpCoreNIOSender.java:271)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:567)
at org.apache.synapse.core.axis2.Axis2Sender.sendBack(Axis2Sender.java:143)
at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.send(Axis2SynapseEnvironment.java:219)
at org.apache.synapse.mediators.builtin.SendMediator.mediate(SendMediator.java:154)
at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:58)
at org.apache.synapse.mediators.filters.OutMediator.mediate(OutMediator.java:60)
at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:58)
at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:125)
at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:176)
at org.apache.synapse.core.axis2.SynapseCallbackReceiver.handleMessage(SynapseCallbackReceiver.java:344)
at org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive(SynapseCallbackReceiver.java:139)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:274)
at org.apache.synapse.transport.nhttp.ClientWorker.run(ClientWorker.java:200)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:713)"
The number of requests synapse handles are logged and strangely that equals to more than the number of requests sent to Synapse. Investigating further on this I could see that Synapse had processed the same request more than once and sometimes, the result hasn't been sent to the client. I'm guessing that this may be due to retransmission of requests from the client due to not having any response from synapse.
The Axis2 client is used an asynchronous client and since the timing of the requests is important, each request is created on a separate thread and use a newly created serviceClient object inside each thread of execution. After the non blocking call is done serviceClient.cleanup() and serviceClient.cleanupTransport() is called. I'm wondering whether the problems are connected with the version of the client I use (I've seen that 1.4 has some problems) and/or the way I'm using the client. Here's a part of the code how it's done,
For synapse issue try to increase the time out value in nhttp properties file.
"ServiceClient serviceClient = new ServiceClient();
Options opts = new Options();
opts.setTo(EPR);
opts.setAction(Action);
opts.setTimeOutInMilliSeconds(5400000);
serviceClient.setOptions(opts);
serviceClient.sendReceiveNonBlocking(this.createPayload(parameter), callback);
...
serviceClient.cleanup();
serviceClient.cleanupTransport();"
you can not clean up the transports before getting the response.
Migrating the Axis2 client libraries to 1.5.1 resulted in exceptions being thrown and requests not reaching Synapse at all. I have a couple of questions on the above,
With axis2 1.5.1 by default you have only 2 connections. Unless you increase this value it may gives exceptions for a high load scenarios.
thanks,
Amila.
1) Could this be due to bugs with the 1.4 version of Axis2?
2) Are there any changes required/suggested in the way the client code works?
3) Could this be happening due to any problems with Synapse? (I posted this to the Axis2 mailing list assuming that it might be to do with the client), in which case please cross post this to the Synapse mailing list, thanks.
Appreciate any help and suggestions you could give in solving this issue.
Thanks,
Vidura
--
Amila Suriarachchi
WSO2 Inc.
blog: http://amilachinthaka.blogspot.com/
Re: Time out problem when using Axis2 and Synapse together
Posted by Amila Suriarachchi <am...@gmail.com>.
On Mon, Mar 1, 2010 at 11:01 AM, Vidura Gamini Abhaya <vi...@gmail.com>wrote:
> Hi All,
>
> I'm using Axis2 and Synapse together for some experiments. Axis2 ver 1.4 is
> used both as a client as well as to host a couple of web services. I use
> Synapse (ver 1.2) as a load balancer/request redirector to 2 or more Axis2
> instances. Axis2 servers run with a modified axis2-kernel where I've done
> changes to AxisEngine and some of the transports. Synapse also uses this
> modified kernel. I generate requests which gets load balanced according to
> some algorithms I implement in Synapse. I change the arrival rates of the
> requests (at Synapse) for various experiments.
>
> I encounter some problems when there's a higher arrival of requests at
> synapse (for lower arrival rates this seem to work fine). After some time
> into the experiments, some of the requests time out on the client side
> complaining there's no response from Synapse. When inspecting Synapse, I
> could see that the following exceptions were thrown,
>
> "[HttpClientWorker-4] ERROR ServerHandler IO error submiting response :
> Connection is closed
> org.apache.http.ConnectionClosedException: Connection is closed
> at
> org.apache.http.impl.nio.NHttpConnectionBase.assertNotClosed(NHttpConnectionBase.java:238)
> at
> org.apache.http.impl.nio.DefaultNHttpServerConnection.submitResponse(DefaultNHttpServerConnection.java:200)
> at
> org.apache.synapse.transport.nhttp.ServerHandler.commitResponse(ServerHandler.java:213)
> at
> org.apache.synapse.transport.nhttp.HttpCoreNIOSender.sendAsyncResponse(HttpCoreNIOSender.java:416)
> at
> org.apache.synapse.transport.nhttp.HttpCoreNIOSender.invoke(HttpCoreNIOSender.java:271)
> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:567)
> at
> org.apache.synapse.core.axis2.Axis2Sender.sendBack(Axis2Sender.java:143)
> at
> org.apache.synapse.core.axis2.Axis2SynapseEnvironment.send(Axis2SynapseEnvironment.java:219)
> at
> org.apache.synapse.mediators.builtin.SendMediator.mediate(SendMediator.java:154)
> at
> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:58)
> at
> org.apache.synapse.mediators.filters.OutMediator.mediate(OutMediator.java:60)
> at
> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:58)
> at
> org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:125)
> at
> org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:176)
> at
> org.apache.synapse.core.axis2.SynapseCallbackReceiver.handleMessage(SynapseCallbackReceiver.java:344)
> at
> org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive(SynapseCallbackReceiver.java:139)
> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:274)
> at
> org.apache.synapse.transport.nhttp.ClientWorker.run(ClientWorker.java:200)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:713)"
>
> The number of requests synapse handles are logged and strangely that equals
> to more than the number of requests sent to Synapse. Investigating further
> on this I could see that Synapse had processed the same request more than
> once and sometimes, the result hasn't been sent to the client. I'm guessing
> that this may be due to retransmission of requests from the client due to
> not having any response from synapse.
>
> The Axis2 client is used an asynchronous client and since the timing of the
> requests is important, each request is created on a separate thread and use
> a newly created serviceClient object inside each thread of execution. After
> the non blocking call is done serviceClient.cleanup() and
> serviceClient.cleanupTransport() is called. I'm wondering whether the
> problems are connected with the version of the client I use (I've seen that
> 1.4 has some problems) and/or the way I'm using the client. Here's a part of
> the code how it's done,
>
For synapse issue try to increase the time out value in nhttp properties
file.
>
> "ServiceClient serviceClient = new ServiceClient();
> Options opts = new Options();
>
> opts.setTo(EPR);
> opts.setAction(Action);
>
> opts.setTimeOutInMilliSeconds(5400000);
> serviceClient.setOptions(opts);
>
> serviceClient.sendReceiveNonBlocking(this.createPayload(parameter),
> callback);
> ...
> serviceClient.cleanup();
> serviceClient.cleanupTransport();"
>
you can not clean up the transports before getting the response.
>
> Migrating the Axis2 client libraries to 1.5.1 resulted in exceptions being
> thrown and requests not reaching Synapse at all. I have a couple of
> questions on the above,
>
With axis2 1.5.1 by default you have only 2 connections. Unless you increase
this value it may gives exceptions for a high load scenarios.
thanks,
Amila.
>
> 1) Could this be due to bugs with the 1.4 version of Axis2?
> 2) Are there any changes required/suggested in the way the client code
> works?
> 3) Could this be happening due to any problems with Synapse? (I posted this
> to the Axis2 mailing list assuming that it might be to do with the client),
> in which case please cross post this to the Synapse mailing list, thanks.
>
> Appreciate any help and suggestions you could give in solving this issue.
>
> Thanks,
>
> Vidura
>
>
--
Amila Suriarachchi
WSO2 Inc.
blog: http://amilachinthaka.blogspot.com/