Synapse is blocked and does not deliver responses

["Hubert, Eric" <er...@jamba.net>]

Hi all,

today we were facing a serious issue on our integration servers.
Suddenly the ESB did not respond to service calls. When not setting a
client-side timeout on the client side we only see the following:

Caused by: org.apache.commons.httpclient.NoHttpResponseException: The
server xxx failed to respond
at
org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBa
se.java:1972)
at
org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase
.java:1727)
at
org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java
:1090)
at
org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMe
thodDirector.java:398)
at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMetho
dDirector.java:171)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:3
97)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:3
23)
at
de.jamba.soa.serviceconnector.hessian.client.JHessianProxy.invoke(JHessi
anProxy.java:244)

I tried to go through all logs and found out that the message is
accepted goes through all incoming sequences gets send to an endpoint,
the endpoints responds but the ESB does not deliver the response for
some reason.

After an instance restart everything works fine. We could not find a
reliable way to reproduce this and get an instance in this state. But it
happened several times. Currently I'm keeping one instance in this
state. I did some thread dumps to see where the instance is hanging.

Please find attached one of the thread dumps in zipped format. Please
help me tracking down the problem. Just tell me if I shall raise a JIRA
for this.


Regards,
  Eric

Re: Synapse is blocked and does not deliver responses

["Asankha C. Perera" <as...@wso2.com>]

Hi all

If at all any of you thought we dropped this email .. thats far from 
reality :-) ..

Infact I have been working closely with Eric in the past couple of days, 
and with his help, we were able to find out and fix a few issues around 
this, which I will list in a JIRA and commit shortly

asankha


Hubert, Eric wrote:
> Hi all,
>
> today we were facing a serious issue on our integration servers.
> Suddenly the ESB did not respond to service calls. When not setting a
> client-side timeout on the client side we only see the following:
>
> Caused by: org.apache.commons.httpclient.NoHttpResponseException: The
> server xxx failed to respond
> at
> .....
> I tried to go through all logs and found out that the message is
> accepted goes through all incoming sequences gets send to an endpoint,
> the endpoints responds but the ESB does not deliver the response for
> some reason.
>
> After an instance restart everything works fine. We could not find a
> reliable way to reproduce this and get an instance in this state. But it
> happened several times. Currently I'm keeping one instance in this
> state. I did some thread dumps to see where the instance is hanging.
>
> Please find attached one of the thread dumps in zipped format. Please
> help me tracking down the problem. Just tell me if I shall raise a JIRA
> for this.
>
>
> Regards,
>   Eric
>   
-- 
Asankha C. Perera

WSO2 - http://wso2.org
http://esbmagic.blogspot.com