You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@servicemix.apache.org by "Ninck Blok, Jeroen" <je...@logica.com> on 2008/04/09 09:12:03 UTC
ConsumerProcessor - Exchange not found
Hi all,
I am getting the Exception "Exchange not found" from the
ConsumerProcessor when I am sending (using separate threads) two new JBI
messages to the same SU nearly simultaneously. When I send the same
message (but only once) it does not cause any problems.
After the Exception ServiceMix locks up completely.
I use the following code (and reuse the existing delivery channel) in an
implementation of MessageExchangeListener:
public InOut sendMessageExchange(QName service, Source content,
Map<String, Object> properties, boolean sendSync)
throws MessagingException
{
logger.debug("Preparing to send message to service " +
service.getLocalPart());
InOut exchange =
channel.createExchangeFactory().createInOutExchange();
NormalizedMessage message = exchange.createMessage();
message.setContent(content);
if(properties != null && properties.size() > 0)
{
logger.debug("Adding " + properties.size() + " properties to
message");
Set<Entry<String, Object>> propertiesSet =
properties.entrySet();
for(Entry<String, Object> propertyEntry: propertiesSet)
{
message.setProperty(propertyEntry.getKey(),
propertyEntry.getValue());
}
}
exchange.setMessage(message, "in");
exchange.setService(service);
if (sendSync)
{
logger.debug("Sending message synchronous");
channel.sendSync(exchange);
}
else
{
logger.debug("Sending message asynchronous");
channel.send(exchange);
}
return exchange;
}
This code is executed in a separate function.
Exception:
WARN - jetty - /stt/
java.lang.IllegalStateException: Exchange not found
at
org.apache.servicemix.http.processors.ConsumerProcessor.process(Consumer
Processor.java:192)
at
org.apache.servicemix.http.HttpBridgeServlet.doPost(HttpBridgeServlet.ja
va:71)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandler
Collection.java:211)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.jav
a:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:313)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
at
org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:375)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:
396)
at
org.mortbay.jetty.nio.SelectChannelConnector$RetryContinuation.run(Selec
tChannelConnector.java:511)
at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.ja
va:442)
I see bug reports reporting something similar (SM-1179, SM-1294), but
there is not enough information in these reports to confirm this problem
for me. Does anybody else have the same problem? How do other people
send JBI messages from an SU?
Thanks,
Jeroen Ninck Blok
Consultant
_________________________________________
Logica - Releasing your potential
Kralingseweg 241 - 249
3062 CE Rotterdam
PO Box 8566
3009 AN Rotterdam
The Netherlands
T: +31 10 253 7343
E: jeroen.ninck.blok@logica.com <ma...@logica.com>
www.logica.com <http://www.logica.com>
-
Logica Nederland BV
Registered office in Amstelveen, The Netherlands
Registration Number Chamber of Commerce: 33136004
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
RE: ConsumerProcessor - Exchange not found
Posted by "Ninck Blok, Jeroen" <je...@logica.com>.
Thanks for the response.
Trial and error on our part have shown that the problem is related to the coolPoolSize property. The other thread pool properties are being ignored somehow in this scenario. What are normal settings for thread pool properties? We used the default settings and we stumbled upon the problem very quick. We raised them to a high level, however this will not prevent any problems on a messive load.
As soon as I am back at the office I will post a fix for SM-1179 (simple copy past from SM-1294), however this will have to wait for more then a week. This fix will only prevent the HTTP consumer endpoint from dying. A timeout Exception will be thrown instead of the Exchange not found.
Jeroen NB
-----Original Message-----
From: Guillaume Nodet [mailto:gnodet@gmail.com]
Sent: Thu 24-7-2008 21:14
To: users@servicemix.apache.org
Subject: Re: ConsumerProcessor - Exchange not found
This could be caused by a combination of problems under high load.
The default configuration of the thread pools is to create new threads
while the number of threads is below a given limit, then queue the
exchanges in a collection until the collection is full. When no
threads are available and the collection is full, the thread sending
the exchange will process it instead of queuing it. The goal is to
slow the consumers and not loose any messages. However, some
problems may appear when using sendSync, because in an InOut exchange,
both components could use sendSync. This work when using different
threads, but when a single thread is involved (as it would be the case
if the queues are full), this completely fail and not error is thrown
anywhere (I don't recall exactly what would happen, but It may the
case that the thread is waiting forever on an event that will never
come).
So raising the number of threads should work around the problem.
I'm not sure however about the "Exchange not found" error.
On Thu, Jul 24, 2008 at 6:05 PM, ninckblokje
<je...@logica.com> wrote:
>
> Hi all,
>
> We have made some progress on the problem I mentioned earlier.
> This problem occurs when the number of threads run out. The number of
> threads can be specified in servicemix.properties. You can increase all
> properties, but the property corePoolSize is the most important one. I am
> not sure what cases this behaviour, but increasing the number of threads is
> a nice work around.
> The HTTP consumer endpoint dies because of the resulting Exception. By
> implementing the suggested fix for SM-1294 you can prevent the HTTP consumer
> endpoint from dying. This could be the fix for SM-1179.
>
> Jeroen NB
> --
> View this message in context: http://www.nabble.com/ConsumerProcessor---Exchange-not-found-tp16581070p18635124.html
> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>
>
--
Cheers,
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
Re: ConsumerProcessor - Exchange not found
Posted by Guillaume Nodet <gn...@gmail.com>.
This could be caused by a combination of problems under high load.
The default configuration of the thread pools is to create new threads
while the number of threads is below a given limit, then queue the
exchanges in a collection until the collection is full. When no
threads are available and the collection is full, the thread sending
the exchange will process it instead of queuing it. The goal is to
slow the consumers and not loose any messages. However, some
problems may appear when using sendSync, because in an InOut exchange,
both components could use sendSync. This work when using different
threads, but when a single thread is involved (as it would be the case
if the queues are full), this completely fail and not error is thrown
anywhere (I don't recall exactly what would happen, but It may the
case that the thread is waiting forever on an event that will never
come).
So raising the number of threads should work around the problem.
I'm not sure however about the "Exchange not found" error.
On Thu, Jul 24, 2008 at 6:05 PM, ninckblokje
<je...@logica.com> wrote:
>
> Hi all,
>
> We have made some progress on the problem I mentioned earlier.
> This problem occurs when the number of threads run out. The number of
> threads can be specified in servicemix.properties. You can increase all
> properties, but the property corePoolSize is the most important one. I am
> not sure what cases this behaviour, but increasing the number of threads is
> a nice work around.
> The HTTP consumer endpoint dies because of the resulting Exception. By
> implementing the suggested fix for SM-1294 you can prevent the HTTP consumer
> endpoint from dying. This could be the fix for SM-1179.
>
> Jeroen NB
> --
> View this message in context: http://www.nabble.com/ConsumerProcessor---Exchange-not-found-tp16581070p18635124.html
> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>
>
--
Cheers,
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
Re: ConsumerProcessor - Exchange not found
Posted by ninckblokje <je...@logica.com>.
Hi all,
We have made some progress on the problem I mentioned earlier.
This problem occurs when the number of threads run out. The number of
threads can be specified in servicemix.properties. You can increase all
properties, but the property corePoolSize is the most important one. I am
not sure what cases this behaviour, but increasing the number of threads is
a nice work around.
The HTTP consumer endpoint dies because of the resulting Exception. By
implementing the suggested fix for SM-1294 you can prevent the HTTP consumer
endpoint from dying. This could be the fix for SM-1179.
Jeroen NB
--
View this message in context: http://www.nabble.com/ConsumerProcessor---Exchange-not-found-tp16581070p18635124.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
Re: ConsumerProcessor - Exchange not found
Posted by ninckblokje <je...@logica.com>.
Hi all,
I have been busy testing several work arounds and I am still unable to fix
the problem.
I have tested (and reproduced) the problem on ServiceMix 3.2.2 incubating. I
have also tried to change use a new ServiceMixClient instead of the
DeliveryChannel provided in the Bean (implementation of
MessageExchangeListener), but with no success. I have also tried to create
to stop using the sendSync function and use the function send. The response
will then be received by the implementation of MessageExchangeListener as a
consumer message. By using the producer / consumer pattern I try to link the
received message to the send message (by using the Exchange Id). However the
problem still remains.
Setting a timeout for the sendSync function will prevent a complete
ServiceMix lock up (the request will then get a timeout Exception). But
increasing the timeout does not fix the problem.
My first guess is that something goes wrong with internal locks or that the
Exchange Id is not unique enough. Though I have not yet tested changing the
Exchange Id. See SM-1179 for a bug report.
I will try to reproduce the issue on ServiceMix 4.0 M1, to see if the issue
remains or not.
Did someone find a workaround / correct code for this problem?
Thanks,
Jeroen NB
ninckblokje wrote:
>
> Hi all,
>
>
>
> I am getting the Exception "Exchange not found" from the
> ConsumerProcessor when I am sending (using separate threads) two new JBI
> messages to the same SU nearly simultaneously. When I send the same
> message (but only once) it does not cause any problems.
>
> After the Exception ServiceMix locks up completely.
>
>
>
> I use the following code (and reuse the existing delivery channel) in an
> implementation of MessageExchangeListener:
>
>
>
> public InOut sendMessageExchange(QName service, Source content,
> Map<String, Object> properties, boolean sendSync)
>
> throws MessagingException
>
> {
>
> logger.debug("Preparing to send message to service " +
> service.getLocalPart());
>
> InOut exchange =
> channel.createExchangeFactory().createInOutExchange();
>
> NormalizedMessage message = exchange.createMessage();
>
> message.setContent(content);
>
>
>
> if(properties != null && properties.size() > 0)
>
> {
>
> logger.debug("Adding " + properties.size() + " properties to
> message");
>
> Set<Entry<String, Object>> propertiesSet =
> properties.entrySet();
>
> for(Entry<String, Object> propertyEntry: propertiesSet)
>
> {
>
> message.setProperty(propertyEntry.getKey(),
> propertyEntry.getValue());
>
> }
>
> }
>
>
>
> exchange.setMessage(message, "in");
>
> exchange.setService(service);
>
>
>
> if (sendSync)
>
> {
>
> logger.debug("Sending message synchronous");
>
> channel.sendSync(exchange);
>
> }
>
> else
>
> {
>
> logger.debug("Sending message asynchronous");
>
> channel.send(exchange);
>
> }
>
>
>
> return exchange;
>
> }
>
>
>
> This code is executed in a separate function.
>
>
>
> Exception:
>
> WARN - jetty - /stt/
>
> java.lang.IllegalStateException: Exchange not found
>
> at
> org.apache.servicemix.http.processors.ConsumerProcessor.process(Consumer
> Processor.java:192)
>
> at
> org.apache.servicemix.http.HttpBridgeServlet.doPost(HttpBridgeServlet.ja
> va:71)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
>
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362)
>
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
>
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandler
> Collection.java:211)
>
> at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.jav
> a:114)
>
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
>
> at org.mortbay.jetty.Server.handle(Server.java:313)
>
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
>
> at
> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:375)
>
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:
> 396)
>
> at
> org.mortbay.jetty.nio.SelectChannelConnector$RetryContinuation.run(Selec
> tChannelConnector.java:511)
>
> at
> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.ja
> va:442)
>
>
>
> I see bug reports reporting something similar (SM-1179, SM-1294), but
> there is not enough information in these reports to confirm this problem
> for me. Does anybody else have the same problem? How do other people
> send JBI messages from an SU?
>
>
>
> Thanks,
>
>
>
> Jeroen Ninck Blok
> Consultant
> _________________________________________
>
> Logica - Releasing your potential
>
> Kralingseweg 241 - 249
> 3062 CE Rotterdam
> PO Box 8566
> 3009 AN Rotterdam
> The Netherlands
> T: +31 10 253 7343
> E: jeroen.ninck.blok@logica.com <ma...@logica.com>
> www.logica.com <http://www.logica.com>
> -
> Logica Nederland BV
> Registered office in Amstelveen, The Netherlands
> Registration Number Chamber of Commerce: 33136004
>
>
>
> This e-mail and any attachment is for authorised use by the intended
> recipient(s) only. It may contain proprietary material, confidential
> information and/or be subject to legal privilege. It should not be copied,
> disclosed to, retained or used by, any other party. If you are not an
> intended recipient then please promptly delete this e-mail and any
> attachment and all copies and inform the sender. Thank you.
>
>
>
--
View this message in context: http://www.nabble.com/ConsumerProcessor---Exchange-not-found-tp16581070p16892024.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.