You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Ilya S <de...@gmail.com> on 2010/04/14 19:41:19 UTC
Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Hi All,
Fist of all, thank you for this great product!
Every new release brings excellent new features just in time.
I'm currently investigating Graceful Shutdown feature, and I'm am
wondering if graceful shutdown is supported for Camel-Jetty component?
I was able to successfully shut down JMS component. However,
JettyHttpCoponent seems to have issues when shutting down.
In my simple test (adopted from camel-example-spring-jms:ServerRoute
example) I have a simple route that looks like this:
from("jetty:http://0.0.0.0:8686/hello")
..
.. do work
...
.to("jms:queue:numbers2?requestTimeout=50000")
.to("multiplier");
My producers simply send numbers to http endpoint..
When I shutdown this context gracefully I'm getting all sort of
exceptions from Jetty, and the context does not seem to wait until
inflight exchanges have finished.
I'm using the defaults everywhere (e.g. default shutdown strategy).
Is shutdown supported for Jetty, and if so how should one configure
jetty to initiate proper shutdown?
Thank you,
Ilya.
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Ilya S <de...@gmail.com>.
Let me add that I see around 5-6 inflight exchanges in the jconsole
(via JMX) before I hit stop(). So there are exchanges, but the log
statements say there are none..
Thanks again,
Ilya
On Thu, Apr 15, 2010 at 8:37 AM, Ilya S <de...@gmail.com> wrote:
> Here is my full route:
>
>
> from("jetty:http://0.0.0.0:8686/hello")
> .process(new Processor() {
> public void process(Exchange exchange) throws Exception {
>
> String body = exchange.getIn().getBody(String.class);
> Integer ii = Integer.parseInt(body);
>
> System.out.println("About to process: " + ii);
> System.out.println("Sending to numbers2 queue to wait...");
>
> exchange.getOut().setBody(ii);
> }
> })
> .to("jms:queue:numbers2?requestTimeout=50000")
> .process(new Processor() {
> public void process(Exchange exchange) throws Exception {
> Integer ii = exchange.getIn().getBody(Integer.class);
> System.out.println("Back to processing of: " + ii);
> exchange.getOut().setBody(ii);
> }
> })
> .to("multiplier");
>
>
> The other queue (jms:queue:numbers2) is a part of another route (in
> another Camel context):
>
> from("jms:queue:numbers2")
> .process(new Processor() {
> public void process(Exchange exchange) throws Exception {
> int sleepTime = (int)(Math.random()*25000);
> System.out.println("Received from numbers2....Going to sleep now
> for " + sleepTime/1000 + " seconds");
> Thread.sleep(sleepTime);
>
> Message in = exchange.getIn();
> exchange.getOut().setBody(in.getBody());
> }
> });
>
>
>
> I initiate the shutdown on the first Camel Context via JMX. If I
> remove the http component, and only use JMS components, the shutdown
> works just fine. With all the INFO messages about waiting for inflight
> exchanges to finish printed.
> As soon as I add Jetty, and initiate a stop() call, I'm getting the
> following output in the logs:
>
> Back to processing of: 1
> About to process: 2
> Sending to numbers2 queue to wait...
> Back to processing of: 2
> About to process: 3
> Sending to numbers2 queue to wait...
> Back to processing of: 2
> About to process: 3
> Sending to numbers2 queue to wait...
> [CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
> Apache Camel 2.2.0 (CamelContext:camel) is stopping
> [CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
> Starting to graceful shutdown routes (timeout 300 seconds)
> [ 6718343@qtp-21758581-2] DefaultErrorHandler ERROR
> Failed delivery for exchangeId: b5e983a8-4777-4db5-997c-3e4daf17c13e.
> On delivery attempt: 0 caught:
> org.apache.camel.ExchangeTimedOutException: The OUT message was not
> received within: 50000 millis. Exchange[Message: 1]
> [ 16889166@qtp-21758581-5] DefaultErrorHandler ERROR
> Failed delivery for exchangeId: b6653535-c32c-49c6-af6f-89fc0d58bbcf.
> Exhausted after delivery attempt: 1 caught:
> org.apache.camel.ExchangeTimedOutException: The OUT message was not
> received within: 50000 millis. Exchange[Message: 2]
> org.apache.camel.ExchangeTimedOutException: The OUT message was not
> received within: 50000 millis. Exchange[Message: 2]
> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:265)
> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> [ 19538521@qtp-21758581-4] DefaultErrorHandler ERROR
> Failed delivery for exchangeId: c5090377-e834-4b79-9b47-2512080c0632.
> Exhausted after delivery attempt: 1 caught:
> org.apache.camel.ExchangeTimedOutException: The OUT message was not
> received within: 50000 millis. Exchange[Message: 3]
> org.apache.camel.ExchangeTimedOutException: The OUT message was not
> received within: 50000 millis. Exchange[Message: 3]
> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:265)
> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy INFO
> Route: route1 shutdown complete.
> [CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
> Graceful shutdown of routes completed in 0 seconds
> [CP Connection(4)-10.58.123.154] DefaultInflightRepository INFO
> Shutting down with no inflight exchanges.
> [CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
> Apache Camel 2.2.0 (CamelContext:camel) stopped
>
>
> I tried to fix this by extending JettyHttpComponent and overriding
> createServer method as follows:
>
> @Override
> protected Server createServer() throws Exception {
> Server server = super.createServer();
> server.setGracefulShutdown(50000);
> server.setStopAtShutdown(true);
>
> return server;
> }
>
> This seemed to work, but I'm not sure if that's the right approach.
>
>
> Any help is greatly appreciated.
>
> Thanks,
> Ilya.
>
>
>
> On Thu, Apr 15, 2010 at 2:17 AM, Claus Ibsen <cl...@gmail.com> wrote:
>> Hi
>>
>> All the unit tests run with graceful shutdown by default, and all unit
>> tests with camel-jetty shut down with no issues.
>>
>> The in flight repository do keep track of all in flights. I can only
>> think of if you use the aggregator or some custom code which
>> can cause the exchange to be completed in Camels point of view.
>>
>> Can you post your entire route?
>>
>> And as always report which version of Camel you are using? And try
>> with the latest stable and even with 2.3-SNAPSHOT.
>>
>>
>> On Wed, Apr 14, 2010 at 7:41 PM, Ilya S <de...@gmail.com> wrote:
>>> Hi All,
>>>
>>> Fist of all, thank you for this great product!
>>> Every new release brings excellent new features just in time.
>>>
>>> I'm currently investigating Graceful Shutdown feature, and I'm am
>>> wondering if graceful shutdown is supported for Camel-Jetty component?
>>> I was able to successfully shut down JMS component. However,
>>> JettyHttpCoponent seems to have issues when shutting down.
>>>
>>> In my simple test (adopted from camel-example-spring-jms:ServerRoute
>>> example) I have a simple route that looks like this:
>>>
>>> from("jetty:http://0.0.0.0:8686/hello")
>>> ..
>>> .. do work
>>> ...
>>> .to("jms:queue:numbers2?requestTimeout=50000")
>>> .to("multiplier");
>>>
>>>
>>> My producers simply send numbers to http endpoint..
>>>
>>>
>>> When I shutdown this context gracefully I'm getting all sort of
>>> exceptions from Jetty, and the context does not seem to wait until
>>> inflight exchanges have finished.
>>> I'm using the defaults everywhere (e.g. default shutdown strategy).
>>>
>>> Is shutdown supported for Jetty, and if so how should one configure
>>> jetty to initiate proper shutdown?
>>>
>>> Thank you,
>>>
>>> Ilya.
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Ilya S <de...@gmail.com>.
Here is my full route:
from("jetty:http://0.0.0.0:8686/hello")
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
String body = exchange.getIn().getBody(String.class);
Integer ii = Integer.parseInt(body);
System.out.println("About to process: " + ii);
System.out.println("Sending to numbers2 queue to wait...");
exchange.getOut().setBody(ii);
}
})
.to("jms:queue:numbers2?requestTimeout=50000")
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
Integer ii = exchange.getIn().getBody(Integer.class);
System.out.println("Back to processing of: " + ii);
exchange.getOut().setBody(ii);
}
})
.to("multiplier");
The other queue (jms:queue:numbers2) is a part of another route (in
another Camel context):
from("jms:queue:numbers2")
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
int sleepTime = (int)(Math.random()*25000);
System.out.println("Received from numbers2....Going to sleep now
for " + sleepTime/1000 + " seconds");
Thread.sleep(sleepTime);
Message in = exchange.getIn();
exchange.getOut().setBody(in.getBody());
}
});
I initiate the shutdown on the first Camel Context via JMX. If I
remove the http component, and only use JMS components, the shutdown
works just fine. With all the INFO messages about waiting for inflight
exchanges to finish printed.
As soon as I add Jetty, and initiate a stop() call, I'm getting the
following output in the logs:
Back to processing of: 1
About to process: 2
Sending to numbers2 queue to wait...
Back to processing of: 2
About to process: 3
Sending to numbers2 queue to wait...
Back to processing of: 2
About to process: 3
Sending to numbers2 queue to wait...
[CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
Apache Camel 2.2.0 (CamelContext:camel) is stopping
[CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
Starting to graceful shutdown routes (timeout 300 seconds)
[ 6718343@qtp-21758581-2] DefaultErrorHandler ERROR
Failed delivery for exchangeId: b5e983a8-4777-4db5-997c-3e4daf17c13e.
On delivery attempt: 0 caught:
org.apache.camel.ExchangeTimedOutException: The OUT message was not
received within: 50000 millis. Exchange[Message: 1]
[ 16889166@qtp-21758581-5] DefaultErrorHandler ERROR
Failed delivery for exchangeId: b6653535-c32c-49c6-af6f-89fc0d58bbcf.
Exhausted after delivery attempt: 1 caught:
org.apache.camel.ExchangeTimedOutException: The OUT message was not
received within: 50000 millis. Exchange[Message: 2]
org.apache.camel.ExchangeTimedOutException: The OUT message was not
received within: 50000 millis. Exchange[Message: 2]
at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:265)
at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
[ 19538521@qtp-21758581-4] DefaultErrorHandler ERROR
Failed delivery for exchangeId: c5090377-e834-4b79-9b47-2512080c0632.
Exhausted after delivery attempt: 1 caught:
org.apache.camel.ExchangeTimedOutException: The OUT message was not
received within: 50000 millis. Exchange[Message: 3]
org.apache.camel.ExchangeTimedOutException: The OUT message was not
received within: 50000 millis. Exchange[Message: 3]
at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:265)
at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
[ Camel thread 11: ShutdownTask] DefaultShutdownStrategy INFO
Route: route1 shutdown complete.
[CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
Graceful shutdown of routes completed in 0 seconds
[CP Connection(4)-10.58.123.154] DefaultInflightRepository INFO
Shutting down with no inflight exchanges.
[CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
Apache Camel 2.2.0 (CamelContext:camel) stopped
I tried to fix this by extending JettyHttpComponent and overriding
createServer method as follows:
@Override
protected Server createServer() throws Exception {
Server server = super.createServer();
server.setGracefulShutdown(50000);
server.setStopAtShutdown(true);
return server;
}
This seemed to work, but I'm not sure if that's the right approach.
Any help is greatly appreciated.
Thanks,
Ilya.
On Thu, Apr 15, 2010 at 2:17 AM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> All the unit tests run with graceful shutdown by default, and all unit
> tests with camel-jetty shut down with no issues.
>
> The in flight repository do keep track of all in flights. I can only
> think of if you use the aggregator or some custom code which
> can cause the exchange to be completed in Camels point of view.
>
> Can you post your entire route?
>
> And as always report which version of Camel you are using? And try
> with the latest stable and even with 2.3-SNAPSHOT.
>
>
> On Wed, Apr 14, 2010 at 7:41 PM, Ilya S <de...@gmail.com> wrote:
>> Hi All,
>>
>> Fist of all, thank you for this great product!
>> Every new release brings excellent new features just in time.
>>
>> I'm currently investigating Graceful Shutdown feature, and I'm am
>> wondering if graceful shutdown is supported for Camel-Jetty component?
>> I was able to successfully shut down JMS component. However,
>> JettyHttpCoponent seems to have issues when shutting down.
>>
>> In my simple test (adopted from camel-example-spring-jms:ServerRoute
>> example) I have a simple route that looks like this:
>>
>> from("jetty:http://0.0.0.0:8686/hello")
>> ..
>> .. do work
>> ...
>> .to("jms:queue:numbers2?requestTimeout=50000")
>> .to("multiplier");
>>
>>
>> My producers simply send numbers to http endpoint..
>>
>>
>> When I shutdown this context gracefully I'm getting all sort of
>> exceptions from Jetty, and the context does not seem to wait until
>> inflight exchanges have finished.
>> I'm using the defaults everywhere (e.g. default shutdown strategy).
>>
>> Is shutdown supported for Jetty, and if so how should one configure
>> jetty to initiate proper shutdown?
>>
>> Thank you,
>>
>> Ilya.
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Claus Ibsen <cl...@gmail.com>.
On Wed, Apr 21, 2010 at 6:30 PM, Ilya S <de...@gmail.com> wrote:
> Hi Claus,
>
> Thanks again for making this work!
>
> I tried the latest code from trunk directly, and everything seems to
> work fine with my test case/prototype.
>
Great to hear it works now.
> Ilya.
>
>
>
> On Wed, Apr 21, 2010 at 8:47 AM, Ilya S <de...@gmail.com> wrote:
>> Hi
>>
>> Just refreshed my 2.3-SNAPSHOT, and I'm still getting the exceptions
>> if I omit Defer option in my route.
>> I'm going to wait some time and try again later, perhaps your changes
>> are not in the latest snapshot yet.
>>
>> Thank you.
>>
>> Ilya.
>>
>> On Wed, Apr 21, 2010 at 2:10 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>> Hi
>>>
>>> I have just committed this to trunk.
>>>
>>> camel-jetty will now return 503 when shutting down. This allows you to
>>> process existing messages while new messages will be denied with 503.
>>> You can also suspend/resume the consumer from JMX console with the
>>> operations for that.
>>>
>>> It would be great if you have the time to test it on your end as well.
>>>
>>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Ilya S <de...@gmail.com>.
Hi Claus,
Thanks again for making this work!
I tried the latest code from trunk directly, and everything seems to
work fine with my test case/prototype.
Ilya.
On Wed, Apr 21, 2010 at 8:47 AM, Ilya S <de...@gmail.com> wrote:
> Hi
>
> Just refreshed my 2.3-SNAPSHOT, and I'm still getting the exceptions
> if I omit Defer option in my route.
> I'm going to wait some time and try again later, perhaps your changes
> are not in the latest snapshot yet.
>
> Thank you.
>
> Ilya.
>
> On Wed, Apr 21, 2010 at 2:10 AM, Claus Ibsen <cl...@gmail.com> wrote:
>> Hi
>>
>> I have just committed this to trunk.
>>
>> camel-jetty will now return 503 when shutting down. This allows you to
>> process existing messages while new messages will be denied with 503.
>> You can also suspend/resume the consumer from JMX console with the
>> operations for that.
>>
>> It would be great if you have the time to test it on your end as well.
>>
>
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Ilya S <de...@gmail.com>.
Hi
Just refreshed my 2.3-SNAPSHOT, and I'm still getting the exceptions
if I omit Defer option in my route.
I'm going to wait some time and try again later, perhaps your changes
are not in the latest snapshot yet.
Thank you.
Ilya.
On Wed, Apr 21, 2010 at 2:10 AM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> I have just committed this to trunk.
>
> camel-jetty will now return 503 when shutting down. This allows you to
> process existing messages while new messages will be denied with 503.
> You can also suspend/resume the consumer from JMX console with the
> operations for that.
>
> It would be great if you have the time to test it on your end as well.
>
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
I have just committed this to trunk.
camel-jetty will now return 503 when shutting down. This allows you to
process existing messages while new messages will be denied with 503.
You can also suspend/resume the consumer from JMX console with the
operations for that.
It would be great if you have the time to test it on your end as well.
On Tue, Apr 20, 2010 at 7:45 PM, Ilya S <de...@gmail.com> wrote:
> Hi Claus,
>
> Thanks for creating a ticket. I think Service Unavaliable should be
> find for load balancer (LB).
>
> In my particular case load balancer will be checking some predefined
> healthcheck/status page with a status info that can be parsed by LB.
> E.g. every 5 seconds the LB will issue a GET request to some url, if
> the returned status is negative, the LB will take the server out of a
> pool. When we are about to perform maintenance, we will indicate in
> our healthcheck page that the server is about to go down. Then, after
> a delay we will initiate a gracefull shutdown.
>
> I now realize that with the health check page I can in fact simply use
> the "Defer" option as you originally suggested. I will let the LB know
> that we are about to shutdown via healthcheck page first, and so it
> should stop redirecting new requests to the machine in question.
> Eventually, all the in flight exchanges will finish, and the machine
> can be completely shutdown.
>
> Thanks again,
>
> Ilya.
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Ilya S <de...@gmail.com>.
Hi Claus,
Thanks for creating a ticket. I think Service Unavaliable should be
find for load balancer (LB).
In my particular case load balancer will be checking some predefined
healthcheck/status page with a status info that can be parsed by LB.
E.g. every 5 seconds the LB will issue a GET request to some url, if
the returned status is negative, the LB will take the server out of a
pool. When we are about to perform maintenance, we will indicate in
our healthcheck page that the server is about to go down. Then, after
a delay we will initiate a gracefull shutdown.
I now realize that with the health check page I can in fact simply use
the "Defer" option as you originally suggested. I will let the LB know
that we are about to shutdown via healthcheck page first, and so it
should stop redirecting new requests to the machine in question.
Eventually, all the in flight exchanges will finish, and the machine
can be completely shutdown.
Thanks again,
Ilya.
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
I have created a ticket
https://issues.apache.org/activemq/browse/CAMEL-2660
And did some experiment but Jetty resets the current in progress
connection so it will fail.
I even tried sending from a webbrowser and have Camel stop while in progress.
Would returning a selected HTTP error code work with your load balancer?
For example HTTP error 503
503 Service Unavailable
This is easy to implement in camel-jetty.
On Tue, Apr 20, 2010 at 7:13 AM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> camel-jetty does not currently support shutting down by stopping
> intaking new requests while allowing existing to be processed to
> completion.
> In fact this is something that each individual component has to implemented.
>
> This can be done by supporting the SuspendableService on the consumer.
> Then the graceful shutdown in camel-core will be able to leverage it.
>
> For example this has been implemented in camel-jms etc.
>
>
>
>
> On Tue, Apr 20, 2010 at 12:10 AM, Ilya S <de...@gmail.com> wrote:
>> Hi Claus,
>>
>> The test cases were helpful. However, the reason I cannot use "Defer"
>> option is because my Jetty input point is public, and if I have this
>> option set, then the new messages will keep comming from the clients
>> and the context will never be done. My goal is stop processesing
>> gracefully while not acceptin new messages. E.g. Jetty entry point
>> should not accept any new messages, while it should still let the
>> pending messages finish.
>> In our use case, once jetty component stops acceptin new messages, our
>> load ballancer will take it off the pool, however, we would still like
>> the in flight messages to finish.
>> The issue seems only occur when I use Jetty and JMS together.
>>
>> The simple route with just one jetty consumer such as the one below
>> seems to shutdown just fine :
>> from("jetty:http://0.0.0.0:8686/hello")
>> .delay(5000)
>> .to("multiplier");
>>
>> If I add a jms producer to the mix as follows, things break:
>> from("jetty:http://0.0.0.0:8686/hello")
>> .delay(5000)
>> .to("jms:queue:numbers2?requestTimeout=50000")
>> .to("multiplier");
>>
>> The other queue is running in a different context, but I don't think
>> this should matter.
>>
>> Please, let me know if you think I'm misusing the API, or if it is an
>> actual bug.
>>
>> Thanks again,
>> Ilya.
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
camel-jetty does not currently support shutting down by stopping
intaking new requests while allowing existing to be processed to
completion.
In fact this is something that each individual component has to implemented.
This can be done by supporting the SuspendableService on the consumer.
Then the graceful shutdown in camel-core will be able to leverage it.
For example this has been implemented in camel-jms etc.
On Tue, Apr 20, 2010 at 12:10 AM, Ilya S <de...@gmail.com> wrote:
> Hi Claus,
>
> The test cases were helpful. However, the reason I cannot use "Defer"
> option is because my Jetty input point is public, and if I have this
> option set, then the new messages will keep comming from the clients
> and the context will never be done. My goal is stop processesing
> gracefully while not acceptin new messages. E.g. Jetty entry point
> should not accept any new messages, while it should still let the
> pending messages finish.
> In our use case, once jetty component stops acceptin new messages, our
> load ballancer will take it off the pool, however, we would still like
> the in flight messages to finish.
> The issue seems only occur when I use Jetty and JMS together.
>
> The simple route with just one jetty consumer such as the one below
> seems to shutdown just fine :
> from("jetty:http://0.0.0.0:8686/hello")
> .delay(5000)
> .to("multiplier");
>
> If I add a jms producer to the mix as follows, things break:
> from("jetty:http://0.0.0.0:8686/hello")
> .delay(5000)
> .to("jms:queue:numbers2?requestTimeout=50000")
> .to("multiplier");
>
> The other queue is running in a different context, but I don't think
> this should matter.
>
> Please, let me know if you think I'm misusing the API, or if it is an
> actual bug.
>
> Thanks again,
> Ilya.
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Ilya S <de...@gmail.com>.
Hi Claus,
The test cases were helpful. However, the reason I cannot use "Defer"
option is because my Jetty input point is public, and if I have this
option set, then the new messages will keep comming from the clients
and the context will never be done. My goal is stop processesing
gracefully while not acceptin new messages. E.g. Jetty entry point
should not accept any new messages, while it should still let the
pending messages finish.
In our use case, once jetty component stops acceptin new messages, our
load ballancer will take it off the pool, however, we would still like
the in flight messages to finish.
The issue seems only occur when I use Jetty and JMS together.
The simple route with just one jetty consumer such as the one below
seems to shutdown just fine :
from("jetty:http://0.0.0.0:8686/hello")
.delay(5000)
.to("multiplier");
If I add a jms producer to the mix as follows, things break:
from("jetty:http://0.0.0.0:8686/hello")
.delay(5000)
.to("jms:queue:numbers2?requestTimeout=50000")
.to("multiplier");
The other queue is running in a different context, but I don't think
this should matter.
Please, let me know if you think I'm misusing the API, or if it is an
actual bug.
Thanks again,
Ilya.
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Ilya S <de...@gmail.com>.
Claus, thanks for your response.
If I use the Defer option on both of my routes (jetty and jms) then
the system does not shut down because new messages are constantly
coming from my clients (producers).
I tried the other scenario, recommended in the documentation:
In my case the jetty route is a 'public' route that should be shutdown
quickly (so that no new messages are accepted). So I removed the
'Defered' option there.
The second route (jms route running in a separate camel context) is
'internal', and I set the 'Defer' option on it.
With this configuration I'm still getting the same errors errors.
Should I try this in Camel 2.3 instead?
Thank you very much.
Ilya.
On Mon, Apr 19, 2010 at 11:09 AM, Claus Ibsen <cl...@gmail.com> wrote:
> See this unit test
> http://svn.apache.org/viewvc?rev=935691&view=rev
>
> On Mon, Apr 19, 2010 at 7:52 PM, Claus Ibsen <cl...@gmail.com> wrote:
>> Hi
>>
>> Just defer shutting down the Jetty route.
>> http://camel.apache.org/graceful-shutdown.html
>>
>> On Mon, Apr 19, 2010 at 7:08 PM, Ilya S <de...@gmail.com> wrote:
>>> I'd like to bump this thread up, and see if anyone has any suggestions.
>>> Thanks.
>>>
>>> On Thu, Apr 15, 2010 at 10:24 AM, Ilya S <de...@gmail.com> wrote:
>>>> Here is a more detailed log (with TRACE level on) if that can help:
>>>>
>>>> [CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
>>>> Apache Camel 2.2.0 (CamelContext:camel) is stopping
>>>> [CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
>>>> Starting to graceful shutdown routes (timeout 600000 seconds)
>>>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy DEBUG
>>>> There are 1 routes to shutdown
>>>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy TRACE
>>>> Shutting down route: route1 with options
>>>> [Default,CompleteCurrentTaskOnly]
>>>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy TRACE
>>>> Shutting down: Consumer[http://0.0.0.0:8686/hello]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service Consumer[http://0.0.0.0:8686/hello]
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> Container Server@8032df - SelectChannelConnector@0.0.0.0:8686 as
>>>> connector
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopped org.mortbay.jetty.nio.SelectChannelConnector$1@28df48
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopped SelectChannelConnector@0.0.0.0:8686
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopping Server@8032df
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopping ContextHandlerCollection@8bfdea
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopping org.mortbay.jetty.servlet.Context@1157f77{/,null}
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopping ServletHandler@6399ae
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopped org.apache.camel.component.http.CamelServlet-29615099
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopped ServletHandler@6399ae
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopping ErrorHandler@56acfa
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopped ErrorHandler@56acfa
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopped org.mortbay.jetty.servlet.Context@1157f77{/,null}
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopped ContextHandlerCollection@8bfdea
>>>> [3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
>>>> to: Recording duration: 2342 millis for exchange: Exchange[Message: 2]
>>>> [3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
>>>> to: Recording duration: 2342 millis for exchange: Exchange[Message: 2]
>>>> [3850238@qtp-9243153-3 - /hello] DefaultExceptionPolicyStrategy TRACE
>>>> Finding best suited exception policy for thrown exception
>>>> java.lang.InterruptedException
>>>> [3850238@qtp-9243153-3 - /hello] DefaultExceptionPolicyStrategy TRACE
>>>> Found 0 candidates
>>>> [2390869@qtp-9243153-4 - /hello] JmsProducer DEBUG
>>>> Future interrupted: java.lang.InterruptedException
>>>> java.lang.InterruptedException
>>>> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:926)
>>>> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1201)
>>>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:210)
>>>> at java.util.concurrent.FutureTask.get(FutureTask.java:85)
>>>> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:227)
>>>> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
>>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
>>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
>>>> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
>>>> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
>>>> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>>> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
>>>> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
>>>> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>>> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
>>>> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
>>>> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
>>>> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
>>>> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
>>>> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
>>>> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>>> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>>>> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
>>>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
>>>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>> at org.mortbay.jetty.Server.handle(Server.java:322)
>>>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
>>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
>>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
>>>> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>>> [1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
>>>> to: Recording duration: 19757 millis for exchange: Exchange[Message:
>>>> 1]
>>>> [3850238@qtp-9243153-3 - /hello] DefaultErrorHandler DEBUG
>>>> Failed delivery for exchangeId: 21b68982-8ec1-405b-97d8-56f6e533115f.
>>>> On delivery attempt: 0 caught: java.lang.InterruptedException
>>>> [1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
>>>> to: Recording duration: 19758 millis for exchange: Exchange[Message:
>>>> 1]
>>>> [3850238@qtp-9243153-3 - /hello] DefaultTypeConverter TRACE
>>>> Converting java.lang.Boolean -> java.lang.Object with value: false
>>>> [3850238@qtp-9243153-3 - /hello] DefaultErrorHandler ERROR
>>>> Failed delivery for exchangeId: 21b68982-8ec1-405b-97d8-56f6e533115f.
>>>> Exhausted after delivery attempt: 1 caught:
>>>> java.lang.InterruptedException
>>>> java.lang.InterruptedException
>>>> at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:154)
>>>> at org.apache.log4j.Category.forcedLog(Category.java:388)
>>>> at org.apache.log4j.Category.log(Category.java:853)
>>>> at org.apache.commons.logging.impl.Log4JLogger.debug(Log4JLogger.java:188)
>>>> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:231)
>>>> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
>>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
>>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
>>>> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
>>>> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
>>>> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>>> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
>>>> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
>>>> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>>> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
>>>> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
>>>> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
>>>> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
>>>> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
>>>> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
>>>> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>>> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>>>> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
>>>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
>>>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>> at org.mortbay.jetty.Server.handle(Server.java:322)
>>>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
>>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
>>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
>>>> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>>> [2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
>>>> to: Recording duration: 1826 millis for exchange: Exchange[Message: 3]
>>>> [1671238@qtp-9243153-2 - /hello] DefaultExceptionPolicyStrategy TRACE
>>>> Finding best suited exception policy for thrown exception
>>>> java.lang.InterruptedException
>>>> [3850238@qtp-9243153-3 - /hello] DefaultTypeConverter TRACE
>>>> Converting java.lang.Boolean -> java.lang.Object with value: false
>>>> [2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
>>>> to: Recording duration: 1827 millis for exchange: Exchange[Message: 3]
>>>> [1671238@qtp-9243153-2 - /hello] DefaultExceptionPolicyStrategy TRACE
>>>> Found 0 candidates
>>>> [3850238@qtp-9243153-3 - /hello] DefaultErrorHandler DEBUG
>>>> This exchange is not handled so its marked as failed:
>>>> Exchange[Message: 2]
>>>> [2390869@qtp-9243153-4 - /hello] DefaultExceptionPolicyStrategy TRACE
>>>> Finding best suited exception policy for thrown exception
>>>> org.apache.camel.ExchangeTimedOutException
>>>> [1671238@qtp-9243153-2 - /hello] DefaultErrorHandler DEBUG
>>>> Failed delivery for exchangeId: 915083b6-4291-4716-929d-aa451e29e495.
>>>> On delivery attempt: 0 caught: java.lang.InterruptedException
>>>> [2390869@qtp-9243153-4 - /hello] DefaultExceptionPolicyStrategy TRACE
>>>> Found 0 candidates
>>>> [3850238@qtp-9243153-3 - /hello] Pipeline DEBUG
>>>> Message exchange has failed so breaking out of pipeline:
>>>> Exchange[Message: 2] Exception: java.lang.InterruptedException
>>>> [1671238@qtp-9243153-2 - /hello] DefaultTypeConverter TRACE
>>>> Converting java.lang.Boolean -> java.lang.Object with value: false
>>>> [2390869@qtp-9243153-4 - /hello] DefaultErrorHandler DEBUG
>>>> Failed delivery for exchangeId: 5903a17f-bff9-4ba5-b8f4-78bc96149195.
>>>> On delivery attempt: 0 caught:
>>>> org.apache.camel.ExchangeTimedOutException: The OUT message was not
>>>> received within: 50000 millis. Exchange[Message: 3]
>>>> [1671238@qtp-9243153-2 - /hello] DefaultErrorHandler ERROR
>>>> Failed delivery for exchangeId: 915083b6-4291-4716-929d-aa451e29e495.
>>>> Exhausted after delivery attempt: 1 caught:
>>>> java.lang.InterruptedException
>>>> java.lang.InterruptedException
>>>> at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:154)
>>>> at org.apache.log4j.Category.forcedLog(Category.java:388)
>>>> at org.apache.log4j.Category.log(Category.java:853)
>>>> at org.apache.commons.logging.impl.Log4JLogger.debug(Log4JLogger.java:188)
>>>> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:231)
>>>> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
>>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
>>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
>>>> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
>>>> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
>>>> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>>> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
>>>> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
>>>> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>>> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
>>>> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
>>>> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
>>>> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
>>>> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
>>>> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
>>>> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>>> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>>>> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
>>>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
>>>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>> at org.mortbay.jetty.Server.handle(Server.java:322)
>>>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
>>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
>>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
>>>> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>>> [2390869@qtp-9243153-4 - /hello] DefaultTypeConverter TRACE
>>>> Converting java.lang.Boolean -> java.lang.Object with value: false
>>>> [3850238@qtp-9243153-3 - /hello] Pipeline TRACE
>>>> Processing complete for exchangeId:
>>>> 21b68982-8ec1-405b-97d8-56f6e533115f >>> Exchange[Message: 2]
>>>> [1671238@qtp-9243153-2 - /hello] DefaultTypeConverter TRACE
>>>> Converting java.lang.Boolean -> java.lang.Object with value: false
>>>> [2390869@qtp-9243153-4 - /hello] DefaultErrorHandler ERROR
>>>> Failed delivery for exchangeId: 5903a17f-bff9-4ba5-b8f4-78bc96149195.
>>>> Exhausted after delivery attempt: 1 caught:
>>>> org.apache.camel.ExchangeTimedOutException: The OUT message was not
>>>> received within: 50000 millis. Exchange[Message: 3]
>>>> org.apache.camel.ExchangeTimedOutException: The OUT message was not
>>>> received within: 50000 millis. Exchange[Message: 3]
>>>> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:265)
>>>> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
>>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
>>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
>>>> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
>>>> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
>>>> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>>> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
>>>> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
>>>> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>>> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
>>>> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
>>>> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
>>>> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
>>>> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
>>>> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
>>>> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>>> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>>>> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
>>>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
>>>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>> at org.mortbay.jetty.Server.handle(Server.java:322)
>>>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
>>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
>>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
>>>> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>>> [1671238@qtp-9243153-2 - /hello] DefaultErrorHandler DEBUG
>>>> This exchange is not handled so its marked as failed:
>>>> Exchange[Message: 1]
>>>> [3850238@qtp-9243153-3 - /hello] DefaultInflightRepository TRACE
>>>> Total 2 inflight exchanges. Last removed:
>>>> 21b68982-8ec1-405b-97d8-56f6e533115f
>>>> [2390869@qtp-9243153-4 - /hello] DefaultTypeConverter TRACE
>>>> Converting java.lang.Boolean -> java.lang.Object with value: false
>>>> [1671238@qtp-9243153-2 - /hello] Pipeline DEBUG
>>>> Message exchange has failed so breaking out of pipeline:
>>>> Exchange[Message: 1] Exception: java.lang.InterruptedException
>>>> [3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
>>>> route: Recording duration: 2347 millis for exchange: Exchange[Message:
>>>> [Body is instance of java.io.InputStream]]
>>>> [2390869@qtp-9243153-4 - /hello] DefaultErrorHandler DEBUG
>>>> This exchange is not handled so its marked as failed:
>>>> Exchange[Message: 3]
>>>> [1671238@qtp-9243153-2 - /hello] Pipeline TRACE
>>>> Processing complete for exchangeId:
>>>> 915083b6-4291-4716-929d-aa451e29e495 >>> Exchange[Message: 1]
>>>> [2390869@qtp-9243153-4 - /hello] Pipeline DEBUG
>>>> Message exchange has failed so breaking out of pipeline:
>>>> Exchange[Message: 3] Exception:
>>>> org.apache.camel.ExchangeTimedOutException: The OUT message was not
>>>> received within: 50000 millis. Exchange[Message: 3]
>>>> [1671238@qtp-9243153-2 - /hello] DefaultInflightRepository TRACE
>>>> Total 1 inflight exchanges. Last removed:
>>>> 915083b6-4291-4716-929d-aa451e29e495
>>>> [2390869@qtp-9243153-4 - /hello] Pipeline TRACE
>>>> Processing complete for exchangeId:
>>>> 5903a17f-bff9-4ba5-b8f4-78bc96149195 >>> Exchange[Message: 3]
>>>> [1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
>>>> route: Recording duration: 19763 millis for exchange:
>>>> Exchange[Message: [Body is instance of java.io.InputStream]]
>>>> [2390869@qtp-9243153-4 - /hello] DefaultInflightRepository TRACE
>>>> Total 0 inflight exchanges. Last removed:
>>>> 5903a17f-bff9-4ba5-b8f4-78bc96149195
>>>> [3850238@qtp-9243153-3 - /hello] log DEBUG
>>>> RESPONSE /hello 500
>>>> [2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
>>>> route: Recording duration: 1830 millis for exchange: Exchange[Message:
>>>> [Body is instance of java.io.InputStream]]
>>>> [ 23850238@qtp-9243153-3] log DEBUG EOF
>>>> [2390869@qtp-9243153-4 - /hello] log DEBUG
>>>> RESPONSE /hello 500
>>>> [ 2390869@qtp-9243153-4] log DEBUG EOF
>>>> [1671238@qtp-9243153-2 - /hello] log DEBUG
>>>> RESPONSE /hello 500
>>>> [ 1671238@qtp-9243153-2] log DEBUG EOF
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopped org.mortbay.thread.QueuedThreadPool@8d0a11
>>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>>> stopped Server@8032df
>>>> [ Camel thread 11: ShutdownTask] HttpConsumer DEBUG
>>>> Stopping consumer: Consumer[http://0.0.0.0:8686/hello]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>>> Channel[sendTo(Endpoint[multiplier])]])]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>>> Channel[sendTo(Endpoint[multiplier])]])
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>>> Channel[sendTo(Endpoint[multiplier])]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Instrumentation:process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> DefaultErrorHandler[Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Instrumentation:to[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> org.apache.camel.impl.SharedProducerServicePool@11d20d3
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> DefaultErrorHandler[Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Instrumentation:process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> DefaultErrorHandler[Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service: Channel[sendTo(Endpoint[multiplier])]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service: Instrumentation:to[TraceInterceptor[To[multiplier]]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service: TraceInterceptor[To[multiplier]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service: Instrumentation:to[sendTo(Endpoint[multiplier])]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service: sendTo(Endpoint[multiplier])
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> org.apache.camel.impl.SharedProducerServicePool@11d20d3
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service:
>>>> DefaultErrorHandler[Instrumentation:to[TraceInterceptor[To[multiplier]]]]
>>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>>> Stopping service: Instrumentation:to[TraceInterceptor[To[multiplier]]]
>>>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy DEBUG
>>>> Shutdown complete for: Consumer[http://0.0.0.0:8686/hello]
>>>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy INFO
>>>> Route: route1 shutdown complete.
>>>> [CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
>>>> Graceful shutdown of routes completed in 0 seconds
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service org.apache.camel.impl.RouteService@dd421f
>>>> [CP Connection(4)-10.58.123.154] RouteService TRACE
>>>> Stopping route:
>>>> EventDrivenConsumerRoute[Endpoint[http://0.0.0.0:8686/hello] ->
>>>> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>>> Channel[sendTo(Endpoint[multiplier])]])]]
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service Consumer[http://0.0.0.0:8686/hello]
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>>> Channel[sendTo(Endpoint[multiplier])]])]
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>>> Channel[sendTo(Endpoint[multiplier])]])
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>>> Channel[sendTo(Endpoint[multiplier])]]
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service Channel[sendTo(Endpoint[multiplier])]
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service sendTo(Endpoint[multiplier])
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> EventDrivenConsumerRoute[Endpoint[http://0.0.0.0:8686/hello] ->
>>>> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>>> Channel[sendTo(Endpoint[multiplier])]])]]
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service Producer[multiplier]
>>>> [CP Connection(4)-10.58.123.154] ProcessorEndpoint$1 DEBUG
>>>> Stopping producer: Producer[multiplier]
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service Producer[jms://queue:numbers2?requestTimeout=50000]
>>>> [CP Connection(4)-10.58.123.154] JmsProducer DEBUG
>>>> Stopping producer: Producer[jms://queue:numbers2?requestTimeout=50000]
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service org.apache.camel.impl.DefaultShutdownStrategy@1c5a33b
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> org.apache.camel.impl.DefaultInflightRepository@d8978c
>>>> [CP Connection(4)-10.58.123.154] DefaultInflightRepository INFO
>>>> Shutting down with no inflight exchanges.
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> org.apache.camel.impl.converter.DefaultTypeConverter@da9067
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> org.apache.activemq.camel.component.ActiveMQComponent@1104da7
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> org.apache.camel.component.event.EventComponent@1031310
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> org.apache.camel.example.server.MyHttpJettyComponent@659812
>>>> [CP Connection(4)-10.58.123.154] SharedProducerServicePool DEBUG
>>>> Stopping service pool:
>>>> org.apache.camel.impl.SharedProducerServicePool@11d20d3
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> org.apache.camel.impl.DefaultInflightRepository@d8978c
>>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>>> Stopping service
>>>> org.apache.camel.management.ManagedManagementStrategy@1894800
>>>> [CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
>>>> Apache Camel 2.2.0 (CamelContext:camel) stopped
>>>> [ InactivityMonitor WriteCheck] InactivityMonitor DEBUG
>>>> 10001 ms elapsed since last write check.
>>>> [ InactivityMonitor WriteCheck] InactivityMonitor TRACE
>>>> Message sent since last write check, resetting flag
>>>> [ InactivityMonitor ReadCheck] InactivityMonitor DEBUG
>>>> 30004 ms elapsed since last read check.
>>>> [ InactivityMonitor ReadCheck] InactivityMonitor TRACE
>>>> Message received since last read check, resetting flag:
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Apr 15, 2010 at 8:42 AM, Ilya S <de...@gmail.com> wrote:
>>>>> Thanks Claus,
>>>>>
>>>>> I'm currently back to using 2.2.0 because I can see the source code.
>>>>> But I tried 2.3-SNAPSHOT also with the same issue.
>>>>> Please, see my full route in previous post.
>>>>>
>>>>>
>>>>> On Thu, Apr 15, 2010 at 2:17 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>>>>> Hi
>>>>>>
>>>>>> All the unit tests run with graceful shutdown by default, and all unit
>>>>>> tests with camel-jetty shut down with no issues.
>>>>>>
>>>>>> The in flight repository do keep track of all in flights. I can only
>>>>>> think of if you use the aggregator or some custom code which
>>>>>> can cause the exchange to be completed in Camels point of view.
>>>>>>
>>>>>> Can you post your entire route?
>>>>>>
>>>>>> And as always report which version of Camel you are using? And try
>>>>>> with the latest stable and even with 2.3-SNAPSHOT.
>>>>>>
>>>>>>
>>>>>> On Wed, Apr 14, 2010 at 7:41 PM, Ilya S <de...@gmail.com> wrote:
>>>>>>> Hi All,
>>>>>>>
>>>>>>> Fist of all, thank you for this great product!
>>>>>>> Every new release brings excellent new features just in time.
>>>>>>>
>>>>>>> I'm currently investigating Graceful Shutdown feature, and I'm am
>>>>>>> wondering if graceful shutdown is supported for Camel-Jetty component?
>>>>>>> I was able to successfully shut down JMS component. However,
>>>>>>> JettyHttpCoponent seems to have issues when shutting down.
>>>>>>>
>>>>>>> In my simple test (adopted from camel-example-spring-jms:ServerRoute
>>>>>>> example) I have a simple route that looks like this:
>>>>>>>
>>>>>>> from("jetty:http://0.0.0.0:8686/hello")
>>>>>>> ..
>>>>>>> .. do work
>>>>>>> ...
>>>>>>> .to("jms:queue:numbers2?requestTimeout=50000")
>>>>>>> .to("multiplier");
>>>>>>>
>>>>>>>
>>>>>>> My producers simply send numbers to http endpoint..
>>>>>>>
>>>>>>>
>>>>>>> When I shutdown this context gracefully I'm getting all sort of
>>>>>>> exceptions from Jetty, and the context does not seem to wait until
>>>>>>> inflight exchanges have finished.
>>>>>>> I'm using the defaults everywhere (e.g. default shutdown strategy).
>>>>>>>
>>>>>>> Is shutdown supported for Jetty, and if so how should one configure
>>>>>>> jetty to initiate proper shutdown?
>>>>>>>
>>>>>>> Thank you,
>>>>>>>
>>>>>>> Ilya.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> Apache Camel Committer
>>>>>>
>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>> Open Source Integration: http://fusesource.com
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Claus Ibsen <cl...@gmail.com>.
See this unit test
http://svn.apache.org/viewvc?rev=935691&view=rev
On Mon, Apr 19, 2010 at 7:52 PM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> Just defer shutting down the Jetty route.
> http://camel.apache.org/graceful-shutdown.html
>
> On Mon, Apr 19, 2010 at 7:08 PM, Ilya S <de...@gmail.com> wrote:
>> I'd like to bump this thread up, and see if anyone has any suggestions.
>> Thanks.
>>
>> On Thu, Apr 15, 2010 at 10:24 AM, Ilya S <de...@gmail.com> wrote:
>>> Here is a more detailed log (with TRACE level on) if that can help:
>>>
>>> [CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
>>> Apache Camel 2.2.0 (CamelContext:camel) is stopping
>>> [CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
>>> Starting to graceful shutdown routes (timeout 600000 seconds)
>>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy DEBUG
>>> There are 1 routes to shutdown
>>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy TRACE
>>> Shutting down route: route1 with options
>>> [Default,CompleteCurrentTaskOnly]
>>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy TRACE
>>> Shutting down: Consumer[http://0.0.0.0:8686/hello]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service Consumer[http://0.0.0.0:8686/hello]
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> Container Server@8032df - SelectChannelConnector@0.0.0.0:8686 as
>>> connector
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopped org.mortbay.jetty.nio.SelectChannelConnector$1@28df48
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopped SelectChannelConnector@0.0.0.0:8686
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopping Server@8032df
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopping ContextHandlerCollection@8bfdea
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopping org.mortbay.jetty.servlet.Context@1157f77{/,null}
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopping ServletHandler@6399ae
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopped org.apache.camel.component.http.CamelServlet-29615099
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopped ServletHandler@6399ae
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopping ErrorHandler@56acfa
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopped ErrorHandler@56acfa
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopped org.mortbay.jetty.servlet.Context@1157f77{/,null}
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopped ContextHandlerCollection@8bfdea
>>> [3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
>>> to: Recording duration: 2342 millis for exchange: Exchange[Message: 2]
>>> [3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
>>> to: Recording duration: 2342 millis for exchange: Exchange[Message: 2]
>>> [3850238@qtp-9243153-3 - /hello] DefaultExceptionPolicyStrategy TRACE
>>> Finding best suited exception policy for thrown exception
>>> java.lang.InterruptedException
>>> [3850238@qtp-9243153-3 - /hello] DefaultExceptionPolicyStrategy TRACE
>>> Found 0 candidates
>>> [2390869@qtp-9243153-4 - /hello] JmsProducer DEBUG
>>> Future interrupted: java.lang.InterruptedException
>>> java.lang.InterruptedException
>>> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:926)
>>> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1201)
>>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:210)
>>> at java.util.concurrent.FutureTask.get(FutureTask.java:85)
>>> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:227)
>>> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
>>> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
>>> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
>>> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
>>> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
>>> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
>>> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
>>> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
>>> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
>>> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
>>> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
>>> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>>> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
>>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
>>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>> at org.mortbay.jetty.Server.handle(Server.java:322)
>>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
>>> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>> [1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
>>> to: Recording duration: 19757 millis for exchange: Exchange[Message:
>>> 1]
>>> [3850238@qtp-9243153-3 - /hello] DefaultErrorHandler DEBUG
>>> Failed delivery for exchangeId: 21b68982-8ec1-405b-97d8-56f6e533115f.
>>> On delivery attempt: 0 caught: java.lang.InterruptedException
>>> [1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
>>> to: Recording duration: 19758 millis for exchange: Exchange[Message:
>>> 1]
>>> [3850238@qtp-9243153-3 - /hello] DefaultTypeConverter TRACE
>>> Converting java.lang.Boolean -> java.lang.Object with value: false
>>> [3850238@qtp-9243153-3 - /hello] DefaultErrorHandler ERROR
>>> Failed delivery for exchangeId: 21b68982-8ec1-405b-97d8-56f6e533115f.
>>> Exhausted after delivery attempt: 1 caught:
>>> java.lang.InterruptedException
>>> java.lang.InterruptedException
>>> at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:154)
>>> at org.apache.log4j.Category.forcedLog(Category.java:388)
>>> at org.apache.log4j.Category.log(Category.java:853)
>>> at org.apache.commons.logging.impl.Log4JLogger.debug(Log4JLogger.java:188)
>>> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:231)
>>> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
>>> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
>>> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
>>> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
>>> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
>>> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
>>> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
>>> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
>>> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
>>> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
>>> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
>>> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>>> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
>>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
>>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>> at org.mortbay.jetty.Server.handle(Server.java:322)
>>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
>>> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>> [2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
>>> to: Recording duration: 1826 millis for exchange: Exchange[Message: 3]
>>> [1671238@qtp-9243153-2 - /hello] DefaultExceptionPolicyStrategy TRACE
>>> Finding best suited exception policy for thrown exception
>>> java.lang.InterruptedException
>>> [3850238@qtp-9243153-3 - /hello] DefaultTypeConverter TRACE
>>> Converting java.lang.Boolean -> java.lang.Object with value: false
>>> [2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
>>> to: Recording duration: 1827 millis for exchange: Exchange[Message: 3]
>>> [1671238@qtp-9243153-2 - /hello] DefaultExceptionPolicyStrategy TRACE
>>> Found 0 candidates
>>> [3850238@qtp-9243153-3 - /hello] DefaultErrorHandler DEBUG
>>> This exchange is not handled so its marked as failed:
>>> Exchange[Message: 2]
>>> [2390869@qtp-9243153-4 - /hello] DefaultExceptionPolicyStrategy TRACE
>>> Finding best suited exception policy for thrown exception
>>> org.apache.camel.ExchangeTimedOutException
>>> [1671238@qtp-9243153-2 - /hello] DefaultErrorHandler DEBUG
>>> Failed delivery for exchangeId: 915083b6-4291-4716-929d-aa451e29e495.
>>> On delivery attempt: 0 caught: java.lang.InterruptedException
>>> [2390869@qtp-9243153-4 - /hello] DefaultExceptionPolicyStrategy TRACE
>>> Found 0 candidates
>>> [3850238@qtp-9243153-3 - /hello] Pipeline DEBUG
>>> Message exchange has failed so breaking out of pipeline:
>>> Exchange[Message: 2] Exception: java.lang.InterruptedException
>>> [1671238@qtp-9243153-2 - /hello] DefaultTypeConverter TRACE
>>> Converting java.lang.Boolean -> java.lang.Object with value: false
>>> [2390869@qtp-9243153-4 - /hello] DefaultErrorHandler DEBUG
>>> Failed delivery for exchangeId: 5903a17f-bff9-4ba5-b8f4-78bc96149195.
>>> On delivery attempt: 0 caught:
>>> org.apache.camel.ExchangeTimedOutException: The OUT message was not
>>> received within: 50000 millis. Exchange[Message: 3]
>>> [1671238@qtp-9243153-2 - /hello] DefaultErrorHandler ERROR
>>> Failed delivery for exchangeId: 915083b6-4291-4716-929d-aa451e29e495.
>>> Exhausted after delivery attempt: 1 caught:
>>> java.lang.InterruptedException
>>> java.lang.InterruptedException
>>> at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:154)
>>> at org.apache.log4j.Category.forcedLog(Category.java:388)
>>> at org.apache.log4j.Category.log(Category.java:853)
>>> at org.apache.commons.logging.impl.Log4JLogger.debug(Log4JLogger.java:188)
>>> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:231)
>>> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
>>> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
>>> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
>>> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
>>> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
>>> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
>>> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
>>> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
>>> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
>>> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
>>> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
>>> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>>> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
>>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
>>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>> at org.mortbay.jetty.Server.handle(Server.java:322)
>>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
>>> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>> [2390869@qtp-9243153-4 - /hello] DefaultTypeConverter TRACE
>>> Converting java.lang.Boolean -> java.lang.Object with value: false
>>> [3850238@qtp-9243153-3 - /hello] Pipeline TRACE
>>> Processing complete for exchangeId:
>>> 21b68982-8ec1-405b-97d8-56f6e533115f >>> Exchange[Message: 2]
>>> [1671238@qtp-9243153-2 - /hello] DefaultTypeConverter TRACE
>>> Converting java.lang.Boolean -> java.lang.Object with value: false
>>> [2390869@qtp-9243153-4 - /hello] DefaultErrorHandler ERROR
>>> Failed delivery for exchangeId: 5903a17f-bff9-4ba5-b8f4-78bc96149195.
>>> Exhausted after delivery attempt: 1 caught:
>>> org.apache.camel.ExchangeTimedOutException: The OUT message was not
>>> received within: 50000 millis. Exchange[Message: 3]
>>> org.apache.camel.ExchangeTimedOutException: The OUT message was not
>>> received within: 50000 millis. Exchange[Message: 3]
>>> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:265)
>>> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
>>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
>>> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
>>> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
>>> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
>>> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
>>> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
>>> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
>>> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
>>> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
>>> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
>>> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
>>> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>>> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>>> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
>>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
>>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>> at org.mortbay.jetty.Server.handle(Server.java:322)
>>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
>>> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>> [1671238@qtp-9243153-2 - /hello] DefaultErrorHandler DEBUG
>>> This exchange is not handled so its marked as failed:
>>> Exchange[Message: 1]
>>> [3850238@qtp-9243153-3 - /hello] DefaultInflightRepository TRACE
>>> Total 2 inflight exchanges. Last removed:
>>> 21b68982-8ec1-405b-97d8-56f6e533115f
>>> [2390869@qtp-9243153-4 - /hello] DefaultTypeConverter TRACE
>>> Converting java.lang.Boolean -> java.lang.Object with value: false
>>> [1671238@qtp-9243153-2 - /hello] Pipeline DEBUG
>>> Message exchange has failed so breaking out of pipeline:
>>> Exchange[Message: 1] Exception: java.lang.InterruptedException
>>> [3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
>>> route: Recording duration: 2347 millis for exchange: Exchange[Message:
>>> [Body is instance of java.io.InputStream]]
>>> [2390869@qtp-9243153-4 - /hello] DefaultErrorHandler DEBUG
>>> This exchange is not handled so its marked as failed:
>>> Exchange[Message: 3]
>>> [1671238@qtp-9243153-2 - /hello] Pipeline TRACE
>>> Processing complete for exchangeId:
>>> 915083b6-4291-4716-929d-aa451e29e495 >>> Exchange[Message: 1]
>>> [2390869@qtp-9243153-4 - /hello] Pipeline DEBUG
>>> Message exchange has failed so breaking out of pipeline:
>>> Exchange[Message: 3] Exception:
>>> org.apache.camel.ExchangeTimedOutException: The OUT message was not
>>> received within: 50000 millis. Exchange[Message: 3]
>>> [1671238@qtp-9243153-2 - /hello] DefaultInflightRepository TRACE
>>> Total 1 inflight exchanges. Last removed:
>>> 915083b6-4291-4716-929d-aa451e29e495
>>> [2390869@qtp-9243153-4 - /hello] Pipeline TRACE
>>> Processing complete for exchangeId:
>>> 5903a17f-bff9-4ba5-b8f4-78bc96149195 >>> Exchange[Message: 3]
>>> [1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
>>> route: Recording duration: 19763 millis for exchange:
>>> Exchange[Message: [Body is instance of java.io.InputStream]]
>>> [2390869@qtp-9243153-4 - /hello] DefaultInflightRepository TRACE
>>> Total 0 inflight exchanges. Last removed:
>>> 5903a17f-bff9-4ba5-b8f4-78bc96149195
>>> [3850238@qtp-9243153-3 - /hello] log DEBUG
>>> RESPONSE /hello 500
>>> [2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
>>> route: Recording duration: 1830 millis for exchange: Exchange[Message:
>>> [Body is instance of java.io.InputStream]]
>>> [ 23850238@qtp-9243153-3] log DEBUG EOF
>>> [2390869@qtp-9243153-4 - /hello] log DEBUG
>>> RESPONSE /hello 500
>>> [ 2390869@qtp-9243153-4] log DEBUG EOF
>>> [1671238@qtp-9243153-2 - /hello] log DEBUG
>>> RESPONSE /hello 500
>>> [ 1671238@qtp-9243153-2] log DEBUG EOF
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopped org.mortbay.thread.QueuedThreadPool@8d0a11
>>> [ Camel thread 11: ShutdownTask] log DEBUG
>>> stopped Server@8032df
>>> [ Camel thread 11: ShutdownTask] HttpConsumer DEBUG
>>> Stopping consumer: Consumer[http://0.0.0.0:8686/hello]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>> Channel[sendTo(Endpoint[multiplier])]])]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>> Channel[sendTo(Endpoint[multiplier])]])
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>> Channel[sendTo(Endpoint[multiplier])]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Instrumentation:process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> DefaultErrorHandler[Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Instrumentation:to[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> org.apache.camel.impl.SharedProducerServicePool@11d20d3
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> DefaultErrorHandler[Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Instrumentation:process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> DefaultErrorHandler[Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service: Channel[sendTo(Endpoint[multiplier])]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service: Instrumentation:to[TraceInterceptor[To[multiplier]]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service: TraceInterceptor[To[multiplier]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service: Instrumentation:to[sendTo(Endpoint[multiplier])]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service: sendTo(Endpoint[multiplier])
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> org.apache.camel.impl.SharedProducerServicePool@11d20d3
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service:
>>> DefaultErrorHandler[Instrumentation:to[TraceInterceptor[To[multiplier]]]]
>>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>>> Stopping service: Instrumentation:to[TraceInterceptor[To[multiplier]]]
>>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy DEBUG
>>> Shutdown complete for: Consumer[http://0.0.0.0:8686/hello]
>>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy INFO
>>> Route: route1 shutdown complete.
>>> [CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
>>> Graceful shutdown of routes completed in 0 seconds
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service org.apache.camel.impl.RouteService@dd421f
>>> [CP Connection(4)-10.58.123.154] RouteService TRACE
>>> Stopping route:
>>> EventDrivenConsumerRoute[Endpoint[http://0.0.0.0:8686/hello] ->
>>> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>> Channel[sendTo(Endpoint[multiplier])]])]]
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service Consumer[http://0.0.0.0:8686/hello]
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>> Channel[sendTo(Endpoint[multiplier])]])]
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>> Channel[sendTo(Endpoint[multiplier])]])
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>> Channel[sendTo(Endpoint[multiplier])]]
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service Channel[sendTo(Endpoint[multiplier])]
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service sendTo(Endpoint[multiplier])
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> EventDrivenConsumerRoute[Endpoint[http://0.0.0.0:8686/hello] ->
>>> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>>> Channel[sendTo(Endpoint[multiplier])]])]]
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service Producer[multiplier]
>>> [CP Connection(4)-10.58.123.154] ProcessorEndpoint$1 DEBUG
>>> Stopping producer: Producer[multiplier]
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service Producer[jms://queue:numbers2?requestTimeout=50000]
>>> [CP Connection(4)-10.58.123.154] JmsProducer DEBUG
>>> Stopping producer: Producer[jms://queue:numbers2?requestTimeout=50000]
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service org.apache.camel.impl.DefaultShutdownStrategy@1c5a33b
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> org.apache.camel.impl.DefaultInflightRepository@d8978c
>>> [CP Connection(4)-10.58.123.154] DefaultInflightRepository INFO
>>> Shutting down with no inflight exchanges.
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> org.apache.camel.impl.converter.DefaultTypeConverter@da9067
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> org.apache.activemq.camel.component.ActiveMQComponent@1104da7
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> org.apache.camel.component.event.EventComponent@1031310
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> org.apache.camel.example.server.MyHttpJettyComponent@659812
>>> [CP Connection(4)-10.58.123.154] SharedProducerServicePool DEBUG
>>> Stopping service pool:
>>> org.apache.camel.impl.SharedProducerServicePool@11d20d3
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> org.apache.camel.impl.DefaultInflightRepository@d8978c
>>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>>> Stopping service
>>> org.apache.camel.management.ManagedManagementStrategy@1894800
>>> [CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
>>> Apache Camel 2.2.0 (CamelContext:camel) stopped
>>> [ InactivityMonitor WriteCheck] InactivityMonitor DEBUG
>>> 10001 ms elapsed since last write check.
>>> [ InactivityMonitor WriteCheck] InactivityMonitor TRACE
>>> Message sent since last write check, resetting flag
>>> [ InactivityMonitor ReadCheck] InactivityMonitor DEBUG
>>> 30004 ms elapsed since last read check.
>>> [ InactivityMonitor ReadCheck] InactivityMonitor TRACE
>>> Message received since last read check, resetting flag:
>>>
>>>
>>>
>>>
>>> On Thu, Apr 15, 2010 at 8:42 AM, Ilya S <de...@gmail.com> wrote:
>>>> Thanks Claus,
>>>>
>>>> I'm currently back to using 2.2.0 because I can see the source code.
>>>> But I tried 2.3-SNAPSHOT also with the same issue.
>>>> Please, see my full route in previous post.
>>>>
>>>>
>>>> On Thu, Apr 15, 2010 at 2:17 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>>>> Hi
>>>>>
>>>>> All the unit tests run with graceful shutdown by default, and all unit
>>>>> tests with camel-jetty shut down with no issues.
>>>>>
>>>>> The in flight repository do keep track of all in flights. I can only
>>>>> think of if you use the aggregator or some custom code which
>>>>> can cause the exchange to be completed in Camels point of view.
>>>>>
>>>>> Can you post your entire route?
>>>>>
>>>>> And as always report which version of Camel you are using? And try
>>>>> with the latest stable and even with 2.3-SNAPSHOT.
>>>>>
>>>>>
>>>>> On Wed, Apr 14, 2010 at 7:41 PM, Ilya S <de...@gmail.com> wrote:
>>>>>> Hi All,
>>>>>>
>>>>>> Fist of all, thank you for this great product!
>>>>>> Every new release brings excellent new features just in time.
>>>>>>
>>>>>> I'm currently investigating Graceful Shutdown feature, and I'm am
>>>>>> wondering if graceful shutdown is supported for Camel-Jetty component?
>>>>>> I was able to successfully shut down JMS component. However,
>>>>>> JettyHttpCoponent seems to have issues when shutting down.
>>>>>>
>>>>>> In my simple test (adopted from camel-example-spring-jms:ServerRoute
>>>>>> example) I have a simple route that looks like this:
>>>>>>
>>>>>> from("jetty:http://0.0.0.0:8686/hello")
>>>>>> ..
>>>>>> .. do work
>>>>>> ...
>>>>>> .to("jms:queue:numbers2?requestTimeout=50000")
>>>>>> .to("multiplier");
>>>>>>
>>>>>>
>>>>>> My producers simply send numbers to http endpoint..
>>>>>>
>>>>>>
>>>>>> When I shutdown this context gracefully I'm getting all sort of
>>>>>> exceptions from Jetty, and the context does not seem to wait until
>>>>>> inflight exchanges have finished.
>>>>>> I'm using the defaults everywhere (e.g. default shutdown strategy).
>>>>>>
>>>>>> Is shutdown supported for Jetty, and if so how should one configure
>>>>>> jetty to initiate proper shutdown?
>>>>>>
>>>>>> Thank you,
>>>>>>
>>>>>> Ilya.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>
>>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Just defer shutting down the Jetty route.
http://camel.apache.org/graceful-shutdown.html
On Mon, Apr 19, 2010 at 7:08 PM, Ilya S <de...@gmail.com> wrote:
> I'd like to bump this thread up, and see if anyone has any suggestions.
> Thanks.
>
> On Thu, Apr 15, 2010 at 10:24 AM, Ilya S <de...@gmail.com> wrote:
>> Here is a more detailed log (with TRACE level on) if that can help:
>>
>> [CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
>> Apache Camel 2.2.0 (CamelContext:camel) is stopping
>> [CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
>> Starting to graceful shutdown routes (timeout 600000 seconds)
>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy DEBUG
>> There are 1 routes to shutdown
>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy TRACE
>> Shutting down route: route1 with options
>> [Default,CompleteCurrentTaskOnly]
>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy TRACE
>> Shutting down: Consumer[http://0.0.0.0:8686/hello]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service Consumer[http://0.0.0.0:8686/hello]
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> Container Server@8032df - SelectChannelConnector@0.0.0.0:8686 as
>> connector
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopped org.mortbay.jetty.nio.SelectChannelConnector$1@28df48
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopped SelectChannelConnector@0.0.0.0:8686
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopping Server@8032df
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopping ContextHandlerCollection@8bfdea
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopping org.mortbay.jetty.servlet.Context@1157f77{/,null}
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopping ServletHandler@6399ae
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopped org.apache.camel.component.http.CamelServlet-29615099
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopped ServletHandler@6399ae
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopping ErrorHandler@56acfa
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopped ErrorHandler@56acfa
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopped org.mortbay.jetty.servlet.Context@1157f77{/,null}
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopped ContextHandlerCollection@8bfdea
>> [3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
>> to: Recording duration: 2342 millis for exchange: Exchange[Message: 2]
>> [3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
>> to: Recording duration: 2342 millis for exchange: Exchange[Message: 2]
>> [3850238@qtp-9243153-3 - /hello] DefaultExceptionPolicyStrategy TRACE
>> Finding best suited exception policy for thrown exception
>> java.lang.InterruptedException
>> [3850238@qtp-9243153-3 - /hello] DefaultExceptionPolicyStrategy TRACE
>> Found 0 candidates
>> [2390869@qtp-9243153-4 - /hello] JmsProducer DEBUG
>> Future interrupted: java.lang.InterruptedException
>> java.lang.InterruptedException
>> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:926)
>> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1201)
>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:210)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:85)
>> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:227)
>> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
>> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
>> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
>> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
>> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
>> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
>> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
>> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
>> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
>> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
>> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
>> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>> at org.mortbay.jetty.Server.handle(Server.java:322)
>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
>> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>> [1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
>> to: Recording duration: 19757 millis for exchange: Exchange[Message:
>> 1]
>> [3850238@qtp-9243153-3 - /hello] DefaultErrorHandler DEBUG
>> Failed delivery for exchangeId: 21b68982-8ec1-405b-97d8-56f6e533115f.
>> On delivery attempt: 0 caught: java.lang.InterruptedException
>> [1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
>> to: Recording duration: 19758 millis for exchange: Exchange[Message:
>> 1]
>> [3850238@qtp-9243153-3 - /hello] DefaultTypeConverter TRACE
>> Converting java.lang.Boolean -> java.lang.Object with value: false
>> [3850238@qtp-9243153-3 - /hello] DefaultErrorHandler ERROR
>> Failed delivery for exchangeId: 21b68982-8ec1-405b-97d8-56f6e533115f.
>> Exhausted after delivery attempt: 1 caught:
>> java.lang.InterruptedException
>> java.lang.InterruptedException
>> at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:154)
>> at org.apache.log4j.Category.forcedLog(Category.java:388)
>> at org.apache.log4j.Category.log(Category.java:853)
>> at org.apache.commons.logging.impl.Log4JLogger.debug(Log4JLogger.java:188)
>> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:231)
>> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
>> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
>> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
>> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
>> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
>> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
>> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
>> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
>> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
>> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
>> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
>> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>> at org.mortbay.jetty.Server.handle(Server.java:322)
>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
>> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>> [2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
>> to: Recording duration: 1826 millis for exchange: Exchange[Message: 3]
>> [1671238@qtp-9243153-2 - /hello] DefaultExceptionPolicyStrategy TRACE
>> Finding best suited exception policy for thrown exception
>> java.lang.InterruptedException
>> [3850238@qtp-9243153-3 - /hello] DefaultTypeConverter TRACE
>> Converting java.lang.Boolean -> java.lang.Object with value: false
>> [2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
>> to: Recording duration: 1827 millis for exchange: Exchange[Message: 3]
>> [1671238@qtp-9243153-2 - /hello] DefaultExceptionPolicyStrategy TRACE
>> Found 0 candidates
>> [3850238@qtp-9243153-3 - /hello] DefaultErrorHandler DEBUG
>> This exchange is not handled so its marked as failed:
>> Exchange[Message: 2]
>> [2390869@qtp-9243153-4 - /hello] DefaultExceptionPolicyStrategy TRACE
>> Finding best suited exception policy for thrown exception
>> org.apache.camel.ExchangeTimedOutException
>> [1671238@qtp-9243153-2 - /hello] DefaultErrorHandler DEBUG
>> Failed delivery for exchangeId: 915083b6-4291-4716-929d-aa451e29e495.
>> On delivery attempt: 0 caught: java.lang.InterruptedException
>> [2390869@qtp-9243153-4 - /hello] DefaultExceptionPolicyStrategy TRACE
>> Found 0 candidates
>> [3850238@qtp-9243153-3 - /hello] Pipeline DEBUG
>> Message exchange has failed so breaking out of pipeline:
>> Exchange[Message: 2] Exception: java.lang.InterruptedException
>> [1671238@qtp-9243153-2 - /hello] DefaultTypeConverter TRACE
>> Converting java.lang.Boolean -> java.lang.Object with value: false
>> [2390869@qtp-9243153-4 - /hello] DefaultErrorHandler DEBUG
>> Failed delivery for exchangeId: 5903a17f-bff9-4ba5-b8f4-78bc96149195.
>> On delivery attempt: 0 caught:
>> org.apache.camel.ExchangeTimedOutException: The OUT message was not
>> received within: 50000 millis. Exchange[Message: 3]
>> [1671238@qtp-9243153-2 - /hello] DefaultErrorHandler ERROR
>> Failed delivery for exchangeId: 915083b6-4291-4716-929d-aa451e29e495.
>> Exhausted after delivery attempt: 1 caught:
>> java.lang.InterruptedException
>> java.lang.InterruptedException
>> at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:154)
>> at org.apache.log4j.Category.forcedLog(Category.java:388)
>> at org.apache.log4j.Category.log(Category.java:853)
>> at org.apache.commons.logging.impl.Log4JLogger.debug(Log4JLogger.java:188)
>> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:231)
>> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
>> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
>> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
>> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
>> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
>> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
>> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
>> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
>> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
>> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
>> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
>> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>> at org.mortbay.jetty.Server.handle(Server.java:322)
>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
>> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>> [2390869@qtp-9243153-4 - /hello] DefaultTypeConverter TRACE
>> Converting java.lang.Boolean -> java.lang.Object with value: false
>> [3850238@qtp-9243153-3 - /hello] Pipeline TRACE
>> Processing complete for exchangeId:
>> 21b68982-8ec1-405b-97d8-56f6e533115f >>> Exchange[Message: 2]
>> [1671238@qtp-9243153-2 - /hello] DefaultTypeConverter TRACE
>> Converting java.lang.Boolean -> java.lang.Object with value: false
>> [2390869@qtp-9243153-4 - /hello] DefaultErrorHandler ERROR
>> Failed delivery for exchangeId: 5903a17f-bff9-4ba5-b8f4-78bc96149195.
>> Exhausted after delivery attempt: 1 caught:
>> org.apache.camel.ExchangeTimedOutException: The OUT message was not
>> received within: 50000 millis. Exchange[Message: 3]
>> org.apache.camel.ExchangeTimedOutException: The OUT message was not
>> received within: 50000 millis. Exchange[Message: 3]
>> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:265)
>> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
>> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
>> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
>> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
>> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
>> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
>> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
>> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
>> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
>> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
>> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
>> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
>> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
>> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
>> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>> at org.mortbay.jetty.Server.handle(Server.java:322)
>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
>> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>> [1671238@qtp-9243153-2 - /hello] DefaultErrorHandler DEBUG
>> This exchange is not handled so its marked as failed:
>> Exchange[Message: 1]
>> [3850238@qtp-9243153-3 - /hello] DefaultInflightRepository TRACE
>> Total 2 inflight exchanges. Last removed:
>> 21b68982-8ec1-405b-97d8-56f6e533115f
>> [2390869@qtp-9243153-4 - /hello] DefaultTypeConverter TRACE
>> Converting java.lang.Boolean -> java.lang.Object with value: false
>> [1671238@qtp-9243153-2 - /hello] Pipeline DEBUG
>> Message exchange has failed so breaking out of pipeline:
>> Exchange[Message: 1] Exception: java.lang.InterruptedException
>> [3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
>> route: Recording duration: 2347 millis for exchange: Exchange[Message:
>> [Body is instance of java.io.InputStream]]
>> [2390869@qtp-9243153-4 - /hello] DefaultErrorHandler DEBUG
>> This exchange is not handled so its marked as failed:
>> Exchange[Message: 3]
>> [1671238@qtp-9243153-2 - /hello] Pipeline TRACE
>> Processing complete for exchangeId:
>> 915083b6-4291-4716-929d-aa451e29e495 >>> Exchange[Message: 1]
>> [2390869@qtp-9243153-4 - /hello] Pipeline DEBUG
>> Message exchange has failed so breaking out of pipeline:
>> Exchange[Message: 3] Exception:
>> org.apache.camel.ExchangeTimedOutException: The OUT message was not
>> received within: 50000 millis. Exchange[Message: 3]
>> [1671238@qtp-9243153-2 - /hello] DefaultInflightRepository TRACE
>> Total 1 inflight exchanges. Last removed:
>> 915083b6-4291-4716-929d-aa451e29e495
>> [2390869@qtp-9243153-4 - /hello] Pipeline TRACE
>> Processing complete for exchangeId:
>> 5903a17f-bff9-4ba5-b8f4-78bc96149195 >>> Exchange[Message: 3]
>> [1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
>> route: Recording duration: 19763 millis for exchange:
>> Exchange[Message: [Body is instance of java.io.InputStream]]
>> [2390869@qtp-9243153-4 - /hello] DefaultInflightRepository TRACE
>> Total 0 inflight exchanges. Last removed:
>> 5903a17f-bff9-4ba5-b8f4-78bc96149195
>> [3850238@qtp-9243153-3 - /hello] log DEBUG
>> RESPONSE /hello 500
>> [2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
>> route: Recording duration: 1830 millis for exchange: Exchange[Message:
>> [Body is instance of java.io.InputStream]]
>> [ 23850238@qtp-9243153-3] log DEBUG EOF
>> [2390869@qtp-9243153-4 - /hello] log DEBUG
>> RESPONSE /hello 500
>> [ 2390869@qtp-9243153-4] log DEBUG EOF
>> [1671238@qtp-9243153-2 - /hello] log DEBUG
>> RESPONSE /hello 500
>> [ 1671238@qtp-9243153-2] log DEBUG EOF
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopped org.mortbay.thread.QueuedThreadPool@8d0a11
>> [ Camel thread 11: ShutdownTask] log DEBUG
>> stopped Server@8032df
>> [ Camel thread 11: ShutdownTask] HttpConsumer DEBUG
>> Stopping consumer: Consumer[http://0.0.0.0:8686/hello]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>> Channel[sendTo(Endpoint[multiplier])]])]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>> Channel[sendTo(Endpoint[multiplier])]])
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>> Channel[sendTo(Endpoint[multiplier])]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Instrumentation:process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> DefaultErrorHandler[Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Instrumentation:to[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> org.apache.camel.impl.SharedProducerServicePool@11d20d3
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> DefaultErrorHandler[Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Instrumentation:process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> DefaultErrorHandler[Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service: Channel[sendTo(Endpoint[multiplier])]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service: Instrumentation:to[TraceInterceptor[To[multiplier]]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service: TraceInterceptor[To[multiplier]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service: Instrumentation:to[sendTo(Endpoint[multiplier])]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service: sendTo(Endpoint[multiplier])
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> org.apache.camel.impl.SharedProducerServicePool@11d20d3
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service:
>> DefaultErrorHandler[Instrumentation:to[TraceInterceptor[To[multiplier]]]]
>> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
>> Stopping service: Instrumentation:to[TraceInterceptor[To[multiplier]]]
>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy DEBUG
>> Shutdown complete for: Consumer[http://0.0.0.0:8686/hello]
>> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy INFO
>> Route: route1 shutdown complete.
>> [CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
>> Graceful shutdown of routes completed in 0 seconds
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service org.apache.camel.impl.RouteService@dd421f
>> [CP Connection(4)-10.58.123.154] RouteService TRACE
>> Stopping route:
>> EventDrivenConsumerRoute[Endpoint[http://0.0.0.0:8686/hello] ->
>> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>> Channel[sendTo(Endpoint[multiplier])]])]]
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service Consumer[http://0.0.0.0:8686/hello]
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>> Channel[sendTo(Endpoint[multiplier])]])]
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>> Channel[sendTo(Endpoint[multiplier])]])
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>> Channel[sendTo(Endpoint[multiplier])]]
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service Channel[sendTo(Endpoint[multiplier])]
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service sendTo(Endpoint[multiplier])
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> EventDrivenConsumerRoute[Endpoint[http://0.0.0.0:8686/hello] ->
>> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
>> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
>> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
>> Channel[sendTo(Endpoint[multiplier])]])]]
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service Producer[multiplier]
>> [CP Connection(4)-10.58.123.154] ProcessorEndpoint$1 DEBUG
>> Stopping producer: Producer[multiplier]
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service Producer[jms://queue:numbers2?requestTimeout=50000]
>> [CP Connection(4)-10.58.123.154] JmsProducer DEBUG
>> Stopping producer: Producer[jms://queue:numbers2?requestTimeout=50000]
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service org.apache.camel.impl.DefaultShutdownStrategy@1c5a33b
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> org.apache.camel.impl.DefaultInflightRepository@d8978c
>> [CP Connection(4)-10.58.123.154] DefaultInflightRepository INFO
>> Shutting down with no inflight exchanges.
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> org.apache.camel.impl.converter.DefaultTypeConverter@da9067
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> org.apache.activemq.camel.component.ActiveMQComponent@1104da7
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> org.apache.camel.component.event.EventComponent@1031310
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> org.apache.camel.example.server.MyHttpJettyComponent@659812
>> [CP Connection(4)-10.58.123.154] SharedProducerServicePool DEBUG
>> Stopping service pool:
>> org.apache.camel.impl.SharedProducerServicePool@11d20d3
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> org.apache.camel.impl.DefaultInflightRepository@d8978c
>> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
>> Stopping service
>> org.apache.camel.management.ManagedManagementStrategy@1894800
>> [CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
>> Apache Camel 2.2.0 (CamelContext:camel) stopped
>> [ InactivityMonitor WriteCheck] InactivityMonitor DEBUG
>> 10001 ms elapsed since last write check.
>> [ InactivityMonitor WriteCheck] InactivityMonitor TRACE
>> Message sent since last write check, resetting flag
>> [ InactivityMonitor ReadCheck] InactivityMonitor DEBUG
>> 30004 ms elapsed since last read check.
>> [ InactivityMonitor ReadCheck] InactivityMonitor TRACE
>> Message received since last read check, resetting flag:
>>
>>
>>
>>
>> On Thu, Apr 15, 2010 at 8:42 AM, Ilya S <de...@gmail.com> wrote:
>>> Thanks Claus,
>>>
>>> I'm currently back to using 2.2.0 because I can see the source code.
>>> But I tried 2.3-SNAPSHOT also with the same issue.
>>> Please, see my full route in previous post.
>>>
>>>
>>> On Thu, Apr 15, 2010 at 2:17 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>>> Hi
>>>>
>>>> All the unit tests run with graceful shutdown by default, and all unit
>>>> tests with camel-jetty shut down with no issues.
>>>>
>>>> The in flight repository do keep track of all in flights. I can only
>>>> think of if you use the aggregator or some custom code which
>>>> can cause the exchange to be completed in Camels point of view.
>>>>
>>>> Can you post your entire route?
>>>>
>>>> And as always report which version of Camel you are using? And try
>>>> with the latest stable and even with 2.3-SNAPSHOT.
>>>>
>>>>
>>>> On Wed, Apr 14, 2010 at 7:41 PM, Ilya S <de...@gmail.com> wrote:
>>>>> Hi All,
>>>>>
>>>>> Fist of all, thank you for this great product!
>>>>> Every new release brings excellent new features just in time.
>>>>>
>>>>> I'm currently investigating Graceful Shutdown feature, and I'm am
>>>>> wondering if graceful shutdown is supported for Camel-Jetty component?
>>>>> I was able to successfully shut down JMS component. However,
>>>>> JettyHttpCoponent seems to have issues when shutting down.
>>>>>
>>>>> In my simple test (adopted from camel-example-spring-jms:ServerRoute
>>>>> example) I have a simple route that looks like this:
>>>>>
>>>>> from("jetty:http://0.0.0.0:8686/hello")
>>>>> ..
>>>>> .. do work
>>>>> ...
>>>>> .to("jms:queue:numbers2?requestTimeout=50000")
>>>>> .to("multiplier");
>>>>>
>>>>>
>>>>> My producers simply send numbers to http endpoint..
>>>>>
>>>>>
>>>>> When I shutdown this context gracefully I'm getting all sort of
>>>>> exceptions from Jetty, and the context does not seem to wait until
>>>>> inflight exchanges have finished.
>>>>> I'm using the defaults everywhere (e.g. default shutdown strategy).
>>>>>
>>>>> Is shutdown supported for Jetty, and if so how should one configure
>>>>> jetty to initiate proper shutdown?
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Ilya.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>
>>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Ilya S <de...@gmail.com>.
I'd like to bump this thread up, and see if anyone has any suggestions.
Thanks.
On Thu, Apr 15, 2010 at 10:24 AM, Ilya S <de...@gmail.com> wrote:
> Here is a more detailed log (with TRACE level on) if that can help:
>
> [CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
> Apache Camel 2.2.0 (CamelContext:camel) is stopping
> [CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
> Starting to graceful shutdown routes (timeout 600000 seconds)
> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy DEBUG
> There are 1 routes to shutdown
> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy TRACE
> Shutting down route: route1 with options
> [Default,CompleteCurrentTaskOnly]
> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy TRACE
> Shutting down: Consumer[http://0.0.0.0:8686/hello]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service Consumer[http://0.0.0.0:8686/hello]
> [ Camel thread 11: ShutdownTask] log DEBUG
> Container Server@8032df - SelectChannelConnector@0.0.0.0:8686 as
> connector
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopped org.mortbay.jetty.nio.SelectChannelConnector$1@28df48
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopped SelectChannelConnector@0.0.0.0:8686
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopping Server@8032df
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopping ContextHandlerCollection@8bfdea
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopping org.mortbay.jetty.servlet.Context@1157f77{/,null}
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopping ServletHandler@6399ae
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopped org.apache.camel.component.http.CamelServlet-29615099
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopped ServletHandler@6399ae
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopping ErrorHandler@56acfa
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopped ErrorHandler@56acfa
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopped org.mortbay.jetty.servlet.Context@1157f77{/,null}
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopped ContextHandlerCollection@8bfdea
> [3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
> to: Recording duration: 2342 millis for exchange: Exchange[Message: 2]
> [3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
> to: Recording duration: 2342 millis for exchange: Exchange[Message: 2]
> [3850238@qtp-9243153-3 - /hello] DefaultExceptionPolicyStrategy TRACE
> Finding best suited exception policy for thrown exception
> java.lang.InterruptedException
> [3850238@qtp-9243153-3 - /hello] DefaultExceptionPolicyStrategy TRACE
> Found 0 candidates
> [2390869@qtp-9243153-4 - /hello] JmsProducer DEBUG
> Future interrupted: java.lang.InterruptedException
> java.lang.InterruptedException
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:926)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1201)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:210)
> at java.util.concurrent.FutureTask.get(FutureTask.java:85)
> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:227)
> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:322)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> [1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
> to: Recording duration: 19757 millis for exchange: Exchange[Message:
> 1]
> [3850238@qtp-9243153-3 - /hello] DefaultErrorHandler DEBUG
> Failed delivery for exchangeId: 21b68982-8ec1-405b-97d8-56f6e533115f.
> On delivery attempt: 0 caught: java.lang.InterruptedException
> [1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
> to: Recording duration: 19758 millis for exchange: Exchange[Message:
> 1]
> [3850238@qtp-9243153-3 - /hello] DefaultTypeConverter TRACE
> Converting java.lang.Boolean -> java.lang.Object with value: false
> [3850238@qtp-9243153-3 - /hello] DefaultErrorHandler ERROR
> Failed delivery for exchangeId: 21b68982-8ec1-405b-97d8-56f6e533115f.
> Exhausted after delivery attempt: 1 caught:
> java.lang.InterruptedException
> java.lang.InterruptedException
> at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:154)
> at org.apache.log4j.Category.forcedLog(Category.java:388)
> at org.apache.log4j.Category.log(Category.java:853)
> at org.apache.commons.logging.impl.Log4JLogger.debug(Log4JLogger.java:188)
> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:231)
> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:322)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> [2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
> to: Recording duration: 1826 millis for exchange: Exchange[Message: 3]
> [1671238@qtp-9243153-2 - /hello] DefaultExceptionPolicyStrategy TRACE
> Finding best suited exception policy for thrown exception
> java.lang.InterruptedException
> [3850238@qtp-9243153-3 - /hello] DefaultTypeConverter TRACE
> Converting java.lang.Boolean -> java.lang.Object with value: false
> [2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
> to: Recording duration: 1827 millis for exchange: Exchange[Message: 3]
> [1671238@qtp-9243153-2 - /hello] DefaultExceptionPolicyStrategy TRACE
> Found 0 candidates
> [3850238@qtp-9243153-3 - /hello] DefaultErrorHandler DEBUG
> This exchange is not handled so its marked as failed:
> Exchange[Message: 2]
> [2390869@qtp-9243153-4 - /hello] DefaultExceptionPolicyStrategy TRACE
> Finding best suited exception policy for thrown exception
> org.apache.camel.ExchangeTimedOutException
> [1671238@qtp-9243153-2 - /hello] DefaultErrorHandler DEBUG
> Failed delivery for exchangeId: 915083b6-4291-4716-929d-aa451e29e495.
> On delivery attempt: 0 caught: java.lang.InterruptedException
> [2390869@qtp-9243153-4 - /hello] DefaultExceptionPolicyStrategy TRACE
> Found 0 candidates
> [3850238@qtp-9243153-3 - /hello] Pipeline DEBUG
> Message exchange has failed so breaking out of pipeline:
> Exchange[Message: 2] Exception: java.lang.InterruptedException
> [1671238@qtp-9243153-2 - /hello] DefaultTypeConverter TRACE
> Converting java.lang.Boolean -> java.lang.Object with value: false
> [2390869@qtp-9243153-4 - /hello] DefaultErrorHandler DEBUG
> Failed delivery for exchangeId: 5903a17f-bff9-4ba5-b8f4-78bc96149195.
> On delivery attempt: 0 caught:
> org.apache.camel.ExchangeTimedOutException: The OUT message was not
> received within: 50000 millis. Exchange[Message: 3]
> [1671238@qtp-9243153-2 - /hello] DefaultErrorHandler ERROR
> Failed delivery for exchangeId: 915083b6-4291-4716-929d-aa451e29e495.
> Exhausted after delivery attempt: 1 caught:
> java.lang.InterruptedException
> java.lang.InterruptedException
> at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:154)
> at org.apache.log4j.Category.forcedLog(Category.java:388)
> at org.apache.log4j.Category.log(Category.java:853)
> at org.apache.commons.logging.impl.Log4JLogger.debug(Log4JLogger.java:188)
> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:231)
> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:322)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> [2390869@qtp-9243153-4 - /hello] DefaultTypeConverter TRACE
> Converting java.lang.Boolean -> java.lang.Object with value: false
> [3850238@qtp-9243153-3 - /hello] Pipeline TRACE
> Processing complete for exchangeId:
> 21b68982-8ec1-405b-97d8-56f6e533115f >>> Exchange[Message: 2]
> [1671238@qtp-9243153-2 - /hello] DefaultTypeConverter TRACE
> Converting java.lang.Boolean -> java.lang.Object with value: false
> [2390869@qtp-9243153-4 - /hello] DefaultErrorHandler ERROR
> Failed delivery for exchangeId: 5903a17f-bff9-4ba5-b8f4-78bc96149195.
> Exhausted after delivery attempt: 1 caught:
> org.apache.camel.ExchangeTimedOutException: The OUT message was not
> received within: 50000 millis. Exchange[Message: 3]
> org.apache.camel.ExchangeTimedOutException: The OUT message was not
> received within: 50000 millis. Exchange[Message: 3]
> at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:265)
> at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
> at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
> at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
> at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
> at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
> at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
> at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:322)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> [1671238@qtp-9243153-2 - /hello] DefaultErrorHandler DEBUG
> This exchange is not handled so its marked as failed:
> Exchange[Message: 1]
> [3850238@qtp-9243153-3 - /hello] DefaultInflightRepository TRACE
> Total 2 inflight exchanges. Last removed:
> 21b68982-8ec1-405b-97d8-56f6e533115f
> [2390869@qtp-9243153-4 - /hello] DefaultTypeConverter TRACE
> Converting java.lang.Boolean -> java.lang.Object with value: false
> [1671238@qtp-9243153-2 - /hello] Pipeline DEBUG
> Message exchange has failed so breaking out of pipeline:
> Exchange[Message: 1] Exception: java.lang.InterruptedException
> [3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
> route: Recording duration: 2347 millis for exchange: Exchange[Message:
> [Body is instance of java.io.InputStream]]
> [2390869@qtp-9243153-4 - /hello] DefaultErrorHandler DEBUG
> This exchange is not handled so its marked as failed:
> Exchange[Message: 3]
> [1671238@qtp-9243153-2 - /hello] Pipeline TRACE
> Processing complete for exchangeId:
> 915083b6-4291-4716-929d-aa451e29e495 >>> Exchange[Message: 1]
> [2390869@qtp-9243153-4 - /hello] Pipeline DEBUG
> Message exchange has failed so breaking out of pipeline:
> Exchange[Message: 3] Exception:
> org.apache.camel.ExchangeTimedOutException: The OUT message was not
> received within: 50000 millis. Exchange[Message: 3]
> [1671238@qtp-9243153-2 - /hello] DefaultInflightRepository TRACE
> Total 1 inflight exchanges. Last removed:
> 915083b6-4291-4716-929d-aa451e29e495
> [2390869@qtp-9243153-4 - /hello] Pipeline TRACE
> Processing complete for exchangeId:
> 5903a17f-bff9-4ba5-b8f4-78bc96149195 >>> Exchange[Message: 3]
> [1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
> route: Recording duration: 19763 millis for exchange:
> Exchange[Message: [Body is instance of java.io.InputStream]]
> [2390869@qtp-9243153-4 - /hello] DefaultInflightRepository TRACE
> Total 0 inflight exchanges. Last removed:
> 5903a17f-bff9-4ba5-b8f4-78bc96149195
> [3850238@qtp-9243153-3 - /hello] log DEBUG
> RESPONSE /hello 500
> [2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
> route: Recording duration: 1830 millis for exchange: Exchange[Message:
> [Body is instance of java.io.InputStream]]
> [ 23850238@qtp-9243153-3] log DEBUG EOF
> [2390869@qtp-9243153-4 - /hello] log DEBUG
> RESPONSE /hello 500
> [ 2390869@qtp-9243153-4] log DEBUG EOF
> [1671238@qtp-9243153-2 - /hello] log DEBUG
> RESPONSE /hello 500
> [ 1671238@qtp-9243153-2] log DEBUG EOF
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopped org.mortbay.thread.QueuedThreadPool@8d0a11
> [ Camel thread 11: ShutdownTask] log DEBUG
> stopped Server@8032df
> [ Camel thread 11: ShutdownTask] HttpConsumer DEBUG
> Stopping consumer: Consumer[http://0.0.0.0:8686/hello]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
> Channel[sendTo(Endpoint[multiplier])]])]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
> Channel[sendTo(Endpoint[multiplier])]])
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
> Channel[sendTo(Endpoint[multiplier])]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Instrumentation:process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> DefaultErrorHandler[Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Instrumentation:to[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> org.apache.camel.impl.SharedProducerServicePool@11d20d3
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> DefaultErrorHandler[Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Instrumentation:process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> DefaultErrorHandler[Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service: Channel[sendTo(Endpoint[multiplier])]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service: Instrumentation:to[TraceInterceptor[To[multiplier]]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service: TraceInterceptor[To[multiplier]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service: Instrumentation:to[sendTo(Endpoint[multiplier])]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service: sendTo(Endpoint[multiplier])
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> org.apache.camel.impl.SharedProducerServicePool@11d20d3
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service:
> DefaultErrorHandler[Instrumentation:to[TraceInterceptor[To[multiplier]]]]
> [ Camel thread 11: ShutdownTask] ServiceHelper TRACE
> Stopping service: Instrumentation:to[TraceInterceptor[To[multiplier]]]
> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy DEBUG
> Shutdown complete for: Consumer[http://0.0.0.0:8686/hello]
> [ Camel thread 11: ShutdownTask] DefaultShutdownStrategy INFO
> Route: route1 shutdown complete.
> [CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
> Graceful shutdown of routes completed in 0 seconds
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service org.apache.camel.impl.RouteService@dd421f
> [CP Connection(4)-10.58.123.154] RouteService TRACE
> Stopping route:
> EventDrivenConsumerRoute[Endpoint[http://0.0.0.0:8686/hello] ->
> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
> Channel[sendTo(Endpoint[multiplier])]])]]
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service Consumer[http://0.0.0.0:8686/hello]
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
> Channel[sendTo(Endpoint[multiplier])]])]
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
> Channel[sendTo(Endpoint[multiplier])]])
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
> Channel[sendTo(Endpoint[multiplier])]]
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service Channel[sendTo(Endpoint[multiplier])]
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service sendTo(Endpoint[multiplier])
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> EventDrivenConsumerRoute[Endpoint[http://0.0.0.0:8686/hello] ->
> Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
> Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
> Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
> Channel[sendTo(Endpoint[multiplier])]])]]
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service Producer[multiplier]
> [CP Connection(4)-10.58.123.154] ProcessorEndpoint$1 DEBUG
> Stopping producer: Producer[multiplier]
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service Producer[jms://queue:numbers2?requestTimeout=50000]
> [CP Connection(4)-10.58.123.154] JmsProducer DEBUG
> Stopping producer: Producer[jms://queue:numbers2?requestTimeout=50000]
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service org.apache.camel.impl.DefaultShutdownStrategy@1c5a33b
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> org.apache.camel.impl.DefaultInflightRepository@d8978c
> [CP Connection(4)-10.58.123.154] DefaultInflightRepository INFO
> Shutting down with no inflight exchanges.
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> org.apache.camel.impl.converter.DefaultTypeConverter@da9067
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> org.apache.activemq.camel.component.ActiveMQComponent@1104da7
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> org.apache.camel.component.event.EventComponent@1031310
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> org.apache.camel.example.server.MyHttpJettyComponent@659812
> [CP Connection(4)-10.58.123.154] SharedProducerServicePool DEBUG
> Stopping service pool:
> org.apache.camel.impl.SharedProducerServicePool@11d20d3
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> org.apache.camel.impl.DefaultInflightRepository@d8978c
> [CP Connection(4)-10.58.123.154] ServiceHelper TRACE
> Stopping service
> org.apache.camel.management.ManagedManagementStrategy@1894800
> [CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
> Apache Camel 2.2.0 (CamelContext:camel) stopped
> [ InactivityMonitor WriteCheck] InactivityMonitor DEBUG
> 10001 ms elapsed since last write check.
> [ InactivityMonitor WriteCheck] InactivityMonitor TRACE
> Message sent since last write check, resetting flag
> [ InactivityMonitor ReadCheck] InactivityMonitor DEBUG
> 30004 ms elapsed since last read check.
> [ InactivityMonitor ReadCheck] InactivityMonitor TRACE
> Message received since last read check, resetting flag:
>
>
>
>
> On Thu, Apr 15, 2010 at 8:42 AM, Ilya S <de...@gmail.com> wrote:
>> Thanks Claus,
>>
>> I'm currently back to using 2.2.0 because I can see the source code.
>> But I tried 2.3-SNAPSHOT also with the same issue.
>> Please, see my full route in previous post.
>>
>>
>> On Thu, Apr 15, 2010 at 2:17 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>> Hi
>>>
>>> All the unit tests run with graceful shutdown by default, and all unit
>>> tests with camel-jetty shut down with no issues.
>>>
>>> The in flight repository do keep track of all in flights. I can only
>>> think of if you use the aggregator or some custom code which
>>> can cause the exchange to be completed in Camels point of view.
>>>
>>> Can you post your entire route?
>>>
>>> And as always report which version of Camel you are using? And try
>>> with the latest stable and even with 2.3-SNAPSHOT.
>>>
>>>
>>> On Wed, Apr 14, 2010 at 7:41 PM, Ilya S <de...@gmail.com> wrote:
>>>> Hi All,
>>>>
>>>> Fist of all, thank you for this great product!
>>>> Every new release brings excellent new features just in time.
>>>>
>>>> I'm currently investigating Graceful Shutdown feature, and I'm am
>>>> wondering if graceful shutdown is supported for Camel-Jetty component?
>>>> I was able to successfully shut down JMS component. However,
>>>> JettyHttpCoponent seems to have issues when shutting down.
>>>>
>>>> In my simple test (adopted from camel-example-spring-jms:ServerRoute
>>>> example) I have a simple route that looks like this:
>>>>
>>>> from("jetty:http://0.0.0.0:8686/hello")
>>>> ..
>>>> .. do work
>>>> ...
>>>> .to("jms:queue:numbers2?requestTimeout=50000")
>>>> .to("multiplier");
>>>>
>>>>
>>>> My producers simply send numbers to http endpoint..
>>>>
>>>>
>>>> When I shutdown this context gracefully I'm getting all sort of
>>>> exceptions from Jetty, and the context does not seem to wait until
>>>> inflight exchanges have finished.
>>>> I'm using the defaults everywhere (e.g. default shutdown strategy).
>>>>
>>>> Is shutdown supported for Jetty, and if so how should one configure
>>>> jetty to initiate proper shutdown?
>>>>
>>>> Thank you,
>>>>
>>>> Ilya.
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Ilya S <de...@gmail.com>.
Here is a more detailed log (with TRACE level on) if that can help:
[CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
Apache Camel 2.2.0 (CamelContext:camel) is stopping
[CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
Starting to graceful shutdown routes (timeout 600000 seconds)
[ Camel thread 11: ShutdownTask] DefaultShutdownStrategy DEBUG
There are 1 routes to shutdown
[ Camel thread 11: ShutdownTask] DefaultShutdownStrategy TRACE
Shutting down route: route1 with options
[Default,CompleteCurrentTaskOnly]
[ Camel thread 11: ShutdownTask] DefaultShutdownStrategy TRACE
Shutting down: Consumer[http://0.0.0.0:8686/hello]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service Consumer[http://0.0.0.0:8686/hello]
[ Camel thread 11: ShutdownTask] log DEBUG
Container Server@8032df - SelectChannelConnector@0.0.0.0:8686 as
connector
[ Camel thread 11: ShutdownTask] log DEBUG
stopped org.mortbay.jetty.nio.SelectChannelConnector$1@28df48
[ Camel thread 11: ShutdownTask] log DEBUG
stopped SelectChannelConnector@0.0.0.0:8686
[ Camel thread 11: ShutdownTask] log DEBUG
stopping Server@8032df
[ Camel thread 11: ShutdownTask] log DEBUG
stopping ContextHandlerCollection@8bfdea
[ Camel thread 11: ShutdownTask] log DEBUG
stopping org.mortbay.jetty.servlet.Context@1157f77{/,null}
[ Camel thread 11: ShutdownTask] log DEBUG
stopping ServletHandler@6399ae
[ Camel thread 11: ShutdownTask] log DEBUG
stopped org.apache.camel.component.http.CamelServlet-29615099
[ Camel thread 11: ShutdownTask] log DEBUG
stopped ServletHandler@6399ae
[ Camel thread 11: ShutdownTask] log DEBUG
stopping ErrorHandler@56acfa
[ Camel thread 11: ShutdownTask] log DEBUG
stopped ErrorHandler@56acfa
[ Camel thread 11: ShutdownTask] log DEBUG
stopped org.mortbay.jetty.servlet.Context@1157f77{/,null}
[ Camel thread 11: ShutdownTask] log DEBUG
stopped ContextHandlerCollection@8bfdea
[3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
to: Recording duration: 2342 millis for exchange: Exchange[Message: 2]
[3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
to: Recording duration: 2342 millis for exchange: Exchange[Message: 2]
[3850238@qtp-9243153-3 - /hello] DefaultExceptionPolicyStrategy TRACE
Finding best suited exception policy for thrown exception
java.lang.InterruptedException
[3850238@qtp-9243153-3 - /hello] DefaultExceptionPolicyStrategy TRACE
Found 0 candidates
[2390869@qtp-9243153-4 - /hello] JmsProducer DEBUG
Future interrupted: java.lang.InterruptedException
java.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:926)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1201)
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:210)
at java.util.concurrent.FutureTask.get(FutureTask.java:85)
at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:227)
at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:322)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
[1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
to: Recording duration: 19757 millis for exchange: Exchange[Message:
1]
[3850238@qtp-9243153-3 - /hello] DefaultErrorHandler DEBUG
Failed delivery for exchangeId: 21b68982-8ec1-405b-97d8-56f6e533115f.
On delivery attempt: 0 caught: java.lang.InterruptedException
[1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
to: Recording duration: 19758 millis for exchange: Exchange[Message:
1]
[3850238@qtp-9243153-3 - /hello] DefaultTypeConverter TRACE
Converting java.lang.Boolean -> java.lang.Object with value: false
[3850238@qtp-9243153-3 - /hello] DefaultErrorHandler ERROR
Failed delivery for exchangeId: 21b68982-8ec1-405b-97d8-56f6e533115f.
Exhausted after delivery attempt: 1 caught:
java.lang.InterruptedException
java.lang.InterruptedException
at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:154)
at org.apache.log4j.Category.forcedLog(Category.java:388)
at org.apache.log4j.Category.log(Category.java:853)
at org.apache.commons.logging.impl.Log4JLogger.debug(Log4JLogger.java:188)
at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:231)
at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:322)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
[2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
to: Recording duration: 1826 millis for exchange: Exchange[Message: 3]
[1671238@qtp-9243153-2 - /hello] DefaultExceptionPolicyStrategy TRACE
Finding best suited exception policy for thrown exception
java.lang.InterruptedException
[3850238@qtp-9243153-3 - /hello] DefaultTypeConverter TRACE
Converting java.lang.Boolean -> java.lang.Object with value: false
[2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
to: Recording duration: 1827 millis for exchange: Exchange[Message: 3]
[1671238@qtp-9243153-2 - /hello] DefaultExceptionPolicyStrategy TRACE
Found 0 candidates
[3850238@qtp-9243153-3 - /hello] DefaultErrorHandler DEBUG
This exchange is not handled so its marked as failed:
Exchange[Message: 2]
[2390869@qtp-9243153-4 - /hello] DefaultExceptionPolicyStrategy TRACE
Finding best suited exception policy for thrown exception
org.apache.camel.ExchangeTimedOutException
[1671238@qtp-9243153-2 - /hello] DefaultErrorHandler DEBUG
Failed delivery for exchangeId: 915083b6-4291-4716-929d-aa451e29e495.
On delivery attempt: 0 caught: java.lang.InterruptedException
[2390869@qtp-9243153-4 - /hello] DefaultExceptionPolicyStrategy TRACE
Found 0 candidates
[3850238@qtp-9243153-3 - /hello] Pipeline DEBUG
Message exchange has failed so breaking out of pipeline:
Exchange[Message: 2] Exception: java.lang.InterruptedException
[1671238@qtp-9243153-2 - /hello] DefaultTypeConverter TRACE
Converting java.lang.Boolean -> java.lang.Object with value: false
[2390869@qtp-9243153-4 - /hello] DefaultErrorHandler DEBUG
Failed delivery for exchangeId: 5903a17f-bff9-4ba5-b8f4-78bc96149195.
On delivery attempt: 0 caught:
org.apache.camel.ExchangeTimedOutException: The OUT message was not
received within: 50000 millis. Exchange[Message: 3]
[1671238@qtp-9243153-2 - /hello] DefaultErrorHandler ERROR
Failed delivery for exchangeId: 915083b6-4291-4716-929d-aa451e29e495.
Exhausted after delivery attempt: 1 caught:
java.lang.InterruptedException
java.lang.InterruptedException
at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:154)
at org.apache.log4j.Category.forcedLog(Category.java:388)
at org.apache.log4j.Category.log(Category.java:853)
at org.apache.commons.logging.impl.Log4JLogger.debug(Log4JLogger.java:188)
at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:231)
at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:322)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
[2390869@qtp-9243153-4 - /hello] DefaultTypeConverter TRACE
Converting java.lang.Boolean -> java.lang.Object with value: false
[3850238@qtp-9243153-3 - /hello] Pipeline TRACE
Processing complete for exchangeId:
21b68982-8ec1-405b-97d8-56f6e533115f >>> Exchange[Message: 2]
[1671238@qtp-9243153-2 - /hello] DefaultTypeConverter TRACE
Converting java.lang.Boolean -> java.lang.Object with value: false
[2390869@qtp-9243153-4 - /hello] DefaultErrorHandler ERROR
Failed delivery for exchangeId: 5903a17f-bff9-4ba5-b8f4-78bc96149195.
Exhausted after delivery attempt: 1 caught:
org.apache.camel.ExchangeTimedOutException: The OUT message was not
received within: 50000 millis. Exchange[Message: 3]
org.apache.camel.ExchangeTimedOutException: The OUT message was not
received within: 50000 millis. Exchange[Message: 3]
at org.apache.camel.component.jms.JmsProducer.processInOut(JmsProducer.java:265)
at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:147)
at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97)
at org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95)
at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146)
at org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53)
at org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82)
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:93)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.processor.RedeliveryErrorHandler.processExchange(RedeliveryErrorHandler.java:177)
at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:143)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:88)
at org.apache.camel.processor.DefaultErrorHandler.process(DefaultErrorHandler.java:49)
at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:228)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:74)
at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:66)
at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.component.http.CamelServlet.service(CamelServlet.java:53)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:322)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
[1671238@qtp-9243153-2 - /hello] DefaultErrorHandler DEBUG
This exchange is not handled so its marked as failed:
Exchange[Message: 1]
[3850238@qtp-9243153-3 - /hello] DefaultInflightRepository TRACE
Total 2 inflight exchanges. Last removed:
21b68982-8ec1-405b-97d8-56f6e533115f
[2390869@qtp-9243153-4 - /hello] DefaultTypeConverter TRACE
Converting java.lang.Boolean -> java.lang.Object with value: false
[1671238@qtp-9243153-2 - /hello] Pipeline DEBUG
Message exchange has failed so breaking out of pipeline:
Exchange[Message: 1] Exception: java.lang.InterruptedException
[3850238@qtp-9243153-3 - /hello] InstrumentationProcessor TRACE
route: Recording duration: 2347 millis for exchange: Exchange[Message:
[Body is instance of java.io.InputStream]]
[2390869@qtp-9243153-4 - /hello] DefaultErrorHandler DEBUG
This exchange is not handled so its marked as failed:
Exchange[Message: 3]
[1671238@qtp-9243153-2 - /hello] Pipeline TRACE
Processing complete for exchangeId:
915083b6-4291-4716-929d-aa451e29e495 >>> Exchange[Message: 1]
[2390869@qtp-9243153-4 - /hello] Pipeline DEBUG
Message exchange has failed so breaking out of pipeline:
Exchange[Message: 3] Exception:
org.apache.camel.ExchangeTimedOutException: The OUT message was not
received within: 50000 millis. Exchange[Message: 3]
[1671238@qtp-9243153-2 - /hello] DefaultInflightRepository TRACE
Total 1 inflight exchanges. Last removed:
915083b6-4291-4716-929d-aa451e29e495
[2390869@qtp-9243153-4 - /hello] Pipeline TRACE
Processing complete for exchangeId:
5903a17f-bff9-4ba5-b8f4-78bc96149195 >>> Exchange[Message: 3]
[1671238@qtp-9243153-2 - /hello] InstrumentationProcessor TRACE
route: Recording duration: 19763 millis for exchange:
Exchange[Message: [Body is instance of java.io.InputStream]]
[2390869@qtp-9243153-4 - /hello] DefaultInflightRepository TRACE
Total 0 inflight exchanges. Last removed:
5903a17f-bff9-4ba5-b8f4-78bc96149195
[3850238@qtp-9243153-3 - /hello] log DEBUG
RESPONSE /hello 500
[2390869@qtp-9243153-4 - /hello] InstrumentationProcessor TRACE
route: Recording duration: 1830 millis for exchange: Exchange[Message:
[Body is instance of java.io.InputStream]]
[ 23850238@qtp-9243153-3] log DEBUG EOF
[2390869@qtp-9243153-4 - /hello] log DEBUG
RESPONSE /hello 500
[ 2390869@qtp-9243153-4] log DEBUG EOF
[1671238@qtp-9243153-2 - /hello] log DEBUG
RESPONSE /hello 500
[ 1671238@qtp-9243153-2] log DEBUG EOF
[ Camel thread 11: ShutdownTask] log DEBUG
stopped org.mortbay.thread.QueuedThreadPool@8d0a11
[ Camel thread 11: ShutdownTask] log DEBUG
stopped Server@8032df
[ Camel thread 11: ShutdownTask] HttpConsumer DEBUG
Stopping consumer: Consumer[http://0.0.0.0:8686/hello]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
Channel[sendTo(Endpoint[multiplier])]])]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
Channel[sendTo(Endpoint[multiplier])]])
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
Channel[sendTo(Endpoint[multiplier])]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Instrumentation:process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
DefaultErrorHandler[Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Instrumentation:to[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
org.apache.camel.impl.SharedProducerServicePool@11d20d3
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
DefaultErrorHandler[Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Instrumentation:to[TraceInterceptor[To[jms://queue:numbers2?requestTimeout=50000]]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Instrumentation:process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
DefaultErrorHandler[Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
Instrumentation:process[TraceInterceptor[process[org.apache.camel.example.server.ServerRoutes$2@1d6b049]]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service: Channel[sendTo(Endpoint[multiplier])]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service: Instrumentation:to[TraceInterceptor[To[multiplier]]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service: TraceInterceptor[To[multiplier]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service: Instrumentation:to[sendTo(Endpoint[multiplier])]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service: sendTo(Endpoint[multiplier])
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
org.apache.camel.impl.SharedProducerServicePool@11d20d3
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service:
DefaultErrorHandler[Instrumentation:to[TraceInterceptor[To[multiplier]]]]
[ Camel thread 11: ShutdownTask] ServiceHelper TRACE
Stopping service: Instrumentation:to[TraceInterceptor[To[multiplier]]]
[ Camel thread 11: ShutdownTask] DefaultShutdownStrategy DEBUG
Shutdown complete for: Consumer[http://0.0.0.0:8686/hello]
[ Camel thread 11: ShutdownTask] DefaultShutdownStrategy INFO
Route: route1 shutdown complete.
[CP Connection(4)-10.58.123.154] DefaultShutdownStrategy INFO
Graceful shutdown of routes completed in 0 seconds
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service org.apache.camel.impl.RouteService@dd421f
[CP Connection(4)-10.58.123.154] RouteService TRACE
Stopping route:
EventDrivenConsumerRoute[Endpoint[http://0.0.0.0:8686/hello] ->
Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
Channel[sendTo(Endpoint[multiplier])]])]]
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service Consumer[http://0.0.0.0:8686/hello]
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
Channel[sendTo(Endpoint[multiplier])]])]
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
Channel[sendTo(Endpoint[multiplier])]])
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
Channel[sendTo(Endpoint[multiplier])]]
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad]
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])]
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049]
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service Channel[sendTo(Endpoint[multiplier])]
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service sendTo(Endpoint[multiplier])
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
EventDrivenConsumerRoute[Endpoint[http://0.0.0.0:8686/hello] ->
Instrumentation:route[UnitOfWork(Pipeline[Channel[org.apache.camel.example.server.ServerRoutes$1@d1f0ad],
Channel[sendTo(Endpoint[jms://queue:numbers2?requestTimeout=50000])],
Channel[org.apache.camel.example.server.ServerRoutes$2@1d6b049],
Channel[sendTo(Endpoint[multiplier])]])]]
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service Producer[multiplier]
[CP Connection(4)-10.58.123.154] ProcessorEndpoint$1 DEBUG
Stopping producer: Producer[multiplier]
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service Producer[jms://queue:numbers2?requestTimeout=50000]
[CP Connection(4)-10.58.123.154] JmsProducer DEBUG
Stopping producer: Producer[jms://queue:numbers2?requestTimeout=50000]
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service org.apache.camel.impl.DefaultShutdownStrategy@1c5a33b
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
org.apache.camel.impl.DefaultInflightRepository@d8978c
[CP Connection(4)-10.58.123.154] DefaultInflightRepository INFO
Shutting down with no inflight exchanges.
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
org.apache.camel.impl.converter.DefaultTypeConverter@da9067
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
org.apache.activemq.camel.component.ActiveMQComponent@1104da7
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
org.apache.camel.component.event.EventComponent@1031310
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
org.apache.camel.example.server.MyHttpJettyComponent@659812
[CP Connection(4)-10.58.123.154] SharedProducerServicePool DEBUG
Stopping service pool:
org.apache.camel.impl.SharedProducerServicePool@11d20d3
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
org.apache.camel.impl.DefaultInflightRepository@d8978c
[CP Connection(4)-10.58.123.154] ServiceHelper TRACE
Stopping service
org.apache.camel.management.ManagedManagementStrategy@1894800
[CP Connection(4)-10.58.123.154] DefaultCamelContext INFO
Apache Camel 2.2.0 (CamelContext:camel) stopped
[ InactivityMonitor WriteCheck] InactivityMonitor DEBUG
10001 ms elapsed since last write check.
[ InactivityMonitor WriteCheck] InactivityMonitor TRACE
Message sent since last write check, resetting flag
[ InactivityMonitor ReadCheck] InactivityMonitor DEBUG
30004 ms elapsed since last read check.
[ InactivityMonitor ReadCheck] InactivityMonitor TRACE
Message received since last read check, resetting flag:
On Thu, Apr 15, 2010 at 8:42 AM, Ilya S <de...@gmail.com> wrote:
> Thanks Claus,
>
> I'm currently back to using 2.2.0 because I can see the source code.
> But I tried 2.3-SNAPSHOT also with the same issue.
> Please, see my full route in previous post.
>
>
> On Thu, Apr 15, 2010 at 2:17 AM, Claus Ibsen <cl...@gmail.com> wrote:
>> Hi
>>
>> All the unit tests run with graceful shutdown by default, and all unit
>> tests with camel-jetty shut down with no issues.
>>
>> The in flight repository do keep track of all in flights. I can only
>> think of if you use the aggregator or some custom code which
>> can cause the exchange to be completed in Camels point of view.
>>
>> Can you post your entire route?
>>
>> And as always report which version of Camel you are using? And try
>> with the latest stable and even with 2.3-SNAPSHOT.
>>
>>
>> On Wed, Apr 14, 2010 at 7:41 PM, Ilya S <de...@gmail.com> wrote:
>>> Hi All,
>>>
>>> Fist of all, thank you for this great product!
>>> Every new release brings excellent new features just in time.
>>>
>>> I'm currently investigating Graceful Shutdown feature, and I'm am
>>> wondering if graceful shutdown is supported for Camel-Jetty component?
>>> I was able to successfully shut down JMS component. However,
>>> JettyHttpCoponent seems to have issues when shutting down.
>>>
>>> In my simple test (adopted from camel-example-spring-jms:ServerRoute
>>> example) I have a simple route that looks like this:
>>>
>>> from("jetty:http://0.0.0.0:8686/hello")
>>> ..
>>> .. do work
>>> ...
>>> .to("jms:queue:numbers2?requestTimeout=50000")
>>> .to("multiplier");
>>>
>>>
>>> My producers simply send numbers to http endpoint..
>>>
>>>
>>> When I shutdown this context gracefully I'm getting all sort of
>>> exceptions from Jetty, and the context does not seem to wait until
>>> inflight exchanges have finished.
>>> I'm using the defaults everywhere (e.g. default shutdown strategy).
>>>
>>> Is shutdown supported for Jetty, and if so how should one configure
>>> jetty to initiate proper shutdown?
>>>
>>> Thank you,
>>>
>>> Ilya.
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Ilya S <de...@gmail.com>.
Thanks Claus,
I'm currently back to using 2.2.0 because I can see the source code.
But I tried 2.3-SNAPSHOT also with the same issue.
Please, see my full route in previous post.
On Thu, Apr 15, 2010 at 2:17 AM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> All the unit tests run with graceful shutdown by default, and all unit
> tests with camel-jetty shut down with no issues.
>
> The in flight repository do keep track of all in flights. I can only
> think of if you use the aggregator or some custom code which
> can cause the exchange to be completed in Camels point of view.
>
> Can you post your entire route?
>
> And as always report which version of Camel you are using? And try
> with the latest stable and even with 2.3-SNAPSHOT.
>
>
> On Wed, Apr 14, 2010 at 7:41 PM, Ilya S <de...@gmail.com> wrote:
>> Hi All,
>>
>> Fist of all, thank you for this great product!
>> Every new release brings excellent new features just in time.
>>
>> I'm currently investigating Graceful Shutdown feature, and I'm am
>> wondering if graceful shutdown is supported for Camel-Jetty component?
>> I was able to successfully shut down JMS component. However,
>> JettyHttpCoponent seems to have issues when shutting down.
>>
>> In my simple test (adopted from camel-example-spring-jms:ServerRoute
>> example) I have a simple route that looks like this:
>>
>> from("jetty:http://0.0.0.0:8686/hello")
>> ..
>> .. do work
>> ...
>> .to("jms:queue:numbers2?requestTimeout=50000")
>> .to("multiplier");
>>
>>
>> My producers simply send numbers to http endpoint..
>>
>>
>> When I shutdown this context gracefully I'm getting all sort of
>> exceptions from Jetty, and the context does not seem to wait until
>> inflight exchanges have finished.
>> I'm using the defaults everywhere (e.g. default shutdown strategy).
>>
>> Is shutdown supported for Jetty, and if so how should one configure
>> jetty to initiate proper shutdown?
>>
>> Thank you,
>>
>> Ilya.
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
All the unit tests run with graceful shutdown by default, and all unit
tests with camel-jetty shut down with no issues.
The in flight repository do keep track of all in flights. I can only
think of if you use the aggregator or some custom code which
can cause the exchange to be completed in Camels point of view.
Can you post your entire route?
And as always report which version of Camel you are using? And try
with the latest stable and even with 2.3-SNAPSHOT.
On Wed, Apr 14, 2010 at 7:41 PM, Ilya S <de...@gmail.com> wrote:
> Hi All,
>
> Fist of all, thank you for this great product!
> Every new release brings excellent new features just in time.
>
> I'm currently investigating Graceful Shutdown feature, and I'm am
> wondering if graceful shutdown is supported for Camel-Jetty component?
> I was able to successfully shut down JMS component. However,
> JettyHttpCoponent seems to have issues when shutting down.
>
> In my simple test (adopted from camel-example-spring-jms:ServerRoute
> example) I have a simple route that looks like this:
>
> from("jetty:http://0.0.0.0:8686/hello")
> ..
> .. do work
> ...
> .to("jms:queue:numbers2?requestTimeout=50000")
> .to("multiplier");
>
>
> My producers simply send numbers to http endpoint..
>
>
> When I shutdown this context gracefully I'm getting all sort of
> exceptions from Jetty, and the context does not seem to wait until
> inflight exchanges have finished.
> I'm using the defaults everywhere (e.g. default shutdown strategy).
>
> Is shutdown supported for Jetty, and if so how should one configure
> jetty to initiate proper shutdown?
>
> Thank you,
>
> Ilya.
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Graceful shutdown of Camel-Jetty component. (Camel 2.2.0)
Posted by Willem Jiang <wi...@gmail.com>.
Hi,
What kind of exception did you get?
Can you show us the stack trace ?
Willem
Ilya S wrote:
> Hi All,
>
> Fist of all, thank you for this great product!
> Every new release brings excellent new features just in time.
>
> I'm currently investigating Graceful Shutdown feature, and I'm am
> wondering if graceful shutdown is supported for Camel-Jetty component?
> I was able to successfully shut down JMS component. However,
> JettyHttpCoponent seems to have issues when shutting down.
>
> In my simple test (adopted from camel-example-spring-jms:ServerRoute
> example) I have a simple route that looks like this:
>
> from("jetty:http://0.0.0.0:8686/hello")
> ..
> .. do work
> ...
> .to("jms:queue:numbers2?requestTimeout=50000")
> .to("multiplier");
>
>
> My producers simply send numbers to http endpoint..
>
>
> When I shutdown this context gracefully I'm getting all sort of
> exceptions from Jetty, and the context does not seem to wait until
> inflight exchanges have finished.
> I'm using the defaults everywhere (e.g. default shutdown strategy).
>
> Is shutdown supported for Jetty, and if so how should one configure
> jetty to initiate proper shutdown?
>
> Thank you,
>
> Ilya.
>