You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by boday <bo...@vektrel.com> on 2010/01/06 18:43:56 UTC
context.stopRoute() API hangs periodically
I'm using Camel 2.1 on SMX 3.3.1. I'm using a timer to periodically
reprocess messages that I've sent to an error queue. While reprocessing I
want to stop the main message processing route as follows (for some specific
client requirements)...
from("direct:main").routeId("mainRoute").process(MessageProcessor());
from("timer://ErrorMessageProcessor?period=60000")
.bean(blockedNodeManager, "processBlockedQueue");
then, in blockedNodeManager stops the route, acts as a polling consumer and
then restarts the route...
ServiceStatus routeStatus = context.getRouteStatus("mainRoute");
if(routeStatus.isStoppable())
{
logger.info("stopping route..." + routeId);
context.stopRoute(routeId);
logger.info("route stopped..." + routeId);
}
while (true)
{
String msg = consumer.receiveBody("activemq:errorQueue", 100,
String.class);
if (msg == null)
{
break;
}
producer.sendBody("direct:main", msg);
}
routeStatus = context.getRouteStatus("mainRoute");
if(routeStatus.isStartable())
{
logger.info("starting route..." + routeId);
context.startRoute(routeId);
logger.info("route started..." + routeId);
}
This seems to work fine for a while (up to 12 hours sometimes), but will
periodically hang while trying to stop the main route. The last message I
see in my log file is...."stopping route mainRoute"...it never reaches the
"route stopped" debug statement. At this point, I'm hosed...I can't even
use JMX to restart the route. I restart the server and all works fine again
(for a while)...
Am I using these APIs incorrectly? If there is a better way to accomplish
this, let me know...
thanks in advance...
-----
Ben - Senior Consultant
using SMX 3.3.1/Camel 2.1
--
View this message in context: http://old.nabble.com/context.stopRoute%28%29-API-hangs-periodically-tp27026922p27026922.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: context.stopRoute() API hangs periodically
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jan 8, 2010 at 10:14 PM, boday <bo...@vektrel.com> wrote:
>
> thanks for the feedback guys...
>
> I think I found the issue. My ErrorMessageProcessor runs every 60 seconds
> and was periodically taking longer than 60 seconds to run. Then, I end up
> with multiple threads trying to stop/start the same route (not a good thing
> I imagine).
>
> So, I increased the time between runs and synchronized the method to make
> sure it is single threaded. Since then, I haven't had any issues. I'm
> hoping this was it, if not...I'll try to get a thread dump, etc.
>
> Also, Claus when will Camel 2.2 be available?
In 2010 :) Ah okay I personally want 2.2 to be a smaller release so I
am hoping for about 3-4 weeks time.
Its really the improved thread pool configuration that I would love to
get in 2.2 but maybe time does not permit and therefore we can cut a
release without it.
The graceful shutdown alone is worth a release IMHO.
>
> thanks again...
>
>
> -----
> Ben - Senior Consultant
> using SMX 3.3.1/Camel 2.1
> --
> View this message in context: http://old.nabble.com/context.stopRoute%28%29-API-hangs-periodically-tp27026922p27082410.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: context.stopRoute() API hangs periodically
Posted by boday <bo...@vektrel.com>.
thanks for the feedback guys...
I think I found the issue. My ErrorMessageProcessor runs every 60 seconds
and was periodically taking longer than 60 seconds to run. Then, I end up
with multiple threads trying to stop/start the same route (not a good thing
I imagine).
So, I increased the time between runs and synchronized the method to make
sure it is single threaded. Since then, I haven't had any issues. I'm
hoping this was it, if not...I'll try to get a thread dump, etc.
Also, Claus when will Camel 2.2 be available?
thanks again...
-----
Ben - Senior Consultant
using SMX 3.3.1/Camel 2.1
--
View this message in context: http://old.nabble.com/context.stopRoute%28%29-API-hangs-periodically-tp27026922p27082410.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: context.stopRoute() API hangs periodically
Posted by Claus Ibsen <cl...@gmail.com>.
On Thu, Jan 7, 2010 at 8:57 AM, Willem Jiang <wi...@gmail.com> wrote:
> Hi,
>
> Can you check if there are deadlock of CamelContext which introduce this
> error?
> Did you use the camelContext as the monitor Object when you try to stop the
> Route ?
>
Yeah try to dump the thread stack list and see where the stopping is waiting.
BTW: In Camel 2.2 you can now suspend a route which is more gentle.
> Willem
>
> boday wrote:
>>
>> I'm using Camel 2.1 on SMX 3.3.1. I'm using a timer to periodically
>> reprocess messages that I've sent to an error queue. While reprocessing I
>> want to stop the main message processing route as follows (for some
>> specific
>> client requirements)...
>>
>> from("direct:main").routeId("mainRoute").process(MessageProcessor());
>>
>> from("timer://ErrorMessageProcessor?period=60000")
>> .bean(blockedNodeManager, "processBlockedQueue");
>>
>> then, in blockedNodeManager stops the route, acts as a polling consumer
>> and
>> then restarts the route...
>>
>> ServiceStatus routeStatus = context.getRouteStatus("mainRoute");
>> if(routeStatus.isStoppable())
>> {
>> logger.info("stopping route..." + routeId);
>> context.stopRoute(routeId);
>> logger.info("route stopped..." + routeId);
>> }
>>
>> while (true) {
>> String msg = consumer.receiveBody("activemq:errorQueue", 100,
>> String.class);
>> if (msg == null)
>> {
>> break;
>> }
>> producer.sendBody("direct:main", msg); }
>>
>> routeStatus = context.getRouteStatus("mainRoute");
>> if(routeStatus.isStartable())
>> {
>> logger.info("starting route..." + routeId);
>> context.startRoute(routeId);
>> logger.info("route started..." + routeId);
>> }
>>
>> This seems to work fine for a while (up to 12 hours sometimes), but will
>> periodically hang while trying to stop the main route. The last message I
>> see in my log file is...."stopping route mainRoute"...it never reaches the
>> "route stopped" debug statement. At this point, I'm hosed...I can't even
>> use JMX to restart the route. I restart the server and all works fine
>> again
>> (for a while)...
>>
>> Am I using these APIs incorrectly? If there is a better way to accomplish
>> this, let me know...
>>
>> thanks in advance...
>>
>>
>>
>>
>> -----
>> Ben - Senior Consultant
>> using SMX 3.3.1/Camel 2.1
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: context.stopRoute() API hangs periodically
Posted by Willem Jiang <wi...@gmail.com>.
Hi,
Can you check if there are deadlock of CamelContext which introduce this
error?
Did you use the camelContext as the monitor Object when you try to stop
the Route ?
Willem
boday wrote:
> I'm using Camel 2.1 on SMX 3.3.1. I'm using a timer to periodically
> reprocess messages that I've sent to an error queue. While reprocessing I
> want to stop the main message processing route as follows (for some specific
> client requirements)...
>
> from("direct:main").routeId("mainRoute").process(MessageProcessor());
>
> from("timer://ErrorMessageProcessor?period=60000")
> .bean(blockedNodeManager, "processBlockedQueue");
>
> then, in blockedNodeManager stops the route, acts as a polling consumer and
> then restarts the route...
>
> ServiceStatus routeStatus = context.getRouteStatus("mainRoute");
> if(routeStatus.isStoppable())
> {
> logger.info("stopping route..." + routeId);
> context.stopRoute(routeId);
> logger.info("route stopped..." + routeId);
> }
>
> while (true)
> {
> String msg = consumer.receiveBody("activemq:errorQueue", 100,
> String.class);
> if (msg == null)
> {
> break;
> }
> producer.sendBody("direct:main", msg);
> }
>
> routeStatus = context.getRouteStatus("mainRoute");
> if(routeStatus.isStartable())
> {
> logger.info("starting route..." + routeId);
> context.startRoute(routeId);
> logger.info("route started..." + routeId);
> }
>
> This seems to work fine for a while (up to 12 hours sometimes), but will
> periodically hang while trying to stop the main route. The last message I
> see in my log file is...."stopping route mainRoute"...it never reaches the
> "route stopped" debug statement. At this point, I'm hosed...I can't even
> use JMX to restart the route. I restart the server and all works fine again
> (for a while)...
>
> Am I using these APIs incorrectly? If there is a better way to accomplish
> this, let me know...
>
> thanks in advance...
>
>
>
>
> -----
> Ben - Senior Consultant
> using SMX 3.3.1/Camel 2.1