You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Anna Maier <A....@topdesk.com> on 2015/09/29 15:31:13 UTC
Problem shutting down activemq client with failover transport
Hi,
I am using Camel to connect to ActiveMQ and it fails to shutdown when the ActiveMQComponent cannot make a connection to the broker with the failover protocol (activemq version 5.12.0).
The problem seems to be that the failover transport tries to connect to the broker in an endless loop on its own thread that cannot be interrupted. I think the problem is in the PooledTaskRunner: there is a shutdown field which is set to true when camel shuts down, but it appears as false to the thread that is running the endless loop.
I am currently at a loss what to do. I want to keep the maximum retries setting for the failover transport on indefinite, since I do not know when the the broker is back online. At the same time, it has to be possible to shut down the application even if there never was a connection to the ActiveMQ broker.
Has anybody seen this problem before and can give some advice? Is this a bug in ActiveMQ?
Attached there is some code that reproduces the problem - it never terminates.
Regards,
Anna
The code:
public class ShutdownBug {
public static void main(String[] args) throws Exception {
DefaultCamelContext camelContext = new DefaultCamelContext();
ActiveMQComponent activeMQComponent = new ActiveMQComponent();
activeMQComponent.setBrokerURL("failover:(http://localhost:61616)");
camelContext.addComponent("activemq", activeMQComponent);
camelContext.addRoutes(new RouteBuilder() {
@Override
public void configure() {
from("activemq:queue:myqueue").to("stream:out");
}
});
Thread startThread = new Thread(new Runnable() {
@Override
public void run() {
try {
camelContext.start();
}
catch (Exception e) {
e.printStackTrace();
}
}
});
startThread.start();
startThread.interrupt();
startThread.join();
camelContext.stop();
}
}
Re: Problem shutting down activemq client with failover transport
Posted by Tim Bain <tb...@alumni.duke.edu>.
Given that the code in question is in DefaultMessageListenerContainer
(Spring), I'd encourage you to post to their mailing list and/or submit a
bug report to them and see if maybe there's something for them to fix.
Setting the timeout sounds like a reasonable workaround, but it would be
good to have the code work properly at some point in the future...
Tim
On Wed, Sep 30, 2015 at 8:44 AM, Anna Maier <A....@topdesk.com> wrote:
> Hi,
>
> I debugged a bit, and it looks like it never calls the shutdown method in
> PooledTaskRunner after all. It gets stuck in
> DefaultMessageListenerContainer.doShutdown() where it calls
> this.lifecycleMonitor.wait(). Apparently, this does not terminate. I am not
> sure if this is a bug in Camel, ActiveMQ or Spring, or if it is even
> intended behaviour.
>
> Anyway, setting the timeout to camel does the trick:
> camelContext.getShutdownStrategy().setTimeout(1L);
> Apparently the default for this is 5 minutes and I was never patient
> enough to wait so long. ;)
>
> Thanks for the help!
> Anna
>
>
> -----Original Message-----
> From: Ganesh Murthy [mailto:gmurthy@redhat.com]
> Sent: Mittwoch, 30. September 2015 15:12
> To: users@activemq.apache.org
> Subject: Re: Problem shutting down activemq client with failover transport
>
> How is Camel setting the shutdown flag on the PooledTaskRunner? Is Camel
> calling the shutdown() function to shut it down? The shutdown() method
> (which in turn calls shutdown(0)) on PooledTaskRunner is guaranteed to shut
> it down because if shutdown is set to true, then iterating is set to false.
>
> Have you tried the "timeout" parameter in FailoverTransport?
>
> http://activemq.apache.org/failover-transport-reference.html
>
> If you are using an IDE like IntelliJ, try putting in breakpoints on only
> that PooledTaskRunner thread and analyze what is going on.
>
> Thanks.
>
> -----Original Message-----
> From: Ganesh Murthy [mailto:gmurthy@redhat.com]
> Sent: Dienstag, 29. September 2015 16:52
> To: users@activemq.apache.org
> Subject: Re: Problem shutting down activemq client with failover transport
>
> Can you please provide a thread dump? That would throw some light on
> exactly what is going on.
>
> Thanks.
>
> ----- Original Message -----
> From: "Anna Maier" <A....@topdesk.com>
> To: users@activemq.apache.org
> Sent: Tuesday, September 29, 2015 9:31:13 AM
> Subject: Problem shutting down activemq client with failover transport
>
> Hi,
>
>
>
> I am using Camel to connect to ActiveMQ and it fails to shutdown when the
> ActiveMQComponent cannot make a connection to the broker with the failover
> protocol (activemq version 5.12.0).
>
> The problem seems to be that the failover transport tries to connect to
> the broker in an endless loop on its own thread that cannot be interrupted.
> I think the problem is in the PooledTaskRunner: there is a shutdown field
> which is set to true when camel shuts down, but it appears as false to the
> thread that is running the endless loop.
>
>
>
> I am currently at a loss what to do. I want to keep the maximum retries
> setting for the failover transport on indefinite, since I do not know when
> the the broker is back online. At the same time, it has to be possible to
> shut down the application even if there never was a connection to the
> ActiveMQ broker.
>
>
>
> Has anybody seen this problem before and can give some advice? Is this a
> bug in ActiveMQ?
>
>
>
> Attached there is some code that reproduces the problem - it never
> terminates.
>
>
>
> Regards,
>
> Anna
>
>
>
> The code:
> public class ShutdownBug {
>
> public static void main(String[] args) throws Exception {
> DefaultCamelContext camelContext = new DefaultCamelContext();
> ActiveMQComponent activeMQComponent = new ActiveMQComponent();
> activeMQComponent.setBrokerURL("failover:(http://localhost:61616
> )");
> camelContext.addComponent("activemq", activeMQComponent);
> camelContext.addRoutes(new RouteBuilder() {
> @Override
> public void configure() {
> from("activemq:queue:myqueue").to("stream:out");
> }
> });
> Thread startThread = new Thread(new Runnable() {
> @Override
> public void run() {
> try {
> camelContext.start();
> }
> catch (Exception e) {
> e.printStackTrace();
> }
> }
> });
> startThread.start();
> startThread.interrupt();
> startThread.join();
> camelContext.stop();
> }
>
> }
>
RE: Problem shutting down activemq client with failover transport
Posted by Anna Maier <A....@topdesk.com>.
Hi,
I debugged a bit, and it looks like it never calls the shutdown method in PooledTaskRunner after all. It gets stuck in DefaultMessageListenerContainer.doShutdown() where it calls this.lifecycleMonitor.wait(). Apparently, this does not terminate. I am not sure if this is a bug in Camel, ActiveMQ or Spring, or if it is even intended behaviour.
Anyway, setting the timeout to camel does the trick:
camelContext.getShutdownStrategy().setTimeout(1L);
Apparently the default for this is 5 minutes and I was never patient enough to wait so long. ;)
Thanks for the help!
Anna
-----Original Message-----
From: Ganesh Murthy [mailto:gmurthy@redhat.com]
Sent: Mittwoch, 30. September 2015 15:12
To: users@activemq.apache.org
Subject: Re: Problem shutting down activemq client with failover transport
How is Camel setting the shutdown flag on the PooledTaskRunner? Is Camel calling the shutdown() function to shut it down? The shutdown() method (which in turn calls shutdown(0)) on PooledTaskRunner is guaranteed to shut it down because if shutdown is set to true, then iterating is set to false.
Have you tried the "timeout" parameter in FailoverTransport?
http://activemq.apache.org/failover-transport-reference.html
If you are using an IDE like IntelliJ, try putting in breakpoints on only that PooledTaskRunner thread and analyze what is going on.
Thanks.
-----Original Message-----
From: Ganesh Murthy [mailto:gmurthy@redhat.com]
Sent: Dienstag, 29. September 2015 16:52
To: users@activemq.apache.org
Subject: Re: Problem shutting down activemq client with failover transport
Can you please provide a thread dump? That would throw some light on exactly what is going on.
Thanks.
----- Original Message -----
From: "Anna Maier" <A....@topdesk.com>
To: users@activemq.apache.org
Sent: Tuesday, September 29, 2015 9:31:13 AM
Subject: Problem shutting down activemq client with failover transport
Hi,
I am using Camel to connect to ActiveMQ and it fails to shutdown when the ActiveMQComponent cannot make a connection to the broker with the failover protocol (activemq version 5.12.0).
The problem seems to be that the failover transport tries to connect to the broker in an endless loop on its own thread that cannot be interrupted. I think the problem is in the PooledTaskRunner: there is a shutdown field which is set to true when camel shuts down, but it appears as false to the thread that is running the endless loop.
I am currently at a loss what to do. I want to keep the maximum retries setting for the failover transport on indefinite, since I do not know when the the broker is back online. At the same time, it has to be possible to shut down the application even if there never was a connection to the ActiveMQ broker.
Has anybody seen this problem before and can give some advice? Is this a bug in ActiveMQ?
Attached there is some code that reproduces the problem - it never terminates.
Regards,
Anna
The code:
public class ShutdownBug {
public static void main(String[] args) throws Exception {
DefaultCamelContext camelContext = new DefaultCamelContext();
ActiveMQComponent activeMQComponent = new ActiveMQComponent();
activeMQComponent.setBrokerURL("failover:(http://localhost:61616)");
camelContext.addComponent("activemq", activeMQComponent);
camelContext.addRoutes(new RouteBuilder() {
@Override
public void configure() {
from("activemq:queue:myqueue").to("stream:out");
}
});
Thread startThread = new Thread(new Runnable() {
@Override
public void run() {
try {
camelContext.start();
}
catch (Exception e) {
e.printStackTrace();
}
}
});
startThread.start();
startThread.interrupt();
startThread.join();
camelContext.stop();
}
}
Re: Problem shutting down activemq client with failover transport
Posted by Ganesh Murthy <gm...@redhat.com>.
How is Camel setting the shutdown flag on the PooledTaskRunner? Is Camel calling the shutdown() function to shut it down? The shutdown() method (which in turn calls shutdown(0)) on PooledTaskRunner is guaranteed to shut it down because if shutdown is set to true, then iterating is set to false.
Have you tried the "timeout" parameter in FailoverTransport?
http://activemq.apache.org/failover-transport-reference.html
If you are using an IDE like IntelliJ, try putting in breakpoints on only that PooledTaskRunner thread and analyze what is going on.
Thanks.
-----Original Message-----
From: Ganesh Murthy [mailto:gmurthy@redhat.com]
Sent: Dienstag, 29. September 2015 16:52
To: users@activemq.apache.org
Subject: Re: Problem shutting down activemq client with failover transport
Can you please provide a thread dump? That would throw some light on exactly what is going on.
Thanks.
----- Original Message -----
From: "Anna Maier" <A....@topdesk.com>
To: users@activemq.apache.org
Sent: Tuesday, September 29, 2015 9:31:13 AM
Subject: Problem shutting down activemq client with failover transport
Hi,
I am using Camel to connect to ActiveMQ and it fails to shutdown when the ActiveMQComponent cannot make a connection to the broker with the failover protocol (activemq version 5.12.0).
The problem seems to be that the failover transport tries to connect to the broker in an endless loop on its own thread that cannot be interrupted. I think the problem is in the PooledTaskRunner: there is a shutdown field which is set to true when camel shuts down, but it appears as false to the thread that is running the endless loop.
I am currently at a loss what to do. I want to keep the maximum retries setting for the failover transport on indefinite, since I do not know when the the broker is back online. At the same time, it has to be possible to shut down the application even if there never was a connection to the ActiveMQ broker.
Has anybody seen this problem before and can give some advice? Is this a bug in ActiveMQ?
Attached there is some code that reproduces the problem - it never terminates.
Regards,
Anna
The code:
public class ShutdownBug {
public static void main(String[] args) throws Exception {
DefaultCamelContext camelContext = new DefaultCamelContext();
ActiveMQComponent activeMQComponent = new ActiveMQComponent();
activeMQComponent.setBrokerURL("failover:(http://localhost:61616)");
camelContext.addComponent("activemq", activeMQComponent);
camelContext.addRoutes(new RouteBuilder() {
@Override
public void configure() {
from("activemq:queue:myqueue").to("stream:out");
}
});
Thread startThread = new Thread(new Runnable() {
@Override
public void run() {
try {
camelContext.start();
}
catch (Exception e) {
e.printStackTrace();
}
}
});
startThread.start();
startThread.interrupt();
startThread.join();
camelContext.stop();
}
}
RE: Problem shutting down activemq client with failover transport
Posted by Anna Maier <A....@topdesk.com>.
Hi,
please find attached the thread dump.
Regards,
Anna
-----Original Message-----
From: Ganesh Murthy [mailto:gmurthy@redhat.com]
Sent: Dienstag, 29. September 2015 16:52
To: users@activemq.apache.org
Subject: Re: Problem shutting down activemq client with failover transport
Can you please provide a thread dump? That would throw some light on exactly what is going on.
Thanks.
----- Original Message -----
From: "Anna Maier" <A....@topdesk.com>
To: users@activemq.apache.org
Sent: Tuesday, September 29, 2015 9:31:13 AM
Subject: Problem shutting down activemq client with failover transport
Hi,
I am using Camel to connect to ActiveMQ and it fails to shutdown when the ActiveMQComponent cannot make a connection to the broker with the failover protocol (activemq version 5.12.0).
The problem seems to be that the failover transport tries to connect to the broker in an endless loop on its own thread that cannot be interrupted. I think the problem is in the PooledTaskRunner: there is a shutdown field which is set to true when camel shuts down, but it appears as false to the thread that is running the endless loop.
I am currently at a loss what to do. I want to keep the maximum retries setting for the failover transport on indefinite, since I do not know when the the broker is back online. At the same time, it has to be possible to shut down the application even if there never was a connection to the ActiveMQ broker.
Has anybody seen this problem before and can give some advice? Is this a bug in ActiveMQ?
Attached there is some code that reproduces the problem - it never terminates.
Regards,
Anna
The code:
public class ShutdownBug {
public static void main(String[] args) throws Exception {
DefaultCamelContext camelContext = new DefaultCamelContext();
ActiveMQComponent activeMQComponent = new ActiveMQComponent();
activeMQComponent.setBrokerURL("failover:(http://localhost:61616)");
camelContext.addComponent("activemq", activeMQComponent);
camelContext.addRoutes(new RouteBuilder() {
@Override
public void configure() {
from("activemq:queue:myqueue").to("stream:out");
}
});
Thread startThread = new Thread(new Runnable() {
@Override
public void run() {
try {
camelContext.start();
}
catch (Exception e) {
e.printStackTrace();
}
}
});
startThread.start();
startThread.interrupt();
startThread.join();
camelContext.stop();
}
}
Re: Problem shutting down activemq client with failover transport
Posted by Ganesh Murthy <gm...@redhat.com>.
Can you please provide a thread dump? That would throw some light on exactly what is going on.
Thanks.
----- Original Message -----
From: "Anna Maier" <A....@topdesk.com>
To: users@activemq.apache.org
Sent: Tuesday, September 29, 2015 9:31:13 AM
Subject: Problem shutting down activemq client with failover transport
Hi,
I am using Camel to connect to ActiveMQ and it fails to shutdown when the ActiveMQComponent cannot make a connection to the broker with the failover protocol (activemq version 5.12.0).
The problem seems to be that the failover transport tries to connect to the broker in an endless loop on its own thread that cannot be interrupted. I think the problem is in the PooledTaskRunner: there is a shutdown field which is set to true when camel shuts down, but it appears as false to the thread that is running the endless loop.
I am currently at a loss what to do. I want to keep the maximum retries setting for the failover transport on indefinite, since I do not know when the the broker is back online. At the same time, it has to be possible to shut down the application even if there never was a connection to the ActiveMQ broker.
Has anybody seen this problem before and can give some advice? Is this a bug in ActiveMQ?
Attached there is some code that reproduces the problem - it never terminates.
Regards,
Anna
The code:
public class ShutdownBug {
public static void main(String[] args) throws Exception {
DefaultCamelContext camelContext = new DefaultCamelContext();
ActiveMQComponent activeMQComponent = new ActiveMQComponent();
activeMQComponent.setBrokerURL("failover:(http://localhost:61616)");
camelContext.addComponent("activemq", activeMQComponent);
camelContext.addRoutes(new RouteBuilder() {
@Override
public void configure() {
from("activemq:queue:myqueue").to("stream:out");
}
});
Thread startThread = new Thread(new Runnable() {
@Override
public void run() {
try {
camelContext.start();
}
catch (Exception e) {
e.printStackTrace();
}
}
});
startThread.start();
startThread.interrupt();
startThread.join();
camelContext.stop();
}
}
Re: Problem shutting down activemq client with failover transport
Posted by Martin Lichtin <li...@yahoo.com.INVALID>.
I've seen this behaviour as well and did not find a way around it besides limiting the number of _startup_ reconnect attempts.
Append something like "?startupMaxReconnectAttempts=5" to the Url.
On 29.09.2015 15:31, Anna Maier wrote:
> Hi,
>
>
>
> I am using Camel to connect to ActiveMQ and it fails to shutdown when the ActiveMQComponent cannot make a connection to the broker with the failover protocol (activemq version 5.12.0).
>
> The problem seems to be that the failover transport tries to connect to the broker in an endless loop on its own thread that cannot be interrupted. I think the problem is in the PooledTaskRunner: there is a shutdown field which is set to true when camel shuts down, but it appears as false to the thread that is running the endless loop.
>
>
>
> I am currently at a loss what to do. I want to keep the maximum retries setting for the failover transport on indefinite, since I do not know when the the broker is back online. At the same time, it has to be possible to shut down the application even if there never was a connection to the ActiveMQ broker.
>
>
>
> Has anybody seen this problem before and can give some advice? Is this a bug in ActiveMQ?
>
>
>
> Attached there is some code that reproduces the problem - it never terminates.
>
>
>
> Regards,
>
> Anna
>
>
>
> The code:
> public class ShutdownBug {
>
> public static void main(String[] args) throws Exception {
> DefaultCamelContext camelContext = new DefaultCamelContext();
> ActiveMQComponent activeMQComponent = new ActiveMQComponent();
> activeMQComponent.setBrokerURL("failover:(http://localhost:61616)");
> camelContext.addComponent("activemq", activeMQComponent);
> camelContext.addRoutes(new RouteBuilder() {
> @Override
> public void configure() {
> from("activemq:queue:myqueue").to("stream:out");
> }
> });
> Thread startThread = new Thread(new Runnable() {
> @Override
> public void run() {
> try {
> camelContext.start();
> }
> catch (Exception e) {
> e.printStackTrace();
> }
> }
> });
> startThread.start();
> startThread.interrupt();
> startThread.join();
> camelContext.stop();
> }
>
> }
>