You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Nick Stolwijk <ni...@gmail.com> on 2017/09/27 21:49:29 UTC
ActiveMQ Cluster in Docker Swarm
Hi,
We are just starting with Docker Swarm and we have 2 services that need to
talk to each other asynchronously. The messages need to be confirmed that
they have been processed. So that when a container goes down, another will
pick it up.
With these requirements I thought to use an MQ and with my experiences with
ActiveMQ in the past, that seemed the logical choice.
After using my Google Fuu for some while, I couldn't find a working example
to run an ActiveMQ cluster in Docker Swarm. I was wondering if there is
someone that got this to work and would share their experiences.
FYI, We have looked at Kubernetes instead of Docker Swarm. We found out
that Docker Swarm is easier to use and configure, although Kubernetes is
more complete. We thought it wisely to start simple and once we mastered
that and need more we can switch to Kubernetes.
With regards,
Nick Stolwijk
~~~ Try to leave this world a little better than you found it and, when
your turn comes to die, you can die happy in feeling that at any rate you
have not wasted your time but have done your best ~~~
Lord Baden-Powell
Re: ActiveMQ Cluster in Docker Swarm
Posted by Lachezar Dobrev <l....@gmail.com>.
Hm.
I have had experience with running (embedded) Artemis (in
Springframework) in a Docker Swarm. Most of what I had tried failed.
The main reason was that the virtual network adapters that are
available (sans any commercial and third-party ones) do not support
broadcast/multicast so dynamic discovery fails. The ones that did
support broadcast/multicast did not work properly when the swarm was
distributed over multiple nodes.
I can not provide any code due to the commercial nature of the application.
I can tell you how I solved it eventually: when running inside a
docker swarm instance doing an 'nslookup tasks.service-name' (replace
service-name with the actual service name) returns the IPs of all the
(current) instances of the service. Using that information (and
excluding the 127.*.*.*) I got the IPs of the other nodes in the
service. With that information and using a dedicated
ArtemisConfigurationCustomizer (from Springframework) I declared
static connectors to all the other nodes, and built the cluster using
static connectors only.
Notice: I do not use XML configuration for Artemis, rather I use
Springframework for configuration.
Parse tcp://x.x.x.x:yyyy/
...artemis.uri.ConnectorTransportConfigurationParser.newObject(URI, ?)
Declare static connectors:
...artemis.core.config.Configuration.addConnectorConfiguration(String,
TransportConfiguration)
Declare static connectors as cluster backbone:
...artemis.core.config.ClusterConnectionConfiguration.setStaticConnectors(List<String>)
Add the static cluster configuration:
...artemis.core.config.Configuration.addClusterConfiguration(ClusterConnectionConfiguration)
That's admittedly a lot of work, in many places: docker for set-up,
bash for lookup, Springframework for configuration, Artemis for
configuration. I believe it *could* be done more concisely, but I did
not see any interest in continuing that (and I still don't).
2017-09-28 11:27 GMT+03:00 Nick Stolwijk <ni...@gmail.com>:
> Thanks.
> I am looking now at Artemis and got it running through Rancher and Docker
> Swarm.
>
> Has anyone got information about service discovery for Artemis with Docker
> Swarm?
>
> Looking at the Kubernetes example I need to use some XSLT templates at
> deployment to create the right override configuration. Is that possible
> with Docker Swarm and Docker Compose?
>
> With regards,
>
> Nick Stolwijk
>
> ~~~ Try to leave this world a little better than you found it and, when
> your turn comes to die, you can die happy in feeling that at any rate you
> have not wasted your time but have done your best ~~~
>
> Lord Baden-Powell
>
> On Thu, Sep 28, 2017 at 4:05 AM, Tim Bain <tb...@alumni.duke.edu> wrote:
>
>> Since you're looking for clustered operation, ActiveMQ Artemis sounds like
>> a better fit for your needs than ActiveMQ 5.x would be.
>>
>> Tim
>>
>> On Sep 27, 2017 3:49 PM, "Nick Stolwijk" <ni...@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > We are just starting with Docker Swarm and we have 2 services that need
>> to
>> > talk to each other asynchronously. The messages need to be confirmed that
>> > they have been processed. So that when a container goes down, another
>> will
>> > pick it up.
>> >
>> > With these requirements I thought to use an MQ and with my experiences
>> with
>> > ActiveMQ in the past, that seemed the logical choice.
>> >
>> > After using my Google Fuu for some while, I couldn't find a working
>> example
>> > to run an ActiveMQ cluster in Docker Swarm. I was wondering if there is
>> > someone that got this to work and would share their experiences.
>> >
>> > FYI, We have looked at Kubernetes instead of Docker Swarm. We found out
>> > that Docker Swarm is easier to use and configure, although Kubernetes is
>> > more complete. We thought it wisely to start simple and once we mastered
>> > that and need more we can switch to Kubernetes.
>> >
>> > With regards,
>> >
>> > Nick Stolwijk
>> >
>> > ~~~ Try to leave this world a little better than you found it and, when
>> > your turn comes to die, you can die happy in feeling that at any rate you
>> > have not wasted your time but have done your best ~~~
>> >
>> > Lord Baden-Powell
>> >
>>
Re: ActiveMQ Cluster in Docker Swarm
Posted by Nick Stolwijk <ni...@gmail.com>.
Thanks.
I am looking now at Artemis and got it running through Rancher and Docker
Swarm.
Has anyone got information about service discovery for Artemis with Docker
Swarm?
Looking at the Kubernetes example I need to use some XSLT templates at
deployment to create the right override configuration. Is that possible
with Docker Swarm and Docker Compose?
With regards,
Nick Stolwijk
~~~ Try to leave this world a little better than you found it and, when
your turn comes to die, you can die happy in feeling that at any rate you
have not wasted your time but have done your best ~~~
Lord Baden-Powell
On Thu, Sep 28, 2017 at 4:05 AM, Tim Bain <tb...@alumni.duke.edu> wrote:
> Since you're looking for clustered operation, ActiveMQ Artemis sounds like
> a better fit for your needs than ActiveMQ 5.x would be.
>
> Tim
>
> On Sep 27, 2017 3:49 PM, "Nick Stolwijk" <ni...@gmail.com> wrote:
>
> > Hi,
> >
> > We are just starting with Docker Swarm and we have 2 services that need
> to
> > talk to each other asynchronously. The messages need to be confirmed that
> > they have been processed. So that when a container goes down, another
> will
> > pick it up.
> >
> > With these requirements I thought to use an MQ and with my experiences
> with
> > ActiveMQ in the past, that seemed the logical choice.
> >
> > After using my Google Fuu for some while, I couldn't find a working
> example
> > to run an ActiveMQ cluster in Docker Swarm. I was wondering if there is
> > someone that got this to work and would share their experiences.
> >
> > FYI, We have looked at Kubernetes instead of Docker Swarm. We found out
> > that Docker Swarm is easier to use and configure, although Kubernetes is
> > more complete. We thought it wisely to start simple and once we mastered
> > that and need more we can switch to Kubernetes.
> >
> > With regards,
> >
> > Nick Stolwijk
> >
> > ~~~ Try to leave this world a little better than you found it and, when
> > your turn comes to die, you can die happy in feeling that at any rate you
> > have not wasted your time but have done your best ~~~
> >
> > Lord Baden-Powell
> >
>
Re: ActiveMQ Cluster in Docker Swarm
Posted by Tim Bain <tb...@alumni.duke.edu>.
Since you're looking for clustered operation, ActiveMQ Artemis sounds like
a better fit for your needs than ActiveMQ 5.x would be.
Tim
On Sep 27, 2017 3:49 PM, "Nick Stolwijk" <ni...@gmail.com> wrote:
> Hi,
>
> We are just starting with Docker Swarm and we have 2 services that need to
> talk to each other asynchronously. The messages need to be confirmed that
> they have been processed. So that when a container goes down, another will
> pick it up.
>
> With these requirements I thought to use an MQ and with my experiences with
> ActiveMQ in the past, that seemed the logical choice.
>
> After using my Google Fuu for some while, I couldn't find a working example
> to run an ActiveMQ cluster in Docker Swarm. I was wondering if there is
> someone that got this to work and would share their experiences.
>
> FYI, We have looked at Kubernetes instead of Docker Swarm. We found out
> that Docker Swarm is easier to use and configure, although Kubernetes is
> more complete. We thought it wisely to start simple and once we mastered
> that and need more we can switch to Kubernetes.
>
> With regards,
>
> Nick Stolwijk
>
> ~~~ Try to leave this world a little better than you found it and, when
> your turn comes to die, you can die happy in feeling that at any rate you
> have not wasted your time but have done your best ~~~
>
> Lord Baden-Powell
>