You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by Victor <vi...@gmail.com> on 2018/06/07 06:53:48 UTC
[artemis] Dynamically add cluster connection using JMX/Jolokia?
Hi all,
I'm having some fun trying to figure out the ideal way to deploy an
ActiveMQ Artemis cluster in Kubernetes.
I started by creating a Helm Chart (think of it as a deb/rpm package for
Kubernetes) where I can just install a Symmetric cluster of 8 nodes with
the following commands:
helm repo add activemq-artemis https://vromero.github.io/
activemq-artemis-helm/
helm install vromero/activemq-artemis --set replicas=8
It works pretty well but I'm creating static-connectors for the cluster
connection at the install time. Meaning that if later the user wants to
scale up or down his cluster it won't work.
I have been exploring these two ideas to overcome it:
- ActiveMQ Artemis Plugin to enable auto-discover by connecting to the
Artemis to the Kubernetes api
- To have an external minimal agent written in Go (given that it has the
best client library for k8s and it has a very small binary / memory
footprint) that connects to k8s and let know Artemis from the outside which
other nodes are available for the cluster.
After a quick look at the first option, the one of the plugin, I decided to
keep it as plan B given that a hypothetically big cluster might end up
pounding too hard the Kubernetes API and especially given that the
Kubernetes client would bring a not too big but neither too small number of
transitive dependencies to Artemis.
So off I went with the external agent to now realize that at least at a
quick glance at the JMX/Jolokia API I see no method to add new
cluster-connections. So my question would be:
Is there any way to add externally cluster-connections to Artemis?
I'm hoping that this belongs to the dev list as it is kind-of advanced if
it actually belongs to user I apologise
Thanks
Re: [artemis] Dynamically add cluster connection using JMX/Jolokia?
Posted by Clebert Suconic <cl...@gmail.com>.
This definitely belongs to Dev.
I want to get more involved with Kubernetes. I am setting time apart
into this for upcoming weeks. Right now I don't have much information
to help you.. but please keep us informed of your discoveries :)
On Thu, Jun 7, 2018 at 2:53 AM, Victor <vi...@gmail.com> wrote:
> Hi all,
>
> I'm having some fun trying to figure out the ideal way to deploy an
> ActiveMQ Artemis cluster in Kubernetes.
>
> I started by creating a Helm Chart (think of it as a deb/rpm package for
> Kubernetes) where I can just install a Symmetric cluster of 8 nodes with
> the following commands:
>
> helm repo add activemq-artemis https://vromero.github.io/
> activemq-artemis-helm/
> helm install vromero/activemq-artemis --set replicas=8
>
> It works pretty well but I'm creating static-connectors for the cluster
> connection at the install time. Meaning that if later the user wants to
> scale up or down his cluster it won't work.
>
> I have been exploring these two ideas to overcome it:
>
> - ActiveMQ Artemis Plugin to enable auto-discover by connecting to the
> Artemis to the Kubernetes api
> - To have an external minimal agent written in Go (given that it has the
> best client library for k8s and it has a very small binary / memory
> footprint) that connects to k8s and let know Artemis from the outside which
> other nodes are available for the cluster.
>
> After a quick look at the first option, the one of the plugin, I decided to
> keep it as plan B given that a hypothetically big cluster might end up
> pounding too hard the Kubernetes API and especially given that the
> Kubernetes client would bring a not too big but neither too small number of
> transitive dependencies to Artemis.
>
> So off I went with the external agent to now realize that at least at a
> quick glance at the JMX/Jolokia API I see no method to add new
> cluster-connections. So my question would be:
>
> Is there any way to add externally cluster-connections to Artemis?
>
> I'm hoping that this belongs to the dev list as it is kind-of advanced if
> it actually belongs to user I apologise
>
> Thanks
--
Clebert Suconic
Re: [artemis] Dynamically add cluster connection using JMX/Jolokia?
Posted by Victor <vi...@gmail.com>.
Thanks guys,
I will definitely try the JGroups configuration with KUBE_PING.
I'll keep an eye on the list for possible clustering updates and in the
meantime, I will try to do my best with the helm chart and JGroups.
In the big scheme of things, what I'm trying to do is to have a Docker
image eventually worthy of being official and a Helm chart good enough to
be included in the official helm charts repository to drive more adoption
of Artemis in the cloud native world.
2018-06-07 8:15 GMT-07:00 Christopher Shannon <
christopher.l.shannon@gmail.com>:
> As Justin says there is currently no way to add a cluster connection
> dynamically. However, I also have a use case where it would be nice to be
> able to dynamically add or destroy a cluster connection. I will probably
> take a look when I get some time to see if there's a way that the behavior
> can be supported but it probably won't be easy to do with the current
> design.
>
> On Thu, Jun 7, 2018 at 10:09 AM Justin Bertram <jb...@apache.org>
> wrote:
>
> > There is no programmatic way to add a cluster-connection. The
> > cluster-connection is such a fundamental piece of the broker's
> > configuration it really has to be there when the broker starts. Also,
> > there is no mechanism to persist a cluster-connection configuration other
> > than in the XML (unlike addresses and queues which can be created at
> > runtime and are persisted, if necessary, in the bindings journal).
> >
> > As for the actual clustering mechanism, have you tried using a JGroups
> > configuration with KUBE_PING [1]?
> >
> >
> > Justin
> >
> > [1] https://github.com/jgroups-extras/jgroups-kubernetes
> >
> > On Thu, Jun 7, 2018 at 1:53 AM, Victor <vi...@gmail.com> wrote:
> >
> > > Hi all,
> > >
> > > I'm having some fun trying to figure out the ideal way to deploy an
> > > ActiveMQ Artemis cluster in Kubernetes.
> > >
> > > I started by creating a Helm Chart (think of it as a deb/rpm package
> for
> > > Kubernetes) where I can just install a Symmetric cluster of 8 nodes
> with
> > > the following commands:
> > >
> > > helm repo add activemq-artemis https://vromero.github.io/
> > > activemq-artemis-helm/
> > > helm install vromero/activemq-artemis --set replicas=8
> > >
> > > It works pretty well but I'm creating static-connectors for the cluster
> > > connection at the install time. Meaning that if later the user wants to
> > > scale up or down his cluster it won't work.
> > >
> > > I have been exploring these two ideas to overcome it:
> > >
> > > - ActiveMQ Artemis Plugin to enable auto-discover by connecting to the
> > > Artemis to the Kubernetes api
> > > - To have an external minimal agent written in Go (given that it has
> the
> > > best client library for k8s and it has a very small binary / memory
> > > footprint) that connects to k8s and let know Artemis from the outside
> > which
> > > other nodes are available for the cluster.
> > >
> > > After a quick look at the first option, the one of the plugin, I
> decided
> > to
> > > keep it as plan B given that a hypothetically big cluster might end up
> > > pounding too hard the Kubernetes API and especially given that the
> > > Kubernetes client would bring a not too big but neither too small
> number
> > of
> > > transitive dependencies to Artemis.
> > >
> > > So off I went with the external agent to now realize that at least at a
> > > quick glance at the JMX/Jolokia API I see no method to add new
> > > cluster-connections. So my question would be:
> > >
> > > Is there any way to add externally cluster-connections to Artemis?
> > >
> > > I'm hoping that this belongs to the dev list as it is kind-of advanced
> if
> > > it actually belongs to user I apologise
> > >
> > > Thanks
> > >
> >
>
Re: [artemis] Dynamically add cluster connection using JMX/Jolokia?
Posted by Christopher Shannon <ch...@gmail.com>.
As Justin says there is currently no way to add a cluster connection
dynamically. However, I also have a use case where it would be nice to be
able to dynamically add or destroy a cluster connection. I will probably
take a look when I get some time to see if there's a way that the behavior
can be supported but it probably won't be easy to do with the current
design.
On Thu, Jun 7, 2018 at 10:09 AM Justin Bertram <jb...@apache.org> wrote:
> There is no programmatic way to add a cluster-connection. The
> cluster-connection is such a fundamental piece of the broker's
> configuration it really has to be there when the broker starts. Also,
> there is no mechanism to persist a cluster-connection configuration other
> than in the XML (unlike addresses and queues which can be created at
> runtime and are persisted, if necessary, in the bindings journal).
>
> As for the actual clustering mechanism, have you tried using a JGroups
> configuration with KUBE_PING [1]?
>
>
> Justin
>
> [1] https://github.com/jgroups-extras/jgroups-kubernetes
>
> On Thu, Jun 7, 2018 at 1:53 AM, Victor <vi...@gmail.com> wrote:
>
> > Hi all,
> >
> > I'm having some fun trying to figure out the ideal way to deploy an
> > ActiveMQ Artemis cluster in Kubernetes.
> >
> > I started by creating a Helm Chart (think of it as a deb/rpm package for
> > Kubernetes) where I can just install a Symmetric cluster of 8 nodes with
> > the following commands:
> >
> > helm repo add activemq-artemis https://vromero.github.io/
> > activemq-artemis-helm/
> > helm install vromero/activemq-artemis --set replicas=8
> >
> > It works pretty well but I'm creating static-connectors for the cluster
> > connection at the install time. Meaning that if later the user wants to
> > scale up or down his cluster it won't work.
> >
> > I have been exploring these two ideas to overcome it:
> >
> > - ActiveMQ Artemis Plugin to enable auto-discover by connecting to the
> > Artemis to the Kubernetes api
> > - To have an external minimal agent written in Go (given that it has the
> > best client library for k8s and it has a very small binary / memory
> > footprint) that connects to k8s and let know Artemis from the outside
> which
> > other nodes are available for the cluster.
> >
> > After a quick look at the first option, the one of the plugin, I decided
> to
> > keep it as plan B given that a hypothetically big cluster might end up
> > pounding too hard the Kubernetes API and especially given that the
> > Kubernetes client would bring a not too big but neither too small number
> of
> > transitive dependencies to Artemis.
> >
> > So off I went with the external agent to now realize that at least at a
> > quick glance at the JMX/Jolokia API I see no method to add new
> > cluster-connections. So my question would be:
> >
> > Is there any way to add externally cluster-connections to Artemis?
> >
> > I'm hoping that this belongs to the dev list as it is kind-of advanced if
> > it actually belongs to user I apologise
> >
> > Thanks
> >
>
Re: [artemis] Dynamically add cluster connection using JMX/Jolokia?
Posted by Justin Bertram <jb...@apache.org>.
There is no programmatic way to add a cluster-connection. The
cluster-connection is such a fundamental piece of the broker's
configuration it really has to be there when the broker starts. Also,
there is no mechanism to persist a cluster-connection configuration other
than in the XML (unlike addresses and queues which can be created at
runtime and are persisted, if necessary, in the bindings journal).
As for the actual clustering mechanism, have you tried using a JGroups
configuration with KUBE_PING [1]?
Justin
[1] https://github.com/jgroups-extras/jgroups-kubernetes
On Thu, Jun 7, 2018 at 1:53 AM, Victor <vi...@gmail.com> wrote:
> Hi all,
>
> I'm having some fun trying to figure out the ideal way to deploy an
> ActiveMQ Artemis cluster in Kubernetes.
>
> I started by creating a Helm Chart (think of it as a deb/rpm package for
> Kubernetes) where I can just install a Symmetric cluster of 8 nodes with
> the following commands:
>
> helm repo add activemq-artemis https://vromero.github.io/
> activemq-artemis-helm/
> helm install vromero/activemq-artemis --set replicas=8
>
> It works pretty well but I'm creating static-connectors for the cluster
> connection at the install time. Meaning that if later the user wants to
> scale up or down his cluster it won't work.
>
> I have been exploring these two ideas to overcome it:
>
> - ActiveMQ Artemis Plugin to enable auto-discover by connecting to the
> Artemis to the Kubernetes api
> - To have an external minimal agent written in Go (given that it has the
> best client library for k8s and it has a very small binary / memory
> footprint) that connects to k8s and let know Artemis from the outside which
> other nodes are available for the cluster.
>
> After a quick look at the first option, the one of the plugin, I decided to
> keep it as plan B given that a hypothetically big cluster might end up
> pounding too hard the Kubernetes API and especially given that the
> Kubernetes client would bring a not too big but neither too small number of
> transitive dependencies to Artemis.
>
> So off I went with the external agent to now realize that at least at a
> quick glance at the JMX/Jolokia API I see no method to add new
> cluster-connections. So my question would be:
>
> Is there any way to add externally cluster-connections to Artemis?
>
> I'm hoping that this belongs to the dev list as it is kind-of advanced if
> it actually belongs to user I apologise
>
> Thanks
>