You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Lachezar Dobrev <l....@gmail.com> on 2016/10/03 13:54:59 UTC
Duplicating messages published to a Topic to a Queue
Hello all,
I have a design problem: a topic is clustered in a network of brokers to
allow multiple nodes to handle events occurring in other nodes: i.e. a user
logs in on one node, but all nodes *might* need to know that.
A new requirement has surfaced, that copies of these messages need to be
sent to an external system (not a messaging system). The trivial solution
is to add a new subscriber to the Topic that handles the delivery, but
since this is a distributed system of equivalent nodes that ends up sending
the message multiple times, one per node in the cluster.
My current solution is to use Springframework's infrastructure (which the
Application is based on), and override the JMSTemplate with a custom one
that sends a copy of every message sent to the Topic to a special Queue
that the deliverer is handling. This seems to work just fine.
I'm wondering if there is any out-of-the-box solution to this problem
provided by Artemis. Like routing/duplication/monitoring that I can turn on
and get the same (or similar) functionality.
Lachezar
Re: Duplicating messages published to a Topic to a Queue
Posted by Martyn Taylor <mt...@redhat.com>.
Good to hear,
Re: Your comments
The "jms.topic." and "jms.queue." prefix requirements are causing a lot of
confusion for many users, hopefully it'll be resolved soon, as part of:
https://issues.apache.org/jira/browse/ARTEMIS-780
Regarding the divert name, we should probably throw an exception on boot to
make this clearer. I've opened a JIRA here:
https://issues.apache.org/jira/browse/ARTEMIS-802
Thanks
On Thu, Oct 13, 2016 at 1:50 PM, Lachezar Dobrev <l....@gmail.com> wrote:
> Thanks!
> Diverting was exactly what I was looking for. Creating a non-exclusive
> divert from the topic to the queue did the trick without any class changes.
> Very nice. Using the Spring support for declarative configuration
> management it is pretty easy to declare diverts.
> A few things bit me on the way though:
> - diver name must be specified, otherwise the diver is ignored with a log
> message (easily missed)
> - to divert topic named "topic.whatever" the address must be
> "jms.topic.topic.whatever"
> - to divert to a query named "query.whatever" the forwarding address must
> be "jms.queue.queue.whatever"
>
> Otherwise the diverts work pretty well, I even managed to catch one queue
> message being processed on a different node than the one sending the topic
> message, which is exactly what I was hoping for.
>
> Thanks again for the directions.
>
> 2016-10-07 13:10 GMT+03:00 Martyn Taylor <mt...@redhat.com>:
>
> > I don't have enough information to understand you problem fully. But
> > perhaps you could use a combination of duplicate detection and diverts
> > could solve your problem.
> >
> > https://activemq.apache.org/artemis/docs/1.4.0/duplicate-detection.htm
> > https://activemq.apache.org/artemis/docs/1.4.0/diverts.html
> >
> > Cheers
> > Martyn
> >
> >
> >
> > On Mon, Oct 3, 2016 at 2:54 PM, Lachezar Dobrev <l....@gmail.com>
> > wrote:
> >
> > > Hello all,
> > >
> > > I have a design problem: a topic is clustered in a network of brokers
> > to
> > > allow multiple nodes to handle events occurring in other nodes: i.e. a
> > user
> > > logs in on one node, but all nodes *might* need to know that.
> > > A new requirement has surfaced, that copies of these messages need to
> > be
> > > sent to an external system (not a messaging system). The trivial
> solution
> > > is to add a new subscriber to the Topic that handles the delivery, but
> > > since this is a distributed system of equivalent nodes that ends up
> > sending
> > > the message multiple times, one per node in the cluster.
> > >
> > > My current solution is to use Springframework's infrastructure (which
> > the
> > > Application is based on), and override the JMSTemplate with a custom
> one
> > > that sends a copy of every message sent to the Topic to a special Queue
> > > that the deliverer is handling. This seems to work just fine.
> > >
> > > I'm wondering if there is any out-of-the-box solution to this problem
> > > provided by Artemis. Like routing/duplication/monitoring that I can
> turn
> > on
> > > and get the same (or similar) functionality.
> > >
> > > Lachezar
> > >
> >
>
Re: Duplicating messages published to a Topic to a Queue
Posted by Lachezar Dobrev <l....@gmail.com>.
Thanks!
Diverting was exactly what I was looking for. Creating a non-exclusive
divert from the topic to the queue did the trick without any class changes.
Very nice. Using the Spring support for declarative configuration
management it is pretty easy to declare diverts.
A few things bit me on the way though:
- diver name must be specified, otherwise the diver is ignored with a log
message (easily missed)
- to divert topic named "topic.whatever" the address must be
"jms.topic.topic.whatever"
- to divert to a query named "query.whatever" the forwarding address must
be "jms.queue.queue.whatever"
Otherwise the diverts work pretty well, I even managed to catch one queue
message being processed on a different node than the one sending the topic
message, which is exactly what I was hoping for.
Thanks again for the directions.
2016-10-07 13:10 GMT+03:00 Martyn Taylor <mt...@redhat.com>:
> I don't have enough information to understand you problem fully. But
> perhaps you could use a combination of duplicate detection and diverts
> could solve your problem.
>
> https://activemq.apache.org/artemis/docs/1.4.0/duplicate-detection.htm
> https://activemq.apache.org/artemis/docs/1.4.0/diverts.html
>
> Cheers
> Martyn
>
>
>
> On Mon, Oct 3, 2016 at 2:54 PM, Lachezar Dobrev <l....@gmail.com>
> wrote:
>
> > Hello all,
> >
> > I have a design problem: a topic is clustered in a network of brokers
> to
> > allow multiple nodes to handle events occurring in other nodes: i.e. a
> user
> > logs in on one node, but all nodes *might* need to know that.
> > A new requirement has surfaced, that copies of these messages need to
> be
> > sent to an external system (not a messaging system). The trivial solution
> > is to add a new subscriber to the Topic that handles the delivery, but
> > since this is a distributed system of equivalent nodes that ends up
> sending
> > the message multiple times, one per node in the cluster.
> >
> > My current solution is to use Springframework's infrastructure (which
> the
> > Application is based on), and override the JMSTemplate with a custom one
> > that sends a copy of every message sent to the Topic to a special Queue
> > that the deliverer is handling. This seems to work just fine.
> >
> > I'm wondering if there is any out-of-the-box solution to this problem
> > provided by Artemis. Like routing/duplication/monitoring that I can turn
> on
> > and get the same (or similar) functionality.
> >
> > Lachezar
> >
>
Re: Duplicating messages published to a Topic to a Queue
Posted by Martyn Taylor <mt...@redhat.com>.
I don't have enough information to understand you problem fully. But
perhaps you could use a combination of duplicate detection and diverts
could solve your problem.
https://activemq.apache.org/artemis/docs/1.4.0/duplicate-detection.htm
https://activemq.apache.org/artemis/docs/1.4.0/diverts.html
Cheers
Martyn
On Mon, Oct 3, 2016 at 2:54 PM, Lachezar Dobrev <l....@gmail.com> wrote:
> Hello all,
>
> I have a design problem: a topic is clustered in a network of brokers to
> allow multiple nodes to handle events occurring in other nodes: i.e. a user
> logs in on one node, but all nodes *might* need to know that.
> A new requirement has surfaced, that copies of these messages need to be
> sent to an external system (not a messaging system). The trivial solution
> is to add a new subscriber to the Topic that handles the delivery, but
> since this is a distributed system of equivalent nodes that ends up sending
> the message multiple times, one per node in the cluster.
>
> My current solution is to use Springframework's infrastructure (which the
> Application is based on), and override the JMSTemplate with a custom one
> that sends a copy of every message sent to the Topic to a special Queue
> that the deliverer is handling. This seems to work just fine.
>
> I'm wondering if there is any out-of-the-box solution to this problem
> provided by Artemis. Like routing/duplication/monitoring that I can turn on
> and get the same (or similar) functionality.
>
> Lachezar
>