You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Calle Andersson <ca...@hotmail.com> on 2023/10/16 20:42:12 UTC
Replacing composite queues with Artemis
I have an old A-MQ with a lot of composite queues, e.g:
<compositeQueue name="TEST.QUEUE.A">
<filteredDestination selector="MyMessageProperty='SYSTEM_X'" queue="TEST.QUEUE.A.SYSTEM_X" />
<filteredDestination selector="MyMessageProperty='SYSTEM_Y'" queue="TEST.QUEUE.A.SYSTEM_Y" />
<filteredDestination selector="NOT(MyMessageProperty='SYSTEM_X') AND NOT(MyMessageProperty='SYSTEM_Y') queue="DLQ.HANTERAUTSKICK.UTSKICKSBEGARAN.REPLY" />
</compositeQueue>
Currently, I'm investigating how to migrate to Artemis. The implementations of the existing clients (which sends and receives messages) should not be adjusted (i.e. the names of the queues should be the same and the clients should continue communicating via queues, not topics).
I have defined the following addresses/queues in broker.xml:
<address name="TEST.QUEUE.A">
<anycast>
<queue name="TEST.QUEUE.A" >
<filter string="UnwantedProperty='ShouldNeverMatch'"/>
</queue>
</anycast>
</address>
<address name="TEST.QUEUE.A.SYSTEM_X">
<anycast>
<queue name="TEST.QUEUE.A.SYSTEM_X" />
</anycast>
</address>
<address name="TEST.QUEUE.A.SYSTEM_Y">
<anycast>
<queue name="TEST.QUEUE.A.SYSTEM_Y" />
</anycast>
</address>
<address name="DLQ">
<anycast>
<queue name="DLQ" />
</anycast>
</address>
<address name="ExpiryQueue">
<anycast>
<queue name="ExpiryQueue" />
</anycast>
</address>
I also added the following diverts:
<divert name="TEST.QUEUE.A.SYSTEM_X-divert">
<routing-name>TEST.QUEUE.A.SYSTEM_X-divert</routing-name>
<address>TEST.QUEUE.A</address>
<filter string="MyMessageProperty='SYSTEM_X'"/>
<forwarding-address>TEST.QUEUE.A.SYSTEM_X</forwarding-address>
<exclusive>true</exclusive>
</divert>
<divert name="TEST.QUEUE.A.SYSTEM_Y-divert">
<routing-name>TEST.QUEUE.A.SYSTEM_Y-divert</routing-name>
<address>TEST.QUEUE.A</address>
<filter string="MyMessageProperty='SYSTEM_Y'"/>
<forwarding-address>TEST.QUEUE.A.SYSTEM_Y</forwarding-address>
<exclusive>true</exclusive>
</divert>
And I also configured this for <address-setting match="#"> (in an attempt to make any messages not diverted by any divert to end up on a DLQ associated with the queue):
<send-to-dla-on-no-route>true</send-to-dla-on-no-route>
<dead-letter-address>DLQ</dead-letter-address>
<dead-letter-queue-prefix>DLQ.</dead-letter-queue-prefix>
<auto-create-dead-letter-resources>true</auto-create-dead-letter-resources>
<max-delivery-attempts>1</max-delivery-attempts>
Everything else is configured in the same way as in a newly created 2.31.0 broker.
Since I added a dummy filter to the TEST.QUEUE.A queue, I hoped that messages which doesn't get filtered to any queues or get diverted by any divert (e.g. if MyMessageProperty is missing or has an unexpected value) should be considered as undeliverable and end up on a newly created queue named "DLQ.TEST.QUEUE.A". To test this, I send messages to "TEST.QUEUE.A" in the following way (the “admin” user has all permissions):
[ARTEMIS_HOME]/bin/artemis producer --message ipsum --message-count 1 --destination TEST.QUEUE.A --url tcp://localhost:61616 --user admin --password admin
However, the original messages simply disappears but for each message sent, there are 6 messages placed on the "DLQ" queue. All messages are empty. In 3 of them, "_AMQ_NotifType" have been set to "SESSION_CREATED". In the rest, "SESSION_CLOSED" are set.
Could anyone explain the result and why it differs from my assumptions? Have I misunderstood some parts of the configuration or does some configurations not work together?
Is there better ways to achieve what I'm trying to do?
Regards,
Calle
Re: Replacing composite queues with Artemis
Posted by Calle Andersson <ca...@hotmail.com>.
Is there anyone who got any thoughts regarding the described behavior? Or is there anyone who has made a similar migration and want to share some suggestions?
Regards,
Calle
________________________________
Från: Calle Andersson <ca...@hotmail.com>
Skickat: måndag, oktober 16, 2023 10:42:33 PM
Till: users@activemq.apache.org <us...@activemq.apache.org>
Ämne: Replacing composite queues with Artemis
I have an old A-MQ with a lot of composite queues, e.g:
<compositeQueue name="TEST.QUEUE.A">
<filteredDestination selector="MyMessageProperty='SYSTEM_X'" queue="TEST.QUEUE.A.SYSTEM_X" />
<filteredDestination selector="MyMessageProperty='SYSTEM_Y'" queue="TEST.QUEUE.A.SYSTEM_Y" />
<filteredDestination selector="NOT(MyMessageProperty='SYSTEM_X') AND NOT(MyMessageProperty='SYSTEM_Y') queue="DLQ.HANTERAUTSKICK.UTSKICKSBEGARAN.REPLY" />
</compositeQueue>
Currently, I'm investigating how to migrate to Artemis. The implementations of the existing clients (which sends and receives messages) should not be adjusted (i.e. the names of the queues should be the same and the clients should continue communicating via queues, not topics).
I have defined the following addresses/queues in broker.xml:
<address name="TEST.QUEUE.A">
<anycast>
<queue name="TEST.QUEUE.A" >
<filter string="UnwantedProperty='ShouldNeverMatch'"/>
</queue>
</anycast>
</address>
<address name="TEST.QUEUE.A.SYSTEM_X">
<anycast>
<queue name="TEST.QUEUE.A.SYSTEM_X" />
</anycast>
</address>
<address name="TEST.QUEUE.A.SYSTEM_Y">
<anycast>
<queue name="TEST.QUEUE.A.SYSTEM_Y" />
</anycast>
</address>
<address name="DLQ">
<anycast>
<queue name="DLQ" />
</anycast>
</address>
<address name="ExpiryQueue">
<anycast>
<queue name="ExpiryQueue" />
</anycast>
</address>
I also added the following diverts:
<divert name="TEST.QUEUE.A.SYSTEM_X-divert">
<routing-name>TEST.QUEUE.A.SYSTEM_X-divert</routing-name>
<address>TEST.QUEUE.A</address>
<filter string="MyMessageProperty='SYSTEM_X'"/>
<forwarding-address>TEST.QUEUE.A.SYSTEM_X</forwarding-address>
<exclusive>true</exclusive>
</divert>
<divert name="TEST.QUEUE.A.SYSTEM_Y-divert">
<routing-name>TEST.QUEUE.A.SYSTEM_Y-divert</routing-name>
<address>TEST.QUEUE.A</address>
<filter string="MyMessageProperty='SYSTEM_Y'"/>
<forwarding-address>TEST.QUEUE.A.SYSTEM_Y</forwarding-address>
<exclusive>true</exclusive>
</divert>
And I also configured this for <address-setting match="#"> (in an attempt to make any messages not diverted by any divert to end up on a DLQ associated with the queue):
<send-to-dla-on-no-route>true</send-to-dla-on-no-route>
<dead-letter-address>DLQ</dead-letter-address>
<dead-letter-queue-prefix>DLQ.</dead-letter-queue-prefix>
<auto-create-dead-letter-resources>true</auto-create-dead-letter-resources>
<max-delivery-attempts>1</max-delivery-attempts>
Everything else is configured in the same way as in a newly created 2.31.0 broker.
Since I added a dummy filter to the TEST.QUEUE.A queue, I hoped that messages which doesn't get filtered to any queues or get diverted by any divert (e.g. if MyMessageProperty is missing or has an unexpected value) should be considered as undeliverable and end up on a newly created queue named "DLQ.TEST.QUEUE.A". To test this, I send messages to "TEST.QUEUE.A" in the following way (the “admin” user has all permissions):
[ARTEMIS_HOME]/bin/artemis producer --message ipsum --message-count 1 --destination TEST.QUEUE.A --url tcp://localhost:61616 --user admin --password admin
However, the original messages simply disappears but for each message sent, there are 6 messages placed on the "DLQ" queue. All messages are empty. In 3 of them, "_AMQ_NotifType" have been set to "SESSION_CREATED". In the rest, "SESSION_CLOSED" are set.
Could anyone explain the result and why it differs from my assumptions? Have I misunderstood some parts of the configuration or does some configurations not work together?
Is there better ways to achieve what I'm trying to do?
Regards,
Calle
Re: Replacing composite queues with Artemis
Posted by Calle Andersson <ca...@hotmail.com>.
Thanks for your suggestion and the comment regarding send-to-dla-on-no-route.
I will probably use your approach. However, I did try to do a little adjustment by defining the DLQ address like this (since that appears to be similar to how auto-create-dead-letter-resources creates queues when set to true):
<address name="DLQ">
<multicast>
<queue name="DLQ.TEST.QUEUE.A">
<filter string="NOT(MyMessageProperty='SYSTEM_X') AND NOT(MyMessageProperty='SYSTEM_Y')"/>
</queue>
</multicast>
</address>
The divert is forwarding to that queue using the fully qualified queue name DLQ::DLQ.TEST.QUEUE.A like this:
<forwarding-address>TEST.QUEUE.A.SYSTEM_X,TEST.QUEUE.A.SYSTEM_Y,DLQ::DLQ.TEST.QUEUE.A</forwarding-address>
When sending a message with MyMessageProperty set as “dummy” it ends up on DLQ::DLQ.TEST.QUEUE.A. However, when setting MyMessageProperty as “'SYSTEM_X” the message ends up on both TEST.QUEUE.A.SYSTEM_X and DLQ::DLQ.TEST.QUEUE.A.
When not using the fully qualified queue name and instead forwarding to an address with an anycast queue using the exact same filter expression, the message with MyMessageProperty set as “'SYSTEM_X” won’t end up on that queue.
Is this a bug or expected behaviour? Should fully qualified queue names be avoided when configuring forwarding-address?
Regards,
Calle
Re: Replacing composite queues with Artemis
Posted by Justin Bertram <jb...@apache.org>.
Here's what I would suggest:
<addresses>
...
<address name="TEST.QUEUE.A">
<anycast>
<queue name="TEST.QUEUE.A"/>
</anycast>
</address>
<address name="TEST.QUEUE.A.SYSTEM_X">
<anycast>
<queue name="TEST.QUEUE.A.SYSTEM_X">
<filter string="MyMessageProperty='SYSTEM_X'"/>
</queue>
</anycast>
</address>
<address name="TEST.QUEUE.A.SYSTEM_Y">
<anycast>
<queue name="TEST.QUEUE.A.SYSTEM_Y">
<filter string="MyMessageProperty='SYSTEM_Y'"/>
</queue>
</anycast>
</address>
<address name="DLQ.HANTERAUTSKICK.UTSKICKSBEGARAN.REPLY">
<anycast>
<queue name="DLQ.HANTERAUTSKICK.UTSKICKSBEGARAN.REPLY">
<filter string="NOT(MyMessageProperty='SYSTEM_X') AND
NOT(MyMessageProperty='SYSTEM_Y')"/>
</queue>
</anycast>
</address>
</addresses>
<diverts>
<divert name="COMPOSITE-TEST.QUEUE.A">
<address>TEST.QUEUE.A</address>
<forwarding-address>TEST.QUEUE.A.SYSTEM_X,TEST.QUEUE.A.SYSTEM_Y,DLQ.HANTERAUTSKICK.UTSKICKSBEGARAN.REPLY</forwarding-address>
<exclusive>true</exclusive>
</divert>
</diverts>
This gets you essentially the exact behavior you had before, including with
the DLQ, and you don't have to configure any extra address settings.
For what it's worth, setting send-to-dla-on-no-route to true on # is going
to cause problems (e.g. the extra notification messages in the DLQ). This
kind of configuration is not recommended since dropping messages that have
no route is standard practice for multicast (i.e. pub/sub) use-cases.
Hope that helps.
Justin
On Mon, Oct 16, 2023 at 3:42 PM Calle Andersson <ca...@hotmail.com>
wrote:
> I have an old A-MQ with a lot of composite queues, e.g:
> <compositeQueue name="TEST.QUEUE.A">
> <filteredDestination selector="MyMessageProperty='SYSTEM_X'"
> queue="TEST.QUEUE.A.SYSTEM_X" />
> <filteredDestination selector="MyMessageProperty='SYSTEM_Y'"
> queue="TEST.QUEUE.A.SYSTEM_Y" />
> <filteredDestination selector="NOT(MyMessageProperty='SYSTEM_X') AND
> NOT(MyMessageProperty='SYSTEM_Y')
> queue="DLQ.HANTERAUTSKICK.UTSKICKSBEGARAN.REPLY" />
> </compositeQueue>
>
> Currently, I'm investigating how to migrate to Artemis. The
> implementations of the existing clients (which sends and receives messages)
> should not be adjusted (i.e. the names of the queues should be the same and
> the clients should continue communicating via queues, not topics).
>
> I have defined the following addresses/queues in broker.xml:
> <address name="TEST.QUEUE.A">
> <anycast>
> <queue
> name="TEST.QUEUE.A" >
>
> <filter string="UnwantedProperty='ShouldNeverMatch'"/>
> </queue>
> </anycast>
> </address>
> <address name="TEST.QUEUE.A.SYSTEM_X">
> <anycast>
> <queue
> name="TEST.QUEUE.A.SYSTEM_X" />
> </anycast>
> </address>
> <address name="TEST.QUEUE.A.SYSTEM_Y">
> <anycast>
> <queue
> name="TEST.QUEUE.A.SYSTEM_Y" />
> </anycast>
> </address>
> <address name="DLQ">
> <anycast>
> <queue name="DLQ" />
> </anycast>
> </address>
> <address name="ExpiryQueue">
> <anycast>
> <queue
> name="ExpiryQueue" />
> </anycast>
> </address>
>
> I also added the following diverts:
> <divert name="TEST.QUEUE.A.SYSTEM_X-divert">
>
> <routing-name>TEST.QUEUE.A.SYSTEM_X-divert</routing-name>
> <address>TEST.QUEUE.A</address>
> <filter string="MyMessageProperty='SYSTEM_X'"/>
>
> <forwarding-address>TEST.QUEUE.A.SYSTEM_X</forwarding-address>
> <exclusive>true</exclusive>
> </divert>
> <divert name="TEST.QUEUE.A.SYSTEM_Y-divert">
>
> <routing-name>TEST.QUEUE.A.SYSTEM_Y-divert</routing-name>
> <address>TEST.QUEUE.A</address>
> <filter string="MyMessageProperty='SYSTEM_Y'"/>
>
> <forwarding-address>TEST.QUEUE.A.SYSTEM_Y</forwarding-address>
> <exclusive>true</exclusive>
> </divert>
>
> And I also configured this for <address-setting match="#"> (in an attempt
> to make any messages not diverted by any divert to end up on a DLQ
> associated with the queue):
> <send-to-dla-on-no-route>true</send-to-dla-on-no-route>
> <dead-letter-address>DLQ</dead-letter-address>
> <dead-letter-queue-prefix>DLQ.</dead-letter-queue-prefix>
> <auto-create-dead-letter-resources>true</auto-create-dead-letter-resources>
> <max-delivery-attempts>1</max-delivery-attempts>
>
> Everything else is configured in the same way as in a newly created 2.31.0
> broker.
>
> Since I added a dummy filter to the TEST.QUEUE.A queue, I hoped that
> messages which doesn't get filtered to any queues or get diverted by any
> divert (e.g. if MyMessageProperty is missing or has an unexpected value)
> should be considered as undeliverable and end up on a newly created queue
> named "DLQ.TEST.QUEUE.A". To test this, I send messages to "TEST.QUEUE.A"
> in the following way (the “admin” user has all permissions):
> [ARTEMIS_HOME]/bin/artemis producer --message ipsum --message-count 1
> --destination TEST.QUEUE.A --url tcp://localhost:61616 --user admin
> --password admin
>
> However, the original messages simply disappears but for each message
> sent, there are 6 messages placed on the "DLQ" queue. All messages are
> empty. In 3 of them, "_AMQ_NotifType" have been set to "SESSION_CREATED".
> In the rest, "SESSION_CLOSED" are set.
>
> Could anyone explain the result and why it differs from my assumptions?
> Have I misunderstood some parts of the configuration or does some
> configurations not work together?
>
> Is there better ways to achieve what I'm trying to do?
>
> Regards,
> Calle
>