You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by The4Summers <xa...@les4etes.com> on 2019/08/08 18:37:58 UTC
VirtualTopic issue with virtualSelectorCacheBrokerPlugin
Hi!
We recently started to use the VirtualTopic concept in ActiveMQ
<https://activemq.apache.org/virtual-destinations> with the selectorAware
attribute set to true, and with /virtualSelectorCacheBrokerPlugin/ to be
able to continue receiving selected (filtered) messages in the destination
queue even if a consumer if down.
We have producer and consumer services defined with Camel endpoints and
running as SpringBoot application .
*In activemq.xml:*
...
<plugins>
<virtualSelectorCacheBrokerPlugin
persistFile="${activemq.data}/selectorcache.data"/>
</plugins>
<destinationInterceptors>
<virtualDestinationInterceptor>
<virtualDestinations>
<virtualTopic name="VirtualTopic.>" prefix="Consumer.*."
selectorAware="true"/>
</virtualDestinations>
</virtualDestinationInterceptor>
</destinationInterceptors>
...
- *In producer, virtual topic is defined with the following Camel URI*:
jms.producer.uri=/activemq:topic:VirtualTopic.DeliveryCompletion?connectionFactory=xaJmsConnectionFactory&transactionManager=#transactionManager/
- *In consumer, the listening queue is defined with the following Camel
URI*:
jms.consumer.uri=/activemq:queue:Consumer.QUEUE_A.VirtualTopic.DeliveryCompletion?connectionFactory=connectionFactory&acknowledgementModeName=CLIENT_ACKNOWLEDGE&selector=DESTINATION_APP_ID='1234567890'/
In our producer test, we are sending in loop same set of messages to the
destination virtual topic with few distinct application Id in the header
(DESTINATION_APP_ID). The selector configured in the consumers is used to
filter messages based on this header containing the application id. So we
expect that when the producer sends a bunch of notification messages to the
virtual topic (VirtualTopic.DeliveryCompletion), then they will be forwarded
to listener queues according to the configured selector expression. The role
of 'virtualSelectorCacheBrokerPlugin' is to cache all the selector
expressions for each consumer listening on the virtual topic queue.
Most of the time (I would say 99%) of the time, all this is working very
well. All messages are properly forwarded to the proper queues according to
the selector, other are discarded. However, after some time, some messages
are not filtered properly and are sent to all listening queues and they stay
there (since not accepted by consumers because of the selector).
When we disable the plugin in ActiveMQ, we do not have this problem, however
if one consumer goes down few minutes for example, it loose some messages of
course. To void loosing messages when consumers are down, we want to use the
plugin. But because we encountered that issue with it, it is not possible to
continue use this approach (for the moment we can reproduce the expected
behavior using the concept of Composite Destination, but is it not as
flexible as the VirtualTopic).
If anyone knows what is going on with this solution (with the
"virtualSelectorCacheBrokerPlugin" plugin?), let us know.
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: VirtualTopic issue with virtualSelectorCacheBrokerPlugin
Posted by Tim Bain <tb...@alumni.duke.edu>.
The next time this happens, could you please use JVisualVM to capture a
minute or so of CPU sampling to give us an understanding of what code paths
are/aren't being executed? Please make sure that messages are coming into
the virtual topic in question during the time you do the sampling. Note
that you'll need to have the JMX port open on the broker process for this
to work.
Once you have that, please create a bug in JIRA and attach the capture to
it, and hopefully someone can figure out how to fix the bug based on the
information in the capture.
Thanks,
Tim
On Thu, Aug 8, 2019, 12:38 PM The4Summers <xa...@les4etes.com>
wrote:
> Hi!
>
> We recently started to use the VirtualTopic concept in ActiveMQ
> <https://activemq.apache.org/virtual-destinations> with the
> selectorAware
> attribute set to true, and with /virtualSelectorCacheBrokerPlugin/ to be
> able to continue receiving selected (filtered) messages in the destination
> queue even if a consumer if down.
>
> We have producer and consumer services defined with Camel endpoints and
> running as SpringBoot application .
>
> *In activemq.xml:*
> ...
> <plugins>
> <virtualSelectorCacheBrokerPlugin
> persistFile="${activemq.data}/selectorcache.data"/>
> </plugins>
>
> <destinationInterceptors>
> <virtualDestinationInterceptor>
> <virtualDestinations>
> <virtualTopic name="VirtualTopic.>" prefix="Consumer.*."
> selectorAware="true"/>
> </virtualDestinations>
> </virtualDestinationInterceptor>
> </destinationInterceptors>
> ...
>
>
> - *In producer, virtual topic is defined with the following Camel URI*:
>
>
> jms.producer.uri=/activemq:topic:VirtualTopic.DeliveryCompletion?connectionFactory=xaJmsConnectionFactory&transactionManager=#transactionManager/
>
>
>
> - *In consumer, the listening queue is defined with the following Camel
> URI*:
>
>
> jms.consumer.uri=/activemq:queue:Consumer.QUEUE_A.VirtualTopic.DeliveryCompletion?connectionFactory=connectionFactory&acknowledgementModeName=CLIENT_ACKNOWLEDGE&selector=DESTINATION_APP_ID='1234567890'/
>
>
> In our producer test, we are sending in loop same set of messages to the
> destination virtual topic with few distinct application Id in the header
> (DESTINATION_APP_ID). The selector configured in the consumers is used to
> filter messages based on this header containing the application id. So we
> expect that when the producer sends a bunch of notification messages to the
> virtual topic (VirtualTopic.DeliveryCompletion), then they will be
> forwarded
> to listener queues according to the configured selector expression. The
> role
> of 'virtualSelectorCacheBrokerPlugin' is to cache all the selector
> expressions for each consumer listening on the virtual topic queue.
>
> Most of the time (I would say 99%) of the time, all this is working very
> well. All messages are properly forwarded to the proper queues according to
> the selector, other are discarded. However, after some time, some messages
> are not filtered properly and are sent to all listening queues and they
> stay
> there (since not accepted by consumers because of the selector).
>
> When we disable the plugin in ActiveMQ, we do not have this problem,
> however
> if one consumer goes down few minutes for example, it loose some messages
> of
> course. To void loosing messages when consumers are down, we want to use
> the
> plugin. But because we encountered that issue with it, it is not possible
> to
> continue use this approach (for the moment we can reproduce the expected
> behavior using the concept of Composite Destination, but is it not as
> flexible as the VirtualTopic).
>
> If anyone knows what is going on with this solution (with the
> "virtualSelectorCacheBrokerPlugin" plugin?), let us know.
>
>
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>