Re: Using ActiveMQ For Distributed Replicated Cache

[pragmaticjdev <am...@gmail.com>]

Thanks Tim for your valuable comments.

1. 

> so you're using message-driven code rather than managing the connection 
> yourself. In that case you'd want to do the following to handle the error 
> (by clearing the cache): 
> https://stackoverflow.com/questions/40654586/spring-jms-set-errorhandler-for-jmslistener-annotated-method

On your suggestion of registering an ErrorHandler, I wanted to confirm my
understand - Does the subscriber underneath continuously poll activemq and a
failure in doing so gets the ErrorHandler implementation invoked? 

2. 


Tim Bain wrote
> @JmsListener is typically used with a 
> DefaultMessageListenerContainer, which interacts poorly with topics if the 
> subscription is non-durable

Are you suggesting we enable durability? Is there an alternative? I ask
because theoretically we don't need durability of the messages. If the
subscriber is down when the message is published to the queue, it is
perfectly ok as the cache is designed to build as requests are received. We
go to the RDBMS store on a cache miss and the subsequent requests then are
handled by the cache.

Thanks!




--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: Using ActiveMQ For Distributed Replicated Cache

[Tim Bain <tb...@alumni.duke.edu>]

On Mon, May 7, 2018 at 8:10 AM, pragmaticjdev <am...@gmail.com> wrote:

> I think there is some misunderstanding that I created. We don't plan to use
> activemq to populate the cache. We build the cache on cache misses. Here's
> how the overall flow looks like. Note there are multiple jvm's involved, I
> have just pictured one for simplicity.
>


Do you truly mean that you don't plan to use ActiveMQ to build the cache?
Or are you just saying that you attempt to build the cache via ActiveMQ
because it's more efficient but that guaranteed delivery isn't required,
because if ActiveMQ misses a message, the cache-miss logic will fill in
whatever gaps might occur? If you're truly not using ActiveMQ to build the
cache, what are you using it for?



> I didn't completely follow your response to the durability question.
> Probably this diagram helps me to explain how the cache behaves in read and
> write operations. Please let me know if you still see problems with
> @JmsListener(DefaultMessageListenerContainer) and non-durabilty.
>
> <http://activemq.2283324.n4.nabble.com/file/t378744/ActiveMQ_Oracle.png>
>
> Thanks.
>


All I meant was that for a non-durable topic subscription, the Spring
JmsTemplate code will disconnect between messages, which results in the
subscription being removed and then re-created the next time the code
connects. If a message is sent when the subscription doesn't exist, the
message will not be delivered when the consumer next creates the
connection, because the subscription did not exist at the time the message
was sent. This is standard topic behavior, but users often don't expect the
JmsTemplate to remove the subscription between calls so it trips up some
users. If you can accept a non-trivial percentage of your updates not
making it into the cache and being filled via cache misses that query the
database, this isn't a problem, but since you've described updating objects
(which will return a stale object from the cache instead of resulting in a
cache miss) you might not be able to live with this behavior. If you can't,
you'll want to use a durable subscription, and you'll need to ensure that
if your consumer disappears for a significant amount of time, your broker
is configured to discard the messages or the subscription (or both).

Tim

Re: Using ActiveMQ For Distributed Replicated Cache

[pragmaticjdev <am...@gmail.com>]

I think there is some misunderstanding that I created. We don't plan to use
activemq to populate the cache. We build the cache on cache misses. Here's
how the overall flow looks like. Note there are multiple jvm's involved, I
have just pictured one for simplicity.

I didn't completely follow your response to the durability question.
Probably this diagram helps me to explain how the cache behaves in read and
write operations. Please let me know if you still see problems with
@JmsListener(DefaultMessageListenerContainer) and non-durabilty. 

<http://activemq.2283324.n4.nabble.com/file/t378744/ActiveMQ_Oracle.png> 

Thanks.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: Using ActiveMQ For Distributed Replicated Cache

[Tim Bain <tb...@alumni.duke.edu>]

On Wed, May 2, 2018, 12:02 AM pragmaticjdev <am...@gmail.com> wrote:

> Thanks Tim for your valuable comments.
>
> 1.
>
> > so you're using message-driven code rather than managing the connection
> > yourself. In that case you'd want to do the following to handle the
> error
> > (by clearing the cache):
> >
> https://stackoverflow.com/questions/40654586/spring-jms-set-errorhandler-for-jmslistener-annotated-method
>
> On your suggestion of registering an ErrorHandler, I wanted to confirm my
> understand - Does the subscriber underneath continuously poll activemq and
> a
> failure in doing so gets the ErrorHandler implementation invoked?
>


That's my understanding, though I'm not an expert on the Spring code. If
you have more detailed questions about the Spring JMS library specifically,
you might want to either take a look at its source code or post a question
on the Spring forum or on StackOverflow.


2.
>
>
> Tim Bain wrote
> > @JmsListener is typically used with a
> > DefaultMessageListenerContainer, which interacts poorly with topics if
> the
> > subscription is non-durable
>
> Are you suggesting we enable durability? Is there an alternative? I ask
> because theoretically we don't need durability of the messages. If the
> subscriber is down when the message is published to the queue, it is
> perfectly ok as the cache is designed to build as requests are received. We
> go to the RDBMS store on a cache miss and the subsequent requests then are
> handled by the cache.
>


A non-durable subscription may increase the likelihood of those cache
misses, but as long as that's acceptable, it's probably better than using a
durable subscription where you would have to configure a mechanism for
dropping the subscription (and the built-up backlog of unconsumed messages)
after some timeout.

Out of curiosity, why use ActiveMQ to populate your cache at all, rather
than just populating it on each cache miss (which is simpler)? I'm not
saying there aren't valid reasons for doing what you're proposing, but I'm
curious what your reasons are for deciding that this is worth the extra
complication.

Tim

>