You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Andrea Arcuri <ar...@gmail.com> on 2013/05/02 11:24:00 UTC
Durable Topic, only most-recent message needed
Hi,
I am new to ActiveMQ. Reading documentation and playing with examples. But
there is a key scenario I need to implement, but not sure if ActiveMQ can
support it, and, if it does, how.
- Client publishes topic to broker
- Only most recent message on topic is needed
- Messages can be very large (in the order of megabytes), so shouldn't store
on disk any old message that is not needed any more. but need persistency in
case producer/embedded broker process is restarted
- At the time the producer does publish a message on topic, there might be
no consumer in the network yet. Consumers might appear hours/days later, and
should pick up (for performance reasons) automatically only the most recent
message, as the previous ones are not needed
- the producer might publish 1-4 messages a day, but there might be periods
of few days with 0 messages. there is only one producer, but an unknown,
unpredictable, dynamic number of consumers
On a MessageProducer, I guess I can set a
setDeliveryMode(DeliveryMode.PERSISTENT) and a very high value for
setTimeToLive (e.g., a month/year). But how to tell ActiveMQ that, once a
new message on a topic is published, then all the previous ones are obsolete
and should be removed/not sent? Does ActiveMQ support such kind of
functionality?
many thanks
Andrea
--
View this message in context: http://activemq.2283324.n4.nabble.com/Durable-Topic-only-most-recent-message-needed-tp4666565.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Durable Topic, only most-recent message needed
Posted by Yin Wang <su...@gmail.com>.
The non-durable retroactive consumer of Topic will do the trick.
http://activemq.apache.org/retroactive-consumer.html
Only one condition is not satisfied,messages can not survive from the
restart of the broker.
2013/5/2 Andrea Arcuri <ar...@gmail.com>
> Hi,
> I am new to ActiveMQ. Reading documentation and playing with examples.
> But
> there is a key scenario I need to implement, but not sure if ActiveMQ can
> support it, and, if it does, how.
>
> - Client publishes topic to broker
> - Only most recent message on topic is needed
> - Messages can be very large (in the order of megabytes), so shouldn't
> store
> on disk any old message that is not needed any more. but need persistency
> in
> case producer/embedded broker process is restarted
> - At the time the producer does publish a message on topic, there might be
> no consumer in the network yet. Consumers might appear hours/days later,
> and
> should pick up (for performance reasons) automatically only the most recent
> message, as the previous ones are not needed
> - the producer might publish 1-4 messages a day, but there might be periods
> of few days with 0 messages. there is only one producer, but an unknown,
> unpredictable, dynamic number of consumers
>
> On a MessageProducer, I guess I can set a
> setDeliveryMode(DeliveryMode.PERSISTENT) and a very high value for
> setTimeToLive (e.g., a month/year). But how to tell ActiveMQ that, once a
> new message on a topic is published, then all the previous ones are
> obsolete
> and should be removed/not sent? Does ActiveMQ support such kind of
> functionality?
>
> many thanks
>
> Andrea
>
>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Durable-Topic-only-most-recent-message-needed-tp4666565.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Durable Topic, only most-recent message needed
Posted by Christian Posta <ch...@gmail.com>.
try retroactive consumer and constant pending limit strategy of 1
On Fri, May 3, 2013 at 1:26 AM, Andrea Arcuri <ar...@gmail.com> wrote:
> Hi,
> thanks for reply. Using ActiveMQ (successfully) for a lot of different
> types of messages. But that is a particular scenario for one particular
> type
> of message. Adding and installing a database on all nodes of a large system
> of systems of variegated nodes is not a viable option (different OS,
> different programming languages). As ActiveMQ is already installed (or easy
> to install) in all of those nodes, if there was an option to keep/consider
> only the most recent message on a persistent topic, then that would be an
> elegant solution with minimal impact on the current architecture.
>
> cheers
>
> Andrea
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Durable-Topic-only-most-recent-message-needed-tp4666565p4666616.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
RE: Durable Topic, only most-recent message needed
Posted by Andrea Arcuri <ar...@gmail.com>.
Hi,
thanks for reply. Using ActiveMQ (successfully) for a lot of different
types of messages. But that is a particular scenario for one particular type
of message. Adding and installing a database on all nodes of a large system
of systems of variegated nodes is not a viable option (different OS,
different programming languages). As ActiveMQ is already installed (or easy
to install) in all of those nodes, if there was an option to keep/consider
only the most recent message on a persistent topic, then that would be an
elegant solution with minimal impact on the current architecture.
cheers
Andrea
--
View this message in context: http://activemq.2283324.n4.nabble.com/Durable-Topic-only-most-recent-message-needed-tp4666565p4666616.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
RE: Durable Topic, only most-recent message needed
Posted by Michael Black <md...@yahoo.com>.
Sound to me like you want a database and not ActiveMq.
Producer inserts a record and deletes all previous (in that order in 1
transaction with rollback to assure that clients can always retrieve a
record).
Use a date/time for the key.
Clients retrieve the last key in the table.
#1 Clients always have a record available even if the producer is not
running
#2 Messages survive producer crash/restart
-----Original Message-----
From: Andrea Arcuri [mailto:arcuri82@gmail.com]
Sent: Thursday, May 02, 2013 4:24 AM
To: users@activemq.apache.org
Subject: Durable Topic, only most-recent message needed
Hi,
I am new to ActiveMQ. Reading documentation and playing with examples. But
there is a key scenario I need to implement, but not sure if ActiveMQ can
support it, and, if it does, how.
- Client publishes topic to broker
- Only most recent message on topic is needed
- Messages can be very large (in the order of megabytes), so shouldn't store
on disk any old message that is not needed any more. but need persistency in
case producer/embedded broker process is restarted
- At the time the producer does publish a message on topic, there might be
no consumer in the network yet. Consumers might appear hours/days later, and
should pick up (for performance reasons) automatically only the most recent
message, as the previous ones are not needed
- the producer might publish 1-4 messages a day, but there might be periods
of few days with 0 messages. there is only one producer, but an unknown,
unpredictable, dynamic number of consumers
On a MessageProducer, I guess I can set a
setDeliveryMode(DeliveryMode.PERSISTENT) and a very high value for
setTimeToLive (e.g., a month/year). But how to tell ActiveMQ that, once a
new message on a topic is published, then all the previous ones are obsolete
and should be removed/not sent? Does ActiveMQ support such kind of
functionality?
many thanks
Andrea
--
View this message in context:
http://activemq.2283324.n4.nabble.com/Durable-Topic-only-most-recent-message
-needed-tp4666565.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.