You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by "Neil Thorne (work)" <ne...@travelex.com> on 2007/09/06 12:50:11 UTC
Recovery in activemq
Hi,
I need some clarification - it's maybe more at the activemq level but here
goes...
We have a situation where we have a consumer where we don't want to process
the same message twice. It's a legacy booking system where this will result
in double booking the same payment.
We are trying to recreate a catastrophic failure of the camel process where
it is using an activemq embedded broker with persistent queues. So "same
message" here is the physical message on the bus.
We may receive another message for the same payment, but we DO want to
process that.
When we kill our java process, any message that is not fully processed is
indeed recovered from activemq's journal and sent processed again.
However when we inspect the second "recovered" underlying jms message it has
a NEW messageId.
I was expecting it to have the original messageId.
I was going to use this as a synthetic idempotency key.
I was also hoping that "recovered" messages from the journal would be tagged
in some way ("redeliveryCounter" goes from 0 to 1?).
This would make it even easier for our system to ignore these messages.
In our case it's if in doubt, don't retry, just manually intervene. If we
could see the message as "recovered" we could route it accordingly.
thanks for any help,
Neil
--
View this message in context: http://www.nabble.com/Recovery-in-activemq-tf4391171s22882.html#a12519648
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Recovery in activemq
Posted by James Strachan <ja...@gmail.com>.
On 9/6/07, Neil Thorne (work) <ne...@travelex.com> wrote:
>
> Sorry guys,
>
> scratch these comments. I just read the Camel TOE section properly and I'm
> using a Spring JmsTransactionManager to create transactional producers and
> consumers by default:
Phew!
I was gonna say, recovered messages should be the same!
You can use the JMSRedelivered headers to know if a message has been
redelivered before or not...
http://java.sun.com/j2ee/1.4/docs/api/javax/jms/Message.html#getJMSRedelivered()
Also ActiveMQ uses a counter: JMSXDeliveryCount when a message has
been redelivered to keep track of the number of times its been
delivered.
> I still would like a neat way of finding out if this message is "recovered"
> though, but both messages are the same.
BTW checking for redelivery flag is a nice way to optimise away
duplicate detection code...
http://activemq.apache.org/should-i-use-xa.html
> We're currenty thinking of going with an idempotent receiver that uses the
> messageIds above.
Sounds good to me.
--
James
-------
http://macstrac.blogspot.com/
Re: Recovery in activemq
Posted by "Neil Thorne (work)" <ne...@travelex.com>.
Sorry guys,
scratch these comments. I just read the Camel TOE section properly and I'm
using a Spring JmsTransactionManager to create transactional producers and
consumers by default:
eg.
ActiveMQConnectionFactory activeMQConnectionFactory = new
ActiveMQConnectionFactory("vm://localhost?broker.persistent=true");
JmsTransactionManager jmsTransactionManager = new
JmsTransactionManager();
jmsTransactionManager.setConnectionFactory(activeMQConnectionFactory);
JmsComponent component =
ActiveMQComponent.jmsComponentTransacted(activeMQConnectionFactory,
jmsTransactionManager);
component.setConcurrentConsumers(1);
camelContext.addComponent("activemq", component);
I'm also now seeing the messageIds are the same for "recovered" messages.
I still would like a neat way of finding out if this message is "recovered"
though, but both messages are the same.
[aultMessageListenerContainer-1] ActiveMQMessageConsumer DEBUG
Received message: MessageDispatch {commandId = 5, responseRequired = false,
consumerId = ID:J06483-2779-1189082228516-3:0:1:1, destination =
queue://test.in, message = ActiveMQObjectMessage {commandId = 6,
responseRequired = false, messageId =
ID:J06483-2779-1189082228516-3:2:1:1:1, originalDestination = null,
originalTransactionId = null, producerId =
ID:J06483-2779-1189082228516-3:2:1:1, destination = queue://test.in,
transactionId = TX:ID:J06483-2779-1189082228516-3:2:1, expiration = 0,
timestamp = 1189082232641, arrival = 0, correlationId = null, replyTo =
null, persistent = true, type = null, priority = 4, groupID = null,
groupSequence = 0, targetConsumerId = null, compressed = false, userID =
null, content = org.apache.activemq.util.ByteSequence@1554233,
marshalledProperties = null, dataStructure = null, redeliveryCounter = 0,
size = 1322, properties = null, readOnlyProperties = true, readOnlyBody =
true, droppable = false}, redeliveryCounter = 0}
[aultMessageListenerContainer-1] ActiveMQMessageConsumer DEBUG
Received message: MessageDispatch {commandId = 4, responseRequired = false,
consumerId = ID:J06483-2787-1189082256703-3:0:1:1, destination =
queue://test.in, message = ActiveMQObjectMessage {commandId = 6,
responseRequired = false, messageId =
ID:J06483-2779-1189082228516-3:2:1:1:1, originalDestination = null,
originalTransactionId = null, producerId =
ID:J06483-2779-1189082228516-3:2:1:1, destination = queue://test.in,
transactionId = TX:ID:J06483-2779-1189082228516-3:2:1, expiration = 0,
timestamp = 1189082232641, arrival = 0, correlationId = null, replyTo =
null, persistent = true, type = null, priority = 4, groupID = null,
groupSequence = 0, targetConsumerId = null, compressed = false, userID =
null, content = org.apache.activemq.util.ByteSequence@1fe4169,
marshalledProperties = null, dataStructure = null, redeliveryCounter = 0,
size = 1322, properties = null, readOnlyProperties = true, readOnlyBody =
true, droppable = false}, redeliveryCounter = 0}
any ideas?
We're currenty thinking of going with an idempotent receiver that uses the
messageIds above.
thanks,
Neil
Neil Thorne (work) wrote:
>
> Hi,
>
> I need some clarification - it's maybe more at the activemq level but here
> goes...
>
> We have a situation where we have a consumer where we don't want to
> process the same message twice. It's a legacy booking system where this
> will result in double booking the same payment.
>
> We are trying to recreate a catastrophic failure of the camel process
> where it is using an activemq embedded broker with persistent queues. So
> "same message" here is the physical message on the bus.
>
> We may receive another message for the same payment, but we DO want to
> process that.
>
> When we kill our java process, any message that is not fully processed is
> indeed recovered from activemq's journal and sent processed again.
>
> However when we inspect the second "recovered" underlying jms message it
> has a NEW messageId.
>
> I was expecting it to have the original messageId.
>
> I was going to use this as a synthetic idempotency key.
>
> I was also hoping that "recovered" messages from the journal would be
> tagged in some way ("redeliveryCounter" goes from 0 to 1?).
>
> This would make it even easier for our system to ignore these messages.
>
> In our case it's if in doubt, don't retry, just manually intervene. If we
> could see the message as "recovered" we could route it accordingly.
>
> thanks for any help,
>
> Neil
>
--
View this message in context: http://www.nabble.com/Recovery-in-activemq-tf4391171s22882.html#a12522295
Sent from the Camel - Users mailing list archive at Nabble.com.