Help needed with duplicate messages...

[Alex_ynr <su...@gmail.com>]

Bear with me cause this might be a total misconfiguration on my part but I
can't seem to figure it out....

I am running Fuse 3.3 but with the latest svn trunk version of activeMQ to
solve the duplex="true" bug i mentioned in a previous thread which was just
fixed...

In fact I am running 2 instances of Fuse on 2 different servers... I have 2
servicemix components as consumers and 2 as producers on either server
publishing on 4 different queues.

Their activeMQ brokers are configured using the duplex functionality of the
networkConnector initiated from one of the brokers since it is behind a
firewall....

So the broker behind the firewall has a networkConnector configured like
this:

<amq:networkConnector conduitSubscriptions="true"
uri="static://(failover:tcp://X.X.X.X:61618)" duplex="true"
dynamicOnly="true" networkTTL="1"/>

The idea is that the servicemix components on one side publish messages to
the queue on their local broker  they are then forwarded to the remote
broker and consumed by the servicemix components there... 

Quite a simple setup really...

The issue I'm having is that each broker stores it's internal message store
fine...i.e. 

If i publish a message on one side with both brokers running, it will be
transferred to the other side and consumed ...If I kill both brokers and
restart the one the consumer was on it will correctly NOT redeliver the
message...

However once the first broker is restarted, it will re-forward the message
to the other broker which in turn will re-deliver it to the consumer....

/*
As some extra information, I was originally using JMSFlow only in servicemix
but (this is a side issue) it would occasionally hang the servicemix NMR on
reconnections (i.e. after a connection/reconnection between the 2 servers,
messages would fail to be transferred between endpoints/components in the
same servicemix instance ...much before they were published to a queue)
stopping the fuse instance then would result in JMSExceptions about the
specified endpoint/component not being active....as if the messages were
stuck waiting to be delivered inside the NMR...

ActiveMQ had the same behaviour of redelivering messages in that case too.

*/

I enabled all flows and that issue disappears, however, now when I stop the
broker, I get the following:

ERROR - Service                        - Async error occurred:
javax.jms.JMSException: Invalid acknowledgment: MessageAck {commandId = 11,
responseRequired = false, ackType = 3, consumerId =
ID:middleware-33580-1192178108973-3:3:-1:2, firstMessageId =
ID:middleware-33580-1192178108973-1:0:0:0:3120, lastMessageId =
ID:middleware-33580-1192178108973-1:0:0:0:3120, destination =
topic://ActiveMQ.Advisory.Consumer.Topic.org.apache.servicemix.JCAFlow,
transactionId = null, messageCount = 1}
javax.jms.JMSException: Invalid acknowledgment: MessageAck {commandId = 11,
responseRequired = false, ackType = 3, consumerId =
ID:middleware-33580-1192178108973-3:3:-1:2, firstMessageId =
ID:middleware-33580-1192178108973-1:0:0:0:3120, lastMessageId =
ID:middleware-33580-1192178108973-1:0:0:0:3120, destination =
topic://ActiveMQ.Advisory.Consumer.Topic.org.apache.servicemix.JCAFlow,
transactionId = null, messageCount = 1}

Which leads me to believe that the messages moved between the 2 brokers are
not properly ack'ed leading to the duplicates but I don't know the reason...

I am using the default persistence mechanism in activeMQ.


/*
I tried using Database persistence with a c3p0 pooled data source but It
would not persist messages cleanly ....BTW there is something seriously
wrong with the shutdown hooks in Fuse, with one of servicemix or activemq
not shuttding down cleanly (depending on the setting of useShutdownHook in
activemq.xml)...

If activemq uses the shutdown hook, it will shut down cleanly but in the
case of the pooled data source above, the data source would be stopped and
connections closed BEFORE activemq wrote the persistence data to the
database so that kinda defeated the whole purpose */

Any help would be appreciated...

Alex
-- 
View this message in context: http://www.nabble.com/Help-needed-with-duplicate-messages...-tf4612226s2354.html#a13171540
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Help needed with duplicate messages...

[Alex_ynr <su...@gmail.com>]

Any ideas on this? I'm still stumped...


Alex_ynr wrote:
> 
> Bear with me cause this might be a total misconfiguration on my part but I
> can't seem to figure it out....
> 
> I am running Fuse 3.3 but with the latest svn trunk version of activeMQ to
> solve the duplex="true" bug i mentioned in a previous thread which was
> just fixed...
> 
> In fact I am running 2 instances of Fuse on 2 different servers... I have
> 2 servicemix components as consumers and 2 as producers on either server
> publishing on 4 different queues.
> 
> Their activeMQ brokers are configured using the duplex functionality of
> the networkConnector initiated from one of the brokers since it is behind
> a firewall....
> 
> So the broker behind the firewall has a networkConnector configured like
> this:
> 
> <amq:networkConnector conduitSubscriptions="true"
> uri="static://(failover:tcp://X.X.X.X:61618)" duplex="true"
> dynamicOnly="true" networkTTL="1"/>
> 
> The idea is that the servicemix components on one side publish messages to
> the queue on their local broker  they are then forwarded to the remote
> broker and consumed by the servicemix components there... 
> 
> Quite a simple setup really...
> 
> The issue I'm having is that each broker stores it's internal message
> store fine...i.e. 
> 
> If i publish a message on one side with both brokers running, it will be
> transferred to the other side and consumed ...If I kill both brokers and
> restart the one the consumer was on it will correctly NOT redeliver the
> message...
> 
> However once the first broker is restarted, it will re-forward the message
> to the other broker which in turn will re-deliver it to the consumer....
> 
> /*
> As some extra information, I was originally using JMSFlow only in
> servicemix but (this is a side issue) it would occasionally hang the
> servicemix NMR on reconnections (i.e. after a connection/reconnection
> between the 2 servers, messages would fail to be transferred between
> endpoints/components in the same servicemix instance ...much before they
> were published to a queue) stopping the fuse instance then would result in
> JMSExceptions about the specified endpoint/component not being
> active....as if the messages were stuck waiting to be delivered inside the
> NMR...
> 
> ActiveMQ had the same behaviour of redelivering messages in that case too.
> 
> */
> 
> I enabled all flows and that issue disappears, however, now when I stop
> the broker, I get the following:
> 
> ERROR - Service                        - Async error occurred:
> javax.jms.JMSException: Invalid acknowledgment: MessageAck {commandId =
> 11, responseRequired = false, ackType = 3, consumerId =
> ID:middleware-33580-1192178108973-3:3:-1:2, firstMessageId =
> ID:middleware-33580-1192178108973-1:0:0:0:3120, lastMessageId =
> ID:middleware-33580-1192178108973-1:0:0:0:3120, destination =
> topic://ActiveMQ.Advisory.Consumer.Topic.org.apache.servicemix.JCAFlow,
> transactionId = null, messageCount = 1}
> javax.jms.JMSException: Invalid acknowledgment: MessageAck {commandId =
> 11, responseRequired = false, ackType = 3, consumerId =
> ID:middleware-33580-1192178108973-3:3:-1:2, firstMessageId =
> ID:middleware-33580-1192178108973-1:0:0:0:3120, lastMessageId =
> ID:middleware-33580-1192178108973-1:0:0:0:3120, destination =
> topic://ActiveMQ.Advisory.Consumer.Topic.org.apache.servicemix.JCAFlow,
> transactionId = null, messageCount = 1}
> 
> Which leads me to believe that the messages moved between the 2 brokers
> are not properly ack'ed leading to the duplicates but I don't know the
> reason...
> 
> I am using the default persistence mechanism in activeMQ.
> 
> 
> /*
> I tried using Database persistence with a c3p0 pooled data source but It
> would not persist messages cleanly ....BTW there is something seriously
> wrong with the shutdown hooks in Fuse, with one of servicemix or activemq
> not shuttding down cleanly (depending on the setting of useShutdownHook in
> activemq.xml)...
> 
> If activemq uses the shutdown hook, it will shut down cleanly but in the
> case of the pooled data source above, the data source would be stopped and
> connections closed BEFORE activemq wrote the persistence data to the
> database so that kinda defeated the whole purpose */
> 
> Any help would be appreciated...
> 
> Alex
> 

-- 
View this message in context: http://www.nabble.com/Help-needed-with-duplicate-messages...-tf4612226s2354.html#a13208161
Sent from the ActiveMQ - User mailing list archive at Nabble.com.