You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by srivmeka <sr...@gmail.com> on 2017/04/14 07:51:51 UTC
Duplicate Messages in the Message Store
Hi,
Am facing issue with broker trying to insert duplicate message into
activemq_msgs.
This was observed during
1) Broker fail over
2) Data center/Application fail over where the primary broker changes per
data center, note there is no broker shutdown here.
I am using JDBC persistent Adapter and
<broker brokerName="partner-broker"
xmlns="http://activemq.apache.org/schema/core">
<persistenceAdapter>
<jdbcPersistenceAdapter dataSource="#amq-dataSource"
useDatabaseLock="false"/>
</persistenceAdapter>
<transportConnectors>
<transportConnector name="openwire1"
uri="tcp://${activemq.broker1.hostname}:${activemq.broker1.portId}?maximumConnections=1000"/>
</transportConnectors>
</broker>
I have difficulty in understanding how broker inserts messages into message
queues.
The code in activemq jar does explain that it gets the max(ID) for next
sequence generator but when is this really happening? Is there any caching
mechanism here? What is the message is already consumed and deleted, how can
it get the MAX(ID)?
Thanks,
Sri
--
View this message in context: http://activemq.2283324.n4.nabble.com/Duplicate-Messages-in-the-Message-Store-tp4724959.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Duplicate Messages in the Message Store
Posted by Tim Bain <tb...@alumni.duke.edu>.
Please submit a bug in JIRA for this issue.
Tim
On Apr 18, 2017 1:16 AM, "srivmeka" <sr...@gmail.com> wrote:
This is what i reproduced on lower environments, I have two instances of
servicemix 5.3.0 each have one broker bundle. Broker 1 on instancce 1 is
primary.
1. Broker1 when up inserted a message in to the queue with ID #2466
2. I brought Broker 1 down and inserted another message via Broker2
expecting the ID to be 2466 +1 (as the message is not consumed yet), but the
message inserted was #2209
3. Brought Broker 1 up and inserted another message, it got inserted with
#2247
4. My Consumer pulls the messages 100 at a time form queue of same type.
Thus was expecting consumer to pull all 3 messages, but it did consume only
the once that inserted via Broker1.
This certainly explains me there is message store cached per Broker, which
is not good and causing problems.
--
View this message in context: http://activemq.2283324.n4.
nabble.com/Duplicate-Messages-in-the-Message-Store-tp4724959p4725019.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Duplicate Messages in the Message Store
Posted by srivmeka <sr...@gmail.com>.
This is what i reproduced on lower environments, I have two instances of
servicemix 5.3.0 each have one broker bundle. Broker 1 on instancce 1 is
primary.
1. Broker1 when up inserted a message in to the queue with ID #2466
2. I brought Broker 1 down and inserted another message via Broker2
expecting the ID to be 2466 +1 (as the message is not consumed yet), but the
message inserted was #2209
3. Brought Broker 1 up and inserted another message, it got inserted with
#2247
4. My Consumer pulls the messages 100 at a time form queue of same type.
Thus was expecting consumer to pull all 3 messages, but it did consume only
the once that inserted via Broker1.
This certainly explains me there is message store cached per Broker, which
is not good and causing problems.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Duplicate-Messages-in-the-Message-Store-tp4724959p4725019.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Duplicate Messages in the Message Store
Posted by Tim Bain <tb...@alumni.duke.edu>.
Based on the doGetLastMessageStoreSequenceId() method in
https://fisheye.apache.org/browse/activemq-6/activemq-jdbc-store/src/main/java/org/apache/activemq/store/jdbc/adapter/DefaultJDBCAdapter.java?r=d54d046b8a8f2e9e5c0a28e1f8c7634b3c8b18e4&r=d54d046b8a8f2e9e5c0a28e1f8c7634b3c8b18e4&r=d54d046b8a8f2e9e5c0a28e1f8c7634b3c8b18e4&r=c5cf03806d646045e01f6cd382504fa73dedfeee&r=d54d046b8a8f2e9e5c0a28e1f8c7634b3c8b18e4&r=d54d046b8a8f2e9e5c0a28e1f8c7634b3c8b18e4&r=d54d046b8a8f2e9e5c0a28e1f8c7634b3c8b18e4&r=d54d046b8a8f2e9e5c0a28e1f8c7634b3c8b18e4,
if all messages have been consumed, acked, and deleted, the ID 0 will be
computed as the max value in the store, so I believe that the next ID used
will be 1. But I'm not sure that that information helps you investigate
the behavior you're seeing during a failover event.
Are you able to reproduce this behavior reliably, in a minimal test
configuration? The smoothest path forward would be for you to find a way
to reliably reproduce it and then submit a JIRA entry with the full details
of how to reproduce the problem, so that someone else could figure out
what's going on and address any bug that turns out to be the root cause of
the behavior.
Tim
On Fri, Apr 14, 2017 at 1:51 AM, srivmeka <sr...@gmail.com> wrote:
> Hi,
> Am facing issue with broker trying to insert duplicate message into
> activemq_msgs.
>
> This was observed during
> 1) Broker fail over
> 2) Data center/Application fail over where the primary broker changes per
> data center, note there is no broker shutdown here.
>
> I am using JDBC persistent Adapter and
> <broker brokerName="partner-broker"
> xmlns="http://activemq.apache.org/schema/core">
> <persistenceAdapter>
> <jdbcPersistenceAdapter
> dataSource="#amq-dataSource"
> useDatabaseLock="false"/>
> </persistenceAdapter>
> <transportConnectors>
>
> <transportConnector name="openwire1"
> uri="tcp://${activemq.broker1.hostname}:${activemq.broker1.
> portId}?maximumConnections=1000"/>
> </transportConnectors>
> </broker>
>
> I have difficulty in understanding how broker inserts messages into message
> queues.
> The code in activemq jar does explain that it gets the max(ID) for next
> sequence generator but when is this really happening? Is there any caching
> mechanism here? What is the message is already consumed and deleted, how
> can
> it get the MAX(ID)?
>
> Thanks,
> Sri
>
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/Duplicate-Messages-in-the-Message-Store-tp4724959.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>