[jira] [Work logged] (AMQ-7425) Messages in journal persistence don't get deleted upon ack

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AMQ-7425?focusedWorklogId=396668&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-396668 ]

ASF GitHub Bot logged work on AMQ-7425:
---------------------------------------

                Author: ASF GitHub Bot
            Created on: 03/Mar/20 06:37
            Start Date: 03/Mar/20 06:37
    Worklog Time Spent: 10m 
      Work Description: jbonofre commented on pull request #495: [AMQ-7425] Simple fix to be sure to delete message from JDBC message store
URL: https://github.com/apache/activemq/pull/495
 
 
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 396668)
    Remaining Estimate: 0h
            Time Spent: 10m

> Messages in journal persistence don't get deleted upon ack
> ----------------------------------------------------------
>
>                 Key: AMQ-7425
>                 URL: https://issues.apache.org/jira/browse/AMQ-7425
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: JDBC
>    Affects Versions: 5.15.11
>            Reporter: Nicolas Grondin
>            Assignee: Jean-Baptiste Onofré
>            Priority: Major
>             Fix For: 5.16.0, 5.15.12
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> When using a journal JDBC persistence, we noticed that some messages in the DB table (activemq_msgs) don't always get deleted upon the messages being consumed. The adverse effect is that if the broker restarts (hosted in a K8S pod, so local journal files also lost) the broker will consider all non-deleted message as still unconsumed and will offer them up for consumption (this was an issue for us in production)
> This seems to be cause by the fact that the persistence adapter does not use the right ID in the SQL statement to delete the line.
> This comes from the fact that sometimes, the messageId has a "futureOrSequenceLong" of 0 as opposed to the correct ID (from the DB). This is because the "futureOrSequenceLong" gets set on that message only at the time the message is being persisted. But in a journal persistence, this happens on a frequency (5 minutes by default).
> If a browse action occurs before the message is persisted, then a copy of the message is taken, which also copies of the wrong "futureOrSequenceLong". Upon consumption (assuming no restarts of the broker in the meantime), it is this copy of the messageId that is used when comes the time to remove the message from the DB.
> The removal code actually caters for the missing "futureOrSequenceLong" in JDBCMessageStore [257] with this line:
>   {color:#FF0000}   long seq = ack.getLastMessageId().getFutureOrSequenceLong() != null ?     long seq = ack.getLastMessageId().getFutureOrSequenceLong() != null ?                (Long) ack.getLastMessageId().getFutureOrSequenceLong() :                persistenceAdapter.getStoreSequenceIdForMessageId(context, ack.getLastMessageId(), destination)[0];{color}
> In which case it will get it from the DB directly.
> But the issue is that in the described case, the "futureOrSequenceLong" is not null, but has a value of 0L.
> This is due to this line in JournalMessageStore.addMessage [142] (called when the message is originally produced)
> {color:#FF0000}message.getMessageId().setFutureOrSequenceLong(0l);{color}
> I actually think that creating a message on a JournalMessageStore should leave the "futureOrSequenceLong" null so that later condition checks can pick it up as such and properly get the sequence from the database.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)