[jira] [Commented] (AMQ-9813) Incorrect QueueSize if Non-Persistent messages with TTL is used

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMQ-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18041849#comment-18041849 ] 

ASF subversion and git services commented on AMQ-9813:
------------------------------------------------------

Commit ecdd5726ee9d098a764381878bbe836d2eb4d8a1 in activemq's branch refs/heads/main from Christopher L. Shannon
[ https://gitbox.apache.org/repos/asf?p=activemq.git;h=ecdd5726ee ]

Merge pull request #1551 from lenoch7/AMQ-9813_IncorrectQueueSizeNonPersistentMessageWithTTL

AMQ-9813 - fix wrong QueueSize for non-persistent message with TTL

> Incorrect QueueSize if Non-Persistent messages with TTL is used
> ---------------------------------------------------------------
>
>                 Key: AMQ-9813
>                 URL: https://issues.apache.org/jira/browse/AMQ-9813
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 6.2.0, 5.19.1
>            Reporter: Radek Kraus
>            Assignee: Christopher L. Shannon
>            Priority: Major
>             Fix For: 6.1.9, 5.19.2, 6.2.1
>
>          Time Spent: 1h
>  Remaining Estimate: 0h
>
> Sometimes, I detect a problem with incorrect QueueSize, when non-persitent messages with TTL is used.
> *UseCase/Symptoms*
>  * produce/consume non-persistent messages with defined TTL (expiration)
>  * producerFlowControl=false
>  * consumers "are slow", it means that some messages are expired (broker side or client side)
>  * QueueSize sometimes reports non-zero value even when all messages are consumed/expired
>  * Broker does not "push" next messages to consumers
>  ** JMX method {{browseAsTable()}} does not provide messages
>  ** JMX method {{purge()}} does not remove messages (QueueSize is still non-zero, nothing is deleted)
>  * It seems like messages are expired, but QueueSize still reports non-zero values
>  * after Broker restart QueueSize is reset to zero (expected for non-persitent messages)
> *Analyse*
>  * I guess, that problem was introduced by AMQ-5785 (by [8b23e07|https://github.com/apache/activemq/commit/8b23e07], where deadlock was solved)
>  * there is a valid AMQ-5785 comment without next reaction
>  * what happens in [8b23e07|https://github.com/apache/activemq/commit/8b23e07] commit (nearly the same what is written in [AMQ-5785 comment|https://issues.apache.org/jira/browse/AMQ-5785?focusedCommentId=15607861&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15607861]):
>  ** contract of {{expireOldMessages()}} method was changed to don't invoke {{discardExpiredMessage(MessageReference)}} (which is finally responsible to "process expired message")
>  ** the invocation of {{discardExpiredMessage(MessageReference)}} was moved out of synchronized section (deadlock prevention), see {{onUsageChanged(Usage, int, int)}} method
>  ** but the *problem is that {{expireOldMessages()}} method is invoked from next methods* ({{{}tryAddMessageLast(MessageReference, long){}}}, {{{}addMessageFirst(MessageReference){}}}), where *{{discardExpiredMessage(MessageReference)}} method is no longer invoked* (after change), *what it is problem from my point of view*
>  * IMHO "message expiration process" can be invoked:
>  ** from {{onUsageChanged(Usage, int, int)}} method ({{{}UsageListener{}}}), asynchronously triggered by Usage.setPercentUsage(int) (thread pool executor)
>  ** or "directly" from {{tryAddMessageLast(MessageReference, long)}} (in case when {{OrderedPendingList}} is used)
>  ** it depends on situations (timings), which method is invoked "at first"
>  ** when {{tryAddMessageLast(MessageReference, long)}} method is invoked, then expired messages are only removed from {{{}memoryList{}}}, but {{discardExpiredMessage(MessageReference)}} is never invoked (what it is the problem)
> *PR* (https://github.com/apache/activemq/pull/1551)
>  * I added "missing" invocation of {{discardExpiredMessage(MessageReference)}} method into {{{}tryAddMessageLast(MessageReference, long){}}}, {{addMessageFirst(MessageReference)}} methods
>  * I tried to use same "postponed" strategy (outside of synchronization) like was already done in original commit (see {{{}onUsageChanged(Usage, int, int){}}}) method
>  * I tried to write JUnit test, but it is complicated, because it depends on timings (from my expiriences TTL must be very small to simulate the problem, see {{{}FullDestinationMemoryMessageExpirationTest{}}})
> Could you please check/analyze my findings?
> In each case (IMHO) [8b23e07|https://github.com/apache/activemq/commit/8b23e07] breaks previous contract of {{expireOldMessages()}} methods, which can caused that expired messages are not "processed", when expiration is triggered by {{{}tryAddMessageLast(MessageReference, long){}}}, {{addMessageFirst(MessageReference)}} methods.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@activemq.apache.org
For additional commands, e-mail: issues-help@activemq.apache.org
For further information, visit: https://activemq.apache.org/contact