[jira] [Created] (AMQ-6524) The MemoryUsage tracker can get out of sync when using concurrentStoreAndDispatch

["Christopher L. Shannon (JIRA)" <ji...@apache.org>]

Christopher L. Shannon created AMQ-6524:
-------------------------------------------

             Summary: The MemoryUsage tracker can get out of sync when using concurrentStoreAndDispatch
                 Key: AMQ-6524
                 URL: https://issues.apache.org/jira/browse/AMQ-6524
             Project: ActiveMQ
          Issue Type: Bug
          Components: Broker
    Affects Versions: 5.14.1
            Reporter: Christopher L. Shannon
            Assignee: Christopher L. Shannon


There is a rare issue with memoryUsage when using concurrentStoreAndDispatch that causes memory usage to get out of sync and go negative or stay too high.  It is hard to reproduce and only occurs on brokers under high load in my testing.

I have finally tracked down the issue to the InnerFutureTask inside of KahaDB that gets used to fire the PendingMarshalUsageTracker on completion.  This task is not thread safe and causes the following 2 problems:
    
    1) Sometimes the listener is never fired and the usage isn't
    decremented. This case seems to be caused by the listener not
    being visible to the thread when run.  (volatile or atomic reference need to be used)
    
    2) Sometimes the listener can run twice. This seems to be because of a
    lack of synchronization between the if(isDone()) check and the done()
    method call.  This can also be fixed by using an atomic reference.

I have a patch that solves both of these problems and I have not seen the issue reoccur in my testing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)