[jira] [Commented] (AMQ-6477) ReduceMemoryFootprint should apply to non-persistent messages and subscriptions

["Gary Tully (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMQ-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602237#comment-15602237 ] 

Gary Tully commented on AMQ-6477:
---------------------------------

[~cshannon] looks good 


one thought: 
I guess it also means that on each dispatch, so when prefetched messages are redelivered, there will be an unmarshall/clear.
it makes me think that maybe doing something to ensure that properties in both states are part of the size. That would tackle directly the OOM.

However the non persistent case is valid and I guess the selector case is less common.

One important aspect is ensuring no sharing of a message when the destructive change to clear the state is made. With composite sends or virtual topics (mostly there is copy) but some care there may be needed. Also consumer that use a composite dest to subscribe to multiple dests. Recall the test that had us add/remove some sync in message in the past.

I think it will be fine, just be sure there is some existing sync point that will protect you in the case of sharing.


> ReduceMemoryFootprint should apply to non-persistent messages and subscriptions
> -------------------------------------------------------------------------------
>
>                 Key: AMQ-6477
>                 URL: https://issues.apache.org/jira/browse/AMQ-6477
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.14.1
>            Reporter: Christopher L. Shannon
>            Assignee: Christopher L. Shannon
>
> There is a flag called reduceMemoryFootprint which will clear out the unmarshalled state of a message after it is added to a queue/topic because that state isn't counted towards the size and can lead to OOM errors.
> However, even setting this flag, I am still seeing some brokers run out of memory.  After analyzing the heap dumps I have found 2 reasons for this:
> 1) Non-persistent messages do not have their unmarshalled state cleared.  This was done because when a message is persisted it is guaranteed to have the marshalled state.  However we can still clear the memory for non-persistent messages as long as we check to make sure the marshalled state exists first, which it will for transports like TCP but won't exist for the VM transport.
> 2) When a message is added to a subscription the properties are needed to check which subscription messages can be dispatched to.  This causes the memory to be unmarshalled again.  
> In the topic case, we should probably defer the clearing of the state until after the message is added to a subscription because messages are immediately dispatched to topic subs so we don't unnecessarily have to convert the data twice.  In the queue case it probably makes sense to clear memory on add to the queue and on add to the subscription.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)