How can memory usage be at 10,000% of available memory?

[rth <ro...@gmail.com>]

I'm seeing messages like this in my logs:

2017-05-17 12:42:23.610+0000 | DEBUG | org.apache.activemq.usage.Usage     
|  | main | default:memory:topic://some.topic:memory: usage change from:
10694% of available memory, to: 10695% of available memory

Flow control is enabled, and messages are being published as persistent.
What should I make of this message?







--
View this message in context: http://activemq.2283324.n4.nabble.com/How-can-memory-usage-be-at-10-000-of-available-memory-tp4726425.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: How can memory usage be at 10,000% of available memory?

[rth <ro...@gmail.com>]

A colleague provided these answers. Is the response to #1 a concern?

(Sorry for the formatting issues. I have no idea how to format raw text in
this UI.)

1. What version of ActiveMQ are you running?

ActiveMQ Broker version: 5.13.1, ActiveMQ Client version: 5.14.2

2. What are the limits you're setting on the memory store, and on whichever
persistence store you're using?

          <policyEntry topic="*************"
                       producerFlowControl="true"
                       memoryLimit="64mb"                   
                       topicPrefetch="32"
                       durableTopicPrefetch="32">
            <pendingSubscriberPolicy>
              <vmCursor/>
            </pendGGkkkingSubscriberPolicy>
            <pendingDurableSubscriberPolicy>
              <vmDurableCursor/>
            </pendingDurableSubscriberPolicy>
          </policyEntry>

    <systemUsage>
      <systemUsage>
        <memoryUsage>
        <memoryUsage limit="2048 mb"/>
        </memoryUsage>
        <storeUsage>
          <storeUsage limit="${*********:80 gb}"/>
        </storeUsage>
        <tempUsage>
          <tempUsage limit="10 gb"/>
        </tempUsage>
      </systemUsage>
    </systemUsage>

3. Have you confirmed that your messages for that topic really are being
written to the persistence store?

We have no access to customer's site so there is no definitive answer

4. How big are your messages?

No idea. [However, given the application, I believe they should be
relatively small, on the order of a few KB.]

5. Do you ever see PFC kick in

Unsure. [However, we know the consumer cannot consume as fast as the
producer, so it is likely.]

6. What type of memory cursor
(http://activemq.apache.org/message-cursors.html) are you using for this
topic?
             <pendingSubscriberPolicy>
              <vmCursor/>
            </pendGGkkkingSubscriberPolicy>
            <pendingDurableSubscriberPolicy>
              <vmDurableCursor/>
            </pendingDurableSubscriberPolicy>

7. Do you see any other errors or warnings in the logs, even if it's not
immediately obvious how/if they would relate to the behavior you described?​

No other errors





--
View this message in context: http://activemq.2283324.n4.nabble.com/How-can-memory-usage-be-at-10-000-of-available-memory-tp4726425p4726477.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: How can memory usage be at 10,000% of available memory?

[Tim Bain <tb...@alumni.duke.edu>]

That does sound suspicious, and I believe I've seen users describe similar
(though not quite so extreme) behavior in the past, so there definitely
might be something here.

Can you please provide some additional information? What version of
ActiveMQ are you running? What are the limits you're setting on the memory
store, and on whichever persistence store you're using? Have you confirmed
that your messages for that topic really are being written to the
persistence store? How big are your messages? Do you ever see PFC kick in
(even though it's not doing so at the point you quoted the logs for), and
if so can you please describe when it does and what's different then if
you're aware of anything? What type of memory cursor (
http://activemq.apache.org/message-cursors.html) are you using for this
topic? Do you see any other errors or warnings in the logs, even if it's
not immediately obvious how/if they would relate to the behavior you
described?

Tim


On May 22, 2017 2:39 PM, "rth" <ro...@gmail.com> wrote:

I'm seeing messages like this in my logs:

2017-05-17 12:42:23.610+0000 | DEBUG | org.apache.activemq.usage.Usage
| | main | default:memory:topic://some.topic:memory: usage change from:
10694% of available memory, to: 10695% of available memory

Flow control is enabled, and messages are being published as persistent.
What should I make of this message?







--
View this message in context: http://activemq.2283324.n4.
nabble.com/How-can-memory-usage-be-at-10-000-of-available-memory-tp4726425.
html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.