You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by Nicky Sandhu <ka...@gmail.com> on 2007/10/05 02:17:17 UTC
Usage Manager for durable offline subscriber
On revision 581885 I turned on debug messages on Usage.java and the commented
lines of System.out on Message.java and I see the memory usage increase
beyond 100% to something like 16500%. This causes a problem for the next
thing that calls Usage.waitForSpace(timeout) and causes an indefinite hang
The Usage.increaseUsage(value) does not check to see if it's full and so can
keep allocating even beyond 100%. This seems to be happening a lot from
Message.incrementReferenceCount() with no corresponding
Message.decrementReferenceCount() happening.
In my case it happens when a publishing to a topic (~ 1000 msgs) with a
offline durable subscribe with multiple hard disconnects from the
subscriber.
--
View this message in context: http://www.nabble.com/Usage-Manager-for-durable-offline-subscriber-tf4572289s2354.html#a13051280
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: Usage Manager for durable offline subscriber
Posted by Nicky Sandhu <ka...@gmail.com>.
I have a test case that reproduces the problem. WARNING: This test case is
not a unit test even though it uses the junit framework.
Please have a look at https://issues.apache.org/activemq/browse/AMQ-1449.
rajdavies wrote:
>
> oops - that's the latest- must drink coffee before reading email ;)
> Any chance of a test case ?
>
> cheers,
>
> Rob
>
> rajdavies wrote:
>>
>> Hi Nicky,
>>
>> which version is that?
>>
>>
>> cheers,
>>
>> Rob
>>
>> http://rajdavies.blogspot.com/
>>
>>
>>
>> On Oct 5, 2007, at 1:17 AM, Nicky Sandhu wrote:
>>
>>>
>>> On revision 581885 I turned on debug messages on Usage.java and the
>>> commented
>>> lines of System.out on Message.java and I see the memory usage
>>> increase
>>> beyond 100% to something like 16500%. This causes a problem for the
>>> next
>>> thing that calls Usage.waitForSpace(timeout) and causes an
>>> indefinite hang
>>>
>>> The Usage.increaseUsage(value) does not check to see if it's full
>>> and so can
>>> keep allocating even beyond 100%. This seems to be happening a lot
>>> from
>>> Message.incrementReferenceCount() with no corresponding
>>> Message.decrementReferenceCount() happening.
>>>
>>> In my case it happens when a publishing to a topic (~ 1000 msgs)
>>> with a
>>> offline durable subscribe with multiple hard disconnects from the
>>> subscriber.
>>> --
>>> View this message in context: http://www.nabble.com/Usage-Manager-
>>> for-durable-offline-subscriber-tf4572289s2354.html#a13051280
>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>>
>>
>>
>>
>
>
--
View this message in context: http://www.nabble.com/Usage-Manager-for-durable-offline-subscriber-tf4572289s2354.html#a13102163
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: Usage Manager for durable offline subscriber
Posted by Nicky Sandhu <ka...@gmail.com>.
Still no luck with a unit test but here's some info from a debugging session
I am using a Journaled persistence adapter with Derby backend. I have
registered a durable subscriber that is then disconnected. A producer
produces ~ 1000 msgs to the topic. The durable subscriber comes on line
(after a broker restart sometimes) and I put a conditional breakpoint
(newPercentUsage > 100) on the running remote broker and here's the stack
trace where it is about to exceed the limit
http://www.nabble.com/file/p13064574/memory.usage.activemq.png
Hope this helps...
Nicky Sandhu wrote:
>
> I have been trying to reproduce it in a unit test. No luck so far :( I
> have hunch its the client disconnections and reconnections without a
> proper session.close
>
> On Oct 5, 2007, at 1:17 AM, Nicky Sandhu wrote:
>
>>
>> On revision 581885 I turned on debug messages on Usage.java and the
>> commented
>> lines of System.out on Message.java and I see the memory usage
>> increase
>> beyond 100% to something like 16500%. This causes a problem for the
>> next
>> thing that calls Usage.waitForSpace(timeout) and causes an
>> indefinite hang
>>
>> The Usage.increaseUsage(value) does not check to see if it's full
>> and so can
>> keep allocating even beyond 100%. This seems to be happening a lot
>> from
>> Message.incrementReferenceCount() with no corresponding
>> Message.decrementReferenceCount() happening.
>>
>> In my case it happens when a publishing to a topic (~ 1000 msgs)
>> with a
>> offline durable subscribe with multiple hard disconnects from the
>> subscriber.
>> --
>> View this message in context: http://www.nabble.com/Usage-Manager-
>> for-durable-offline-subscriber-tf4572289s2354.html#a13051280
>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>
>
>
>
--
View this message in context: http://www.nabble.com/Usage-Manager-for-durable-offline-subscriber-tf4572289s2354.html#a13064574
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: Usage Manager for durable offline subscriber
Posted by Nicky Sandhu <ka...@gmail.com>.
I have been trying to reproduce it in a unit test. No luck so far :( I have
hunch its the client disconnections and reconnections without a proper
session.close
It would be great to have Usage exposed via MBeans. Any plans to do that?
IMHO, reference counting is extremely difficult to implement especially when
increment and decrements are async. Any other strategies under consideration
?
rajdavies wrote:
>
> oops - that's the latest- must drink coffee before reading email ;)
> Any chance of a test case ?
>
> cheers,
>
> Rob
>
> rajdavies wrote:
>>
>> Hi Nicky,
>>
>> which version is that?
>>
>>
>> cheers,
>>
>> Rob
>>
>> http://rajdavies.blogspot.com/
>>
>>
>>
>> On Oct 5, 2007, at 1:17 AM, Nicky Sandhu wrote:
>>
>>>
>>> On revision 581885 I turned on debug messages on Usage.java and the
>>> commented
>>> lines of System.out on Message.java and I see the memory usage
>>> increase
>>> beyond 100% to something like 16500%. This causes a problem for the
>>> next
>>> thing that calls Usage.waitForSpace(timeout) and causes an
>>> indefinite hang
>>>
>>> The Usage.increaseUsage(value) does not check to see if it's full
>>> and so can
>>> keep allocating even beyond 100%. This seems to be happening a lot
>>> from
>>> Message.incrementReferenceCount() with no corresponding
>>> Message.decrementReferenceCount() happening.
>>>
>>> In my case it happens when a publishing to a topic (~ 1000 msgs)
>>> with a
>>> offline durable subscribe with multiple hard disconnects from the
>>> subscriber.
>>> --
>>> View this message in context: http://www.nabble.com/Usage-Manager-
>>> for-durable-offline-subscriber-tf4572289s2354.html#a13051280
>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>>
>>
>>
>>
>
>
--
View this message in context: http://www.nabble.com/Usage-Manager-for-durable-offline-subscriber-tf4572289s2354.html#a13063664
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: Usage Manager for durable offline subscriber
Posted by rajdavies <ra...@gmail.com>.
oops - that's the latest- must drink coffee before reading email ;)
Any chance of a test case ?
cheers,
Rob
rajdavies wrote:
>
> Hi Nicky,
>
> which version is that?
>
>
> cheers,
>
> Rob
>
> http://rajdavies.blogspot.com/
>
>
>
> On Oct 5, 2007, at 1:17 AM, Nicky Sandhu wrote:
>
>>
>> On revision 581885 I turned on debug messages on Usage.java and the
>> commented
>> lines of System.out on Message.java and I see the memory usage
>> increase
>> beyond 100% to something like 16500%. This causes a problem for the
>> next
>> thing that calls Usage.waitForSpace(timeout) and causes an
>> indefinite hang
>>
>> The Usage.increaseUsage(value) does not check to see if it's full
>> and so can
>> keep allocating even beyond 100%. This seems to be happening a lot
>> from
>> Message.incrementReferenceCount() with no corresponding
>> Message.decrementReferenceCount() happening.
>>
>> In my case it happens when a publishing to a topic (~ 1000 msgs)
>> with a
>> offline durable subscribe with multiple hard disconnects from the
>> subscriber.
>> --
>> View this message in context: http://www.nabble.com/Usage-Manager-
>> for-durable-offline-subscriber-tf4572289s2354.html#a13051280
>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>
>
>
>
--
View this message in context: http://www.nabble.com/Usage-Manager-for-durable-offline-subscriber-tf4572289s2354.html#a13053965
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: Usage Manager for durable offline subscriber
Posted by Rob Davies <ra...@gmail.com>.
Hi Nicky,
which version is that?
cheers,
Rob
http://rajdavies.blogspot.com/
On Oct 5, 2007, at 1:17 AM, Nicky Sandhu wrote:
>
> On revision 581885 I turned on debug messages on Usage.java and the
> commented
> lines of System.out on Message.java and I see the memory usage
> increase
> beyond 100% to something like 16500%. This causes a problem for the
> next
> thing that calls Usage.waitForSpace(timeout) and causes an
> indefinite hang
>
> The Usage.increaseUsage(value) does not check to see if it's full
> and so can
> keep allocating even beyond 100%. This seems to be happening a lot
> from
> Message.incrementReferenceCount() with no corresponding
> Message.decrementReferenceCount() happening.
>
> In my case it happens when a publishing to a topic (~ 1000 msgs)
> with a
> offline durable subscribe with multiple hard disconnects from the
> subscriber.
> --
> View this message in context: http://www.nabble.com/Usage-Manager-
> for-durable-offline-subscriber-tf4572289s2354.html#a13051280
> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>