You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Rajith Attapattu <ra...@gmail.com> on 2010/06/30 15:27:34 UTC
Re: [jira] Commented: (QPID-2694) Memory leak in 0-8/0-9 AMQSessions
on channel close
I agree with Rafi.
Memory leaks are not the sort that could be verified with a unit test
that runs under a min.
You need to run for a reasonably long time to detect trends.
I used to have a nightly cron tasks that runs a producer consumer pair
with transactions etc for 10 hours and records memory usage.
We also ran a similar test for over a week before to do wrap around
testing, but I also recorded memory usage and graphed it.
If there is a memory leak it will be visible and we did infact find
one issue where we were stuffing stuff into a map that grew very very
slowly over time.
However this wasn't even visible during the 10 hour run, but was only
detected during the week long test.
I don't think it's unreasonable run such a test once a release cycle.
Once you identify an issue then maybe you could try and write a test
that could reproduce it in a short time frame and then analyse using a
profiler etc..
But again in order to verify I would still recommend running for at
least 10 hrs plus.
Just my 2 cents and hope it helps.
Rajith
On Wed, Jun 30, 2010 at 8:00 AM, Rafael Schloming <ra...@redhat.com> wrote:
> Emmanuel Bourg wrote:
>>
>> Le 30/06/2010 09:20, Andrew Kennedy a écrit :
>>
>>> So, what do people think would be the best and simplest way to test this
>>> leak is fixed? Or am I over complicating this for myself before I've had
>>> enough coffee?
>>
>> I admit I don't remember seeing a unit test for a memory leak. This is
>> usually verified once by a profiler. I see two approaches to write such a
>> test:
>> - generate a heap dump and inspect it
>> - if the test can get a reference on the object that leaks, the garbage
>> collection can be verified by a WeakReference combined with a ReferenceQueue
>>
>> public void testMemoryLeak() {
>> Object leakingObject = getLeakingObject();
>>
>> ReferenceQueue queue = new ReferenceQueue();
>> WeakReference reference = new WeakReference(leakingObject, queue);
>> leakingObject = null;
>>
>> // so some operations supposed to dereference the object
>>
>> System.gc();
>>
>> assertTrue(reference.isEnqueued());
>> }
>
> FWIW, I don't know how portable a test like this would be. According to the
> doc, System.gc() only "suggests" that the JVM runs the garbage collector.
>
> It's probably also not that robust since you wouldn't detect leakage of any
> ancillary objects.
>
> I think a targeted stress test is likely to be way more robust for this sort
> of thing than a unit test. Of course that's the sort of thing you don't
> really want to run once per checkin, more like once (hopefully) per release.
>
> Perhaps it would be useful to start a suite/category of longer running
> "release" tests.
>
> --Rafael
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
>
--
Regards,
Rajith Attapattu
Red Hat
http://rajith.2rlabs.com/
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: [jira] Commented: (QPID-2694) Memory leak in 0-8/0-9 AMQSessions
on channel close
Posted by Rafael Schloming <ra...@redhat.com>.
Marnie McCormack wrote:
> How do we currently test that for various operations (like Session close)
> the correct messages are dispatched i.e. so we avoid this case where the
> close-ok didn't go ?
>
> That was really what I was driving at on the test - obv we can stress
> test/profile on a one off basis outside of the codebase/
I haven't followed the details of this Jira, but I would say that in
general these sorts of protocol operations effect state changes in some
entity, e.g. the session and/or the conection, and we should test that
the state change has occurred. This probably involves testing against
the implementing class(es) directly as the JMS interface may not expose
the accessors necessary to verify that the state change has occurred.
--Rafael
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: [jira] Commented: (QPID-2694) Memory leak in 0-8/0-9 AMQSessions
on channel close
Posted by Marnie McCormack <ma...@googlemail.com>.
How do we currently test that for various operations (like Session close)
the correct messages are dispatched i.e. so we avoid this case where the
close-ok didn't go ?
That was really what I was driving at on the test - obv we can stress
test/profile on a one off basis outside of the codebase/
Marnie
On Wed, Jun 30, 2010 at 2:27 PM, Rajith Attapattu <ra...@gmail.com>wrote:
> I agree with Rafi.
> Memory leaks are not the sort that could be verified with a unit test
> that runs under a min.
> You need to run for a reasonably long time to detect trends.
>
> I used to have a nightly cron tasks that runs a producer consumer pair
> with transactions etc for 10 hours and records memory usage.
> We also ran a similar test for over a week before to do wrap around
> testing, but I also recorded memory usage and graphed it.
> If there is a memory leak it will be visible and we did infact find
> one issue where we were stuffing stuff into a map that grew very very
> slowly over time.
> However this wasn't even visible during the 10 hour run, but was only
> detected during the week long test.
> I don't think it's unreasonable run such a test once a release cycle.
>
> Once you identify an issue then maybe you could try and write a test
> that could reproduce it in a short time frame and then analyse using a
> profiler etc..
> But again in order to verify I would still recommend running for at
> least 10 hrs plus.
>
> Just my 2 cents and hope it helps.
>
> Rajith
>
> On Wed, Jun 30, 2010 at 8:00 AM, Rafael Schloming <ra...@redhat.com>
> wrote:
> > Emmanuel Bourg wrote:
> >>
> >> Le 30/06/2010 09:20, Andrew Kennedy a écrit :
> >>
> >>> So, what do people think would be the best and simplest way to test
> this
> >>> leak is fixed? Or am I over complicating this for myself before I've
> had
> >>> enough coffee?
> >>
> >> I admit I don't remember seeing a unit test for a memory leak. This is
> >> usually verified once by a profiler. I see two approaches to write such
> a
> >> test:
> >> - generate a heap dump and inspect it
> >> - if the test can get a reference on the object that leaks, the garbage
> >> collection can be verified by a WeakReference combined with a
> ReferenceQueue
> >>
> >> public void testMemoryLeak() {
> >> Object leakingObject = getLeakingObject();
> >>
> >> ReferenceQueue queue = new ReferenceQueue();
> >> WeakReference reference = new WeakReference(leakingObject, queue);
> >> leakingObject = null;
> >>
> >> // so some operations supposed to dereference the object
> >>
> >> System.gc();
> >>
> >> assertTrue(reference.isEnqueued());
> >> }
> >
> > FWIW, I don't know how portable a test like this would be. According to
> the
> > doc, System.gc() only "suggests" that the JVM runs the garbage collector.
> >
> > It's probably also not that robust since you wouldn't detect leakage of
> any
> > ancillary objects.
> >
> > I think a targeted stress test is likely to be way more robust for this
> sort
> > of thing than a unit test. Of course that's the sort of thing you don't
> > really want to run once per checkin, more like once (hopefully) per
> release.
> >
> > Perhaps it would be useful to start a suite/category of longer running
> > "release" tests.
> >
> > --Rafael
> >
> > ---------------------------------------------------------------------
> > Apache Qpid - AMQP Messaging Implementation
> > Project: http://qpid.apache.org
> > Use/Interact: mailto:dev-subscribe@qpid.apache.org
> >
> >
>
>
>
> --
> Regards,
>
> Rajith Attapattu
> Red Hat
> http://rajith.2rlabs.com/
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
>