You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@archiva.apache.org by Olivier Lamy <ol...@apache.org> on 2014/05/01 06:41:22 UTC
Re: Memory leak in Archiva 2.0.1
fixed.
You can test a build from last master.
On 30 April 2014 16:51, Sascha Vogt <sa...@gmail.com> wrote:
> Great, thanks.
> Just needed to get a cup of coffee before writing it up ;)
>
> I'll add my findings over there
>
> Greetings
> -Sascha-
>
> Am 30.04.2014 08:44, schrieb Olivier Lamy:
>> done here: http://jira.codehaus.org/browse/MRM-1837
>>
>>
>>
>> On 30 April 2014 16:24, Olivier Lamy <ol...@apache.org> wrote:
>>> good catch!!!!
>>> Can you create a Jira for that, I have some ideas to fix it.
>>> OMG sometimes I write comments finally :-)
>>> Anyway this one could improve a bit but it's not the real problem.
>>> The main issue with this refactoring is to rewrite the content
>>> consumer and break backward compat for folks who wrote their own
>>> consumers :-(
>>>
>>>
>>> On 30 April 2014 02:12, Sascha Vogt <sa...@gmail.com> wrote:
>>>> Hi all,
>>>>
>>>> I think we found a memory leak.
>>>>
>>>> In a heap dump taken while Archiva was pretty busy doing GCs, we found
>>>> DefaultArchivaConfiguration to be holding references to 10.000
>>>> registryListeners (which in turn had Jackrabbit XASessionImpls) and
>>>> occupied 84 of our 4GB heap.
>>>>
>>>> Is this related to the following comment we found:
>>>>
>>>> /**
>>>> * A consumer of content (files) in the repository.
>>>> *
>>>> * olamy: TODO/FIXME we must review this api, in the current situation we
>>>> use prototype beans rather than singletons
>>>> * this is a bit memory consuming the better will be to ConsumerContext
>>>> bean to transport repository context etc...
>>>> */
>>>> public interface RepositoryContentConsumer
>>>>
>>>> Will the registryListeners ever be cleaned up? Any ideas how to address
>>>> that?
>
--
Olivier Lamy
Ecetera: http://ecetera.com.au
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Memory leak in Archiva 2.0.1
Posted by Sascha Vogt <sa...@gmail.com>.
Hi Olivier,
it still looks good. Very nice.
Greetings
-Sascha-
Am 13.05.2014 18:58, schrieb Sascha Vogt:
> I can :)
>
> just upgraded... Let's see where this is tomorrow after a full day of
> builds and deployments *hehe*.
>
> Thanks again for fixing so quickly.
>
> Greetings
> -Sascha-
>
> Am 13.05.2014 09:14, schrieb Olivier Lamy:
>> Ok I pushed a fix in master.
>> Let me know if you can test it :-)
>>
>> Thanks!!
>> Olivier
>>
Re: Memory leak in Archiva 2.0.1
Posted by Sascha Vogt <sa...@gmail.com>.
I can :)
just upgraded... Let's see where this is tomorrow after a full day of
builds and deployments *hehe*.
Thanks again for fixing so quickly.
Greetings
-Sascha-
Am 13.05.2014 09:14, schrieb Olivier Lamy:
> Ok I pushed a fix in master.
> Let me know if you can test it :-)
>
> Thanks!!
> Olivier
>
> On 12 May 2014 18:53, Sascha Vogt <sa...@gmail.com> wrote:
>> Hi,
>>
>> Am 12.05.2014 05:20, schrieb Olivier Lamy:
>>> Hi,
>>> Just to correctly understand.
>>> org.apache.archiva.consumers.core.MetadataUpdaterConsumer | 45.052
>>> This mean the jvm retains 45052 instances of this class?
>> Yes, and today we're at 59k ;) So growth over the weekend was slow (so
>> it probably has to do something with uploads to Archiva).
>>
>>> Are they attached to any other object(s)?
>>
>> If I read the heapdump correctly, then I think they are entries in the
>> registryListener set of DefaultArchivaConfiguration.
>> Just verified via Debugger, the registryListener set contains 130k
>> entries (59k MetadataUpdteConsumer + 59k ArtifactMissingChecksumsConsumer)
>>
>> Greetings
>> -Sascha-
>>
>
>
>
Re: Memory leak in Archiva 2.0.1
Posted by Olivier Lamy <ol...@apache.org>.
Ok I pushed a fix in master.
Let me know if you can test it :-)
Thanks!!
Olivier
On 12 May 2014 18:53, Sascha Vogt <sa...@gmail.com> wrote:
> Hi,
>
> Am 12.05.2014 05:20, schrieb Olivier Lamy:
>> Hi,
>> Just to correctly understand.
>> org.apache.archiva.consumers.core.MetadataUpdaterConsumer | 45.052
>> This mean the jvm retains 45052 instances of this class?
> Yes, and today we're at 59k ;) So growth over the weekend was slow (so
> it probably has to do something with uploads to Archiva).
>
>> Are they attached to any other object(s)?
>
> If I read the heapdump correctly, then I think they are entries in the
> registryListener set of DefaultArchivaConfiguration.
> Just verified via Debugger, the registryListener set contains 130k
> entries (59k MetadataUpdteConsumer + 59k ArtifactMissingChecksumsConsumer)
>
> Greetings
> -Sascha-
>
--
Olivier Lamy
Ecetera: http://ecetera.com.au
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Memory leak in Archiva 2.0.1
Posted by Sascha Vogt <sa...@gmail.com>.
Hi,
Am 12.05.2014 05:20, schrieb Olivier Lamy:
> Hi,
> Just to correctly understand.
> org.apache.archiva.consumers.core.MetadataUpdaterConsumer | 45.052
> This mean the jvm retains 45052 instances of this class?
Yes, and today we're at 59k ;) So growth over the weekend was slow (so
it probably has to do something with uploads to Archiva).
> Are they attached to any other object(s)?
If I read the heapdump correctly, then I think they are entries in the
registryListener set of DefaultArchivaConfiguration.
Just verified via Debugger, the registryListener set contains 130k
entries (59k MetadataUpdteConsumer + 59k ArtifactMissingChecksumsConsumer)
Greetings
-Sascha-
Re: Memory leak in Archiva 2.0.1
Posted by Olivier Lamy <ol...@apache.org>.
Hi,
Just to correctly understand.
org.apache.archiva.consumers.core.MetadataUpdaterConsumer | 45.052
This mean the jvm retains 45052 instances of this class?
Are they attached to any other object(s)?
Olivier
On 9 May 2014 17:09, Sascha Vogt <sa...@gmail.com> wrote:
> Hi,
>
> Unfortunately there is still a leak. Current heap dump shows:
>
>> Class Name | Objects | Shallow Heap | Retained Heap
>> -----------------------------------------------------------------------------------------------------------
>> org.apache.archiva.consumers.core.MetadataUpdaterConsumer | 45.052 | 2.883.328 | >= 10.812.760
>> org.apache.archiva.consumers.core.ArtifactMissingChecksumsConsumer| 45.294 | 2.174.112 | >= 10.185.104
>> -----------------------------------------------------------------------------------------------------------
>
> And no repo scans or otherwise are currently running.
>
> @Olivier: do you have an idea where these instances may come from, or
> should I investigate further?
>
> Greetings
> -Sascha-
>
>
--
Olivier Lamy
Ecetera: http://ecetera.com.au
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Memory leak in Archiva 2.0.1
Posted by Sascha Vogt <sa...@gmail.com>.
Hi,
Unfortunately there is still a leak. Current heap dump shows:
> Class Name | Objects | Shallow Heap | Retained Heap
> -----------------------------------------------------------------------------------------------------------
> org.apache.archiva.consumers.core.MetadataUpdaterConsumer | 45.052 | 2.883.328 | >= 10.812.760
> org.apache.archiva.consumers.core.ArtifactMissingChecksumsConsumer| 45.294 | 2.174.112 | >= 10.185.104
> -----------------------------------------------------------------------------------------------------------
And no repo scans or otherwise are currently running.
@Olivier: do you have an idea where these instances may come from, or
should I investigate further?
Greetings
-Sascha-
Re: Memory leak in Archiva 2.0.1
Posted by Sascha Vogt <sa...@gmail.com>.
Hi Olivier,
Am 01.05.2014 06:41, schrieb Olivier Lamy:
> fixed.
> You can test a build from last master.
sorry for the late response - took a while for a timeslot to upgrade our
system. It looks definetly better currently. I'll keep a close eye on it
though, as I currently see
> Class Name | Objects | Shallow Heap | Retained Heap
> -----------------------------------------------------------------------------------------------------------
> org.apache.archiva.consumers.core.MetadataUpdaterConsumer | 10.511 | 672.704 | >= 2.522.968
> org.apache.archiva.consumers.core.ArtifactMissingChecksumsConsumer| 10.573 | 507.504 | >= 2.378.688
> -----------------------------------------------------------------------------------------------------------
This count looks still a bit high, though I cannot tell if it's going to
decrease. Overall heap is currently at 295 MB which is perfectly ok
(heap dump 2,5 h earlier was at 292 MB, so not much increasing and
currently a full scan is running and the scan queue is at 14k entries)
THANKS a lot && Greetings
-Sascha-