You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by ji...@codehaus.org on 2004/05/26 10:13:15 UTC
[jira] Created: (MAVEN-1294) Maven is leaking much memory
Message:
A new issue has been created in JIRA.
---------------------------------------------------------------------
View the issue:
http://jira.codehaus.org/browse/MAVEN-1294
Here is an overview of the issue:
---------------------------------------------------------------------
Key: MAVEN-1294
Summary: Maven is leaking much memory
Type: Bug
Status: Unassigned
Priority: Major
Original Estimate: Unknown
Time Spent: Unknown
Remaining: Unknown
Project: maven
Components:
jelly/ant integration
Versions:
1.0-rc3
Assignee:
Reporter: Rafal Krzewski
Created: Wed, 26 May 2004 4:13 AM
Updated: Wed, 26 May 2004 4:13 AM
Description:
The test was done as follows. I took ledge-components project http://objectledge.org/modules/ledge-components, cleaned out the target, then ran statcvs goal. Statcvs generated 160 xdoc documents, 2.2MB of markup total. Then I ran xdocs goal, to tranfrom the xdocs into html, under hprof profiler build into JDK. Settings were: allocation site analysis, heap dump at exit, stacktrace depth 16. The run took about 65 minutes on Athlon 1700XP, 512 phisical RAM. Maximum memory usage was 530MB virtual/240MB resident during the run, it surged some additional 300MB both during heap dump. The generated dump file is about 350MB, 23MB when bzip2 compressed.
I'm going to post an xls file on Jira containg the statistics of objects allocated, and objects alive at the time of VM generation. From what I'm seeing none of the Jelly TagScript objects created gets ever garbage collected. Wading through the heap dump I was able to determine that all of them are connected to the GC root through the main Thread, and a ThreadLocal variable tagHolder declared in jelly's TagScript.java, line 115. Despite calls to tagHolder.set(null), the TagScript object stays connected to the Thread by means of ThreadLocal$ThreadLocalMap$Entry object which is a subclass of java.lang.ref.WeakReference. ThreadLocal implementation in JDK works like that. Now, AFAIK the GC is free to harvest all objects that are not connected to a GC root through hard reference, or a SoftReference. I wasn't able to trace all references in the object graph regarding TagScript objects (there are 21k+ of them!) so I am not sure if there are any other references that connect the TagScripts to a GC root through a permanent refernce, like a static field in some class. I cant tell you it's a dog's work to do that by hand. If I only had a proper memory analysys tool! Alas all the good ones are commercial... On the other hand GC is not *required* to harvest these objects. Maybe it just didn't bother to harvest because it was not pressed for memory? In my test it had still many megabytes left. Maybe a better test would be running it with too few memory and checking what was left alive, despite the fact GC was desperate to find the space?
At any rate, TagScript.tagHolder is suspicious, and if it's even not the culprit itself (could be if WeakRefernce implementation was broken) it is hiding the real problem (at least in the environment of my test). Note that the variable is not static, which means all TagScript object instances get tied to the Thread. I don't know if the possibility of using each instance concurrently from multiple threads is useful at all, especially that the instances have more state information, not only the tagHolder... Anyway, it's a wrong forum to discuss that. I also heard that Jelly was abandoned by the original author so there is a good chance that noone understands why this tagHolder thingy was introduced and what it does...
---------------------------------------------------------------------
JIRA INFORMATION:
This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Updated: (MAVEN-1294) Maven is leaking much memory
Posted by ji...@codehaus.org.
The following issue has been updated:
Updater: Rafal Krzewski (mailto:rkrzewsk@users.sourceforge.net)
Date: Wed, 26 May 2004 4:16 AM
Comment:
Spreadsheet with analysys of objects allocated vs objects alive at the time of VM termination.
Notice that there was no forced GC run before terminating, and GC was not pressed for memory - the test completed successfully.
Changes:
Attachment changed to sites2.xls
---------------------------------------------------------------------
For a full history of the issue, see:
http://jira.codehaus.org/browse/MAVEN-1294?page=history
---------------------------------------------------------------------
View the issue:
http://jira.codehaus.org/browse/MAVEN-1294
Here is an overview of the issue:
---------------------------------------------------------------------
Key: MAVEN-1294
Summary: Maven is leaking much memory
Type: Bug
Status: Unassigned
Priority: Major
Original Estimate: Unknown
Time Spent: Unknown
Remaining: Unknown
Project: maven
Components:
jelly/ant integration
Versions:
1.0-rc3
Assignee:
Reporter: Rafal Krzewski
Created: Wed, 26 May 2004 4:13 AM
Updated: Wed, 26 May 2004 4:16 AM
Description:
The test was done as follows. I took ledge-components project http://objectledge.org/modules/ledge-components, cleaned out the target, then ran statcvs goal. Statcvs generated 160 xdoc documents, 2.2MB of markup total. Then I ran xdocs goal, to tranfrom the xdocs into html, under hprof profiler build into JDK. Settings were: allocation site analysis, heap dump at exit, stacktrace depth 16. The run took about 65 minutes on Athlon 1700XP, 512 phisical RAM. Maximum memory usage was 530MB virtual/240MB resident during the run, it surged some additional 300MB both during heap dump. The generated dump file is about 350MB, 23MB when bzip2 compressed.
I'm going to post an xls file on Jira containg the statistics of objects allocated, and objects alive at the time of VM generation. From what I'm seeing none of the Jelly TagScript objects created gets ever garbage collected. Wading through the heap dump I was able to determine that all of them are connected to the GC root through the main Thread, and a ThreadLocal variable tagHolder declared in jelly's TagScript.java, line 115. Despite calls to tagHolder.set(null), the TagScript object stays connected to the Thread by means of ThreadLocal$ThreadLocalMap$Entry object which is a subclass of java.lang.ref.WeakReference. ThreadLocal implementation in JDK works like that. Now, AFAIK the GC is free to harvest all objects that are not connected to a GC root through hard reference, or a SoftReference. I wasn't able to trace all references in the object graph regarding TagScript objects (there are 21k+ of them!) so I am not sure if there are any other references that connect the TagScripts to a GC root through a permanent refernce, like a static field in some class. I cant tell you it's a dog's work to do that by hand. If I only had a proper memory analysys tool! Alas all the good ones are commercial... On the other hand GC is not *required* to harvest these objects. Maybe it just didn't bother to harvest because it was not pressed for memory? In my test it had still many megabytes left. Maybe a better test would be running it with too few memory and checking what was left alive, despite the fact GC was desperate to find the space?
At any rate, TagScript.tagHolder is suspicious, and if it's even not the culprit itself (could be if WeakRefernce implementation was broken) it is hiding the real problem (at least in the environment of my test). Note that the variable is not static, which means all TagScript object instances get tied to the Thread. I don't know if the possibility of using each instance concurrently from multiple threads is useful at all, especially that the instances have more state information, not only the tagHolder... Anyway, it's a wrong forum to discuss that. I also heard that Jelly was abandoned by the original author so there is a good chance that noone understands why this tagHolder thingy was introduced and what it does...
---------------------------------------------------------------------
JIRA INFORMATION:
This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MAVEN-1294) Maven is leaking much memory
Posted by "Florian Nierhaus (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MAVEN-1294?page=comments#action_31523 ]
Florian Nierhaus commented on MAVEN-1294:
-----------------------------------------
Hi,
Basically to make maven work for my project (using multi-project) I had to hack multi-project to call a separate maven process for each "site" goal, but now I am at a new breaking point. Just running the dashboard report to collect the statistics runs maven out of memory when I set 512MB, if I set 1Gig of memory it finishes, but I plan to add more sub-projects...
Vincent asked me to close the report against dashboard, since he thinks it is a generic maven issue http://jira.codehaus.org/browse/MPDASHBOARD-21
best regards,
Florian
> Maven is leaking much memory
> ----------------------------
>
> Key: MAVEN-1294
> URL: http://jira.codehaus.org/browse/MAVEN-1294
> Project: maven
> Type: Bug
> Components: jelly/ant integration
> Versions: 1.0-rc3
> Reporter: Rafal Krzewski
> Attachments: sites2.xls
>
>
> The test was done as follows. I took ledge-components project http://objectledge.org/modules/ledge-components, cleaned out the target, then ran statcvs goal. Statcvs generated 160 xdoc documents, 2.2MB of markup total. Then I ran xdocs goal, to tranfrom the xdocs into html, under hprof profiler build into JDK. Settings were: allocation site analysis, heap dump at exit, stacktrace depth 16. The run took about 65 minutes on Athlon 1700XP, 512 phisical RAM. Maximum memory usage was 530MB virtual/240MB resident during the run, it surged some additional 300MB both during heap dump. The generated dump file is about 350MB, 23MB when bzip2 compressed.
> I'm going to post an xls file on Jira containg the statistics of objects allocated, and objects alive at the time of VM generation. From what I'm seeing none of the Jelly TagScript objects created gets ever garbage collected. Wading through the heap dump I was able to determine that all of them are connected to the GC root through the main Thread, and a ThreadLocal variable tagHolder declared in jelly's TagScript.java, line 115. Despite calls to tagHolder.set(null), the TagScript object stays connected to the Thread by means of ThreadLocal$ThreadLocalMap$Entry object which is a subclass of java.lang.ref.WeakReference. ThreadLocal implementation in JDK works like that. Now, AFAIK the GC is free to harvest all objects that are not connected to a GC root through hard reference, or a SoftReference. I wasn't able to trace all references in the object graph regarding TagScript objects (there are 21k+ of them!) so I am not sure if there are any other references that connect the TagScripts to a GC root through a permanent refernce, like a static field in some class. I cant tell you it's a dog's work to do that by hand. If I only had a proper memory analysys tool! Alas all the good ones are commercial... On the other hand GC is not *required* to harvest these objects. Maybe it just didn't bother to harvest because it was not pressed for memory? In my test it had still many megabytes left. Maybe a better test would be running it with too few memory and checking what was left alive, despite the fact GC was desperate to find the space?
> At any rate, TagScript.tagHolder is suspicious, and if it's even not the culprit itself (could be if WeakRefernce implementation was broken) it is hiding the real problem (at least in the environment of my test). Note that the variable is not static, which means all TagScript object instances get tied to the Thread. I don't know if the possibility of using each instance concurrently from multiple threads is useful at all, especially that the instances have more state information, not only the tagHolder... Anyway, it's a wrong forum to discuss that. I also heard that Jelly was abandoned by the original author so there is a good chance that noone understands why this tagHolder thingy was introduced and what it does...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MAVEN-1294) Maven is leaking much memory
Posted by "Eric Lapierre (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MAVEN-1294?page=comments#action_38845 ]
Eric Lapierre commented on MAVEN-1294:
--------------------------------------
This problem has been fixed in jelly 1.0
http://issues.apache.org/jira/browse/JELLY-148
We have continually increased the -Xmx parameter until now where we have reached the limit on our system when we set it at 3500m. Our only alternative is to run the multiproject:site in smaller chunks of subprojects :-(
> Maven is leaking much memory
> ----------------------------
>
> Key: MAVEN-1294
> URL: http://jira.codehaus.org/browse/MAVEN-1294
> Project: maven
> Type: Bug
> Components: jelly/ant integration
> Versions: 1.0-rc3
> Reporter: Rafal Krzewski
> Attachments: sites2.xls
>
>
> The test was done as follows. I took ledge-components project http://objectledge.org/modules/ledge-components, cleaned out the target, then ran statcvs goal. Statcvs generated 160 xdoc documents, 2.2MB of markup total. Then I ran xdocs goal, to tranfrom the xdocs into html, under hprof profiler build into JDK. Settings were: allocation site analysis, heap dump at exit, stacktrace depth 16. The run took about 65 minutes on Athlon 1700XP, 512 phisical RAM. Maximum memory usage was 530MB virtual/240MB resident during the run, it surged some additional 300MB both during heap dump. The generated dump file is about 350MB, 23MB when bzip2 compressed.
> I'm going to post an xls file on Jira containg the statistics of objects allocated, and objects alive at the time of VM generation. From what I'm seeing none of the Jelly TagScript objects created gets ever garbage collected. Wading through the heap dump I was able to determine that all of them are connected to the GC root through the main Thread, and a ThreadLocal variable tagHolder declared in jelly's TagScript.java, line 115. Despite calls to tagHolder.set(null), the TagScript object stays connected to the Thread by means of ThreadLocal$ThreadLocalMap$Entry object which is a subclass of java.lang.ref.WeakReference. ThreadLocal implementation in JDK works like that. Now, AFAIK the GC is free to harvest all objects that are not connected to a GC root through hard reference, or a SoftReference. I wasn't able to trace all references in the object graph regarding TagScript objects (there are 21k+ of them!) so I am not sure if there are any other references that connect the TagScripts to a GC root through a permanent refernce, like a static field in some class. I cant tell you it's a dog's work to do that by hand. If I only had a proper memory analysys tool! Alas all the good ones are commercial... On the other hand GC is not *required* to harvest these objects. Maybe it just didn't bother to harvest because it was not pressed for memory? In my test it had still many megabytes left. Maybe a better test would be running it with too few memory and checking what was left alive, despite the fact GC was desperate to find the space?
> At any rate, TagScript.tagHolder is suspicious, and if it's even not the culprit itself (could be if WeakRefernce implementation was broken) it is hiding the real problem (at least in the environment of my test). Note that the variable is not static, which means all TagScript object instances get tied to the Thread. I don't know if the possibility of using each instance concurrently from multiple threads is useful at all, especially that the instances have more state information, not only the tagHolder... Anyway, it's a wrong forum to discuss that. I also heard that Jelly was abandoned by the original author so there is a good chance that noone understands why this tagHolder thingy was introduced and what it does...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Closed: (MAVEN-1294) Maven is leaking much memory
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MAVEN-1294?page=all ]
Brett Porter closed MAVEN-1294:
-------------------------------
Fix Version: 1.1-beta-1
Resolution: Fixed
we've upgraded to Jelly 1.0-rc2 on Maven 1.1's dev branch. It wouldn't work with the 1.0.x branch.
> Maven is leaking much memory
> ----------------------------
>
> Key: MAVEN-1294
> URL: http://jira.codehaus.org/browse/MAVEN-1294
> Project: maven
> Type: Bug
> Components: jelly/ant integration
> Versions: 1.0-rc3
> Reporter: Rafal Krzewski
> Fix For: 1.1-beta-1
> Attachments: sites2.xls
>
>
> The test was done as follows. I took ledge-components project http://objectledge.org/modules/ledge-components, cleaned out the target, then ran statcvs goal. Statcvs generated 160 xdoc documents, 2.2MB of markup total. Then I ran xdocs goal, to tranfrom the xdocs into html, under hprof profiler build into JDK. Settings were: allocation site analysis, heap dump at exit, stacktrace depth 16. The run took about 65 minutes on Athlon 1700XP, 512 phisical RAM. Maximum memory usage was 530MB virtual/240MB resident during the run, it surged some additional 300MB both during heap dump. The generated dump file is about 350MB, 23MB when bzip2 compressed.
> I'm going to post an xls file on Jira containg the statistics of objects allocated, and objects alive at the time of VM generation. From what I'm seeing none of the Jelly TagScript objects created gets ever garbage collected. Wading through the heap dump I was able to determine that all of them are connected to the GC root through the main Thread, and a ThreadLocal variable tagHolder declared in jelly's TagScript.java, line 115. Despite calls to tagHolder.set(null), the TagScript object stays connected to the Thread by means of ThreadLocal$ThreadLocalMap$Entry object which is a subclass of java.lang.ref.WeakReference. ThreadLocal implementation in JDK works like that. Now, AFAIK the GC is free to harvest all objects that are not connected to a GC root through hard reference, or a SoftReference. I wasn't able to trace all references in the object graph regarding TagScript objects (there are 21k+ of them!) so I am not sure if there are any other references that connect the TagScripts to a GC root through a permanent refernce, like a static field in some class. I cant tell you it's a dog's work to do that by hand. If I only had a proper memory analysys tool! Alas all the good ones are commercial... On the other hand GC is not *required* to harvest these objects. Maybe it just didn't bother to harvest because it was not pressed for memory? In my test it had still many megabytes left. Maybe a better test would be running it with too few memory and checking what was left alive, despite the fact GC was desperate to find the space?
> At any rate, TagScript.tagHolder is suspicious, and if it's even not the culprit itself (could be if WeakRefernce implementation was broken) it is hiding the real problem (at least in the environment of my test). Note that the variable is not static, which means all TagScript object instances get tied to the Thread. I don't know if the possibility of using each instance concurrently from multiple threads is useful at all, especially that the instances have more state information, not only the tagHolder... Anyway, it's a wrong forum to discuss that. I also heard that Jelly was abandoned by the original author so there is a good chance that noone understands why this tagHolder thingy was introduced and what it does...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MAVEN-1294) Maven is leaking much memory
Posted by ji...@codehaus.org.
The following comment has been added to this issue:
Author: Rafal Krzewski
Created: Wed, 26 May 2004 4:34 AM
Body:
Memory dump contents are available at http://caltha.pl/~rafal/java.hprof.txt.bz2 23MB, 350MB after uncompressing.
---------------------------------------------------------------------
View this comment:
http://jira.codehaus.org/browse/MAVEN-1294?page=comments#action_20077
---------------------------------------------------------------------
View the issue:
http://jira.codehaus.org/browse/MAVEN-1294
Here is an overview of the issue:
---------------------------------------------------------------------
Key: MAVEN-1294
Summary: Maven is leaking much memory
Type: Bug
Status: Unassigned
Priority: Major
Original Estimate: Unknown
Time Spent: Unknown
Remaining: Unknown
Project: maven
Components:
jelly/ant integration
Versions:
1.0-rc3
Assignee:
Reporter: Rafal Krzewski
Created: Wed, 26 May 2004 4:13 AM
Updated: Wed, 26 May 2004 4:34 AM
Description:
The test was done as follows. I took ledge-components project http://objectledge.org/modules/ledge-components, cleaned out the target, then ran statcvs goal. Statcvs generated 160 xdoc documents, 2.2MB of markup total. Then I ran xdocs goal, to tranfrom the xdocs into html, under hprof profiler build into JDK. Settings were: allocation site analysis, heap dump at exit, stacktrace depth 16. The run took about 65 minutes on Athlon 1700XP, 512 phisical RAM. Maximum memory usage was 530MB virtual/240MB resident during the run, it surged some additional 300MB both during heap dump. The generated dump file is about 350MB, 23MB when bzip2 compressed.
I'm going to post an xls file on Jira containg the statistics of objects allocated, and objects alive at the time of VM generation. From what I'm seeing none of the Jelly TagScript objects created gets ever garbage collected. Wading through the heap dump I was able to determine that all of them are connected to the GC root through the main Thread, and a ThreadLocal variable tagHolder declared in jelly's TagScript.java, line 115. Despite calls to tagHolder.set(null), the TagScript object stays connected to the Thread by means of ThreadLocal$ThreadLocalMap$Entry object which is a subclass of java.lang.ref.WeakReference. ThreadLocal implementation in JDK works like that. Now, AFAIK the GC is free to harvest all objects that are not connected to a GC root through hard reference, or a SoftReference. I wasn't able to trace all references in the object graph regarding TagScript objects (there are 21k+ of them!) so I am not sure if there are any other references that connect the TagScripts to a GC root through a permanent refernce, like a static field in some class. I cant tell you it's a dog's work to do that by hand. If I only had a proper memory analysys tool! Alas all the good ones are commercial... On the other hand GC is not *required* to harvest these objects. Maybe it just didn't bother to harvest because it was not pressed for memory? In my test it had still many megabytes left. Maybe a better test would be running it with too few memory and checking what was left alive, despite the fact GC was desperate to find the space?
At any rate, TagScript.tagHolder is suspicious, and if it's even not the culprit itself (could be if WeakRefernce implementation was broken) it is hiding the real problem (at least in the environment of my test). Note that the variable is not static, which means all TagScript object instances get tied to the Thread. I don't know if the possibility of using each instance concurrently from multiple threads is useful at all, especially that the instances have more state information, not only the tagHolder... Anyway, it's a wrong forum to discuss that. I also heard that Jelly was abandoned by the original author so there is a good chance that noone understands why this tagHolder thingy was introduced and what it does...
---------------------------------------------------------------------
JIRA INFORMATION:
This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MAVEN-1294) Maven is leaking much memory
Posted by "Mykel Alvis (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MAVEN-1294?page=comments#action_30545 ]
Mykel Alvis commented on MAVEN-1294:
------------------------------------
I'm having this issue using the site plugin on a very large source code base. I posted to users@maven.apache.org so this comment may be irrelevant.
> Maven is leaking much memory
> ----------------------------
>
> Key: MAVEN-1294
> URL: http://jira.codehaus.org/browse/MAVEN-1294
> Project: maven
> Type: Bug
> Components: jelly/ant integration
> Versions: 1.0-rc3
> Reporter: Rafal Krzewski
> Attachments: sites2.xls
>
>
> The test was done as follows. I took ledge-components project http://objectledge.org/modules/ledge-components, cleaned out the target, then ran statcvs goal. Statcvs generated 160 xdoc documents, 2.2MB of markup total. Then I ran xdocs goal, to tranfrom the xdocs into html, under hprof profiler build into JDK. Settings were: allocation site analysis, heap dump at exit, stacktrace depth 16. The run took about 65 minutes on Athlon 1700XP, 512 phisical RAM. Maximum memory usage was 530MB virtual/240MB resident during the run, it surged some additional 300MB both during heap dump. The generated dump file is about 350MB, 23MB when bzip2 compressed.
> I'm going to post an xls file on Jira containg the statistics of objects allocated, and objects alive at the time of VM generation. From what I'm seeing none of the Jelly TagScript objects created gets ever garbage collected. Wading through the heap dump I was able to determine that all of them are connected to the GC root through the main Thread, and a ThreadLocal variable tagHolder declared in jelly's TagScript.java, line 115. Despite calls to tagHolder.set(null), the TagScript object stays connected to the Thread by means of ThreadLocal$ThreadLocalMap$Entry object which is a subclass of java.lang.ref.WeakReference. ThreadLocal implementation in JDK works like that. Now, AFAIK the GC is free to harvest all objects that are not connected to a GC root through hard reference, or a SoftReference. I wasn't able to trace all references in the object graph regarding TagScript objects (there are 21k+ of them!) so I am not sure if there are any other references that connect the TagScripts to a GC root through a permanent refernce, like a static field in some class. I cant tell you it's a dog's work to do that by hand. If I only had a proper memory analysys tool! Alas all the good ones are commercial... On the other hand GC is not *required* to harvest these objects. Maybe it just didn't bother to harvest because it was not pressed for memory? In my test it had still many megabytes left. Maybe a better test would be running it with too few memory and checking what was left alive, despite the fact GC was desperate to find the space?
> At any rate, TagScript.tagHolder is suspicious, and if it's even not the culprit itself (could be if WeakRefernce implementation was broken) it is hiding the real problem (at least in the environment of my test). Note that the variable is not static, which means all TagScript object instances get tied to the Thread. I don't know if the possibility of using each instance concurrently from multiple threads is useful at all, especially that the instances have more state information, not only the tagHolder... Anyway, it's a wrong forum to discuss that. I also heard that Jelly was abandoned by the original author so there is a good chance that noone understands why this tagHolder thingy was introduced and what it does...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MAVEN-1294) Maven is leaking much memory
Posted by "Felipe Leme (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MAVEN-1294?page=comments#action_39133 ]
Felipe Leme commented on MAVEN-1294:
------------------------------------
Brett,
Why wouldn't it work? I mean, I was planning to change a local Maven 1.0.2 installation to Jelly 1.0 in order to fix this issue, but it wouldn't work, right?
So, the only solution is to use Maven 1.1 from CVS then?
-- Felipe
> Maven is leaking much memory
> ----------------------------
>
> Key: MAVEN-1294
> URL: http://jira.codehaus.org/browse/MAVEN-1294
> Project: maven
> Type: Bug
> Components: jelly/ant integration
> Versions: 1.0-rc3
> Reporter: Rafal Krzewski
> Fix For: 1.1-beta-1
> Attachments: sites2.xls
>
>
> The test was done as follows. I took ledge-components project http://objectledge.org/modules/ledge-components, cleaned out the target, then ran statcvs goal. Statcvs generated 160 xdoc documents, 2.2MB of markup total. Then I ran xdocs goal, to tranfrom the xdocs into html, under hprof profiler build into JDK. Settings were: allocation site analysis, heap dump at exit, stacktrace depth 16. The run took about 65 minutes on Athlon 1700XP, 512 phisical RAM. Maximum memory usage was 530MB virtual/240MB resident during the run, it surged some additional 300MB both during heap dump. The generated dump file is about 350MB, 23MB when bzip2 compressed.
> I'm going to post an xls file on Jira containg the statistics of objects allocated, and objects alive at the time of VM generation. From what I'm seeing none of the Jelly TagScript objects created gets ever garbage collected. Wading through the heap dump I was able to determine that all of them are connected to the GC root through the main Thread, and a ThreadLocal variable tagHolder declared in jelly's TagScript.java, line 115. Despite calls to tagHolder.set(null), the TagScript object stays connected to the Thread by means of ThreadLocal$ThreadLocalMap$Entry object which is a subclass of java.lang.ref.WeakReference. ThreadLocal implementation in JDK works like that. Now, AFAIK the GC is free to harvest all objects that are not connected to a GC root through hard reference, or a SoftReference. I wasn't able to trace all references in the object graph regarding TagScript objects (there are 21k+ of them!) so I am not sure if there are any other references that connect the TagScripts to a GC root through a permanent refernce, like a static field in some class. I cant tell you it's a dog's work to do that by hand. If I only had a proper memory analysys tool! Alas all the good ones are commercial... On the other hand GC is not *required* to harvest these objects. Maybe it just didn't bother to harvest because it was not pressed for memory? In my test it had still many megabytes left. Maybe a better test would be running it with too few memory and checking what was left alive, despite the fact GC was desperate to find the space?
> At any rate, TagScript.tagHolder is suspicious, and if it's even not the culprit itself (could be if WeakRefernce implementation was broken) it is hiding the real problem (at least in the environment of my test). Note that the variable is not static, which means all TagScript object instances get tied to the Thread. I don't know if the possibility of using each instance concurrently from multiple threads is useful at all, especially that the instances have more state information, not only the tagHolder... Anyway, it's a wrong forum to discuss that. I also heard that Jelly was abandoned by the original author so there is a good chance that noone understands why this tagHolder thingy was introduced and what it does...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MAVEN-1294) Maven is leaking much memory
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MAVEN-1294?page=comments#action_31529 ]
Brett Porter commented on MAVEN-1294:
-------------------------------------
just FYI - this has been mostly fixed by the Jelly team and Maven 1.1 (SVN) is better (about half), but not perfect. They are still looking into it.
> Maven is leaking much memory
> ----------------------------
>
> Key: MAVEN-1294
> URL: http://jira.codehaus.org/browse/MAVEN-1294
> Project: maven
> Type: Bug
> Components: jelly/ant integration
> Versions: 1.0-rc3
> Reporter: Rafal Krzewski
> Attachments: sites2.xls
>
>
> The test was done as follows. I took ledge-components project http://objectledge.org/modules/ledge-components, cleaned out the target, then ran statcvs goal. Statcvs generated 160 xdoc documents, 2.2MB of markup total. Then I ran xdocs goal, to tranfrom the xdocs into html, under hprof profiler build into JDK. Settings were: allocation site analysis, heap dump at exit, stacktrace depth 16. The run took about 65 minutes on Athlon 1700XP, 512 phisical RAM. Maximum memory usage was 530MB virtual/240MB resident during the run, it surged some additional 300MB both during heap dump. The generated dump file is about 350MB, 23MB when bzip2 compressed.
> I'm going to post an xls file on Jira containg the statistics of objects allocated, and objects alive at the time of VM generation. From what I'm seeing none of the Jelly TagScript objects created gets ever garbage collected. Wading through the heap dump I was able to determine that all of them are connected to the GC root through the main Thread, and a ThreadLocal variable tagHolder declared in jelly's TagScript.java, line 115. Despite calls to tagHolder.set(null), the TagScript object stays connected to the Thread by means of ThreadLocal$ThreadLocalMap$Entry object which is a subclass of java.lang.ref.WeakReference. ThreadLocal implementation in JDK works like that. Now, AFAIK the GC is free to harvest all objects that are not connected to a GC root through hard reference, or a SoftReference. I wasn't able to trace all references in the object graph regarding TagScript objects (there are 21k+ of them!) so I am not sure if there are any other references that connect the TagScripts to a GC root through a permanent refernce, like a static field in some class. I cant tell you it's a dog's work to do that by hand. If I only had a proper memory analysys tool! Alas all the good ones are commercial... On the other hand GC is not *required* to harvest these objects. Maybe it just didn't bother to harvest because it was not pressed for memory? In my test it had still many megabytes left. Maybe a better test would be running it with too few memory and checking what was left alive, despite the fact GC was desperate to find the space?
> At any rate, TagScript.tagHolder is suspicious, and if it's even not the culprit itself (could be if WeakRefernce implementation was broken) it is hiding the real problem (at least in the environment of my test). Note that the variable is not static, which means all TagScript object instances get tied to the Thread. I don't know if the possibility of using each instance concurrently from multiple threads is useful at all, especially that the instances have more state information, not only the tagHolder... Anyway, it's a wrong forum to discuss that. I also heard that Jelly was abandoned by the original author so there is a good chance that noone understands why this tagHolder thingy was introduced and what it does...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[jira] Commented: (MAVEN-1294) Maven is leaking much memory
Posted by "Rafal Krzewski (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MAVEN-1294?page=comments#action_38868 ]
Rafal Krzewski commented on MAVEN-1294:
---------------------------------------
This is excelent news!
Is there any chance to take advantage of fixed Jelly in Maven 1.0.x stream?
> Maven is leaking much memory
> ----------------------------
>
> Key: MAVEN-1294
> URL: http://jira.codehaus.org/browse/MAVEN-1294
> Project: maven
> Type: Bug
> Components: jelly/ant integration
> Versions: 1.0-rc3
> Reporter: Rafal Krzewski
> Attachments: sites2.xls
>
>
> The test was done as follows. I took ledge-components project http://objectledge.org/modules/ledge-components, cleaned out the target, then ran statcvs goal. Statcvs generated 160 xdoc documents, 2.2MB of markup total. Then I ran xdocs goal, to tranfrom the xdocs into html, under hprof profiler build into JDK. Settings were: allocation site analysis, heap dump at exit, stacktrace depth 16. The run took about 65 minutes on Athlon 1700XP, 512 phisical RAM. Maximum memory usage was 530MB virtual/240MB resident during the run, it surged some additional 300MB both during heap dump. The generated dump file is about 350MB, 23MB when bzip2 compressed.
> I'm going to post an xls file on Jira containg the statistics of objects allocated, and objects alive at the time of VM generation. From what I'm seeing none of the Jelly TagScript objects created gets ever garbage collected. Wading through the heap dump I was able to determine that all of them are connected to the GC root through the main Thread, and a ThreadLocal variable tagHolder declared in jelly's TagScript.java, line 115. Despite calls to tagHolder.set(null), the TagScript object stays connected to the Thread by means of ThreadLocal$ThreadLocalMap$Entry object which is a subclass of java.lang.ref.WeakReference. ThreadLocal implementation in JDK works like that. Now, AFAIK the GC is free to harvest all objects that are not connected to a GC root through hard reference, or a SoftReference. I wasn't able to trace all references in the object graph regarding TagScript objects (there are 21k+ of them!) so I am not sure if there are any other references that connect the TagScripts to a GC root through a permanent refernce, like a static field in some class. I cant tell you it's a dog's work to do that by hand. If I only had a proper memory analysys tool! Alas all the good ones are commercial... On the other hand GC is not *required* to harvest these objects. Maybe it just didn't bother to harvest because it was not pressed for memory? In my test it had still many megabytes left. Maybe a better test would be running it with too few memory and checking what was left alive, despite the fact GC was desperate to find the space?
> At any rate, TagScript.tagHolder is suspicious, and if it's even not the culprit itself (could be if WeakRefernce implementation was broken) it is hiding the real problem (at least in the environment of my test). Note that the variable is not static, which means all TagScript object instances get tied to the Thread. I don't know if the possibility of using each instance concurrently from multiple threads is useful at all, especially that the instances have more state information, not only the tagHolder... Anyway, it's a wrong forum to discuss that. I also heard that Jelly was abandoned by the original author so there is a good chance that noone understands why this tagHolder thingy was introduced and what it does...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org