You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Ryan Sonnek (JIRA)" <ji...@codehaus.org> on 2008/04/14 21:51:58 UTC
[jira] Created: (MCLEAN-32) Out of Memory Error cleaning target
directory with *large* number of sub-directories
Out of Memory Error cleaning target directory with *large* number of sub-directories
------------------------------------------------------------------------------------
Key: MCLEAN-32
URL: http://jira.codehaus.org/browse/MCLEAN-32
Project: Maven 2.x Clean Plugin
Issue Type: Bug
Affects Versions: 2.2
Environment: maven-2.0.8, solaris, jdk 1.6.0_05
Reporter: Ryan Sonnek
Priority: Critical
Attachments: output.txt
running mvn clean on a target directory with a *large* number of sub directories causes an OutOfMemoryError. The plexus DirectoryScanner seems to be the major culprit from the stacktrace.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MCLEAN-32) Out of Memory Error cleaning target
directory with *large* number of sub-directories
Posted by "Benjamin Bentmann (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MCLEAN-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=206099#action_206099 ]
Benjamin Bentmann commented on MCLEAN-32:
-----------------------------------------
Also, my testing indicates that {{followSymlinks=false}} requires more memory than {{followSymlinks=true}}. So using the latter option value for projects with huge directories but knowingly no symlinks could also help.
> Out of Memory Error cleaning target directory with *large* number of sub-directories
> ------------------------------------------------------------------------------------
>
> Key: MCLEAN-32
> URL: http://jira.codehaus.org/browse/MCLEAN-32
> Project: Maven 2.x Clean Plugin
> Issue Type: Bug
> Affects Versions: 2.2
> Environment: maven-2.0.8, solaris, jdk 1.6.0_05
> Reporter: Ryan Sonnek
> Assignee: Benjamin Bentmann
> Priority: Critical
> Fix For: 2.4
>
> Attachments: output.txt
>
>
> running mvn clean on a target directory with a *large* number of sub directories causes an OutOfMemoryError. The plexus DirectoryScanner seems to be the major culprit from the stacktrace.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MCLEAN-32) Out of Memory Error cleaning target
directory with *large* number of sub-directories
Posted by "Ryan Sonnek (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MCLEAN-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=130878#action_130878 ]
Ryan Sonnek commented on MCLEAN-32:
-----------------------------------
We're running with 512M for the JVM heap size which should be *more* than enough for this operation.
The question is why is the directory scanner caching references to files so aggressively? This seems *very* inefficient.
> Out of Memory Error cleaning target directory with *large* number of sub-directories
> ------------------------------------------------------------------------------------
>
> Key: MCLEAN-32
> URL: http://jira.codehaus.org/browse/MCLEAN-32
> Project: Maven 2.x Clean Plugin
> Issue Type: Bug
> Affects Versions: 2.2
> Environment: maven-2.0.8, solaris, jdk 1.6.0_05
> Reporter: Ryan Sonnek
> Priority: Critical
> Attachments: output.txt
>
>
> running mvn clean on a target directory with a *large* number of sub directories causes an OutOfMemoryError. The plexus DirectoryScanner seems to be the major culprit from the stacktrace.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MCLEAN-32) Out of Memory Error cleaning target
directory with *large* number of sub-directories
Posted by "Vincent Siveton (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MCLEAN-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=137878#action_137878 ]
Vincent Siveton commented on MCLEAN-32:
---------------------------------------
I created a dir in target with 100 sub dirs in 3 levels so 100 * 100 *100 sub dirs. The plugin is able to clean the target. with MAVEN_OPTS=-Xms256m -Xmx256m
So what is your large number of sub dir? Could you send us a test case with the env to reproduce it?
> Out of Memory Error cleaning target directory with *large* number of sub-directories
> ------------------------------------------------------------------------------------
>
> Key: MCLEAN-32
> URL: http://jira.codehaus.org/browse/MCLEAN-32
> Project: Maven 2.x Clean Plugin
> Issue Type: Bug
> Affects Versions: 2.2
> Environment: maven-2.0.8, solaris, jdk 1.6.0_05
> Reporter: Ryan Sonnek
> Priority: Critical
> Attachments: output.txt
>
>
> running mvn clean on a target directory with a *large* number of sub directories causes an OutOfMemoryError. The plexus DirectoryScanner seems to be the major culprit from the stacktrace.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MCLEAN-32) Out of Memory Error cleaning target
directory with *large* number of sub-directories
Posted by "Bouiaw (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MCLEAN-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=174025#action_174025 ]
Bouiaw commented on MCLEAN-32:
------------------------------
It seems to be linked with number AND the size of the data to clean. I don't know if it load in memory all the content to delete, but try to copy a large amount of data in target, and run mvn clean ...
> Out of Memory Error cleaning target directory with *large* number of sub-directories
> ------------------------------------------------------------------------------------
>
> Key: MCLEAN-32
> URL: http://jira.codehaus.org/browse/MCLEAN-32
> Project: Maven 2.x Clean Plugin
> Issue Type: Bug
> Affects Versions: 2.2
> Environment: maven-2.0.8, solaris, jdk 1.6.0_05
> Reporter: Ryan Sonnek
> Priority: Critical
> Attachments: output.txt
>
>
> running mvn clean on a target directory with a *large* number of sub directories causes an OutOfMemoryError. The plexus DirectoryScanner seems to be the major culprit from the stacktrace.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (MCLEAN-32) Out of Memory Error cleaning target
directory with *large* number of sub-directories
Posted by "Benjamin Bentmann (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MCLEAN-32?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Bentmann closed MCLEAN-32.
-----------------------------------
Resolution: Fixed
Fix Version/s: 2.4
Assignee: Benjamin Bentmann
Fixed in [r897682|http://svn.apache.org/viewvc?view=revision&revision=897682].
On a test structure that resembles a binary tree of depth 14 with two files at each level besides the two child directories, I observed the following improvements:
{noformat}
[INFO] --- maven-clean-plugin:2.3:clean (default-clean) @ test ---
[INFO] Deleting file set: M:\test\. (included: [], excluded: [pom.xml])
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3:50.797s
[INFO] Finished at: Sun Jan 10 18:34:49 CET 2010
[INFO] Final Memory: 1M/42M
[INFO] ------------------------------------------------------------------------
{noformat}
{noformat}
[INFO] --- maven-clean-plugin:2.4-SNAPSHOT:clean (default-clean) @ test ---
[INFO] Deleting M:\test (includes = [], excludes = [pom.xml])
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 41.938s
[INFO] Finished at: Sun Jan 10 18:39:39 CET 2010
[INFO] Final Memory: 1M/4M
[INFO] ------------------------------------------------------------------------
{noformat}
i.e. the memory used dropped from 42 MB to 4MB.
> Out of Memory Error cleaning target directory with *large* number of sub-directories
> ------------------------------------------------------------------------------------
>
> Key: MCLEAN-32
> URL: http://jira.codehaus.org/browse/MCLEAN-32
> Project: Maven 2.x Clean Plugin
> Issue Type: Bug
> Affects Versions: 2.2
> Environment: maven-2.0.8, solaris, jdk 1.6.0_05
> Reporter: Ryan Sonnek
> Assignee: Benjamin Bentmann
> Priority: Critical
> Fix For: 2.4
>
> Attachments: output.txt
>
>
> running mvn clean on a target directory with a *large* number of sub directories causes an OutOfMemoryError. The plexus DirectoryScanner seems to be the major culprit from the stacktrace.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MCLEAN-32) Out of Memory Error cleaning target
directory with *large* number of sub-directories
Posted by "Vincent Siveton (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MCLEAN-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=130818#action_130818 ]
Vincent Siveton commented on MCLEAN-32:
---------------------------------------
What is your MAVEN_OPTS? Did you try to increase the memory?
> Out of Memory Error cleaning target directory with *large* number of sub-directories
> ------------------------------------------------------------------------------------
>
> Key: MCLEAN-32
> URL: http://jira.codehaus.org/browse/MCLEAN-32
> Project: Maven 2.x Clean Plugin
> Issue Type: Bug
> Affects Versions: 2.2
> Environment: maven-2.0.8, solaris, jdk 1.6.0_05
> Reporter: Ryan Sonnek
> Priority: Critical
> Attachments: output.txt
>
>
> running mvn clean on a target directory with a *large* number of sub directories causes an OutOfMemoryError. The plexus DirectoryScanner seems to be the major culprit from the stacktrace.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira