You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Rakesh Arora (JIRA)" <ji...@codehaus.org> on 2009/06/30 17:16:05 UTC
[jira] Created: (WAGON-272) OutOfMemoryError when coping large
files
OutOfMemoryError when coping large files
----------------------------------------
Key: WAGON-272
URL: http://jira.codehaus.org/browse/WAGON-272
Project: Maven Wagon
Issue Type: Bug
Components: wagon-http-lightweight
Affects Versions: 1.0-beta-5
Reporter: Rakesh Arora
Getting OutOfMemory issue when coping/deploying large files. Check the stack trace below.
We traced the issue to wagon-http-lightweight using PosterOutputStream/ByteArrayOutputStream.write(). This method consumes lot of memory when dealing with large files as reported here:
https://issues.alfresco.com/jira/browse/ETHREEOH-974
In our case, we are trying to deploy a around 600M file and have set the maximum heap space to 1024M (-Xmx1024m). We are still running out of memory. We are using maven 2.0.8 but had the same issue when tried with 2.1.0 version.
;-------------------------------------STACK TRACE-----
[DEBUG] -- end configuration -- [INFO] [deploy:deploy-file]
Uploading: http://<repo_url>/<group>/foo/1.0/foo-1.0.zip
[INFO]
------------------------------------------------------------------------
[ERROR] FATAL ERROR
[INFO]
------------------------------------------------------------------------
[INFO] Java heap space
[INFO]
------------------------------------------------------------------------
[DEBUG] Trace
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2786)
at
java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
at
sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
at
org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:338)
at
org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:305)
at
org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:267)
at
org.apache.maven.wagon.AbstractWagon.putTransfer(AbstractWagon.java:2
38)
at org.apache.maven.wagon.StreamWagon.put(StreamWagon.java:143)
at
org.apache.maven.wagon.providers.http.LightweightHttpWagon.put(Lightw
eightHttpWagon.java:148)
at
org.apache.maven.artifact.manager.DefaultWagonManager.putRemoteFile(D
efaultWagonManager.java:237)
at
org.apache.maven.artifact.manager.DefaultWagonManager.putArtifact(Def
aultWagonManager.java:153)
at
org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(Def
aultArtifactDeployer.java:80)
at
org.apache.maven.plugin.deploy.DeployFileMojo.execute(DeployFileMojo.
java:240)
at
org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPlugi
nManager.java:447)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(Defa
ultLifecycleExecutor.java:539)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandalone
Goal(DefaultLifecycleExecutor.java:493)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(Defau
ltLifecycleExecutor.java:463)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHan
dleFailures(DefaultLifecycleExecutor.java:311)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegmen
ts(DefaultLifecycleExecutor.java:224)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLi
fecycleExecutor.java:143)
at
org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:333)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:126)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:282)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at
org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 1 minute 22 seconds
[INFO] Finished at: Fri Jun 26 15:10:36 EDT 2009 [INFO] Final Memory: 4M/1016M [INFO]
------------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (WAGON-272) OutOfMemoryError when coping large
files
Posted by "Rakesh Arora (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/WAGON-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=182004#action_182004 ]
Rakesh Arora commented on WAGON-272:
------------------------------------
Workaround is to use dav:http protocol instead of http for deploying artifacts using maven.
> OutOfMemoryError when coping large files
> ----------------------------------------
>
> Key: WAGON-272
> URL: http://jira.codehaus.org/browse/WAGON-272
> Project: Maven Wagon
> Issue Type: Bug
> Components: wagon-http-lightweight
> Affects Versions: 1.0-beta-5
> Reporter: Rakesh Arora
>
> Getting OutOfMemory issue when coping/deploying large files. Check the stack trace below.
> We traced the issue to wagon-http-lightweight using PosterOutputStream/ByteArrayOutputStream.write(). This method consumes lot of memory when dealing with large files as reported here:
> https://issues.alfresco.com/jira/browse/ETHREEOH-974
> In our case, we are trying to deploy a around 600M file and have set the maximum heap space to 1024M (-Xmx1024m). We are still running out of memory. We are using maven 2.0.8 but had the same issue when tried with 2.1.0 version.
> ;-------------------------------------STACK TRACE-----
> [DEBUG] -- end configuration -- [INFO] [deploy:deploy-file]
> Uploading: http://<repo_url>/<group>/foo/1.0/foo-1.0.zip
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] FATAL ERROR
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Java heap space
> [INFO]
> ------------------------------------------------------------------------
> [DEBUG] Trace
> java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:2786)
> at
> java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
> at
> sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
> at
> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:338)
> at
> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:305)
> at
> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:267)
> at
> org.apache.maven.wagon.AbstractWagon.putTransfer(AbstractWagon.java:2
> 38)
> at org.apache.maven.wagon.StreamWagon.put(StreamWagon.java:143)
> at
> org.apache.maven.wagon.providers.http.LightweightHttpWagon.put(Lightw
> eightHttpWagon.java:148)
> at
> org.apache.maven.artifact.manager.DefaultWagonManager.putRemoteFile(D
> efaultWagonManager.java:237)
> at
> org.apache.maven.artifact.manager.DefaultWagonManager.putArtifact(Def
> aultWagonManager.java:153)
> at
> org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(Def
> aultArtifactDeployer.java:80)
> at
> org.apache.maven.plugin.deploy.DeployFileMojo.execute(DeployFileMojo.
> java:240)
> at
> org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPlugi
> nManager.java:447)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(Defa
> ultLifecycleExecutor.java:539)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandalone
> Goal(DefaultLifecycleExecutor.java:493)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(Defau
> ltLifecycleExecutor.java:463)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHan
> dleFailures(DefaultLifecycleExecutor.java:311)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegmen
> ts(DefaultLifecycleExecutor.java:224)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLi
> fecycleExecutor.java:143)
> at
> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:333)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:126)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:282)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
> java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> sorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at
> org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 1 minute 22 seconds
> [INFO] Finished at: Fri Jun 26 15:10:36 EDT 2009 [INFO] Final Memory: 4M/1016M [INFO]
> ------------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (WAGON-272) OutOfMemoryError when coping large files
Posted by "Brett Porter (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/WAGON-272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brett Porter closed WAGON-272.
------------------------------
Assignee: Brett Porter
Resolution: Won't Fix
this is a limitation of the JDK classes. You can use the HTTPClient version (instructions on the Maven site, requires Maven 2.2.1), or dav:http as you've said
> OutOfMemoryError when coping large files
> ----------------------------------------
>
> Key: WAGON-272
> URL: http://jira.codehaus.org/browse/WAGON-272
> Project: Maven Wagon
> Issue Type: Bug
> Components: wagon-http-lightweight
> Affects Versions: 1.0-beta-5
> Reporter: Rakesh Arora
> Assignee: Brett Porter
>
> Getting OutOfMemory issue when coping/deploying large files. Check the stack trace below.
> We traced the issue to wagon-http-lightweight using PosterOutputStream/ByteArrayOutputStream.write(). This method consumes lot of memory when dealing with large files as reported here:
> https://issues.alfresco.com/jira/browse/ETHREEOH-974
> In our case, we are trying to deploy a around 600M file and have set the maximum heap space to 1024M (-Xmx1024m). We are still running out of memory. We are using maven 2.0.8 but had the same issue when tried with 2.1.0 version.
> ;-------------------------------------STACK TRACE-----
> [DEBUG] -- end configuration -- [INFO] [deploy:deploy-file]
> Uploading: http://<repo_url>/<group>/foo/1.0/foo-1.0.zip
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] FATAL ERROR
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Java heap space
> [INFO]
> ------------------------------------------------------------------------
> [DEBUG] Trace
> java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:2786)
> at
> java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
> at
> sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
> at
> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:338)
> at
> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:305)
> at
> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:267)
> at
> org.apache.maven.wagon.AbstractWagon.putTransfer(AbstractWagon.java:2
> 38)
> at org.apache.maven.wagon.StreamWagon.put(StreamWagon.java:143)
> at
> org.apache.maven.wagon.providers.http.LightweightHttpWagon.put(Lightw
> eightHttpWagon.java:148)
> at
> org.apache.maven.artifact.manager.DefaultWagonManager.putRemoteFile(D
> efaultWagonManager.java:237)
> at
> org.apache.maven.artifact.manager.DefaultWagonManager.putArtifact(Def
> aultWagonManager.java:153)
> at
> org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(Def
> aultArtifactDeployer.java:80)
> at
> org.apache.maven.plugin.deploy.DeployFileMojo.execute(DeployFileMojo.
> java:240)
> at
> org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPlugi
> nManager.java:447)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(Defa
> ultLifecycleExecutor.java:539)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandalone
> Goal(DefaultLifecycleExecutor.java:493)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(Defau
> ltLifecycleExecutor.java:463)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHan
> dleFailures(DefaultLifecycleExecutor.java:311)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegmen
> ts(DefaultLifecycleExecutor.java:224)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLi
> fecycleExecutor.java:143)
> at
> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:333)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:126)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:282)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
> java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> sorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at
> org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 1 minute 22 seconds
> [INFO] Finished at: Fri Jun 26 15:10:36 EDT 2009 [INFO] Final Memory: 4M/1016M [INFO]
> ------------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (WAGON-272) OutOfMemoryError when coping large files
Posted by "Martin Todorov (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/WAGON-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=315321#comment-315321 ]
Martin Todorov commented on WAGON-272:
--------------------------------------
A workaround for this is illustrated [here|http://stackoverflow.com/questions/13767525/my-artifact-seems-to-big-to-be-deployed-by-maven-what-can-i-do].
> OutOfMemoryError when coping large files
> ----------------------------------------
>
> Key: WAGON-272
> URL: https://jira.codehaus.org/browse/WAGON-272
> Project: Maven Wagon
> Issue Type: Bug
> Components: wagon-http-lightweight
> Affects Versions: 1.0-beta-5
> Reporter: Rakesh Arora
> Assignee: Brett Porter
>
> Getting OutOfMemory issue when coping/deploying large files. Check the stack trace below.
> We traced the issue to wagon-http-lightweight using PosterOutputStream/ByteArrayOutputStream.write(). This method consumes lot of memory when dealing with large files as reported here:
> https://issues.alfresco.com/jira/browse/ETHREEOH-974
> In our case, we are trying to deploy a around 600M file and have set the maximum heap space to 1024M (-Xmx1024m). We are still running out of memory. We are using maven 2.0.8 but had the same issue when tried with 2.1.0 version.
> ;-------------------------------------STACK TRACE-----
> [DEBUG] -- end configuration -- [INFO] [deploy:deploy-file]
> Uploading: http://<repo_url>/<group>/foo/1.0/foo-1.0.zip
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] FATAL ERROR
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Java heap space
> [INFO]
> ------------------------------------------------------------------------
> [DEBUG] Trace
> java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:2786)
> at
> java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
> at
> sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
> at
> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:338)
> at
> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:305)
> at
> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:267)
> at
> org.apache.maven.wagon.AbstractWagon.putTransfer(AbstractWagon.java:2
> 38)
> at org.apache.maven.wagon.StreamWagon.put(StreamWagon.java:143)
> at
> org.apache.maven.wagon.providers.http.LightweightHttpWagon.put(Lightw
> eightHttpWagon.java:148)
> at
> org.apache.maven.artifact.manager.DefaultWagonManager.putRemoteFile(D
> efaultWagonManager.java:237)
> at
> org.apache.maven.artifact.manager.DefaultWagonManager.putArtifact(Def
> aultWagonManager.java:153)
> at
> org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(Def
> aultArtifactDeployer.java:80)
> at
> org.apache.maven.plugin.deploy.DeployFileMojo.execute(DeployFileMojo.
> java:240)
> at
> org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPlugi
> nManager.java:447)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(Defa
> ultLifecycleExecutor.java:539)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandalone
> Goal(DefaultLifecycleExecutor.java:493)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(Defau
> ltLifecycleExecutor.java:463)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHan
> dleFailures(DefaultLifecycleExecutor.java:311)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegmen
> ts(DefaultLifecycleExecutor.java:224)
> at
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLi
> fecycleExecutor.java:143)
> at
> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:333)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:126)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:282)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
> java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> sorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at
> org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 1 minute 22 seconds
> [INFO] Finished at: Fri Jun 26 15:10:36 EDT 2009 [INFO] Final Memory: 4M/1016M [INFO]
> ------------------------------------------------------------------------
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira