You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Ryan Connolly <ry...@gmail.com> on 2009/07/01 15:33:20 UTC
Re: OutOfMemoryError when deploying large files
Rakesh:
We are experiencing this very same issue. Would you mind emailing
me a pom snippet for the dav config you used to get around this? Your
assistance would be most appreciated.
Regards,
Ryan
On 6/30/09, Rakesh Arora <ra...@nortel.com> wrote:
> We have resolved the problem by using dav:http protocol instead of http
> protocol.
>
> I have raised a jira against wagon-http-lightweight:
> http://jira.codehaus.org/browse/WAGON-272
>
> -Rakesh
>
>
>
>> -----Original Message-----
>> From: Arora, Rakesh (CAR:9S00)
>> Sent: Friday, June 26, 2009 4:19 PM
>> To: users@maven.apache.org
>> Subject: OutOfMemoryError when deploying large files
>>
>> When deploying a large artifact we get the OutOfMemoryError
>> (please check the attached stack trace). This is similar to
>> error reported here:
>> http://www.mail-archive.com/users@maven.apache.org/msg99157.html
>>
>> We traced the issue to wagon-http-lightweight using
>> PosterOutputStream/ByteArrayOutputStream.write(). This method
>> consumes lot of memory when dealing with large files as reported here:
>> https://issues.alfresco.com/jira/browse/ETHREEOH-974
>>
>> In our case, we are trying to deploy a around 600M file and
>> have set the maximum heap space to 1024M (-Xmx1024m). We are
>> still running out of memory. We are using maven 2.0.8 but had
>> the same issue when tried with 2.1.0 version.
>>
>> Is there a way to use another Wagon provider (wagon-http ?)
>> that can deal better with large files? If yes, how can we
>> configure this?
>>
>> Should I raise a jira for this (against which component
>> maven-deploy-plugin or wagon-http-lightweight)?
>>
>> Thanks,
>> -Rakesh
>>
>>
>>
>> ;-------------------------------------STACK TRACE-----
>> [DEBUG] -- end configuration -- [INFO] [deploy:deploy-file]
>> Uploading: http://<repo_url>/<group>/foo/1.0/foo-1.0.zip
>> [INFO]
>> --------------------------------------------------------------
>> ----------
>> [ERROR] FATAL ERROR
>> [INFO]
>> --------------------------------------------------------------
>> ----------
>> [INFO] Java heap space
>> [INFO]
>> --------------------------------------------------------------
>> ----------
>> [DEBUG] Trace
>> java.lang.OutOfMemoryError: Java heap space
>> at java.util.Arrays.copyOf(Arrays.java:2786)
>> at
>> java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
>> at
>> sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
>>
>> at
>> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:338)
>>
>> at
>> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:305)
>>
>> at
>> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:267)
>>
>> at
>> org.apache.maven.wagon.AbstractWagon.putTransfer(AbstractWagon.java:2
>> 38)
>> at
>> org.apache.maven.wagon.StreamWagon.put(StreamWagon.java:143)
>> at
>> org.apache.maven.wagon.providers.http.LightweightHttpWagon.put(Lightw
>> eightHttpWagon.java:148)
>> at
>> org.apache.maven.artifact.manager.DefaultWagonManager.putRemoteFile(D
>> efaultWagonManager.java:237)
>> at
>> org.apache.maven.artifact.manager.DefaultWagonManager.putArtifact(Def
>> aultWagonManager.java:153)
>> at
>> org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(Def
>> aultArtifactDeployer.java:80)
>> at
>> org.apache.maven.plugin.deploy.DeployFileMojo.execute(DeployFileMojo.
>> java:240)
>> at
>> org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPlugi
>> nManager.java:447)
>> at
>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(Defa
>> ultLifecycleExecutor.java:539)
>> at
>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandalone
>> Goal(DefaultLifecycleExecutor.java:493)
>> at
>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(Defau
>> ltLifecycleExecutor.java:463)
>> at
>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHan
>> dleFailures(DefaultLifecycleExecutor.java:311)
>> at
>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegmen
>> ts(DefaultLifecycleExecutor.java:224)
>> at
>> org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLi
>> fecycleExecutor.java:143)
>> at
>> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:333)
>> at
>> org.apache.maven.DefaultMaven.execute(DefaultMaven.java:126)
>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:282)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
>> java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
>> sorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
>> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
>> at
>> org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
>>
>> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
>> [INFO]
>> --------------------------------------------------------------
>> ----------
>> [INFO] Total time: 1 minute 22 seconds
>> [INFO] Finished at: Fri Jun 26 15:10:36 EDT 2009 [INFO] Final
>> Memory: 4M/1016M [INFO]
>> --------------------------------------------------------------
>> ----------
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
--
Sent from my mobile device
®¥@N
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: OutOfMemoryError when deploying large files
Posted by Ryan Connolly <ry...@gmail.com>.
Thanks, Rakesh... I was just about to respond to let you know that I finally
got this worked out this morning. However, in addition to the change in
protocol in our distributionManagement elements I found that I needed to add
the following extensions to my parent pom:
<extensions>
<extension>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-webdav</artifactId>
<version>1.0-beta-2</version>
</extension>
</extensions>
Thanks again, I really appreciate you getting back to me on this!
-Ryan
On Thu, Jul 2, 2009 at 11:31 AM, Rakesh Arora <ra...@nortel.com>wrote:
> Ryan,
>
> Here is what we now have in the pom.xml to deploy large files
>
> <distributionManagement>
> <repository>
> <id>releases</id>
> <url>dav:http://<nexus_server>/content/repositories/releases</url>
> </repository>
> <snapshotRepository>
> <id>snapshots</id>
> <url>dav:http://<nexus_server>/content/repositories/snapshots</url>
> </snapshotRepository>
> </distributionManagement>
>
> So we have changed the protocol to dav:http from http.
>
> -Rakesh
>
>
>
> > -----Original Message-----
> > From: Ryan Connolly [mailto:rynam0@gmail.com]
> > Sent: Wednesday, July 01, 2009 9:33 AM
> > To: Maven Users List
> > Subject: Re: OutOfMemoryError when deploying large files
> >
> > Rakesh:
> > We are experiencing this very same issue. Would you mind
> > emailing me a pom snippet for the dav config you used to get
> > around this? Your assistance would be most appreciated.
> >
> > Regards,
> > Ryan
> >
> > On 6/30/09, Rakesh Arora <ra...@nortel.com> wrote:
> > > We have resolved the problem by using dav:http protocol instead of
> > > http protocol.
> > >
> > > I have raised a jira against wagon-http-lightweight:
> > > http://jira.codehaus.org/browse/WAGON-272
> > >
> > > -Rakesh
> > >
> > >
> > >
> > >> -----Original Message-----
> > >> From: Arora, Rakesh (CAR:9S00)
> > >> Sent: Friday, June 26, 2009 4:19 PM
> > >> To: users@maven.apache.org
> > >> Subject: OutOfMemoryError when deploying large files
> > >>
> > >> When deploying a large artifact we get the
> > OutOfMemoryError (please
> > >> check the attached stack trace). This is similar to error reported
> > >> here:
> > >> http://www.mail-archive.com/users@maven.apache.org/msg99157.html
> > >>
> > >> We traced the issue to wagon-http-lightweight using
> > >> PosterOutputStream/ByteArrayOutputStream.write(). This method
> > >> consumes lot of memory when dealing with large files as
> > reported here:
> > >> https://issues.alfresco.com/jira/browse/ETHREEOH-974
> > >>
> > >> In our case, we are trying to deploy a around 600M file
> > and have set
> > >> the maximum heap space to 1024M (-Xmx1024m). We are still
> > running out
> > >> of memory. We are using maven 2.0.8 but had the same issue
> > when tried
> > >> with 2.1.0 version.
> > >>
> > >> Is there a way to use another Wagon provider (wagon-http
> > ?) that can
> > >> deal better with large files? If yes, how can we configure this?
> > >>
> > >> Should I raise a jira for this (against which component
> > >> maven-deploy-plugin or wagon-http-lightweight)?
> > >>
> > >> Thanks,
> > >> -Rakesh
> > >>
> > >>
> > >>
> > >> ;-------------------------------------STACK TRACE-----
> > [DEBUG] -- end
> > >> configuration -- [INFO] [deploy:deploy-file]
> > >> Uploading: http://<repo_url>/<group>/foo/1.0/foo-1.0.zip
> > >> [INFO]
> > >> --------------------------------------------------------------
> > >> ----------
> > >> [ERROR] FATAL ERROR
> > >> [INFO]
> > >> --------------------------------------------------------------
> > >> ----------
> > >> [INFO] Java heap space
> > >> [INFO]
> > >> --------------------------------------------------------------
> > >> ----------
> > >> [DEBUG] Trace
> > >> java.lang.OutOfMemoryError: Java heap space
> > >> at java.util.Arrays.copyOf(Arrays.java:2786)
> > >> at
> > >> java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
> > >> at
> > >>
> > sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
> > >>
> > >> at
> > >>
> > org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:338)
> > >>
> > >> at
> > >>
> > org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:305)
> > >>
> > >> at
> > >>
> > org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:267)
> > >>
> > >> at
> > >>
> > org.apache.maven.wagon.AbstractWagon.putTransfer(AbstractWagon.java:2
> > >> 38)
> > >> at
> > >> org.apache.maven.wagon.StreamWagon.put(StreamWagon.java:143)
> > >> at
> > >>
> > org.apache.maven.wagon.providers.http.LightweightHttpWagon.put(Lightw
> > >> eightHttpWagon.java:148)
> > >> at
> > >>
> > org.apache.maven.artifact.manager.DefaultWagonManager.putRemoteFile(D
> > >> efaultWagonManager.java:237)
> > >> at
> > >>
> > org.apache.maven.artifact.manager.DefaultWagonManager.putArtifact(Def
> > >> aultWagonManager.java:153)
> > >> at
> > >>
> > org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(Def
> > >> aultArtifactDeployer.java:80)
> > >> at
> > >>
> > org.apache.maven.plugin.deploy.DeployFileMojo.execute(DeployFileMojo.
> > >> java:240)
> > >> at
> > >>
> > org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPlugi
> > >> nManager.java:447)
> > >> at
> > >>
> > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(Defa
> > >> ultLifecycleExecutor.java:539)
> > >> at
> > >>
> > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandalone
> > >> Goal(DefaultLifecycleExecutor.java:493)
> > >> at
> > >>
> > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(Defau
> > >> ltLifecycleExecutor.java:463)
> > >> at
> > >>
> > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHan
> > >> dleFailures(DefaultLifecycleExecutor.java:311)
> > >> at
> > >>
> > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegmen
> > >> ts(DefaultLifecycleExecutor.java:224)
> > >> at
> > >>
> > org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLi
> > >> fecycleExecutor.java:143)
> > >> at
> > >> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:333)
> > >> at
> > >> org.apache.maven.DefaultMaven.execute(DefaultMaven.java:126)
> > >> at org.apache.maven.cli.MavenCli.main(MavenCli.java:282)
> > >> at
> > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >> at
> > >>
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
> > >> java:39)
> > >> at
> > >>
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> > >> sorImpl.java:25)
> > >> at java.lang.reflect.Method.invoke(Method.java:597)
> > >> at
> > >> org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> > >> at
> > org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> > >> at
> > >>
> > org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> > >>
> > >> at
> > org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> > >> [INFO]
> > >> --------------------------------------------------------------
> > >> ----------
> > >> [INFO] Total time: 1 minute 22 seconds [INFO] Finished at:
> > Fri Jun 26
> > >> 15:10:36 EDT 2009 [INFO] Final
> > >> Memory: 4M/1016M [INFO]
> > >> --------------------------------------------------------------
> > >> ----------
> > >>
> > >
> > >
> > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: users-help@maven.apache.org
> > >
> > >
> >
> > --
> > Sent from my mobile device
> >
> > ®¥@N
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > For additional commands, e-mail: users-help@maven.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
--
®¥@N
RE: OutOfMemoryError when deploying large files
Posted by Rakesh Arora <ra...@nortel.com>.
Ryan,
Here is what we now have in the pom.xml to deploy large files
<distributionManagement>
<repository>
<id>releases</id>
<url>dav:http://<nexus_server>/content/repositories/releases</url>
</repository>
<snapshotRepository>
<id>snapshots</id>
<url>dav:http://<nexus_server>/content/repositories/snapshots</url>
</snapshotRepository>
</distributionManagement>
So we have changed the protocol to dav:http from http.
-Rakesh
> -----Original Message-----
> From: Ryan Connolly [mailto:rynam0@gmail.com]
> Sent: Wednesday, July 01, 2009 9:33 AM
> To: Maven Users List
> Subject: Re: OutOfMemoryError when deploying large files
>
> Rakesh:
> We are experiencing this very same issue. Would you mind
> emailing me a pom snippet for the dav config you used to get
> around this? Your assistance would be most appreciated.
>
> Regards,
> Ryan
>
> On 6/30/09, Rakesh Arora <ra...@nortel.com> wrote:
> > We have resolved the problem by using dav:http protocol instead of
> > http protocol.
> >
> > I have raised a jira against wagon-http-lightweight:
> > http://jira.codehaus.org/browse/WAGON-272
> >
> > -Rakesh
> >
> >
> >
> >> -----Original Message-----
> >> From: Arora, Rakesh (CAR:9S00)
> >> Sent: Friday, June 26, 2009 4:19 PM
> >> To: users@maven.apache.org
> >> Subject: OutOfMemoryError when deploying large files
> >>
> >> When deploying a large artifact we get the
> OutOfMemoryError (please
> >> check the attached stack trace). This is similar to error reported
> >> here:
> >> http://www.mail-archive.com/users@maven.apache.org/msg99157.html
> >>
> >> We traced the issue to wagon-http-lightweight using
> >> PosterOutputStream/ByteArrayOutputStream.write(). This method
> >> consumes lot of memory when dealing with large files as
> reported here:
> >> https://issues.alfresco.com/jira/browse/ETHREEOH-974
> >>
> >> In our case, we are trying to deploy a around 600M file
> and have set
> >> the maximum heap space to 1024M (-Xmx1024m). We are still
> running out
> >> of memory. We are using maven 2.0.8 but had the same issue
> when tried
> >> with 2.1.0 version.
> >>
> >> Is there a way to use another Wagon provider (wagon-http
> ?) that can
> >> deal better with large files? If yes, how can we configure this?
> >>
> >> Should I raise a jira for this (against which component
> >> maven-deploy-plugin or wagon-http-lightweight)?
> >>
> >> Thanks,
> >> -Rakesh
> >>
> >>
> >>
> >> ;-------------------------------------STACK TRACE-----
> [DEBUG] -- end
> >> configuration -- [INFO] [deploy:deploy-file]
> >> Uploading: http://<repo_url>/<group>/foo/1.0/foo-1.0.zip
> >> [INFO]
> >> --------------------------------------------------------------
> >> ----------
> >> [ERROR] FATAL ERROR
> >> [INFO]
> >> --------------------------------------------------------------
> >> ----------
> >> [INFO] Java heap space
> >> [INFO]
> >> --------------------------------------------------------------
> >> ----------
> >> [DEBUG] Trace
> >> java.lang.OutOfMemoryError: Java heap space
> >> at java.util.Arrays.copyOf(Arrays.java:2786)
> >> at
> >> java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
> >> at
> >>
> sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
> >>
> >> at
> >>
> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:338)
> >>
> >> at
> >>
> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:305)
> >>
> >> at
> >>
> org.apache.maven.wagon.AbstractWagon.transfer(AbstractWagon.java:267)
> >>
> >> at
> >>
> org.apache.maven.wagon.AbstractWagon.putTransfer(AbstractWagon.java:2
> >> 38)
> >> at
> >> org.apache.maven.wagon.StreamWagon.put(StreamWagon.java:143)
> >> at
> >>
> org.apache.maven.wagon.providers.http.LightweightHttpWagon.put(Lightw
> >> eightHttpWagon.java:148)
> >> at
> >>
> org.apache.maven.artifact.manager.DefaultWagonManager.putRemoteFile(D
> >> efaultWagonManager.java:237)
> >> at
> >>
> org.apache.maven.artifact.manager.DefaultWagonManager.putArtifact(Def
> >> aultWagonManager.java:153)
> >> at
> >>
> org.apache.maven.artifact.deployer.DefaultArtifactDeployer.deploy(Def
> >> aultArtifactDeployer.java:80)
> >> at
> >>
> org.apache.maven.plugin.deploy.DeployFileMojo.execute(DeployFileMojo.
> >> java:240)
> >> at
> >>
> org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPlugi
> >> nManager.java:447)
> >> at
> >>
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(Defa
> >> ultLifecycleExecutor.java:539)
> >> at
> >>
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandalone
> >> Goal(DefaultLifecycleExecutor.java:493)
> >> at
> >>
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(Defau
> >> ltLifecycleExecutor.java:463)
> >> at
> >>
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHan
> >> dleFailures(DefaultLifecycleExecutor.java:311)
> >> at
> >>
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegmen
> >> ts(DefaultLifecycleExecutor.java:224)
> >> at
> >>
> org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLi
> >> fecycleExecutor.java:143)
> >> at
> >> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:333)
> >> at
> >> org.apache.maven.DefaultMaven.execute(DefaultMaven.java:126)
> >> at org.apache.maven.cli.MavenCli.main(MavenCli.java:282)
> >> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
> >> java:39)
> >> at
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> >> sorImpl.java:25)
> >> at java.lang.reflect.Method.invoke(Method.java:597)
> >> at
> >> org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> >> at
> org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> >> at
> >>
> org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> >>
> >> at
> org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> >> [INFO]
> >> --------------------------------------------------------------
> >> ----------
> >> [INFO] Total time: 1 minute 22 seconds [INFO] Finished at:
> Fri Jun 26
> >> 15:10:36 EDT 2009 [INFO] Final
> >> Memory: 4M/1016M [INFO]
> >> --------------------------------------------------------------
> >> ----------
> >>
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > For additional commands, e-mail: users-help@maven.apache.org
> >
> >
>
> --
> Sent from my mobile device
>
> ®¥@N
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org