You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@felix.apache.org by Stuart McCulloch <st...@jayway.net> on 2007/11/18 14:33:49 UTC
Bnd memory usage (was: Error when building xml-api (in felix/commons) with the latest maven-bundle-plugin)
Hi Peter,
The new versions of Bnd (post 0.0.198) appear to use a lot more memory
than they used to... for example almost 80m when bundling XML jarfiles.
I've tracked this down to additional data held in Clazz instances, which
are kept alive by the "classSpace" map in the Processor when analyzing
the bundle classpath.
Does the processor need to keep all this data, or can some of it be freed
after the class has been analyzed? (there is a reset method in Clazz - if
this is called after the class is processed then the memory usage drops
dramatically, but I'm not sure if calling reset affects later processing...)
Anyway, would appreciate it if you could look into this issue - thanks!
On 18/11/2007, Stuart McCulloch <st...@jayway.net> wrote:
>
> FYI, Bnd v0.0.198 uses 16m whereas v0.0.199 uses 79m while bundling
> xml-api
>
> On 18/11/2007, Stuart McCulloch < stuart.mcculloch@jayway.net> wrote:
> >
> > On 18/11/2007, Clement Escoffier <cl...@gmail.com> wrote:
> > >
> > > Hello,
> > >
> > >
> > >
> > > I am trying to build commons bundle with the latest
> > > maven-bundle-plugin (to
> > > get OBR support). It works fine except for xml-api where the error is
> > > really
> > > strange. It throws an OutOfMemory error (java heap space) when
> > > "bundlelizing" the project. The build works when using the
> > > maven-bundle-plugin 1.0.0. Somebody know how to solve this error ?
> >
> >
> > well I tried the latest bundle-plugin code with the old Bnd library (
> > 0.0.160)
> > and didn't get any OOM, so it looks like the new version is more memory
> > hungry... but it does do a lot more checking
> >
> > you could try increasing the heap available to Maven using MAVEN_OPTS
> > looks like "export MAVEN_OPTS=-Xmx128m" is enough, although I then
> > see a build error due to a missing import...
> >
> > Clement
> > >
> > >
> > >
> > > [WARNING] Include-Resource: overriding
> > >
> > > META-INF\LICENSE=src/main/resources/META-INF/LICENSE,META-INF\NOTICE=src/mai
> > > n/resources/META-INF/NOTICE with src/main/resources,
> > > @xercesImpl-2.8.1.jar!/META-INF/services/** ,
> > > @xalan-2.7.0.jar!/META-INF/services/** (add {maven-resources} if you
> > > want to
> > > include the maven resources)
> > >
> > > java.lang.OutOfMemoryError: Java heap space
> > >
> > > at java.util.Arrays.copyOfRange(Arrays.java :3209)
> > >
> > > at java.lang.String.<init>(String.java:216)
> > >
> > > at java.io.DataInputStream.readUTF(DataInputStream.java:644)
> > >
> > > at java.io.DataInputStream.readUTF(DataInputStream.java:547)
> > >
> > > at aQute.lib.osgi.Clazz.constantUtf8(Clazz.java:308)
> > >
> > > at aQute.lib.osgi.Clazz.parseClassFile(Clazz.java:92)
> > >
> > > at aQute.lib.osgi.Clazz.<init>(Clazz.java:66)
> > >
> > > at aQute.lib.osgi.Processor.analyzeJar(Processor.java:153)
> > >
> > > at
> > > aQute.lib.osgi.Processor.analyzeBundleClasspath(Processor.java:71)
> > >
> > > at aQute.lib.osgi.Verifier.verify(Verifier.java:354)
> > >
> > > at aQute.lib.osgi.Builder.doVerify(Builder.java:207)
> > >
> > > at aQute.lib.osgi.Builder.build(Builder.java:50)
> > >
> > > at
> > > org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java
> > > :276)
> > >
> > > at
> > > org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java
> > > :172)
> > >
> > > at
> > > org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java
> > > :164)
> > >
> > > at
> > > org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManage
> > > r.java:443)
> > >
> > > at
> > > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals
> > > (DefaultLife
> > > cycleExecutor.java:539)
> > >
> > > at
> > >
> > > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle
> > > (DefaultLifecycleExecutor.java:480)
> > >
> > > at
> > > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal
> > > (DefaultLifec
> > > ycleExecutor.java:459)
> > >
> > > at
> > >
> > > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFail
> > > ures(DefaultLifecycleExecutor.java:311)
> > >
> > > at
> > >
> > > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments
> > > (Defa
> > > ultLifecycleExecutor.java:278)
> > >
> > > at
> > > org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycle
> > > Executor.java:143)
> > >
> > > at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java
> > > :334)
> > >
> > > at org.apache.maven.DefaultMaven.execute(DefaultMaven.java
> > > :125)
> > >
> > > at org.apache.maven.cli.MavenCli.main (MavenCli.java:280)
> > >
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >
> > > at
> > > sun.reflect.NativeMethodAccessorImpl.invoke(
> > > NativeMethodAccessorImpl.java:39
> > > )
> > >
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke
> > > (DelegatingMethodAccessorImpl
> > > .java:25)
> > >
> > > at java.lang.reflect.Method.invoke(Method.java:597)
> > >
> > > at
> > > org.codehaus.classworlds.Launcher.launchEnhanced (Launcher.java:315)
> > >
> > > at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> > >
> > > at
> > > org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> > >
> > > java.lang.OutOfMemoryError : Java heap space
> > >
> > > at aQute.lib.osgi.Clazz.parseClassFile(Clazz.java:84)
> > >
> > > at aQute.lib.osgi.Clazz.<init>(Clazz.java:66)
> > >
> > > at aQute.lib.osgi.Processor.analyzeJar(Processor.java :153)
> > >
> > > at
> > > aQute.lib.osgi.Processor.analyzeBundleClasspath(Processor.java:71)
> > >
> > > at aQute.lib.osgi.Verifier.verify(Verifier.java:354)
> > >
> > > at aQute.lib.osgi.Builder.doVerify(Builder.java :207)
> > >
> > > at aQute.lib.osgi.Builder.build(Builder.java:50)
> > >
> > > at
> > > org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java
> > > :276)
> > >
> > > at
> > > org.apache.felix.bundleplugin.BundlePlugin.execute (BundlePlugin.java
> > > :172)
> > >
> > > at
> > > org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java
> > > :164)
> > >
> > > at
> > > org.apache.maven.plugin.DefaultPluginManager.executeMojo
> > > (DefaultPluginManage
> > > r.java :443)
> > >
> > > at
> > > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals
> > > (DefaultLife
> > > cycleExecutor.java:539)
> > >
> > > at
> > >
> > > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle
> > > (DefaultLifecycleExecutor.java:480)
> > >
> > > at
> > > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal
> > > (DefaultLifec
> > > ycleExecutor.java:459)
> > >
> > > at
> > >
> > > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFail
> > > ures(DefaultLifecycleExecutor.java:311)
> > >
> > > at
> > >
> > > org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments
> > > (Defa
> > > ultLifecycleExecutor.java:278)
> > >
> > > at
> > > org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycle
> > > Executor.java:143)
> > >
> > > at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java
> > > :334)
> > >
> > > at org.apache.maven.DefaultMaven.execute(DefaultMaven.java
> > > :125)
> > >
> > > at org.apache.maven.cli.MavenCli.main (MavenCli.java:280)
> > >
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >
> > > at
> > > sun.reflect.NativeMethodAccessorImpl.invoke(
> > > NativeMethodAccessorImpl.java:39
> > > )
> > >
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke
> > > (DelegatingMethodAccessorImpl
> > > .java:25)
> > >
> > > at java.lang.reflect.Method.invoke(Method.java:597)
> > >
> > > at
> > > org.codehaus.classworlds.Launcher.launchEnhanced (Launcher.java:315)
> > >
> > > at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> > >
> > > at
> > > org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> > >
> > > at org.codehaus.classworlds.Launcher.main (Launcher.java:375)
> > >
> > > java.lang.OutOfMemoryError: Java heap space
> > >
> > > at java.util.Arrays.copyOfRange(Arrays.java:3209)
> > >
> > > at java.lang.String.<init>(String.java:216)
> > >
> > > at java.io.DataInputStream.readUTF (DataInputStream.java:644)
> > >
> > > at java.io.DataInputStream.readUTF(DataInputStream.java:547)
> > >
> > > at aQute.lib.osgi.Clazz.constantUtf8(Clazz.java:308)
> > >
> > > at aQute.lib.osgi.Clazz.parseClassFile (Clazz.java:92)
> > >
> > > at aQute.lib.osgi.Clazz.<init>(Clazz.java:66)
> > >
> > > .
> > >
> > >
> > >
> > > --
> > >
> > > Clement Escoffier
> > >
> > > Grenoble University
> > >
> > > +33 (0) 4 76 51 40 24
> > >
> > > http://clement.plop-plop.net
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Cheers, Stuart
>
>
>
>
> --
> Cheers, Stuart
--
Cheers, Stuart
Re: Bnd memory usage (was: Error when building xml-api (in felix/commons) with the latest maven-bundle-plugin)
Posted by Stuart McCulloch <st...@jayway.net>.
On 22/11/2007, Peter Kriens <Pe...@aqute.biz> wrote:
>
> I was trying to cache the results for later use. However, I have
> undone that change. Latest version is 0.0.213
thanks Peter, I've tested it locally and it's much better - I'll update the
bundle-plugin snapshot to use the new version
Kind regards,
>
> Peter Kriens
>
--
Cheers, Stuart
Re: Bnd memory usage (was: Error when building xml-api (in felix/commons) with the latest maven-bundle-plugin)
Posted by Peter Kriens <Pe...@aQute.biz>.
I was trying to cache the results for later use. However, I have
undone that change. Latest version is 0.0.213
Kind regards,
Peter Kriens
SM> Hi Peter,
SM> The new versions of Bnd (post 0.0.198) appear to use a lot more memory
SM> than they used to... for example almost 80m when bundling XML jarfiles.
SM> I've tracked this down to additional data held in Clazz instances, which
SM> are kept alive by the "classSpace" map in the Processor when analyzing
SM> the bundle classpath.
SM> Does the processor need to keep all this data, or can some of it be freed
SM> after the class has been analyzed? (there is a reset method in Clazz - if
SM> this is called after the class is processed then the memory usage drops
SM> dramatically, but I'm not sure if calling reset affects later processing...)
SM> Anyway, would appreciate it if you could look into this issue - thanks!
SM> On 18/11/2007, Stuart McCulloch <st...@jayway.net> wrote:
SM> FYI, Bnd v0.0.198 uses 16m whereas v0.0.199 uses 79m while bundling xml-api
SM> On 18/11/2007, Stuart McCulloch < stuart.mcculloch@jayway.net> wrote:
SM> On 18/11/2007, Clement Escoffier <cl...@gmail.com> wrote:
SM>
SM> Hello,
SM> I am trying to build commons bundle with the latest maven-bundle-plugin (to
SM> get OBR support). It works fine except for xml-api where the error is really
SM> strange. It throws an OutOfMemory error (java heap space) when
SM> "bundlelizing" the project. The build works when using the
SM> maven-bundle-plugin 1.0.0. Somebody know how to solve this error ?
SM> well I tried the latest bundle-plugin code with the old Bnd library ( 0.0.160)
SM> and didn't get any OOM, so it looks like the new version is more memory
SM> hungry... but it does do a lot more checking
SM> you could try increasing the heap available to Maven using MAVEN_OPTS
SM> looks like "export MAVEN_OPTS=-Xmx128m" is enough, although I then
SM> see a build error due to a missing import...
SM> Clement
SM> [WARNING] Include-Resource: overriding
SM> META-INF\LICENSE=src/main/resources/META-INF/LICENSE,META-INF\NOTICE=src/mai
SM> n/resources/META-INF/NOTICE with src/main/resources,
SM> @xercesImpl-2.8.1.jar!/META-INF/services/** ,
SM> @xalan-2.7.0.jar!/META-INF/services/** (add {maven-resources} if you want to
SM> include the maven resources)
SM> java.lang.OutOfMemoryError: Java heap space
SM> at java.util.Arrays.copyOfRange(Arrays.java :3209)
SM> at java.lang.String.<init>(String.java:216)
SM> at java.io.DataInputStream.readUTF(DataInputStream.java:644)
SM> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
SM> at aQute.lib.osgi.Clazz.constantUtf8(Clazz.java:308)
SM> at aQute.lib.osgi.Clazz.parseClassFile(Clazz.java:92)
SM> at aQute.lib.osgi.Clazz.<init>(Clazz.java:66)
SM> at aQute.lib.osgi.Processor.analyzeJar(Processor.java:153)
SM> at
SM> aQute.lib.osgi.Processor.analyzeBundleClasspath(Processor.java:71)
SM> at aQute.lib.osgi.Verifier.verify(Verifier.java:354)
SM> at aQute.lib.osgi.Builder.doVerify(Builder.java:207)
SM> at aQute.lib.osgi.Builder.build(Builder.java:50)
SM> at
SM> org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:276)
SM> at
SM> org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:172)
SM> at
SM> org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:164)
SM> at
SM> org.apache.maven.plugin.DefaultPluginManager.executeMojo (DefaultPluginManage
SM> r.java:443)
SM> at
SM> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLife
SM> cycleExecutor.java:539)
SM> at
SM> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle
SM> (DefaultLifecycleExecutor.java:480)
SM> at
SM> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifec
SM> ycleExecutor.java:459)
SM> at
SM> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFail
SM> ures(DefaultLifecycleExecutor.java:311)
SM> at
SM> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(Defa
SM> ultLifecycleExecutor.java:278)
SM> at
SM> org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute (DefaultLifecycle
SM> Executor.java:143)
SM> at
SM> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:334)
SM> at
SM> org.apache.maven.DefaultMaven.execute(DefaultMaven.java:125)
SM> at org.apache.maven.cli.MavenCli.main (MavenCli.java:280)
SM> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
SM> at
SM> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39
SM> )
SM> at
SM>
SM> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
SM> .java:25)
SM> at java.lang.reflect.Method.invoke(Method.java:597)
SM> at
SM> org.codehaus.classworlds.Launcher.launchEnhanced (Launcher.java:315)
SM> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
SM> at
SM> org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
SM> java.lang.OutOfMemoryError : Java heap space
SM> at aQute.lib.osgi.Clazz.parseClassFile(Clazz.java:84)
SM> at aQute.lib.osgi.Clazz.<init>(Clazz.java:66)
SM> at aQute.lib.osgi.Processor.analyzeJar(Processor.java :153)
SM> at
SM> aQute.lib.osgi.Processor.analyzeBundleClasspath(Processor.java:71)
SM> at aQute.lib.osgi.Verifier.verify(Verifier.java:354)
SM> at aQute.lib.osgi.Builder.doVerify(Builder.java :207)
SM> at aQute.lib.osgi.Builder.build(Builder.java:50)
SM> at
SM> org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:276)
SM> at
SM> org.apache.felix.bundleplugin.BundlePlugin.execute (BundlePlugin.java:172)
SM> at
SM> org.apache.felix.bundleplugin.BundlePlugin.execute(BundlePlugin.java:164)
SM> at
SM> org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManage
SM> r.java :443)
SM> at
SM> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLife
SM> cycleExecutor.java:539)
SM> at
SM> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle
SM> (DefaultLifecycleExecutor.java:480)
SM> at
SM> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifec
SM> ycleExecutor.java:459)
SM> at
SM> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFail
SM> ures(DefaultLifecycleExecutor.java:311)
SM> at
SM> org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(Defa
SM> ultLifecycleExecutor.java:278)
SM> at
SM> org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute (DefaultLifecycle
SM> Executor.java:143)
SM> at
SM> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:334)
SM> at
SM> org.apache.maven.DefaultMaven.execute(DefaultMaven.java:125)
SM> at org.apache.maven.cli.MavenCli.main (MavenCli.java:280)
SM> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
SM> at
SM> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39
SM> )
SM> at
SM>
SM> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
SM> .java:25)
SM> at java.lang.reflect.Method.invoke(Method.java:597)
SM> at
SM> org.codehaus.classworlds.Launcher.launchEnhanced (Launcher.java:315)
SM> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
SM> at
SM> org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
SM> at org.codehaus.classworlds.Launcher.main (Launcher.java:375)
SM> java.lang.OutOfMemoryError: Java heap space
SM> at java.util.Arrays.copyOfRange(Arrays.java:3209)
SM> at java.lang.String.<init>(String.java:216)
SM> at java.io.DataInputStream.readUTF (DataInputStream.java:644)
SM> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
SM> at aQute.lib.osgi.Clazz.constantUtf8(Clazz.java:308)
SM> at aQute.lib.osgi.Clazz.parseClassFile (Clazz.java:92)
SM> at aQute.lib.osgi.Clazz.<init>(Clazz.java:66)
SM> .
SM> --
SM> Clement Escoffier
SM> Grenoble University
SM> +33 (0) 4 76 51 40 24
SM> http://clement.plop-plop.net
SM> --
SM> Cheers, Stuart
SM> --
SM> Cheers, Stuart
--
Peter Kriens Tel +33467542167
9C, Avenue St. Drézéry AOL,Yahoo: pkriens
34160 Beaulieu, France ICQ 255570717
Skype pkriens Fax +1 8153772599