You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Altin Papa (JIRA)" <ji...@codehaus.org> on 2010/09/07 15:42:32 UTC
[jira] Created: (MASSEMBLY-499) Poor performance in
DirectoryArchiver due to unnecessary native calls
Poor performance in DirectoryArchiver due to unnecessary native calls
---------------------------------------------------------------------
Key: MASSEMBLY-499
URL: http://jira.codehaus.org/browse/MASSEMBLY-499
Project: Maven 2.x Assembly Plugin
Issue Type: Bug
Affects Versions: 2.2-beta-5
Environment: Unix/Linux
Reporter: Altin Papa
1)
maven-assembly-plugin calls DirectoryArchiver.copyFile( ArchiveEntry entry, String vPath ), which calls
ArchiveEntryUtils.chmod( outFile, entry.getMode(), getLogger() );
resulting in a native child Process being forked to do a chmod (on Unix only)
The performance impact on a large assembly is enormous: patching DirectoryArchiver to not do this resulted in a performance improvement by more than 5 times.
I've raised this with Plexus as http://jira.codehaus.org/browse/PLXCOMP-156
2)
AssemblyFormatUtils should cache the environment, as AssemblyInterpolator does. Both, though, should use the latest *plexus* CommandLineUtils, as opposed to maven-assembly-plugin's own version. The plexus class uses System.getenv() (JDK5+) when available.
Generally, please see if you can review the use of native command calls, esp. in light of JDK5/6 APIs. Forked native calls had a horrible performance impact on our (large, highly modular) assembly; which has only been corrected by local patches of maven-assembly/plexus-archiver.
Thanks!
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MASSEMBLY-499) Poor performance in
DirectoryArchiver due to unnecessary native calls
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olivier Lamy updated MASSEMBLY-499:
-----------------------------------
Comment: was deleted
(was: ok so basically it works for user and all.
But as some junit test group permissions too (ie ZipArchiverTest int logMode = 0640) the result of the test will depend on the current shell umask.)
> Poor performance in DirectoryArchiver due to unnecessary native calls
> ---------------------------------------------------------------------
>
> Key: MASSEMBLY-499
> URL: http://jira.codehaus.org/browse/MASSEMBLY-499
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-5
> Environment: Unix/Linux
> Reporter: Altin Papa
>
> 1)
> maven-assembly-plugin calls DirectoryArchiver.copyFile( ArchiveEntry entry, String vPath ), which calls
> ArchiveEntryUtils.chmod( outFile, entry.getMode(), getLogger() );
> resulting in a native child Process being forked to do a chmod (on Unix only)
> The performance impact on a large assembly is enormous: patching DirectoryArchiver to not do this resulted in a performance improvement by more than 5 times.
> I've raised this with Plexus as http://jira.codehaus.org/browse/PLXCOMP-156
> 2)
> AssemblyFormatUtils should cache the environment, as AssemblyInterpolator does. Both, though, should use the latest *plexus* CommandLineUtils, as opposed to maven-assembly-plugin's own version. The plexus class uses System.getenv() (JDK5+) when available.
> Generally, please see if you can review the use of native command calls, esp. in light of JDK5/6 APIs. Forked native calls had a horrible performance impact on our (large, highly modular) assembly; which has only been corrected by local patches of maven-assembly/plexus-archiver.
> Thanks!
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Issue Comment Edited: (MASSEMBLY-499) Poor performance in
DirectoryArchiver due to unnecessary native calls
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=236641#action_236641 ]
Olivier Lamy edited comment on MASSEMBLY-499 at 9/27/10 4:26 PM:
-----------------------------------------------------------------
fixed rev 1001903
{code}
-Dassembly.useJvmChmod=true
or mojo configuration
useJvmChmod
{code}
was (Author: olamy):
fixed rev 1001903
> Poor performance in DirectoryArchiver due to unnecessary native calls
> ---------------------------------------------------------------------
>
> Key: MASSEMBLY-499
> URL: http://jira.codehaus.org/browse/MASSEMBLY-499
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-5
> Environment: Unix/Linux
> Reporter: Altin Papa
> Assignee: Olivier Lamy
> Fix For: 2.2-beta-6
>
>
> 1)
> maven-assembly-plugin calls DirectoryArchiver.copyFile( ArchiveEntry entry, String vPath ), which calls
> ArchiveEntryUtils.chmod( outFile, entry.getMode(), getLogger() );
> resulting in a native child Process being forked to do a chmod (on Unix only)
> The performance impact on a large assembly is enormous: patching DirectoryArchiver to not do this resulted in a performance improvement by more than 5 times.
> I've raised this with Plexus as http://jira.codehaus.org/browse/PLXCOMP-156
> 2)
> AssemblyFormatUtils should cache the environment, as AssemblyInterpolator does. Both, though, should use the latest *plexus* CommandLineUtils, as opposed to maven-assembly-plugin's own version. The plexus class uses System.getenv() (JDK5+) when available.
> Generally, please see if you can review the use of native command calls, esp. in light of JDK5/6 APIs. Forked native calls had a horrible performance impact on our (large, highly modular) assembly; which has only been corrected by local patches of maven-assembly/plexus-archiver.
> Thanks!
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-499) Poor performance in
DirectoryArchiver due to unnecessary native calls
Posted by "Arnaud Heritier (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=236648#action_236648 ]
Arnaud Heritier commented on MASSEMBLY-499:
-------------------------------------------
No better in my case and for MASSEMBLY-424.
Results with Maven 3.0-RC3 on http://svn.exoplatform.org/projects/jcr-benchmark/trunk/
{code}
$ mvn clean install -Dversion.assembly.plugin=2.1 -o
[INFO] Total time: 11.683s
$ mvn clean install -Dversion.assembly.plugin=2.2-beta-5 -o
[INFO] Total time: 33.996s
$ mvn clean install -Dversion.assembly.plugin=2.2-beta-6-SNAPSHOT -o
[INFO] Total time: 27.486s
$ mvn clean install -Dversion.assembly.plugin=2.2-beta-6-SNAPSHOT -o -Dassembly.useJvmChmod=false
[INFO] Total time: 28.103s
{code}
Note that the plugin doesn't work with maven 2.2.1 :
{code}
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Internal error in the plugin manager executing goal 'org.apache.maven.plugins:maven-assembly-plugin:2.2-beta-6-SNAPSHOT:directory-inline': Unable to find the mojo 'directory-inline' (or one of its required components) in the plugin 'org.apache.maven.plugins:maven-assembly-plugin'
Component descriptor cannot be found in the component repository: org.apache.maven.artifact.resolver.ArtifactResolverdefault.
[INFO] ------------------------------------------------------------------------
[INFO] Trace
org.apache.maven.lifecycle.LifecycleExecutionException: Internal error in the plugin manager executing goal 'org.apache.maven.plugins:maven-assembly-plugin:2.2-beta-6-SNAPSHOT:directory-inline': Unable to find the mojo 'directory-inline' (or one of its required components) in the plugin 'org.apache.maven.plugins:maven-assembly-plugin'
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:698)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.apache.maven.plugin.PluginManagerException: Unable to find the mojo 'directory-inline' (or one of its required components) in the plugin 'org.apache.maven.plugins:maven-assembly-plugin'
at org.apache.maven.plugin.DefaultPluginManager.getConfiguredMojo(DefaultPluginManager.java:674)
at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:468)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
... 17 more
Caused by: org.codehaus.plexus.component.repository.exception.ComponentLookupException: Unable to lookup component 'org.apache.maven.plugin.Mojoorg.apache.maven.plugins:maven-assembly-plugin:2.2-beta-6-SNAPSHOT:directory-inline', it could not be started
at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:339)
at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:440)
at org.apache.maven.plugin.DefaultPluginManager.getConfiguredMojo(DefaultPluginManager.java:652)
... 19 more
Caused by: org.codehaus.plexus.component.repository.exception.ComponentLifecycleException: Error starting component
at org.codehaus.plexus.component.manager.AbstractComponentManager.startComponentLifecycle(AbstractComponentManager.java:109)
at org.codehaus.plexus.component.manager.AbstractComponentManager.createComponentInstance(AbstractComponentManager.java:95)
at org.codehaus.plexus.component.manager.PerLookupComponentManager.getComponent(PerLookupComponentManager.java:48)
at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:331)
... 21 more
Caused by: org.codehaus.plexus.personality.plexus.lifecycle.phase.PhaseExecutionException: Error composing component
at org.codehaus.plexus.personality.plexus.lifecycle.phase.CompositionPhase.execute(CompositionPhase.java:33)
at org.codehaus.plexus.lifecycle.AbstractLifecycleHandler.start(AbstractLifecycleHandler.java:101)
at org.codehaus.plexus.component.manager.AbstractComponentManager.startComponentLifecycle(AbstractComponentManager.java:105)
... 24 more
Caused by: org.codehaus.plexus.component.composition.CompositionException: Composition failed of field assemblyArchiver in object of type org.apache.maven.plugin.assembly.mojos.DirectoryInlineMojo because the requirement ComponentRequirement{role='org.apache.maven.plugin.assembly.archive.AssemblyArchiver', roleHint='null', fieldName='assemblyArchiver'} was missing
at org.codehaus.plexus.component.composition.FieldComponentComposer.assignRequirementToField(FieldComponentComposer.java:154)
at org.codehaus.plexus.component.composition.FieldComponentComposer.assembleComponent(FieldComponentComposer.java:73)
at org.codehaus.plexus.component.composition.DefaultComponentComposerManager.assembleComponent(DefaultComponentComposerManager.java:68)
at org.codehaus.plexus.DefaultPlexusContainer.composeComponent(DefaultPlexusContainer.java:1486)
at org.codehaus.plexus.personality.plexus.lifecycle.phase.CompositionPhase.execute(CompositionPhase.java:29)
... 26 more
Caused by: org.codehaus.plexus.component.repository.exception.ComponentLookupException: Unable to lookup component 'org.apache.maven.plugin.assembly.archive.AssemblyArchiver', it could not be started
at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:339)
at org.codehaus.plexus.component.composition.FieldComponentComposer.assignRequirementToField(FieldComponentComposer.java:129)
... 30 more
Caused by: org.codehaus.plexus.component.repository.exception.ComponentLifecycleException: Error starting component
at org.codehaus.plexus.component.manager.AbstractComponentManager.startComponentLifecycle(AbstractComponentManager.java:109)
at org.codehaus.plexus.component.manager.AbstractComponentManager.createComponentInstance(AbstractComponentManager.java:95)
at org.codehaus.plexus.component.manager.ClassicSingletonComponentManager.getComponent(ClassicSingletonComponentManager.java:92)
at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:331)
... 31 more
Caused by: org.codehaus.plexus.personality.plexus.lifecycle.phase.PhaseExecutionException: Error composing component
at org.codehaus.plexus.personality.plexus.lifecycle.phase.CompositionPhase.execute(CompositionPhase.java:33)
at org.codehaus.plexus.lifecycle.AbstractLifecycleHandler.start(AbstractLifecycleHandler.java:101)
at org.codehaus.plexus.component.manager.AbstractComponentManager.startComponentLifecycle(AbstractComponentManager.java:105)
... 34 more
Caused by: org.codehaus.plexus.component.composition.CompositionException: Composition failed of field dependencyResolver in object of type org.apache.maven.plugin.assembly.archive.DefaultAssemblyArchiver because the requirement ComponentRequirement{role='org.apache.maven.plugin.assembly.artifact.DependencyResolver', roleHint='default', fieldName='dependencyResolver'} was missing
at org.codehaus.plexus.component.composition.FieldComponentComposer.assignRequirementToField(FieldComponentComposer.java:154)
at org.codehaus.plexus.component.composition.FieldComponentComposer.assembleComponent(FieldComponentComposer.java:73)
at org.codehaus.plexus.component.composition.DefaultComponentComposerManager.assembleComponent(DefaultComponentComposerManager.java:68)
at org.codehaus.plexus.DefaultPlexusContainer.composeComponent(DefaultPlexusContainer.java:1486)
at org.codehaus.plexus.personality.plexus.lifecycle.phase.CompositionPhase.execute(CompositionPhase.java:29)
... 36 more
Caused by: org.codehaus.plexus.component.repository.exception.ComponentLookupException: Unable to lookup component 'org.apache.maven.plugin.assembly.artifact.DependencyResolverdefault', it could not be started
at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:339)
at org.codehaus.plexus.component.composition.FieldComponentComposer.assignRequirementToField(FieldComponentComposer.java:129)
... 40 more
Caused by: org.codehaus.plexus.component.repository.exception.ComponentLifecycleException: Error starting component
at org.codehaus.plexus.component.manager.AbstractComponentManager.startComponentLifecycle(AbstractComponentManager.java:109)
at org.codehaus.plexus.component.manager.AbstractComponentManager.createComponentInstance(AbstractComponentManager.java:95)
at org.codehaus.plexus.component.manager.ClassicSingletonComponentManager.getComponent(ClassicSingletonComponentManager.java:92)
at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:331)
... 41 more
Caused by: org.codehaus.plexus.personality.plexus.lifecycle.phase.PhaseExecutionException: Error composing component
at org.codehaus.plexus.personality.plexus.lifecycle.phase.CompositionPhase.execute(CompositionPhase.java:33)
at org.codehaus.plexus.lifecycle.AbstractLifecycleHandler.start(AbstractLifecycleHandler.java:101)
at org.codehaus.plexus.component.manager.AbstractComponentManager.startComponentLifecycle(AbstractComponentManager.java:105)
... 44 more
Caused by: org.codehaus.plexus.component.composition.CompositionException: Composition failed of field resolver in object of type org.apache.maven.plugin.assembly.artifact.DefaultDependencyResolver because the requirement ComponentRequirement{role='org.apache.maven.artifact.resolver.ArtifactResolver', roleHint='default', fieldName='resolver'} was missing
at org.codehaus.plexus.component.composition.FieldComponentComposer.assignRequirementToField(FieldComponentComposer.java:154)
at org.codehaus.plexus.component.composition.FieldComponentComposer.assembleComponent(FieldComponentComposer.java:73)
at org.codehaus.plexus.component.composition.DefaultComponentComposerManager.assembleComponent(DefaultComponentComposerManager.java:68)
at org.codehaus.plexus.DefaultPlexusContainer.composeComponent(DefaultPlexusContainer.java:1486)
at org.codehaus.plexus.personality.plexus.lifecycle.phase.CompositionPhase.execute(CompositionPhase.java:29)
... 46 more
Caused by: org.codehaus.plexus.component.repository.exception.ComponentLookupException: Component descriptor cannot be found in the component repository: org.apache.maven.artifact.resolver.ArtifactResolverdefault.
at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:323)
at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:312)
at org.codehaus.plexus.component.composition.FieldComponentComposer.assignRequirementToField(FieldComponentComposer.java:129)
... 50 more
{code}
And when it works, with m3, I have many lines like :
{code}
[INFO] mode 33261, chmod 755
[INFO] mode 33261, chmod 755
[INFO] mode 33261, chmod 755
[INFO] mode 33188, chmod 644
[INFO] mode 33188, chmod 644
[INFO] mode 33188, chmod 644
[INFO] mode 33188, chmod 644
[INFO] mode 33188, chmod 644
...
{code}
> Poor performance in DirectoryArchiver due to unnecessary native calls
> ---------------------------------------------------------------------
>
> Key: MASSEMBLY-499
> URL: http://jira.codehaus.org/browse/MASSEMBLY-499
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-5
> Environment: Unix/Linux
> Reporter: Altin Papa
> Assignee: Olivier Lamy
> Fix For: 2.2-beta-6
>
>
> 1)
> maven-assembly-plugin calls DirectoryArchiver.copyFile( ArchiveEntry entry, String vPath ), which calls
> ArchiveEntryUtils.chmod( outFile, entry.getMode(), getLogger() );
> resulting in a native child Process being forked to do a chmod (on Unix only)
> The performance impact on a large assembly is enormous: patching DirectoryArchiver to not do this resulted in a performance improvement by more than 5 times.
> I've raised this with Plexus as http://jira.codehaus.org/browse/PLXCOMP-156
> 2)
> AssemblyFormatUtils should cache the environment, as AssemblyInterpolator does. Both, though, should use the latest *plexus* CommandLineUtils, as opposed to maven-assembly-plugin's own version. The plexus class uses System.getenv() (JDK5+) when available.
> Generally, please see if you can review the use of native command calls, esp. in light of JDK5/6 APIs. Forked native calls had a horrible performance impact on our (large, highly modular) assembly; which has only been corrected by local patches of maven-assembly/plexus-archiver.
> Thanks!
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MASSEMBLY-499) Poor performance in
DirectoryArchiver due to unnecessary native calls
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olivier Lamy updated MASSEMBLY-499:
-----------------------------------
Fix Version/s: 2.2-beta-6
Assignee: Olivier Lamy
> Poor performance in DirectoryArchiver due to unnecessary native calls
> ---------------------------------------------------------------------
>
> Key: MASSEMBLY-499
> URL: http://jira.codehaus.org/browse/MASSEMBLY-499
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-5
> Environment: Unix/Linux
> Reporter: Altin Papa
> Assignee: Olivier Lamy
> Fix For: 2.2-beta-6
>
>
> 1)
> maven-assembly-plugin calls DirectoryArchiver.copyFile( ArchiveEntry entry, String vPath ), which calls
> ArchiveEntryUtils.chmod( outFile, entry.getMode(), getLogger() );
> resulting in a native child Process being forked to do a chmod (on Unix only)
> The performance impact on a large assembly is enormous: patching DirectoryArchiver to not do this resulted in a performance improvement by more than 5 times.
> I've raised this with Plexus as http://jira.codehaus.org/browse/PLXCOMP-156
> 2)
> AssemblyFormatUtils should cache the environment, as AssemblyInterpolator does. Both, though, should use the latest *plexus* CommandLineUtils, as opposed to maven-assembly-plugin's own version. The plexus class uses System.getenv() (JDK5+) when available.
> Generally, please see if you can review the use of native command calls, esp. in light of JDK5/6 APIs. Forked native calls had a horrible performance impact on our (large, highly modular) assembly; which has only been corrected by local patches of maven-assembly/plexus-archiver.
> Thanks!
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-499) Poor performance in
DirectoryArchiver due to unnecessary native calls
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=236453#action_236453 ]
Olivier Lamy commented on MASSEMBLY-499:
----------------------------------------
Note : with this feature, group permissions won't be honored anymore as it's not possible with the jvm "chmod" mechanism.
> Poor performance in DirectoryArchiver due to unnecessary native calls
> ---------------------------------------------------------------------
>
> Key: MASSEMBLY-499
> URL: http://jira.codehaus.org/browse/MASSEMBLY-499
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-5
> Environment: Unix/Linux
> Reporter: Altin Papa
> Assignee: Olivier Lamy
> Fix For: 2.2-beta-6
>
>
> 1)
> maven-assembly-plugin calls DirectoryArchiver.copyFile( ArchiveEntry entry, String vPath ), which calls
> ArchiveEntryUtils.chmod( outFile, entry.getMode(), getLogger() );
> resulting in a native child Process being forked to do a chmod (on Unix only)
> The performance impact on a large assembly is enormous: patching DirectoryArchiver to not do this resulted in a performance improvement by more than 5 times.
> I've raised this with Plexus as http://jira.codehaus.org/browse/PLXCOMP-156
> 2)
> AssemblyFormatUtils should cache the environment, as AssemblyInterpolator does. Both, though, should use the latest *plexus* CommandLineUtils, as opposed to maven-assembly-plugin's own version. The plexus class uses System.getenv() (JDK5+) when available.
> Generally, please see if you can review the use of native command calls, esp. in light of JDK5/6 APIs. Forked native calls had a horrible performance impact on our (large, highly modular) assembly; which has only been corrected by local patches of maven-assembly/plexus-archiver.
> Thanks!
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-499) Poor performance in
DirectoryArchiver due to unnecessary native calls
Posted by "Altin Papa (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=236501#action_236501 ]
Altin Papa commented on MASSEMBLY-499:
--------------------------------------
I agree, a mojo parameter would be easier for activation
> Poor performance in DirectoryArchiver due to unnecessary native calls
> ---------------------------------------------------------------------
>
> Key: MASSEMBLY-499
> URL: http://jira.codehaus.org/browse/MASSEMBLY-499
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-5
> Environment: Unix/Linux
> Reporter: Altin Papa
> Assignee: Olivier Lamy
> Fix For: 2.2-beta-6
>
>
> 1)
> maven-assembly-plugin calls DirectoryArchiver.copyFile( ArchiveEntry entry, String vPath ), which calls
> ArchiveEntryUtils.chmod( outFile, entry.getMode(), getLogger() );
> resulting in a native child Process being forked to do a chmod (on Unix only)
> The performance impact on a large assembly is enormous: patching DirectoryArchiver to not do this resulted in a performance improvement by more than 5 times.
> I've raised this with Plexus as http://jira.codehaus.org/browse/PLXCOMP-156
> 2)
> AssemblyFormatUtils should cache the environment, as AssemblyInterpolator does. Both, though, should use the latest *plexus* CommandLineUtils, as opposed to maven-assembly-plugin's own version. The plexus class uses System.getenv() (JDK5+) when available.
> Generally, please see if you can review the use of native command calls, esp. in light of JDK5/6 APIs. Forked native calls had a horrible performance impact on our (large, highly modular) assembly; which has only been corrected by local patches of maven-assembly/plexus-archiver.
> Thanks!
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-499) Poor performance in
DirectoryArchiver due to unnecessary native calls
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=236452#action_236452 ]
Olivier Lamy commented on MASSEMBLY-499:
----------------------------------------
fixed rev 1001205.
can be tested using -DuseJvmChmod=true.
I wonder if having a mojo parameter to activate this could be better.
WDYT ?
> Poor performance in DirectoryArchiver due to unnecessary native calls
> ---------------------------------------------------------------------
>
> Key: MASSEMBLY-499
> URL: http://jira.codehaus.org/browse/MASSEMBLY-499
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-5
> Environment: Unix/Linux
> Reporter: Altin Papa
> Assignee: Olivier Lamy
> Fix For: 2.2-beta-6
>
>
> 1)
> maven-assembly-plugin calls DirectoryArchiver.copyFile( ArchiveEntry entry, String vPath ), which calls
> ArchiveEntryUtils.chmod( outFile, entry.getMode(), getLogger() );
> resulting in a native child Process being forked to do a chmod (on Unix only)
> The performance impact on a large assembly is enormous: patching DirectoryArchiver to not do this resulted in a performance improvement by more than 5 times.
> I've raised this with Plexus as http://jira.codehaus.org/browse/PLXCOMP-156
> 2)
> AssemblyFormatUtils should cache the environment, as AssemblyInterpolator does. Both, though, should use the latest *plexus* CommandLineUtils, as opposed to maven-assembly-plugin's own version. The plexus class uses System.getenv() (JDK5+) when available.
> Generally, please see if you can review the use of native command calls, esp. in light of JDK5/6 APIs. Forked native calls had a horrible performance impact on our (large, highly modular) assembly; which has only been corrected by local patches of maven-assembly/plexus-archiver.
> Thanks!
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (MASSEMBLY-499) Poor performance in
DirectoryArchiver due to unnecessary native calls
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Casey closed MASSEMBLY-499.
--------------------------------
Resolution: Fixed
Assignee: John Casey (was: Olivier Lamy)
Added ignorePermissions flag to Abstract mojo, so all mojos get to use it. To improve your performance (if you don't care about chmodding):
{code:xml}
<configuration>
[...]
<ignorePermissions>true</ignorePermissions>
</configuration>
{code}
> Poor performance in DirectoryArchiver due to unnecessary native calls
> ---------------------------------------------------------------------
>
> Key: MASSEMBLY-499
> URL: http://jira.codehaus.org/browse/MASSEMBLY-499
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-5
> Environment: Unix/Linux
> Reporter: Altin Papa
> Assignee: John Casey
> Fix For: 2.2-beta-6
>
>
> 1)
> maven-assembly-plugin calls DirectoryArchiver.copyFile( ArchiveEntry entry, String vPath ), which calls
> ArchiveEntryUtils.chmod( outFile, entry.getMode(), getLogger() );
> resulting in a native child Process being forked to do a chmod (on Unix only)
> The performance impact on a large assembly is enormous: patching DirectoryArchiver to not do this resulted in a performance improvement by more than 5 times.
> I've raised this with Plexus as http://jira.codehaus.org/browse/PLXCOMP-156
> 2)
> AssemblyFormatUtils should cache the environment, as AssemblyInterpolator does. Both, though, should use the latest *plexus* CommandLineUtils, as opposed to maven-assembly-plugin's own version. The plexus class uses System.getenv() (JDK5+) when available.
> Generally, please see if you can review the use of native command calls, esp. in light of JDK5/6 APIs. Forked native calls had a horrible performance impact on our (large, highly modular) assembly; which has only been corrected by local patches of maven-assembly/plexus-archiver.
> Thanks!
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-499) Poor performance in
DirectoryArchiver due to unnecessary native calls
Posted by "Altin Papa (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=236653#action_236653 ]
Altin Papa commented on MASSEMBLY-499:
--------------------------------------
The bulk of the slowdown is due to DirectoryArchiver calling this for every file:
{code}
ArchiveEntryUtils.chmod( outFile, entry.getMode(), getLogger() );
{code}
Until this is fixed, I don't think we'll see significant improvements. Even with JDK 6's own permission handling, setting them on every file is pretty onerous.
At a minimum, the above line should only be called if the entry's mode is NOT 'default'.
But perhaps there ought to be a dedicated mojo param to turn off permissions entirely. (Or vice versa, if you want them off by default.)
Right now, you have no way of avoiding the overhead, even if you don't care about setting permissions in the assembly,
> Poor performance in DirectoryArchiver due to unnecessary native calls
> ---------------------------------------------------------------------
>
> Key: MASSEMBLY-499
> URL: http://jira.codehaus.org/browse/MASSEMBLY-499
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-5
> Environment: Unix/Linux
> Reporter: Altin Papa
> Assignee: Olivier Lamy
> Fix For: 2.2-beta-6
>
>
> 1)
> maven-assembly-plugin calls DirectoryArchiver.copyFile( ArchiveEntry entry, String vPath ), which calls
> ArchiveEntryUtils.chmod( outFile, entry.getMode(), getLogger() );
> resulting in a native child Process being forked to do a chmod (on Unix only)
> The performance impact on a large assembly is enormous: patching DirectoryArchiver to not do this resulted in a performance improvement by more than 5 times.
> I've raised this with Plexus as http://jira.codehaus.org/browse/PLXCOMP-156
> 2)
> AssemblyFormatUtils should cache the environment, as AssemblyInterpolator does. Both, though, should use the latest *plexus* CommandLineUtils, as opposed to maven-assembly-plugin's own version. The plexus class uses System.getenv() (JDK5+) when available.
> Generally, please see if you can review the use of native command calls, esp. in light of JDK5/6 APIs. Forked native calls had a horrible performance impact on our (large, highly modular) assembly; which has only been corrected by local patches of maven-assembly/plexus-archiver.
> Thanks!
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-499) Poor performance in
DirectoryArchiver due to unnecessary native calls
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=236641#action_236641 ]
Olivier Lamy commented on MASSEMBLY-499:
----------------------------------------
fixed rev 1001903
> Poor performance in DirectoryArchiver due to unnecessary native calls
> ---------------------------------------------------------------------
>
> Key: MASSEMBLY-499
> URL: http://jira.codehaus.org/browse/MASSEMBLY-499
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-5
> Environment: Unix/Linux
> Reporter: Altin Papa
> Assignee: Olivier Lamy
> Fix For: 2.2-beta-6
>
>
> 1)
> maven-assembly-plugin calls DirectoryArchiver.copyFile( ArchiveEntry entry, String vPath ), which calls
> ArchiveEntryUtils.chmod( outFile, entry.getMode(), getLogger() );
> resulting in a native child Process being forked to do a chmod (on Unix only)
> The performance impact on a large assembly is enormous: patching DirectoryArchiver to not do this resulted in a performance improvement by more than 5 times.
> I've raised this with Plexus as http://jira.codehaus.org/browse/PLXCOMP-156
> 2)
> AssemblyFormatUtils should cache the environment, as AssemblyInterpolator does. Both, though, should use the latest *plexus* CommandLineUtils, as opposed to maven-assembly-plugin's own version. The plexus class uses System.getenv() (JDK5+) when available.
> Generally, please see if you can review the use of native command calls, esp. in light of JDK5/6 APIs. Forked native calls had a horrible performance impact on our (large, highly modular) assembly; which has only been corrected by local patches of maven-assembly/plexus-archiver.
> Thanks!
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-499) Poor performance in
DirectoryArchiver due to unnecessary native calls
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=236441#action_236441 ]
Olivier Lamy commented on MASSEMBLY-499:
----------------------------------------
ok so basically it works for user and all.
But as some junit test group permissions too (ie ZipArchiverTest int logMode = 0640) the result of the test will depend on the current shell umask.
> Poor performance in DirectoryArchiver due to unnecessary native calls
> ---------------------------------------------------------------------
>
> Key: MASSEMBLY-499
> URL: http://jira.codehaus.org/browse/MASSEMBLY-499
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-5
> Environment: Unix/Linux
> Reporter: Altin Papa
>
> 1)
> maven-assembly-plugin calls DirectoryArchiver.copyFile( ArchiveEntry entry, String vPath ), which calls
> ArchiveEntryUtils.chmod( outFile, entry.getMode(), getLogger() );
> resulting in a native child Process being forked to do a chmod (on Unix only)
> The performance impact on a large assembly is enormous: patching DirectoryArchiver to not do this resulted in a performance improvement by more than 5 times.
> I've raised this with Plexus as http://jira.codehaus.org/browse/PLXCOMP-156
> 2)
> AssemblyFormatUtils should cache the environment, as AssemblyInterpolator does. Both, though, should use the latest *plexus* CommandLineUtils, as opposed to maven-assembly-plugin's own version. The plexus class uses System.getenv() (JDK5+) when available.
> Generally, please see if you can review the use of native command calls, esp. in light of JDK5/6 APIs. Forked native calls had a horrible performance impact on our (large, highly modular) assembly; which has only been corrected by local patches of maven-assembly/plexus-archiver.
> Thanks!
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira