You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Cameron Fieber (JIRA)" <ji...@codehaus.org> on 2009/06/18 23:31:15 UTC
[jira] Created: (MASSEMBLY-424) poor performance of dependencySet
in assembly descriptor (compared to using maven-dependency-plugin +
fileSet)
poor performance of dependencySet in assembly descriptor (compared to using maven-dependency-plugin + fileSet)
--------------------------------------------------------------------------------------------------------------
Key: MASSEMBLY-424
URL: http://jira.codehaus.org/browse/MASSEMBLY-424
Project: Maven 2.x Assembly Plugin
Issue Type: Improvement
Affects Versions: 2.2-beta-4
Environment: maven 2.1.0, java 6u13, os x 10.5.6, macbook pro 5400rpm disk
Reporter: Cameron Fieber
Priority: Minor
The performance of the dependencySet element in the assembly descriptor is significantly worse than achieving the equivalent result by doing an execution of dependency:copy-dependencies and including the target/dependencies folder as a fileSet in the assembly descriptor
replacing:
<assembly>
...
<dependencySets>
<dependencySet>
<outputDirectory>lib</outputDirectory>
</dependencySet>
</dependencySets>
...
</assembly>
with:
<assembly>
...
<fileSet>
<directory>${project.build.directory}/dependency</directory>
<outputDirectory>lib</outputDirectory>
</fileSet>
...
</assembly>
and (in pom.xml):
...
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<includeScope>runtime</includeScope>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-424) poor performance of dependencySet
in assembly descriptor (compared to using maven-dependency-plugin +
fileSet)
Posted by "David Boden (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MASSEMBLY-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=285016#comment-285016 ]
David Boden commented on MASSEMBLY-424:
---------------------------------------
Please note that this issue is *not* fixed in version 2.2.2 of maven-assembly-plugin. I'm sticking with my DIY 2.2.1.1 version for now.
> poor performance of dependencySet in assembly descriptor (compared to using maven-dependency-plugin + fileSet)
> --------------------------------------------------------------------------------------------------------------
>
> Key: MASSEMBLY-424
> URL: https://jira.codehaus.org/browse/MASSEMBLY-424
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-4
> Environment: maven 2.1.0, java 6u13, os x 10.5.6, macbook pro 5400rpm disk
> Reporter: Cameron Fieber
> Assignee: John Casey
> Priority: Critical
> Fix For: 2.2
>
>
> The performance of the dependencySet element in the assembly descriptor is significantly worse than achieving the equivalent result by doing an execution of dependency:copy-dependencies and including the target/dependencies folder as a fileSet in the assembly descriptor
> replacing:
> <assembly>
> ...
> <dependencySets>
> <dependencySet>
> <outputDirectory>lib</outputDirectory>
> </dependencySet>
> </dependencySets>
> ...
> </assembly>
> with:
> <assembly>
> ...
> <fileSet>
> <directory>${project.build.directory}/dependency</directory>
> <outputDirectory>lib</outputDirectory>
> </fileSet>
> ...
> </assembly>
> and (in pom.xml):
> ...
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-dependency-plugin</artifactId>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>copy-dependencies</goal>
> </goals>
> <configuration>
> <includeScope>runtime</includeScope>
> </configuration>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-assembly-plugin</artifactId>
> ...
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-424) poor performance of dependencySet
in assembly descriptor (compared to using maven-dependency-plugin +
fileSet)
Posted by "Arnaud Heritier (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=210913#action_210913 ]
Arnaud Heritier commented on MASSEMBLY-424:
-------------------------------------------
Same problem here : http://svn.exoplatform.org/projects/jcr-benchmark/trunk/
This is a regression in the plugin.
Here are some tests (mvn clean install) with this "little" project with an assembly using a dependencySet (tested at least 3 times per version to validate the time) :
with assembly plugin 2.1 : 10s
with assembly plugin 2.2-beta-1 : 11s
with assembly plugin 2.2-beta-2 : 22s
with assembly plugin 2.2-beta-3 : 36s
with assembly plugin 2.2-beta-4 : 36s
with assembly plugin 2.2-beta-5 : 36s
I change the priority to critical because the plugin is quite useless if we use this feature on a real project. I have a case where the plugin takes several minutes to build the assembly.
On some environments where IOs are worst than on mine laptop, the ratio is more important.
> poor performance of dependencySet in assembly descriptor (compared to using maven-dependency-plugin + fileSet)
> --------------------------------------------------------------------------------------------------------------
>
> Key: MASSEMBLY-424
> URL: http://jira.codehaus.org/browse/MASSEMBLY-424
> Project: Maven 2.x Assembly Plugin
> Issue Type: Improvement
> Affects Versions: 2.2-beta-4
> Environment: maven 2.1.0, java 6u13, os x 10.5.6, macbook pro 5400rpm disk
> Reporter: Cameron Fieber
> Priority: Minor
>
> The performance of the dependencySet element in the assembly descriptor is significantly worse than achieving the equivalent result by doing an execution of dependency:copy-dependencies and including the target/dependencies folder as a fileSet in the assembly descriptor
> replacing:
> <assembly>
> ...
> <dependencySets>
> <dependencySet>
> <outputDirectory>lib</outputDirectory>
> </dependencySet>
> </dependencySets>
> ...
> </assembly>
> with:
> <assembly>
> ...
> <fileSet>
> <directory>${project.build.directory}/dependency</directory>
> <outputDirectory>lib</outputDirectory>
> </fileSet>
> ...
> </assembly>
> and (in pom.xml):
> ...
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-dependency-plugin</artifactId>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>copy-dependencies</goal>
> </goals>
> <configuration>
> <includeScope>runtime</includeScope>
> </configuration>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-assembly-plugin</artifactId>
> ...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Issue Comment Edited: (MASSEMBLY-424) poor performance of
dependencySet in assembly descriptor (compared to using
maven-dependency-plugin + fileSet)
Posted by "David Boden (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MASSEMBLY-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=280380#comment-280380 ]
David Boden edited comment on MASSEMBLY-424 at 9/30/11 9:53 AM:
----------------------------------------------------------------
I'd be grateful if someone "in the know" could either reopen this issue or spin my comments off into a separate Jira.
With a large build, forking (twice!) to execute an "ls" on Linux to find out the file attributes (owner, permissions) results in thousands of forked processes and a huge amount of added build time.
I've cut down my build time by a factor of 10 by:
* Checking out tag 2.2.1 of maven-assembly-plugin
* Changing the following dependency versions:
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-io</artifactId>
<version>2.0.1</version>
AND
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-utils</artifactId>
<version>3.0</version>
* Upgrading to Java 7 - The reason for this is in PlexusIoResourceAttributeUtils.java line 224, under the newer plexus-io-2.0.1 jar.
* The plexus io utility checks whether we're building under Java 7 and if we are, uses (fast) Java calls to look at the file properties rather than forking an "ls".
* It uses Java 7's: http://download.oracle.com/javase/7/docs/api/java/nio/file/attribute/PosixFilePermissions.html
Hopefully this is very simple to do on the assembly plugin trunk and, by switching to Java 7, other users can speed up their builds. Any chance of a proper build with these dependencies upgraded? I've created my own 2.2.1.1 for now.
was (Author: david.boden@nomura.com):
I'd be grateful if an someone "in the know" could either reopen this issue or spin my comments off into a separate Jira.
With a large build, forking (twice!) to execute an "ls" on Linux to find out the file attributes (owner, permissions) results in thousands of forked processes and a huge amount of added build time.
I've cut down my build time by a factor of 10 by:
* Checking out tag 2.2.1 of maven-assembly-plugin
* Changing the following dependency versions:
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-io</artifactId>
<version>2.0.1</version>
AND
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-utils</artifactId>
<version>3.0</version>
* Upgrading to Java 7 - The reason for this is in PlexusIoResourceAttributeUtils.java line 224, under the newer plexus-io-2.0.1 jar.
* The plexus io utility checks whether we're building under Java 7 and if we are, uses (fast) Java calls to look at the file properties rather than forking an "ls".
* It uses Java 7's: http://download.oracle.com/javase/7/docs/api/java/nio/file/attribute/PosixFilePermissions.html
Hopefully this is very simple to do on the assembly plugin trunk and, by switching to Java 7, other users can speed up their builds. Any chance of a proper build with these dependencies upgraded? I've created my own 2.2.1.1 for now.
> poor performance of dependencySet in assembly descriptor (compared to using maven-dependency-plugin + fileSet)
> --------------------------------------------------------------------------------------------------------------
>
> Key: MASSEMBLY-424
> URL: https://jira.codehaus.org/browse/MASSEMBLY-424
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-4
> Environment: maven 2.1.0, java 6u13, os x 10.5.6, macbook pro 5400rpm disk
> Reporter: Cameron Fieber
> Assignee: John Casey
> Priority: Critical
> Fix For: 2.2
>
>
> The performance of the dependencySet element in the assembly descriptor is significantly worse than achieving the equivalent result by doing an execution of dependency:copy-dependencies and including the target/dependencies folder as a fileSet in the assembly descriptor
> replacing:
> <assembly>
> ...
> <dependencySets>
> <dependencySet>
> <outputDirectory>lib</outputDirectory>
> </dependencySet>
> </dependencySets>
> ...
> </assembly>
> with:
> <assembly>
> ...
> <fileSet>
> <directory>${project.build.directory}/dependency</directory>
> <outputDirectory>lib</outputDirectory>
> </fileSet>
> ...
> </assembly>
> and (in pom.xml):
> ...
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-dependency-plugin</artifactId>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>copy-dependencies</goal>
> </goals>
> <configuration>
> <includeScope>runtime</includeScope>
> </configuration>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-assembly-plugin</artifactId>
> ...
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (MASSEMBLY-424) poor performance of dependencySet in
assembly descriptor (compared to using maven-dependency-plugin + fileSet)
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Casey closed MASSEMBLY-424.
--------------------------------
Resolution: Fixed
Fix Version/s: 2.2-beta-6
Assignee: John Casey
dependency resolution only happens once with the new code, so it should be on par with other plugins now once 2.2-beta-6 (or, hopefully, we'll call it 2.2 final) comes out.
> poor performance of dependencySet in assembly descriptor (compared to using maven-dependency-plugin + fileSet)
> --------------------------------------------------------------------------------------------------------------
>
> Key: MASSEMBLY-424
> URL: http://jira.codehaus.org/browse/MASSEMBLY-424
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-4
> Environment: maven 2.1.0, java 6u13, os x 10.5.6, macbook pro 5400rpm disk
> Reporter: Cameron Fieber
> Assignee: John Casey
> Priority: Critical
> Fix For: 2.2-beta-6
>
>
> The performance of the dependencySet element in the assembly descriptor is significantly worse than achieving the equivalent result by doing an execution of dependency:copy-dependencies and including the target/dependencies folder as a fileSet in the assembly descriptor
> replacing:
> <assembly>
> ...
> <dependencySets>
> <dependencySet>
> <outputDirectory>lib</outputDirectory>
> </dependencySet>
> </dependencySets>
> ...
> </assembly>
> with:
> <assembly>
> ...
> <fileSet>
> <directory>${project.build.directory}/dependency</directory>
> <outputDirectory>lib</outputDirectory>
> </fileSet>
> ...
> </assembly>
> and (in pom.xml):
> ...
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-dependency-plugin</artifactId>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>copy-dependencies</goal>
> </goals>
> <configuration>
> <includeScope>runtime</includeScope>
> </configuration>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-assembly-plugin</artifactId>
> ...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Issue Comment Edited: (MASSEMBLY-424) poor performance of
dependencySet in assembly descriptor (compared to using
maven-dependency-plugin + fileSet)
Posted by "David Boden (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MASSEMBLY-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=280380#comment-280380 ]
David Boden edited comment on MASSEMBLY-424 at 9/30/11 9:53 AM:
----------------------------------------------------------------
I'd be grateful if an someone "in the know" could either reopen this issue or spin my comments off into a separate Jira.
With a large build, forking (twice!) to execute an "ls" on Linux to find out the file attributes (owner, permissions) results in thousands of forked processes and a huge amount of added build time.
I've cut down my build time by a factor of 10 by:
* Checking out tag 2.2.1 of maven-assembly-plugin
* Changing the following dependency versions:
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-io</artifactId>
<version>2.0.1</version>
AND
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-utils</artifactId>
<version>3.0</version>
* Upgrading to Java 7 - The reason for this is in PlexusIoResourceAttributeUtils.java line 224, under the newer plexus-io-2.0.1 jar.
* The plexus io utility checks whether we're building under Java 7 and if we are, uses (fast) Java calls to look at the file properties rather than forking an "ls".
* It uses Java 7's: http://download.oracle.com/javase/7/docs/api/java/nio/file/attribute/PosixFilePermissions.html
Hopefully this is very simple to do on the assembly plugin trunk and, by switching to Java 7, other users can speed up their builds.
was (Author: david.boden@nomura.com):
I'd be grateful if an someone "in the know" could either reopen this issue or spin my comments off into a separate Jira.
With a large build, forking (twice!) to execute an "ls" on Linux to find out the file attributes (owner, permissions) results in thousands of forked processes and a huge amount of added build time.
I've cut down my build time by a factor of 10 by:
* Checking out tag 2.2.1 of maven-assembly-plugin
* Changing the following dependency versions:
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-io</artifactId>
<version>2.0.1</version>
AND
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-utils</artifactId>
<version>3.0</version>
* Upgrading to Java 7 - The reason for this is in PlexusIoResourceAttributeUtils.java line 224, under the newer plexus-io-2.0.1 jar.
* The plexus io utility checks whether we're building under Java 7 and if we are, uses (fast) Java calls to look at the file properties rather than forking an "ls".
Hopefully this is very simple to do on the assembly plugin trunk and, by switching to Java 7, other users can speed up their builds.
> poor performance of dependencySet in assembly descriptor (compared to using maven-dependency-plugin + fileSet)
> --------------------------------------------------------------------------------------------------------------
>
> Key: MASSEMBLY-424
> URL: https://jira.codehaus.org/browse/MASSEMBLY-424
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-4
> Environment: maven 2.1.0, java 6u13, os x 10.5.6, macbook pro 5400rpm disk
> Reporter: Cameron Fieber
> Assignee: John Casey
> Priority: Critical
> Fix For: 2.2
>
>
> The performance of the dependencySet element in the assembly descriptor is significantly worse than achieving the equivalent result by doing an execution of dependency:copy-dependencies and including the target/dependencies folder as a fileSet in the assembly descriptor
> replacing:
> <assembly>
> ...
> <dependencySets>
> <dependencySet>
> <outputDirectory>lib</outputDirectory>
> </dependencySet>
> </dependencySets>
> ...
> </assembly>
> with:
> <assembly>
> ...
> <fileSet>
> <directory>${project.build.directory}/dependency</directory>
> <outputDirectory>lib</outputDirectory>
> </fileSet>
> ...
> </assembly>
> and (in pom.xml):
> ...
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-dependency-plugin</artifactId>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>copy-dependencies</goal>
> </goals>
> <configuration>
> <includeScope>runtime</includeScope>
> </configuration>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-assembly-plugin</artifactId>
> ...
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-424) poor performance of dependencySet
in assembly descriptor (compared to using maven-dependency-plugin +
fileSet)
Posted by "David Boden (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MASSEMBLY-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=280380#comment-280380 ]
David Boden commented on MASSEMBLY-424:
---------------------------------------
I'd be grateful if an someone "in the know" could either reopen this issue or spin my comments off into a separate Jira.
With a large build, forking (twice!) to execute an "ls" on Linux to find out the file attributes (owner, permissions) results in thousands of forked processes and a huge amount of added build time.
I've cut down my build time by a factor of 10 by:
* Checking out tag 2.2.1 of maven-assembly-plugin
* Changing the following dependency versions:
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-io</artifactId>
<version>2.0.1</version>
AND
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-utils</artifactId>
<version>3.0</version>
* Upgrading to Java 7 - The reason for this is in PlexusIoResourceAttributeUtils.java line 224, under the newer plexus-io-2.0.1 jar.
* The plexus io utility checks whether we're building under Java 7 and if we are, uses (fast) Java calls to look at the file properties rather than forking an "ls".
Hopefully this is very simple to do on the assembly plugin trunk and, by switching to Java 7, other users can speed up their builds.
> poor performance of dependencySet in assembly descriptor (compared to using maven-dependency-plugin + fileSet)
> --------------------------------------------------------------------------------------------------------------
>
> Key: MASSEMBLY-424
> URL: https://jira.codehaus.org/browse/MASSEMBLY-424
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-4
> Environment: maven 2.1.0, java 6u13, os x 10.5.6, macbook pro 5400rpm disk
> Reporter: Cameron Fieber
> Assignee: John Casey
> Priority: Critical
> Fix For: 2.2
>
>
> The performance of the dependencySet element in the assembly descriptor is significantly worse than achieving the equivalent result by doing an execution of dependency:copy-dependencies and including the target/dependencies folder as a fileSet in the assembly descriptor
> replacing:
> <assembly>
> ...
> <dependencySets>
> <dependencySet>
> <outputDirectory>lib</outputDirectory>
> </dependencySet>
> </dependencySets>
> ...
> </assembly>
> with:
> <assembly>
> ...
> <fileSet>
> <directory>${project.build.directory}/dependency</directory>
> <outputDirectory>lib</outputDirectory>
> </fileSet>
> ...
> </assembly>
> and (in pom.xml):
> ...
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-dependency-plugin</artifactId>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>copy-dependencies</goal>
> </goals>
> <configuration>
> <includeScope>runtime</includeScope>
> </configuration>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-assembly-plugin</artifactId>
> ...
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-424) poor performance of dependencySet
in assembly descriptor (compared to using maven-dependency-plugin +
fileSet)
Posted by "Cameron Fieber (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=180867#action_180867 ]
Cameron Fieber commented on MASSEMBLY-424:
------------------------------------------
rough timing differences that I've found
simple jar project with assembly zip that includes its libs (approximately 12 jars as transitive dependencies in runtime scope):
dependencySet: 18s
copy-dependencies + fileset: 12s
complex installer project with approximatly 40 jars as transitive dependencies in runtime scope:
dependencySet: 60s
copy-dependencies + fileset: 40s
> poor performance of dependencySet in assembly descriptor (compared to using maven-dependency-plugin + fileSet)
> --------------------------------------------------------------------------------------------------------------
>
> Key: MASSEMBLY-424
> URL: http://jira.codehaus.org/browse/MASSEMBLY-424
> Project: Maven 2.x Assembly Plugin
> Issue Type: Improvement
> Affects Versions: 2.2-beta-4
> Environment: maven 2.1.0, java 6u13, os x 10.5.6, macbook pro 5400rpm disk
> Reporter: Cameron Fieber
> Priority: Minor
>
> The performance of the dependencySet element in the assembly descriptor is significantly worse than achieving the equivalent result by doing an execution of dependency:copy-dependencies and including the target/dependencies folder as a fileSet in the assembly descriptor
> replacing:
> <assembly>
> ...
> <dependencySets>
> <dependencySet>
> <outputDirectory>lib</outputDirectory>
> </dependencySet>
> </dependencySets>
> ...
> </assembly>
> with:
> <assembly>
> ...
> <fileSet>
> <directory>${project.build.directory}/dependency</directory>
> <outputDirectory>lib</outputDirectory>
> </fileSet>
> ...
> </assembly>
> and (in pom.xml):
> ...
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-dependency-plugin</artifactId>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>copy-dependencies</goal>
> </goals>
> <configuration>
> <includeScope>runtime</includeScope>
> </configuration>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-assembly-plugin</artifactId>
> ...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MASSEMBLY-424) poor performance of dependencySet
in assembly descriptor (compared to using maven-dependency-plugin +
fileSet)
Posted by "Arnaud Heritier (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arnaud Heritier updated MASSEMBLY-424:
--------------------------------------
Priority: Critical (was: Minor)
Issue Type: Bug (was: Improvement)
> poor performance of dependencySet in assembly descriptor (compared to using maven-dependency-plugin + fileSet)
> --------------------------------------------------------------------------------------------------------------
>
> Key: MASSEMBLY-424
> URL: http://jira.codehaus.org/browse/MASSEMBLY-424
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-4
> Environment: maven 2.1.0, java 6u13, os x 10.5.6, macbook pro 5400rpm disk
> Reporter: Cameron Fieber
> Priority: Critical
>
> The performance of the dependencySet element in the assembly descriptor is significantly worse than achieving the equivalent result by doing an execution of dependency:copy-dependencies and including the target/dependencies folder as a fileSet in the assembly descriptor
> replacing:
> <assembly>
> ...
> <dependencySets>
> <dependencySet>
> <outputDirectory>lib</outputDirectory>
> </dependencySet>
> </dependencySets>
> ...
> </assembly>
> with:
> <assembly>
> ...
> <fileSet>
> <directory>${project.build.directory}/dependency</directory>
> <outputDirectory>lib</outputDirectory>
> </fileSet>
> ...
> </assembly>
> and (in pom.xml):
> ...
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-dependency-plugin</artifactId>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>copy-dependencies</goal>
> </goals>
> <configuration>
> <includeScope>runtime</includeScope>
> </configuration>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-assembly-plugin</artifactId>
> ...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Issue Comment Edited: (MASSEMBLY-424) poor performance of
dependencySet in assembly descriptor (compared to using
maven-dependency-plugin + fileSet)
Posted by "David Boden (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MASSEMBLY-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=280380#comment-280380 ]
David Boden edited comment on MASSEMBLY-424 at 9/30/11 9:53 AM:
----------------------------------------------------------------
I'd be grateful if an someone "in the know" could either reopen this issue or spin my comments off into a separate Jira.
With a large build, forking (twice!) to execute an "ls" on Linux to find out the file attributes (owner, permissions) results in thousands of forked processes and a huge amount of added build time.
I've cut down my build time by a factor of 10 by:
* Checking out tag 2.2.1 of maven-assembly-plugin
* Changing the following dependency versions:
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-io</artifactId>
<version>2.0.1</version>
AND
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-utils</artifactId>
<version>3.0</version>
* Upgrading to Java 7 - The reason for this is in PlexusIoResourceAttributeUtils.java line 224, under the newer plexus-io-2.0.1 jar.
* The plexus io utility checks whether we're building under Java 7 and if we are, uses (fast) Java calls to look at the file properties rather than forking an "ls".
* It uses Java 7's: http://download.oracle.com/javase/7/docs/api/java/nio/file/attribute/PosixFilePermissions.html
Hopefully this is very simple to do on the assembly plugin trunk and, by switching to Java 7, other users can speed up their builds. Any chance of a proper build with these dependencies upgraded? I've created my own 2.2.1.1 for now.
was (Author: david.boden@nomura.com):
I'd be grateful if an someone "in the know" could either reopen this issue or spin my comments off into a separate Jira.
With a large build, forking (twice!) to execute an "ls" on Linux to find out the file attributes (owner, permissions) results in thousands of forked processes and a huge amount of added build time.
I've cut down my build time by a factor of 10 by:
* Checking out tag 2.2.1 of maven-assembly-plugin
* Changing the following dependency versions:
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-io</artifactId>
<version>2.0.1</version>
AND
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-utils</artifactId>
<version>3.0</version>
* Upgrading to Java 7 - The reason for this is in PlexusIoResourceAttributeUtils.java line 224, under the newer plexus-io-2.0.1 jar.
* The plexus io utility checks whether we're building under Java 7 and if we are, uses (fast) Java calls to look at the file properties rather than forking an "ls".
* It uses Java 7's: http://download.oracle.com/javase/7/docs/api/java/nio/file/attribute/PosixFilePermissions.html
Hopefully this is very simple to do on the assembly plugin trunk and, by switching to Java 7, other users can speed up their builds.
> poor performance of dependencySet in assembly descriptor (compared to using maven-dependency-plugin + fileSet)
> --------------------------------------------------------------------------------------------------------------
>
> Key: MASSEMBLY-424
> URL: https://jira.codehaus.org/browse/MASSEMBLY-424
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-4
> Environment: maven 2.1.0, java 6u13, os x 10.5.6, macbook pro 5400rpm disk
> Reporter: Cameron Fieber
> Assignee: John Casey
> Priority: Critical
> Fix For: 2.2
>
>
> The performance of the dependencySet element in the assembly descriptor is significantly worse than achieving the equivalent result by doing an execution of dependency:copy-dependencies and including the target/dependencies folder as a fileSet in the assembly descriptor
> replacing:
> <assembly>
> ...
> <dependencySets>
> <dependencySet>
> <outputDirectory>lib</outputDirectory>
> </dependencySet>
> </dependencySets>
> ...
> </assembly>
> with:
> <assembly>
> ...
> <fileSet>
> <directory>${project.build.directory}/dependency</directory>
> <outputDirectory>lib</outputDirectory>
> </fileSet>
> ...
> </assembly>
> and (in pom.xml):
> ...
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-dependency-plugin</artifactId>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>copy-dependencies</goal>
> </goals>
> <configuration>
> <includeScope>runtime</includeScope>
> </configuration>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-assembly-plugin</artifactId>
> ...
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-424) poor performance of dependencySet
in assembly descriptor (compared to using maven-dependency-plugin +
fileSet)
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=215977#action_215977 ]
Olivier Lamy commented on MASSEMBLY-424:
----------------------------------------
probably linked to PLXUTILS-127 .
Can you test your projects with p-u 2.0.2 ?
> poor performance of dependencySet in assembly descriptor (compared to using maven-dependency-plugin + fileSet)
> --------------------------------------------------------------------------------------------------------------
>
> Key: MASSEMBLY-424
> URL: http://jira.codehaus.org/browse/MASSEMBLY-424
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-4
> Environment: maven 2.1.0, java 6u13, os x 10.5.6, macbook pro 5400rpm disk
> Reporter: Cameron Fieber
> Priority: Critical
>
> The performance of the dependencySet element in the assembly descriptor is significantly worse than achieving the equivalent result by doing an execution of dependency:copy-dependencies and including the target/dependencies folder as a fileSet in the assembly descriptor
> replacing:
> <assembly>
> ...
> <dependencySets>
> <dependencySet>
> <outputDirectory>lib</outputDirectory>
> </dependencySet>
> </dependencySets>
> ...
> </assembly>
> with:
> <assembly>
> ...
> <fileSet>
> <directory>${project.build.directory}/dependency</directory>
> <outputDirectory>lib</outputDirectory>
> </fileSet>
> ...
> </assembly>
> and (in pom.xml):
> ...
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-dependency-plugin</artifactId>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>copy-dependencies</goal>
> </goals>
> <configuration>
> <includeScope>runtime</includeScope>
> </configuration>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-assembly-plugin</artifactId>
> ...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-424) poor performance of dependencySet
in assembly descriptor (compared to using maven-dependency-plugin +
fileSet)
Posted by "Arnaud Heritier (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=217000#action_217000 ]
Arnaud Heritier commented on MASSEMBLY-424:
-------------------------------------------
New set of tests always based on http://svn.exoplatform.org/projects/jcr-benchmark/trunk/ with mvn clean install -o
Env :
Apache Maven 3.0-alpha-7 (r921173; 2010-03-09 23:31:07+0100)
Java version: 1.6.0_17
Java home: /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home
Default locale: en_US, platform encoding: MacRoman
OS name: "mac os x" version: "10.6.3" arch: "x86_64" Family: "mac"
Each test is done twice and the best value is taken.
with assembly plugin 2.1 : 11.022s
with assembly plugin 2.2-beta-1 : 11.670s
with assembly plugin 2.2-beta-2 : 19.772s
with assembly plugin 2.2-beta-3 : 30.480s
with assembly plugin 2.2-beta-4 : 30.049s
with assembly plugin 2.2-beta-5 : 30.927s
with assembly plugin 2.2-beta-6-SNAPSHOT : 31.460s
with assembly plugin 2.2-beta-6-SNAPSHOT with p-u 2.0.2 : 23.964s
It's better but not perfect.
I tried to upgrade p-u to 2.0.3, p-io to 1.0-alpha-5, maven-common-artifact-filters to 1.2 ... but it didn't changed the result.
> poor performance of dependencySet in assembly descriptor (compared to using maven-dependency-plugin + fileSet)
> --------------------------------------------------------------------------------------------------------------
>
> Key: MASSEMBLY-424
> URL: http://jira.codehaus.org/browse/MASSEMBLY-424
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-4
> Environment: maven 2.1.0, java 6u13, os x 10.5.6, macbook pro 5400rpm disk
> Reporter: Cameron Fieber
> Priority: Critical
>
> The performance of the dependencySet element in the assembly descriptor is significantly worse than achieving the equivalent result by doing an execution of dependency:copy-dependencies and including the target/dependencies folder as a fileSet in the assembly descriptor
> replacing:
> <assembly>
> ...
> <dependencySets>
> <dependencySet>
> <outputDirectory>lib</outputDirectory>
> </dependencySet>
> </dependencySets>
> ...
> </assembly>
> with:
> <assembly>
> ...
> <fileSet>
> <directory>${project.build.directory}/dependency</directory>
> <outputDirectory>lib</outputDirectory>
> </fileSet>
> ...
> </assembly>
> and (in pom.xml):
> ...
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-dependency-plugin</artifactId>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>copy-dependencies</goal>
> </goals>
> <configuration>
> <includeScope>runtime</includeScope>
> </configuration>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-assembly-plugin</artifactId>
> ...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MASSEMBLY-424) poor performance of dependencySet
in assembly descriptor (compared to using maven-dependency-plugin +
fileSet)
Posted by "Paul Smith (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/MASSEMBLY-424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=214127#action_214127 ]
Paul Smith commented on MASSEMBLY-424:
--------------------------------------
through a set of thread dumps taken during an assembly run, I notice frequent time spent in this state:
{noformat}
"main" prio=5 tid=0x0000000102800000 nid=0x100501000 runnable [0x00000001004fe000]
java.lang.Thread.State: RUNNABLE
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:53)
at java.lang.ProcessImpl.start(ProcessImpl.java:91)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
at java.lang.Runtime.exec(Runtime.java:593)
at java.lang.Runtime.exec(Runtime.java:509)
at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:636)
at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:140)
at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:103)
at org.codehaus.plexus.components.io.attributes.PlexusIoResourceAttributeUtils.executeLs(PlexusIoResourceAttributeUtils.java:277)
at org.codehaus.plexus.components.io.attributes.PlexusIoResourceAttributeUtils.getFileAttributesByPath(PlexusIoResourceAttributeUtils.java:237)
at org.codehaus.plexus.components.io.attributes.PlexusIoResourceAttributeUtils.getFileAttributes(PlexusIoResourceAttributeUtils.java:167)
at org.codehaus.plexus.archiver.ArchiveEntry.createFileEntry(ArchiveEntry.java:157)
at org.codehaus.plexus.archiver.AbstractArchiver.addFile(AbstractArchiver.java:374)
at org.codehaus.plexus.archiver.AbstractArchiver.addFile(AbstractArchiver.java:290)
at org.apache.maven.plugin.assembly.archive.archiver.AssemblyProxyArchiver.addFile(AssemblyProxyArchiver.java:474)
at org.apache.maven.plugin.assembly.archive.task.AddArtifactTask.execute(AddArtifactTask.java:203)
at org.apache.maven.plugin.assembly.archive.task.AddDependencySetsTask.addDependencySet(AddDependencySetsTask.java:195)
at org.apache.maven.plugin.assembly.archive.task.AddDependencySetsTask.execute(AddDependencySetsTask.java:122)
at org.apache.maven.plugin.assembly.archive.phase.ModuleSetAssemblyPhase.addModuleBinaries(ModuleSetAssemblyPhase.java:236)
at org.apache.maven.plugin.assembly.archive.phase.ModuleSetAssemblyPhase.execute(ModuleSetAssemblyPhase.java:131)
at org.apache.maven.plugin.assembly.archive.DefaultAssemblyArchiver.createArchive(DefaultAssemblyArchiver.java:190)
at org.apache.maven.plugin.assembly.mojos.AbstractDirectoryMojo.createDirectory(AbstractDirectoryMojo.java:85)
at org.apache.maven.plugin.assembly.mojos.AbstractDirectoryMojo.execute(AbstractDirectoryMojo.java:63)
at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:483)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:678)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandaloneGoal(DefaultLifecycleExecutor.java:553)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:523)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:371)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:268)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:181)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:356)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:137)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:41)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
{noformat}
>From looking into the PlexusIoResourceAttributeUtils.getFileAttributesByPath(..) method used by the addFile method during the dependency archive, it is doing a Process Fork per-Dependency to get the file attributes, which is a bit yucky.... that's not going to scale very well at all.
> poor performance of dependencySet in assembly descriptor (compared to using maven-dependency-plugin + fileSet)
> --------------------------------------------------------------------------------------------------------------
>
> Key: MASSEMBLY-424
> URL: http://jira.codehaus.org/browse/MASSEMBLY-424
> Project: Maven 2.x Assembly Plugin
> Issue Type: Bug
> Affects Versions: 2.2-beta-4
> Environment: maven 2.1.0, java 6u13, os x 10.5.6, macbook pro 5400rpm disk
> Reporter: Cameron Fieber
> Priority: Critical
>
> The performance of the dependencySet element in the assembly descriptor is significantly worse than achieving the equivalent result by doing an execution of dependency:copy-dependencies and including the target/dependencies folder as a fileSet in the assembly descriptor
> replacing:
> <assembly>
> ...
> <dependencySets>
> <dependencySet>
> <outputDirectory>lib</outputDirectory>
> </dependencySet>
> </dependencySets>
> ...
> </assembly>
> with:
> <assembly>
> ...
> <fileSet>
> <directory>${project.build.directory}/dependency</directory>
> <outputDirectory>lib</outputDirectory>
> </fileSet>
> ...
> </assembly>
> and (in pom.xml):
> ...
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-dependency-plugin</artifactId>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>copy-dependencies</goal>
> </goals>
> <configuration>
> <includeScope>runtime</includeScope>
> </configuration>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-assembly-plugin</artifactId>
> ...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira