You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Scott Glajch (JIRA)" <ji...@codehaus.org> on 2012/03/01 20:50:02 UTC
[jira] (MDEP-346) maven-dependency-plugin violates artifact
dependencies, causing unstable incremental builds
Scott Glajch created MDEP-346:
---------------------------------
Summary: maven-dependency-plugin violates artifact dependencies, causing unstable incremental builds
Key: MDEP-346
URL: https://jira.codehaus.org/browse/MDEP-346
Project: Maven 2.x Dependency Plugin
Issue Type: Bug
Components: copy, copy-dependencies, unpack, unpack-dependencies
Affects Versions: 2.4
Environment: Tested on Ubuntu Linux 10.10 (Maverick) with maven 3.0.4
Reporter: Scott Glajch
If you tell the dependency plugin to unpack or copy a dependency, you are for some reason allowed to specify and artifact that's not in the project's dependency listing, whether it be transitively or specifically called out in the pom file.
What this then means is that your project can depend an another module (since the dependency plugin will use it), but as far as maven knows, it is not a dependency of the project. Therefore the reactor will not consider it when ordering modules on a build, as well as ignore it when running incremental builds (such as the -amd flag).
In our case we have a very large project (200+ modules) that we build incrementally, with help from the "incremental build" option in jenkins. This will determine which projects need building by looking at the recent changes in source control, thus building a list of projects to build. It passes those projects into maven using the project list option (i.e. -pl groupId1:artifactId1,groupId2:artifactId2). It also passes the -amd flag to then build all the modules that those projects depend on.
I have provided a very simple test case to show the problem. There is a toplevel project named "maven-toplevel-test", and 3 sub-modules named component1, component2, and component3.
Component1 depends on component3 the normal way, as an explicit dependency, and you can see that maven knows about this because when you run, from the toplevel, "mvn clean install -pl :component3 -amd", maven will correctly build component3 and any component that depends on it, which will be component1.
Component1 also depends on component2, though not explicity, but instead by running the "copy" goal of the maven-dependency-plugin and specifying component2 there. If you try run "mvn clean install -pl :component2 -amd", you'll see that only component2 gets built, because maven doesn't know enough to dig into the plugin configuration for component1 and see that component2 should be a dependency.
Now there is another similar bug open (since 2008), but that has a very different focus. http://jira.codehaus.org/browse/MDEP-153
You'll see in that bug it is trying to somehow fix the maven reactor to dig into these plugin configurations and inject dependencies.
I argue for a much different solution. I think that the maven-dependency-plugin shouldn't let you specify an artifact that isn't already a dependency of your project, or at the very least, have a flag ("strict mode" or something?) that requires all of the artifacts you are trying to copy/unpack to be dependencies of your project.
The obvious fix for our project for now is to go through all of the poms that use maven-dependency-plugin, figure out which artifacts are being manipulated and add them as dependencies to the projects, but this solution makes it hard to future-proof our projects from further mistakes. Who's to say that developers won't just keep adding new copy/unpack goals to their projects and not add those dependencies to the pom file? After all it won't break the build so the problem will only manifest itself as an unstable incremental build further down the line.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-346) maven-dependency-plugin violates artifact
dependencies, causing unstable incremental builds
Posted by "Joerg Schaible (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=293149#comment-293149 ]
Joerg Schaible commented on MDEP-346:
-------------------------------------
It's easy: Use copy-dependencies (resp. unpack-dependencies) if you rely on artifacts created within the same multi-project. Goals copy/unpack are only for released artifacts that can be resolved from a repository.
> maven-dependency-plugin violates artifact dependencies, causing unstable incremental builds
> -------------------------------------------------------------------------------------------
>
> Key: MDEP-346
> URL: https://jira.codehaus.org/browse/MDEP-346
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: copy, copy-dependencies, unpack, unpack-dependencies
> Affects Versions: 2.4
> Environment: Tested on Ubuntu Linux 10.10 (Maverick) with maven 3.0.4
> Reporter: Scott Glajch
>
> If you tell the dependency plugin to unpack or copy a dependency, you are for some reason allowed to specify and artifact that's not in the project's dependency listing, whether it be transitively or specifically called out in the pom file.
> What this then means is that your project can depend an another module (since the dependency plugin will use it), but as far as maven knows, it is not a dependency of the project. Therefore the reactor will not consider it when ordering modules on a build, as well as ignore it when running incremental builds (such as the -amd flag).
> In our case we have a very large project (200+ modules) that we build incrementally, with help from the "incremental build" option in jenkins. This will determine which projects need building by looking at the recent changes in source control, thus building a list of projects to build. It passes those projects into maven using the project list option (i.e. -pl groupId1:artifactId1,groupId2:artifactId2). It also passes the -amd flag to then build all the modules that those projects depend on.
> I have provided a very simple test case to show the problem. There is a toplevel project named "maven-toplevel-test", and 3 sub-modules named component1, component2, and component3.
> Component1 depends on component3 the normal way, as an explicit dependency, and you can see that maven knows about this because when you run, from the toplevel, "mvn clean install -pl :component3 -amd", maven will correctly build component3 and any component that depends on it, which will be component1.
> Component1 also depends on component2, though not explicity, but instead by running the "copy" goal of the maven-dependency-plugin and specifying component2 there. If you try run "mvn clean install -pl :component2 -amd", you'll see that only component2 gets built, because maven doesn't know enough to dig into the plugin configuration for component1 and see that component2 should be a dependency.
> Now there is another similar bug open (since 2008), but that has a very different focus. http://jira.codehaus.org/browse/MDEP-153
> You'll see in that bug it is trying to somehow fix the maven reactor to dig into these plugin configurations and inject dependencies.
> I argue for a much different solution. I think that the maven-dependency-plugin shouldn't let you specify an artifact that isn't already a dependency of your project, or at the very least, have a flag ("strict mode" or something?) that requires all of the artifacts you are trying to copy/unpack to be dependencies of your project.
> The obvious fix for our project for now is to go through all of the poms that use maven-dependency-plugin, figure out which artifacts are being manipulated and add them as dependencies to the projects, but this solution makes it hard to future-proof our projects from further mistakes. Who's to say that developers won't just keep adding new copy/unpack goals to their projects and not add those dependencies to the pom file? After all it won't break the build so the problem will only manifest itself as an unstable incremental build further down the line.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-346) maven-dependency-plugin violates artifact
dependencies, causing unstable incremental builds
Posted by "Scott Glajch (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=293150#comment-293150 ]
Scott Glajch commented on MDEP-346:
-----------------------------------
First off, after doing more investigation it seems that the bulk versions of the goals, copy-dependencies and unpack-dependencies do not have this bug. Those versions seem to be looping through the projects list of dependencies to do their matching, so I think only copy and unpack are affected.
I think I found a place where a pretty simple fix could be put in. In org.apache.maven.plugin.dependency.AbstractFromConfigurationMojo at about line 176 (this is from the downloaded 2.4 source zip) you could put this or a similar section of code.
java.util.Set map = project.getDependencyArtifacts();
boolean isDependencyDefined = false;
for(java.util.Iterator i = map.iterator(); i.hasNext();) {
org.apache.maven.artifact.Artifact f = (org.apache.maven.artifact.Artifact) i.next();
if (f.getArtifactId().equals(artifactItem.getArtifactId()) &&
f.getGroupId().equals(artifactItem.getGroupId()) &&
f.getVersion().equals(artifactItem.getVersion()) &&
f.getType().equals(artifactItem.getType()) &&
f.getClass().equals(artifactItem.getClassifier())) {
isDependencyDefined = true;
break;
}
}
if (!isDependencyDefined) {
throw new MojoExecutionException( "You must define the artifact " + artifactItem + " as a dependency of your project.");
}
I'm sure there's some utility somewhere in maven's code to do a quicker artifact comparison, but being unfamiliar with the code I couldn't find it easily. Also if you were to make this based off of a flag, you could put that whole comparison section in an if statement "if (useStrictDependencyChecking)" or whatever you'd want to call it, though I would still prefer that this check always exist.
> maven-dependency-plugin violates artifact dependencies, causing unstable incremental builds
> -------------------------------------------------------------------------------------------
>
> Key: MDEP-346
> URL: https://jira.codehaus.org/browse/MDEP-346
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: copy, copy-dependencies, unpack, unpack-dependencies
> Affects Versions: 2.4
> Environment: Tested on Ubuntu Linux 10.10 (Maverick) with maven 3.0.4
> Reporter: Scott Glajch
>
> If you tell the dependency plugin to unpack or copy a dependency, you are for some reason allowed to specify and artifact that's not in the project's dependency listing, whether it be transitively or specifically called out in the pom file.
> What this then means is that your project can depend an another module (since the dependency plugin will use it), but as far as maven knows, it is not a dependency of the project. Therefore the reactor will not consider it when ordering modules on a build, as well as ignore it when running incremental builds (such as the -amd flag).
> In our case we have a very large project (200+ modules) that we build incrementally, with help from the "incremental build" option in jenkins. This will determine which projects need building by looking at the recent changes in source control, thus building a list of projects to build. It passes those projects into maven using the project list option (i.e. -pl groupId1:artifactId1,groupId2:artifactId2). It also passes the -amd flag to then build all the modules that those projects depend on.
> I have provided a very simple test case to show the problem. There is a toplevel project named "maven-toplevel-test", and 3 sub-modules named component1, component2, and component3.
> Component1 depends on component3 the normal way, as an explicit dependency, and you can see that maven knows about this because when you run, from the toplevel, "mvn clean install -pl :component3 -amd", maven will correctly build component3 and any component that depends on it, which will be component1.
> Component1 also depends on component2, though not explicity, but instead by running the "copy" goal of the maven-dependency-plugin and specifying component2 there. If you try run "mvn clean install -pl :component2 -amd", you'll see that only component2 gets built, because maven doesn't know enough to dig into the plugin configuration for component1 and see that component2 should be a dependency.
> Now there is another similar bug open (since 2008), but that has a very different focus. http://jira.codehaus.org/browse/MDEP-153
> You'll see in that bug it is trying to somehow fix the maven reactor to dig into these plugin configurations and inject dependencies.
> I argue for a much different solution. I think that the maven-dependency-plugin shouldn't let you specify an artifact that isn't already a dependency of your project, or at the very least, have a flag ("strict mode" or something?) that requires all of the artifacts you are trying to copy/unpack to be dependencies of your project.
> The obvious fix for our project for now is to go through all of the poms that use maven-dependency-plugin, figure out which artifacts are being manipulated and add them as dependencies to the projects, but this solution makes it hard to future-proof our projects from further mistakes. Who's to say that developers won't just keep adding new copy/unpack goals to their projects and not add those dependencies to the pom file? After all it won't break the build so the problem will only manifest itself as an unstable incremental build further down the line.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-346) maven-dependency-plugin violates artifact
dependencies, causing unstable incremental builds
Posted by "Scott Glajch (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Scott Glajch updated MDEP-346:
------------------------------
Attachment: maven_dependency_test.zip
Test code mentioned in the original bug report.
> maven-dependency-plugin violates artifact dependencies, causing unstable incremental builds
> -------------------------------------------------------------------------------------------
>
> Key: MDEP-346
> URL: https://jira.codehaus.org/browse/MDEP-346
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: copy, copy-dependencies, unpack, unpack-dependencies
> Affects Versions: 2.4
> Environment: Tested on Ubuntu Linux 10.10 (Maverick) with maven 3.0.4
> Reporter: Scott Glajch
> Attachments: maven_dependency_test.zip
>
>
> If you tell the dependency plugin to unpack or copy a dependency, you are for some reason allowed to specify and artifact that's not in the project's dependency listing, whether it be transitively or specifically called out in the pom file.
> What this then means is that your project can depend an another module (since the dependency plugin will use it), but as far as maven knows, it is not a dependency of the project. Therefore the reactor will not consider it when ordering modules on a build, as well as ignore it when running incremental builds (such as the -amd flag).
> In our case we have a very large project (200+ modules) that we build incrementally, with help from the "incremental build" option in jenkins. This will determine which projects need building by looking at the recent changes in source control, thus building a list of projects to build. It passes those projects into maven using the project list option (i.e. -pl groupId1:artifactId1,groupId2:artifactId2). It also passes the -amd flag to then build all the modules that those projects depend on.
> I have provided a very simple test case to show the problem. There is a toplevel project named "maven-toplevel-test", and 3 sub-modules named component1, component2, and component3.
> Component1 depends on component3 the normal way, as an explicit dependency, and you can see that maven knows about this because when you run, from the toplevel, "mvn clean install -pl :component3 -amd", maven will correctly build component3 and any component that depends on it, which will be component1.
> Component1 also depends on component2, though not explicity, but instead by running the "copy" goal of the maven-dependency-plugin and specifying component2 there. If you try run "mvn clean install -pl :component2 -amd", you'll see that only component2 gets built, because maven doesn't know enough to dig into the plugin configuration for component1 and see that component2 should be a dependency.
> Now there is another similar bug open (since 2008), but that has a very different focus. http://jira.codehaus.org/browse/MDEP-153
> You'll see in that bug it is trying to somehow fix the maven reactor to dig into these plugin configurations and inject dependencies.
> I argue for a much different solution. I think that the maven-dependency-plugin shouldn't let you specify an artifact that isn't already a dependency of your project, or at the very least, have a flag ("strict mode" or something?) that requires all of the artifacts you are trying to copy/unpack to be dependencies of your project.
> The obvious fix for our project for now is to go through all of the poms that use maven-dependency-plugin, figure out which artifacts are being manipulated and add them as dependencies to the projects, but this solution makes it hard to future-proof our projects from further mistakes. Who's to say that developers won't just keep adding new copy/unpack goals to their projects and not add those dependencies to the pom file? After all it won't break the build so the problem will only manifest itself as an unstable incremental build further down the line.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-346) maven-dependency-plugin violates artifact
dependencies, causing unstable incremental builds
Posted by "Scott Glajch (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=293157#comment-293157 ]
Scott Glajch commented on MDEP-346:
-----------------------------------
I didn't realize this was intended to be the goal of copy/unpack. The multiple goal is very catch-all and a little tedious when you try to include any semi-complicated set of dependencies. If I had known this was how copy/unpack was supposed to work, I'd ban it's use across our entire project if it was possible.
The idea of removing duplication by not requiring the dependency to be defined at the project-level as well is admirable, but we can't just leave the rest of the system in the dark about it, which depends on that project-level dependency model.
Until you can get maven and all the features therein to include parsing the maven-dependency-plugin configuration to find project dependencies when it goes to create its project model, this seems too broken to leave as is. And even then I think the maven users who are trying to understand how it works, even if just enough to create a good build system for themselves, will not easily jump to the conclusion that maven has somehow figured out that a plugin has dynamically added a dependency to the pom.
I can sympathize with any project that wants something a little simpler and doesn't care about their multi-module build work correctly. It makes it that much quicker for them to copy/unpack a singular jar here and there, but that's not an argument for rejecting the feature, that's an argument for making the logic conditionalized, and perhaps even defaulted to off.
> maven-dependency-plugin violates artifact dependencies, causing unstable incremental builds
> -------------------------------------------------------------------------------------------
>
> Key: MDEP-346
> URL: https://jira.codehaus.org/browse/MDEP-346
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: copy, copy-dependencies, unpack, unpack-dependencies
> Affects Versions: 2.4
> Environment: Tested on Ubuntu Linux 10.10 (Maverick) with maven 3.0.4
> Reporter: Scott Glajch
> Attachments: maven_dependency_test.zip
>
>
> If you tell the dependency plugin to unpack or copy a dependency, you are for some reason allowed to specify and artifact that's not in the project's dependency listing, whether it be transitively or specifically called out in the pom file.
> What this then means is that your project can depend an another module (since the dependency plugin will use it), but as far as maven knows, it is not a dependency of the project. Therefore the reactor will not consider it when ordering modules on a build, as well as ignore it when running incremental builds (such as the -amd flag).
> In our case we have a very large project (200+ modules) that we build incrementally, with help from the "incremental build" option in jenkins. This will determine which projects need building by looking at the recent changes in source control, thus building a list of projects to build. It passes those projects into maven using the project list option (i.e. -pl groupId1:artifactId1,groupId2:artifactId2). It also passes the -amd flag to then build all the modules that those projects depend on.
> I have provided a very simple test case to show the problem. There is a toplevel project named "maven-toplevel-test", and 3 sub-modules named component1, component2, and component3.
> Component1 depends on component3 the normal way, as an explicit dependency, and you can see that maven knows about this because when you run, from the toplevel, "mvn clean install -pl :component3 -amd", maven will correctly build component3 and any component that depends on it, which will be component1.
> Component1 also depends on component2, though not explicity, but instead by running the "copy" goal of the maven-dependency-plugin and specifying component2 there. If you try run "mvn clean install -pl :component2 -amd", you'll see that only component2 gets built, because maven doesn't know enough to dig into the plugin configuration for component1 and see that component2 should be a dependency.
> Now there is another similar bug open (since 2008), but that has a very different focus. http://jira.codehaus.org/browse/MDEP-153
> You'll see in that bug it is trying to somehow fix the maven reactor to dig into these plugin configurations and inject dependencies.
> I argue for a much different solution. I think that the maven-dependency-plugin shouldn't let you specify an artifact that isn't already a dependency of your project, or at the very least, have a flag ("strict mode" or something?) that requires all of the artifacts you are trying to copy/unpack to be dependencies of your project.
> The obvious fix for our project for now is to go through all of the poms that use maven-dependency-plugin, figure out which artifacts are being manipulated and add them as dependencies to the projects, but this solution makes it hard to future-proof our projects from further mistakes. Who's to say that developers won't just keep adding new copy/unpack goals to their projects and not add those dependencies to the pom file? After all it won't break the build so the problem will only manifest itself as an unstable incremental build further down the line.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-346) maven-dependency-plugin violates artifact
dependencies, causing unstable incremental builds
Posted by "Joerg Schaible (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=293154#comment-293154 ]
Joerg Schaible commented on MDEP-346:
-------------------------------------
But this *is* the whole point: With copy/unpack you don't have to declare the artifacts as project dependencies. This is why these goals are duplicated with the "-dependencies" postfix.
> maven-dependency-plugin violates artifact dependencies, causing unstable incremental builds
> -------------------------------------------------------------------------------------------
>
> Key: MDEP-346
> URL: https://jira.codehaus.org/browse/MDEP-346
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: copy, copy-dependencies, unpack, unpack-dependencies
> Affects Versions: 2.4
> Environment: Tested on Ubuntu Linux 10.10 (Maverick) with maven 3.0.4
> Reporter: Scott Glajch
>
> If you tell the dependency plugin to unpack or copy a dependency, you are for some reason allowed to specify and artifact that's not in the project's dependency listing, whether it be transitively or specifically called out in the pom file.
> What this then means is that your project can depend an another module (since the dependency plugin will use it), but as far as maven knows, it is not a dependency of the project. Therefore the reactor will not consider it when ordering modules on a build, as well as ignore it when running incremental builds (such as the -amd flag).
> In our case we have a very large project (200+ modules) that we build incrementally, with help from the "incremental build" option in jenkins. This will determine which projects need building by looking at the recent changes in source control, thus building a list of projects to build. It passes those projects into maven using the project list option (i.e. -pl groupId1:artifactId1,groupId2:artifactId2). It also passes the -amd flag to then build all the modules that those projects depend on.
> I have provided a very simple test case to show the problem. There is a toplevel project named "maven-toplevel-test", and 3 sub-modules named component1, component2, and component3.
> Component1 depends on component3 the normal way, as an explicit dependency, and you can see that maven knows about this because when you run, from the toplevel, "mvn clean install -pl :component3 -amd", maven will correctly build component3 and any component that depends on it, which will be component1.
> Component1 also depends on component2, though not explicity, but instead by running the "copy" goal of the maven-dependency-plugin and specifying component2 there. If you try run "mvn clean install -pl :component2 -amd", you'll see that only component2 gets built, because maven doesn't know enough to dig into the plugin configuration for component1 and see that component2 should be a dependency.
> Now there is another similar bug open (since 2008), but that has a very different focus. http://jira.codehaus.org/browse/MDEP-153
> You'll see in that bug it is trying to somehow fix the maven reactor to dig into these plugin configurations and inject dependencies.
> I argue for a much different solution. I think that the maven-dependency-plugin shouldn't let you specify an artifact that isn't already a dependency of your project, or at the very least, have a flag ("strict mode" or something?) that requires all of the artifacts you are trying to copy/unpack to be dependencies of your project.
> The obvious fix for our project for now is to go through all of the poms that use maven-dependency-plugin, figure out which artifacts are being manipulated and add them as dependencies to the projects, but this solution makes it hard to future-proof our projects from further mistakes. Who's to say that developers won't just keep adding new copy/unpack goals to their projects and not add those dependencies to the pom file? After all it won't break the build so the problem will only manifest itself as an unstable incremental build further down the line.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira