You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Laird Nelson <lj...@gmail.com> on 2014/01/03 23:03:22 UTC
Topologically sorted list of Artifacts for use by a plugin?
Hello; in Maven 3.0.5 and 3.1.x, what is the preferred way, given a
MavenProject representing the current pom.xml/project, to obtain a list of
its resolved dependency Artifacts in test scope that has been topologically
sorted such that the Artifact at the head of the list has the fewest
dependencies and the artifact at the tail of the list has the most?
I asked this question many moons ago (
http://maven.40175.n5.nabble.com/Topologically-sorting-dependencies-tp3384898p3387803.html)
and after no answers, came up with what felt like an awful hack to produce
what I wanted, whose code I pasted in that message.
If you follow the link, you'll see that I used the ProjectSorter class,
which is what Maven's reactor uses when processing a multi-module
project--which gives me exactly the order I wanted--but seems like overkill
and like misusing a tool, and maybe isn't the Right Way given all the new
Aether stuff and whatnot. Nevertheless, it is still how Maven's reactor
works, so...who knows.
Anyway, I started there and "worked backwards"--needed a List of
MavenProjects to feed to it, so ended up using the MavenProjectBuilder, as
you can see in the link.
That class has since been deprecated and moved to the compat layer, though,
and its replacement (another class named DefaultMavenProjectBuilder) has
removed the convenience method that I was using. I'd like to update my
plugin to avoid depending on this deprecated subsystem.
So then: suppose I'm writing a Maven plugin that needs to fetch a
particular well-known classpath resource from all artifacts available in
test scope, and needs that list to be in topological order (starting with
an artifact that has the fewest dependencies and ending with the artifact
that has the most). What is the preferred mechanism for doing this these
days? Is it still by way of the ProjectSorter hammer? Or is there a more
elegant way to do it?
(As background: I am working with Liquibase (http://www.liquibase.org/).
Each of my .jar projects has a META-INF/liquibase/changelog.xml file in
it. Various of these .jar projects already depend on each other--for unit
and integration testing, I'd like to use this dependency order, harvest the
changelog.xml resources in each .jar file, and then combine them to create
only the database tables actually needed for the test. If a.jar has A's
tables, and b.jar has B's tables and b.jar depends on a.jar, then I'd like
to run a.jar!/META-INF/liquibase/changelog.xml first, then
b.jar!/META-INF/liquibase/changelog.xml next, and I'd like to not have to
specify this in any place in my pom.xmls, since the dependency order is
already captured there.)
Best,
Happy 2014,
Laird
--
http://about.me/lairdnelson
Re: Topologically sorted list of Artifacts for use by a plugin?
Posted by Laird Nelson <lj...@gmail.com>.
On Fri, Jan 3, 2014 at 2:03 PM, Laird Nelson <lj...@gmail.com> wrote:
> Hello; in Maven 3.0.5 and 3.1.x, what is the preferred way, given a
> MavenProject representing the current pom.xml/project, to obtain a list of
> its resolved dependency Artifacts in test scope that has been topologically
> sorted such that the Artifact at the head of the list has the fewest
> dependencies and the artifact at the tail of the list has the most?
>
As an aside, I found
http://maven.40175.n5.nabble.com/Sort-dependencies-topologically-tp4971787p5036212.html,
but am not clear if that follows the same rules that Maven does (regarding
things like dependency management, pom inheritance, settings.xmls etc.).
Thanks,
Best,
Laird
--
http://about.me/lairdnelson
Re: Topologically sorted list of Artifacts for use by a plugin?
Posted by Laird Nelson <lj...@gmail.com>.
On Sun, Jan 5, 2014 at 3:24 PM, Martin Gainty <mg...@hotmail.com> wrote:
>
> MG_01_05>Thanks for taking time to 'think this through'
>
More thinking and coding. Here's my final answer:
https://github.com/ljnelson/maven-artifacts/blob/master/src/main/java/com/edugility/maven/Artifacts.java#L80
That method builds the dependency graph in a way that is friendly to both
Maven 3.1.1 and Maven 3.0.5, using only non-deprecated classes and methods,
and using the same dependency graph building and resolution strategy that
Maven uses internally. The output is a non-null unmodifiable Collection of
non-null resolved Artifacts sorted from most standalone to most dependent
that contains a MavenProject's associated Artifact and all of the Artifacts
that he depends on. I'm putting this here in the mailing list for
posterity; there have been several questions about it (not just mine) and
so hopefully I've helped someone out.
Best,
Laird
--
http://about.me/lairdnelson
RE: Topologically sorted list of Artifacts for use by a plugin?
Posted by Martin Gainty <mg...@hotmail.com>.
> Date: Sat, 4 Jan 2014 16:11:02 -0800
> Subject: Re: Topologically sorted list of Artifacts for use by a plugin?
> From: ljnelson@gmail.com
> To: users@maven.apache.org
>
> On Fri, Jan 3, 2014 at 6:17 PM, Martin Gainty <mg...@hotmail.com> wrote:
>
> > MG>so I would say you are definitely on the right track
> >
>
> Thanks; I'm not sure I am.
>
>
> > MG>any reason for topological sorting of dependencies?
> >
>
> In my original post, I wrote:
>
> (As background: I am working with Liquibase (http://www.liquibase.org/).
> > Each of my .jar projects has a META-INF/liquibase/changelog.xml file in
> > it. Various of these .jar projects already depend on each other--for unit
> > and integration testing, I'd like to use this dependency order, harvest the
> > changelog.xml resources in each .jar file, and then combine them to create
> > only the database tables actually needed for the test. If a.jar has A's
> > tables, and b.jar has B's tables and b.jar depends on a.jar, then I'd like
> > to run a.jar!/META-INF/liquibase/changelog.xml first, then
> > b.jar!/META-INF/liquibase/changelog.xml next, and I'd like to not have to
> > specify this in any place in my pom.xmls, since the dependency order is
> > already captured there.)
>
>
> So the fact that I'm using
> ProjectSorter<http://maven.apache.org/ref/3.0.5/maven-core/apidocs/org/apache/maven/project/ProjectSorter.html>strikes
> me as wrong—worse than wrong: an awful hack, in fact—because I am
> basically using the reactor to sort MavenProjects that I've roped in from
> elsewhere. Yucko.
>
> Various people have noted tersely that the maven-dependency-plugin is what
> I want. Of course, its *goals* are completely useless for this task
MG_01_05>I went the same tack but I couldnt find a goal to collect all the dependencies so I deprecated that thought process
>—but
> buried in an obscure StackOverflow comment tangentially related to the
> maven-dependency-plugin I found a reference to this class:
> http://maven.apache.org/shared/maven-dependency-tree/apidocs/index.html?org/apache/maven/shared/dependency/graph/DependencyGraphBuilder.html
> If I understand it right, then code like the following should do the
> trick
> (untested, but you get the general idea):
>
> public class Artifacts {
>
> private final DependencyGraphBuilder dependencyGraphBuilder;
>
> public Artifacts(final DependencyGraphBuilder dependencyGraphBuilder) {
> super();
> this.dependencyGraphBuilder = dependencyGraphBuilder;
> }
>
> public Collection<? extends Artifact>
> getDependencyArtifactsInTopologicalOrder(final MavenProject project) throws
> DependencyGraphBuilderException {
> List<Artifact> returnValue = null;
> final DependencyNode projectNode =
> this.dependencyGraphBuilder.buildDependencyGraph(project, null /* no
> ArtifactFilter; prob. need one */);
> assert projectNode != null;
> final CollectingDependencyNodeVisitor visitor = new
> CollectingDependencyNodeVisitor();
> projectNode.accept(visitor);
> final Collection<? extends DependencyNode> nodes = visitor.getNodes();
> if (nodes != null && !nodes.isEmpty()) {
> returnValue = new ArrayList<Artifact>();
> for (final DependencyNode node : nodes) {
> if (node != null) {
> final Artifact artifact = node.getArtifact();
> if (artifact != null) {
> returnValue.add(artifact);
> }
> }
> }
> if (!returnValue.isEmpty()) {
> Collections.reverse(returnValue);
> }
> }
> return returnValue;
> }
>
> }
MG_01_05>An excellent test-case for maven-dependency-tree
>
> I think that might be the right way to do it. It's unclear to me how
> Maven's internals themselves make use of this class, but it is probably
> used somewhere, and it clearly hides and mediates between the Sonatype
> Aether and
MG_01_05>most of the Original Sonatype authors monitor this list..
> Eclipse Aether kerfuffle that occurred between 3.0.5 and 3.1.x,
MG_01_05>a fair amount of confusion once IBM orphaned Eclipse to OpenSource mavens
MG_01_05>The one 'positive' we 'gain' is seamless integration to Eclipse IDE..
> so this also seems like something that isn't going to be deprecated any
> time soon.
MG_01_05>lets hope..!
> IWBNI there were a quick primer in the documentation somewhere about how
> Maven loads its own guts. I know that fundamentally Maven is just a bundle
> of components that get instantiated and wired together by Plexus but it
> sure is confusing to try to figure out how and where something is called—as
> a result when there are problems you have to go digging through, well,
> obscure StackOverflow questions. :-)
MG_01_05>usually if we can funnel the permutations of thought processes to one specific question an answer will be MG_01_05>shortly forthcoming for a real adventure in managing maven repositories thru Embedding I invite you to take ride on MG_01_05>the Aether train
MG_01_05>http://blog.sonatype.com/2010/08/introducing-aether/
> Best,
> Laird
MG_01_05>Thanks for taking time to 'think this through'
MG_01_05>All the Best, Martin-
>
> --
> http://about.me/lairdnelson
Re: Topologically sorted list of Artifacts for use by a plugin?
Posted by Laird Nelson <lj...@gmail.com>.
On Fri, Jan 3, 2014 at 6:17 PM, Martin Gainty <mg...@hotmail.com> wrote:
> MG>so I would say you are definitely on the right track
>
Thanks; I'm not sure I am.
> MG>any reason for topological sorting of dependencies?
>
In my original post, I wrote:
(As background: I am working with Liquibase (http://www.liquibase.org/).
> Each of my .jar projects has a META-INF/liquibase/changelog.xml file in
> it. Various of these .jar projects already depend on each other--for unit
> and integration testing, I'd like to use this dependency order, harvest the
> changelog.xml resources in each .jar file, and then combine them to create
> only the database tables actually needed for the test. If a.jar has A's
> tables, and b.jar has B's tables and b.jar depends on a.jar, then I'd like
> to run a.jar!/META-INF/liquibase/changelog.xml first, then
> b.jar!/META-INF/liquibase/changelog.xml next, and I'd like to not have to
> specify this in any place in my pom.xmls, since the dependency order is
> already captured there.)
So the fact that I'm using
ProjectSorter<http://maven.apache.org/ref/3.0.5/maven-core/apidocs/org/apache/maven/project/ProjectSorter.html>strikes
me as wrong—worse than wrong: an awful hack, in fact—because I am
basically using the reactor to sort MavenProjects that I've roped in from
elsewhere. Yucko.
Various people have noted tersely that the maven-dependency-plugin is what
I want. Of course, its *goals* are completely useless for this task—but
buried in an obscure StackOverflow comment tangentially related to the
maven-dependency-plugin I found a reference to this class:
http://maven.apache.org/shared/maven-dependency-tree/apidocs/index.html?org/apache/maven/shared/dependency/graph/DependencyGraphBuilder.html
If I understand it right, then code like the following should do the
trick
(untested, but you get the general idea):
public class Artifacts {
private final DependencyGraphBuilder dependencyGraphBuilder;
public Artifacts(final DependencyGraphBuilder dependencyGraphBuilder) {
super();
this.dependencyGraphBuilder = dependencyGraphBuilder;
}
public Collection<? extends Artifact>
getDependencyArtifactsInTopologicalOrder(final MavenProject project) throws
DependencyGraphBuilderException {
List<Artifact> returnValue = null;
final DependencyNode projectNode =
this.dependencyGraphBuilder.buildDependencyGraph(project, null /* no
ArtifactFilter; prob. need one */);
assert projectNode != null;
final CollectingDependencyNodeVisitor visitor = new
CollectingDependencyNodeVisitor();
projectNode.accept(visitor);
final Collection<? extends DependencyNode> nodes = visitor.getNodes();
if (nodes != null && !nodes.isEmpty()) {
returnValue = new ArrayList<Artifact>();
for (final DependencyNode node : nodes) {
if (node != null) {
final Artifact artifact = node.getArtifact();
if (artifact != null) {
returnValue.add(artifact);
}
}
}
if (!returnValue.isEmpty()) {
Collections.reverse(returnValue);
}
}
return returnValue;
}
}
I think that might be the right way to do it. It's unclear to me how
Maven's internals themselves make use of this class, but it is probably
used somewhere, and it clearly hides and mediates between the Sonatype
Aether and Eclipse Aether kerfuffle that occurred between 3.0.5 and 3.1.x,
so this also seems like something that isn't going to be deprecated any
time soon.
IWBNI there were a quick primer in the documentation somewhere about how
Maven loads its own guts. I know that fundamentally Maven is just a bundle
of components that get instantiated and wired together by Plexus but it
sure is confusing to try to figure out how and where something is called—as
a result when there are problems you have to go digging through, well,
obscure StackOverflow questions. :-)
Best,
Laird
--
http://about.me/lairdnelson
RE: Topologically sorted list of Artifacts for use by a plugin?
Posted by Martin Gainty <mg...@hotmail.com>.
> Date: Fri, 3 Jan 2014 14:03:22 -0800
> Subject: Topologically sorted list of Artifacts for use by a plugin?
> From: ljnelson@gmail.com
> To: users@maven.apache.org
>
> Hello; in Maven 3.0.5 and 3.1.x, what is the preferred way, given a
> MavenProject representing the current pom.xml/project, to obtain a list of
> its resolved dependency Artifacts in test scope that has been topologically
> sorted such that the Artifact at the head of the list has the fewest
> dependencies and the artifact at the tail of the list has the most?
>
> I asked this question many moons ago (
> http://maven.40175.n5.nabble.com/Topologically-sorting-dependencies-tp3384898p3387803.html)
> and after no answers, came up with what felt like an awful hack to produce
> what I wanted, whose code I pasted in that message.
>
> If you follow the link, you'll see that I used the ProjectSorter class,
> which is what Maven's reactor uses when processing a multi-module
> project--which gives me exactly the order I wanted--but seems like overkill
> and like misusing a tool, and maybe isn't the Right Way given all the new
> Aether stuff and whatnot. Nevertheless, it is still how Maven's reactor
> works, so...who knows.
>
> Anyway, I started there and "worked backwards"--needed a List of
> MavenProjects to feed to it, so ended up using the MavenProjectBuilder, as
> you can see in the link.
>
> That class has since been deprecated and moved to the compat layer, though,
> and its replacement (another class named DefaultMavenProjectBuilder) has
> removed the convenience method that I was using. I'd like to update my
> plugin to avoid depending on this deprecated subsystem.
>
> So then: suppose I'm writing a Maven plugin that needs to fetch a
> particular well-known classpath resource from all artifacts available in
> test scope, and needs that list to be in topological order (starting with
> an artifact that has the fewest dependencies and ending with the artifact
> that has the most). What is the preferred mechanism for doing this these
> days? Is it still by way of the ProjectSorter hammer? Or is there a more
> elegant way to do it?
>
> (As background: I am working with Liquibase (http://www.liquibase.org/).
> Each of my .jar projects has a META-INF/liquibase/changelog.xml file in
> it. Various of these .jar projects already depend on each other--for unit
> and integration testing, I'd like to use this dependency order, harvest the
> changelog.xml resources in each .jar file, and then combine them to create
> only the database tables actually needed for the test. If a.jar has A's
> tables, and b.jar has B's tables and b.jar depends on a.jar, then I'd like
> to run a.jar!/META-INF/liquibase/changelog.xml first, then
> b.jar!/META-INF/liquibase/changelog.xml next, and I'd like to not have to
> specify this in any place in my pom.xmls, since the dependency order is
> already captured there.)
MG>so I would say you are definitely on the right track
MG>so ....ProjectSorter.java.....
MG>public List getDependents( String id ) { return dag.getParentLabels( id ); }
MG>will get you the required count of dependents you want
MG>
MG>so processing each child module strategy might implement something like:
MG>"include in my reactor a known subset of selected artifacts" which would be implemented by..
MG>mvn reactor:make-dependents -Dmake.folders=childModule1
MG>(from features of maven-reactor-plugin)
MG>http://maven.apache.org/plugins/maven-reactor-plugin/examples.html
MG>
MG>so a way of assessing all of the modules by executing each module ..one at a time
MG>
MG>for dir in /tmp/*
MG>do
MG> dir=${dir%*/}
MG> mvn reactor:make-dependents -Dmake.folders=${dir##*/}
MG>done
MG>
MG>*may* work as a Q&D test-harness to exercise each of your child modules
MG>any reason for topological sorting of dependencies?
> Best,
> Happy 2014,
> Laird
MG>Happy 2014 Laird,
MG>Martin
> --
> http://about.me/lairdnelson