You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Stephane Nicoll <st...@gmail.com> on 2014/01/31 11:20:56 UTC
dependency management across projects
Hi,
I was wondering if anyone knows or has experience with a system that would
track and consolidate dependencies on a project composed of several
sub-projects.
Assume a project P with 10 sub-projects (SP1...SP10). Each of those
sub-projects are located in its own space and have its own lifecycle
(release versioning, etc). But ultimately, the project P needs to be
released as a set of coherent *tested* dependencies. For various reasons,
some sub-projects have to keep different dependencies of the target for
backward compatibility reason.
There are two deliverables to this:
1. Provide a single/coherent dependencyManagement section so that users
using P do not have to care about the different versions of the
sub-projects: they use P and all the required dependencies are pulled
automatically
2. Build configs that *force* each sub-project to run with the list of
dependencies for the project (to ensure all tests pass, etc). This is to be
used alongside the regular build job for validation purpose
I started to look at this and my first trial was to generate a report with
all the dependencies of each project and build a consolidated report that I
can match against the candidates. This would help manage the first goal as
if a dependency gets added, removed or updated, the global
dependencyManagement has to be impacted manually (do we upgrade or not,
etc).
For the second part, it's not easy to force a dependency change in Maven,
especially if the version has been specified at the project level.
Thanks for reading that far. If you have any idea or know any organisation
that tried to implement that, I'd be interested
Thanks!
S.
Re: dependency management across projects
Posted by Stephane Nicoll <st...@gmail.com>.
Thanks for the feedback. This looks like something that the
versions-maven-plugin could do. There's something similar that is
advertised by the plugin documentation but it's not implemented. I'll have
a look to that too.
Thanks!
S.
On Mon, Feb 3, 2014 at 6:39 PM, Curtis Rueden <ct...@wisc.edu> wrote:
> Hi everyone,
>
> > > The very point I am trying to make here is
> > > "how do you manage that manual BOM on a daily basis".
> >
> > There is no automatic solution for this that I know of.
>
> Maybe not exactly what you are looking for, but sort of similar:
>
> My group uses a script [1] to automatically bump the version of our parent
> POM [2].
>
> In that parent POM, we declare many version properties, plugin
> configuration in pluginManagement, etc., and we like to keep all our
> projects across various Git repositories using the newest available version
> of the parent, to minimize version clashes when mixing and matching
> libraries.
>
> We set up a parameterized Jenkins job [3] to run the parent bump for us,
> which provides checkboxes for all the downstream projects so the bump can
> be controlled individually.
>
> It's not perfect but it does save a lot of manual maintenance.
>
> Regards,
> Curtis
>
> [1]
>
> https://github.com/scijava/scijava-scripts/blob/a0fc8006741e0216c74c82866fd1bb1a7d364d55/bump-pom-scijava.sh
> [2]
>
> https://github.com/scijava/pom-scijava/blob/0de0676f7731b98baa89379dc8b92f8b26a5d086/pom.xml
> [3] http://jenkins.imagej.net/view/SciJava/job/Bump-POM-SciJava/
>
>
> On Mon, Feb 3, 2014 at 1:21 AM, Anders Hammar <an...@hammar.net> wrote:
>
> > >
> > > > The release of the BOM would be that release of "a single coherent
> > unit"
> > > > then. It would specify the (marketing) version of the "platform" P.
> > > > For example, P v1.0.0 will include v1.2.3 of SP1 (sub-product 1),
> > v1.4.3
> > > of
> > > > SP2, etc.
> > >
> > > Isn't it what I just write in my original post? (without mentioning the
> > > BOM)
> > >
> >
> > I believe so, yes. The key thing in my "solution" is the BOM. And the BOM
> > will keep the appropriate version of the the sub-products together.
> >
> >
> >
> > > The very point I am trying to make here is
> > > "how do you manage that manual BOM on a daily basis". Each sub-project
> > has
> > > its own release cycle and we cannot force the versions it has to use
> for
> > a
> > > specific branch. For instance, the product might state that the
> > dependency
> > > D should be 2.2.0 (because that's the latest or the one that people
> > > generally use) but for backward compatibility reason SP2 has to use
> > 1.8.0.
> > >
> >
> > There is no automatic solution for this that I know of. I suppose that
> > tolls could be created, but keep in mind that in the end, "for backward
> > compatibility reason SP2 has to use 1.8.0" is normally a human decision.
> >
> > /Anders
> >
> >
> > >
> > > Creating manually the first BOM for P v1.0.0 isn't a problem: i've
> > created
> > > a set of tools that I am happy to share once they are fully ready. But
> > > maintaining that BOM in the long run is more of a challenge (because we
> > > can't force the sub-projects to use those versions so we have to
> monitor
> > > what happens in each sub-project to take the appropriate version at the
> > > product level).
> > >
> > > Thanks again for the feedback!
> > >
> > > S.
> > >
> > >
> > > >
> > > > /Anders
> > > >
> > > >
> > > > >
> > > > >
> > > > > >
> > > > > >
> > > > > > There is also the possibility of creating a "grouping pom", which
> > > lists
> > > > > all
> > > > > > artifacts as dependencies. You would then declare a dependency to
> > > that
> > > > > > grouping pom and get all deps magically sucked in. However, this
> is
> > > not
> > > > > > really the "Maven way" in my opinion as you wouldn't specify your
> > > > direct
> > > > > > deps bu sort of relly on transitive deps. There are some fans of
> > this
> > > > > > approach though here on this list.
> > > > > >
> > > > > >
> > > > > > > 2. Build configs that *force* each sub-project to run with the
> > list
> > > > of
> > > > > > > dependencies for the project (to ensure all tests pass, etc).
> > This
> > > is
> > > > > to
> > > > > > be
> > > > > > > used alongside the regular build job for validation purpose
> > > > > > >
> > > > > >
> > > > > > Maybe some enforcer rule?
> > > > > >
> > > > >
> > > > > Like I said, this is to be used alongside the regular build job. So
> > my
> > > > SP4
> > > > > 1.2.0-SNAPSHOT is building with a set of dependencies on its own
> and
> > I
> > > > want
> > > > > to validate that with the dependencies of the target release for P,
> > it
> > > is
> > > > > also working just fine. It may just be the same ideally or slightly
> > > > > different (or not slightly at all which requires an explicit
> > > validation).
> > > > >
> > > > > So I need to be able to swap those versions for validation purposes
> > and
> > > > run
> > > > > the build with that.
> > > > >
> > > > > S.
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > > /Anders
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > I started to look at this and my first trial was to generate a
> > > report
> > > > > > with
> > > > > > > all the dependencies of each project and build a consolidated
> > > report
> > > > > > that I
> > > > > > > can match against the candidates. This would help manage the
> > first
> > > > goal
> > > > > > as
> > > > > > > if a dependency gets added, removed or updated, the global
> > > > > > > dependencyManagement has to be impacted manually (do we upgrade
> > or
> > > > not,
> > > > > > > etc).
> > > > > > >
> > > > > > > For the second part, it's not easy to force a dependency change
> > in
> > > > > Maven,
> > > > > > > especially if the version has been specified at the project
> > level.
> > > > > > >
> > > > > > > Thanks for reading that far. If you have any idea or know any
> > > > > > organisation
> > > > > > > that tried to implement that, I'd be interested
> > > > > > >
> > > > > > > Thanks!
> > > > > > > S.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: dependency management across projects
Posted by Curtis Rueden <ct...@wisc.edu>.
Hi Jörg,
> What do you mean with "irreproducible builds"?
I mean that you may build the same source code today, and then again next
week, and the two build results may differ.
With our projects we wanted the certainty that if you checked out *any*
revision from the SCM history, you would still be able to build it in, say,
5 years... at least as long as you use the same versions of the tooling
(Java version etc.).
With a SNAPSHOT parent, that is not guaranteed. You may introduce a change
in the parent that causes a build to fail when it used to succeed, which
makes "bisect" style debugging no longer feasible. That is, if old SCM
revisions no longer build properly due to changes in the parent, you will
no longer be able to perform a binary search through the SCM history to
determine the exact commit which introduced a bug.
> This scheme works well for us now for several years.
Sure, and I'm not dissing it, just pointing out the pros and cons. We
wanted fully reproducible builds, which meant that even our SNAPSHOT
revisions have only release dependencies, including the parent POM
reference. This indeed has some negative consequences, such as needing that
script to bump things for us, rather than everything "Just Working" with no
changes. And you're right that *release* version builds are still
reproducible either way.
Regards,
Curtis
On Wed, Feb 5, 2014 at 1:54 AM, Jörg Schaible
<jo...@swisspost.com>wrote:
> Hi Curtis,
>
> Curtis Rueden wrote:
>
> > Hi Jörg,
> >
> >> We use simply "x-SNAPSHOT" for the parent version. That way no
> >> unreleased project has to be touched, it simply uses the "lastest"
> >> SNAPSHOT of the parent. No script required.
> >
> > Yeah, that is very convenient, if you are willing to accept the two-edged
> > sword of irreproducible builds.
>
> What do you mean with "irreproducible builds"? Definitely, if you project
> relies on a SNAPSHOT parent, it does not matter if you call it 1-SNAPSHOT
> or
> x-SNAPSHOT.
>
> > A breaking change in the parent will break
> > all builds that consume it... but can be fixed just as easily too without
> > touching the downstream projects.
>
> Well, somebody obviously made the choice for the SNAPSHOT parent ... with
> all consequences.
>
> > Of course, it's a decision every project has to make for themselves. My
> > projects opted for build reproducibility by using only release linkages
> > (for parent POMs, dependencies and plugins). This has many advantages:
> > e.g., "git bisect" fully works as one might hope, allowing you to
> > resurrect the state of the code even from years prior and still build
> > successfully. The downside is that you must make releases in order to be
> > able to pin to them.
>
> Once a project is released, you also have a released parent. No surprises
> anymore. And the project can then make its own decision if it creates also
> a
> branch for the parent and rely on e.g. 254.x-SNAPSHOT either for
> maintenance
> releases or further development.
>
> This scheme works well for us now for several years.
>
> Cheers,
> Jörg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
Re: dependency management across projects
Posted by Jörg Schaible <jo...@swisspost.com>.
Hi Curtis,
Curtis Rueden wrote:
> Hi Jörg,
>
>> We use simply "x-SNAPSHOT" for the parent version. That way no
>> unreleased project has to be touched, it simply uses the "lastest"
>> SNAPSHOT of the parent. No script required.
>
> Yeah, that is very convenient, if you are willing to accept the two-edged
> sword of irreproducible builds.
What do you mean with "irreproducible builds"? Definitely, if you project
relies on a SNAPSHOT parent, it does not matter if you call it 1-SNAPSHOT or
x-SNAPSHOT.
> A breaking change in the parent will break
> all builds that consume it... but can be fixed just as easily too without
> touching the downstream projects.
Well, somebody obviously made the choice for the SNAPSHOT parent ... with
all consequences.
> Of course, it's a decision every project has to make for themselves. My
> projects opted for build reproducibility by using only release linkages
> (for parent POMs, dependencies and plugins). This has many advantages:
> e.g., "git bisect" fully works as one might hope, allowing you to
> resurrect the state of the code even from years prior and still build
> successfully. The downside is that you must make releases in order to be
> able to pin to them.
Once a project is released, you also have a released parent. No surprises
anymore. And the project can then make its own decision if it creates also a
branch for the parent and rely on e.g. 254.x-SNAPSHOT either for maintenance
releases or further development.
This scheme works well for us now for several years.
Cheers,
Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: dependency management across projects
Posted by Curtis Rueden <ct...@wisc.edu>.
Hi Jörg,
> We use simply "x-SNAPSHOT" for the parent version. That way no
> unreleased project has to be touched, it simply uses the "lastest"
> SNAPSHOT of the parent. No script required.
Yeah, that is very convenient, if you are willing to accept the two-edged
sword of irreproducible builds. A breaking change in the parent will break
all builds that consume it... but can be fixed just as easily too without
touching the downstream projects.
Of course, it's a decision every project has to make for themselves. My
projects opted for build reproducibility by using only release linkages
(for parent POMs, dependencies and plugins). This has many advantages:
e.g., "git bisect" fully works as one might hope, allowing you to resurrect
the state of the code even from years prior and still build successfully.
The downside is that you must make releases in order to be able to pin to
them.
Regards,
Curtis
On Tue, Feb 4, 2014 at 4:38 AM, Jörg Schaible
<jo...@swisspost.com>wrote:
> Curtis Rueden wrote:
>
> > Hi everyone,
> >
> >> > The very point I am trying to make here is
> >> > "how do you manage that manual BOM on a daily basis".
> >>
> >> There is no automatic solution for this that I know of.
> >
> > Maybe not exactly what you are looking for, but sort of similar:
> >
> > My group uses a script [1] to automatically bump the version of our
> parent
> > POM [2].
>
> We use simply "x-SNAPSHOT" for the parent version. That way no unreleased
> project has to be touched, it simply uses the "lastest" SNAPSHOT of the
> parent. No script required.
>
> [snip]
>
> - Jörg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
Re: dependency management across projects
Posted by Jörg Schaible <jo...@swisspost.com>.
Curtis Rueden wrote:
> Hi everyone,
>
>> > The very point I am trying to make here is
>> > "how do you manage that manual BOM on a daily basis".
>>
>> There is no automatic solution for this that I know of.
>
> Maybe not exactly what you are looking for, but sort of similar:
>
> My group uses a script [1] to automatically bump the version of our parent
> POM [2].
We use simply "x-SNAPSHOT" for the parent version. That way no unreleased
project has to be touched, it simply uses the "lastest" SNAPSHOT of the
parent. No script required.
[snip]
- Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: dependency management across projects
Posted by Curtis Rueden <ct...@wisc.edu>.
Hi everyone,
> > The very point I am trying to make here is
> > "how do you manage that manual BOM on a daily basis".
>
> There is no automatic solution for this that I know of.
Maybe not exactly what you are looking for, but sort of similar:
My group uses a script [1] to automatically bump the version of our parent
POM [2].
In that parent POM, we declare many version properties, plugin
configuration in pluginManagement, etc., and we like to keep all our
projects across various Git repositories using the newest available version
of the parent, to minimize version clashes when mixing and matching
libraries.
We set up a parameterized Jenkins job [3] to run the parent bump for us,
which provides checkboxes for all the downstream projects so the bump can
be controlled individually.
It's not perfect but it does save a lot of manual maintenance.
Regards,
Curtis
[1]
https://github.com/scijava/scijava-scripts/blob/a0fc8006741e0216c74c82866fd1bb1a7d364d55/bump-pom-scijava.sh
[2]
https://github.com/scijava/pom-scijava/blob/0de0676f7731b98baa89379dc8b92f8b26a5d086/pom.xml
[3] http://jenkins.imagej.net/view/SciJava/job/Bump-POM-SciJava/
On Mon, Feb 3, 2014 at 1:21 AM, Anders Hammar <an...@hammar.net> wrote:
> >
> > > The release of the BOM would be that release of "a single coherent
> unit"
> > > then. It would specify the (marketing) version of the "platform" P.
> > > For example, P v1.0.0 will include v1.2.3 of SP1 (sub-product 1),
> v1.4.3
> > of
> > > SP2, etc.
> >
> > Isn't it what I just write in my original post? (without mentioning the
> > BOM)
> >
>
> I believe so, yes. The key thing in my "solution" is the BOM. And the BOM
> will keep the appropriate version of the the sub-products together.
>
>
>
> > The very point I am trying to make here is
> > "how do you manage that manual BOM on a daily basis". Each sub-project
> has
> > its own release cycle and we cannot force the versions it has to use for
> a
> > specific branch. For instance, the product might state that the
> dependency
> > D should be 2.2.0 (because that's the latest or the one that people
> > generally use) but for backward compatibility reason SP2 has to use
> 1.8.0.
> >
>
> There is no automatic solution for this that I know of. I suppose that
> tolls could be created, but keep in mind that in the end, "for backward
> compatibility reason SP2 has to use 1.8.0" is normally a human decision.
>
> /Anders
>
>
> >
> > Creating manually the first BOM for P v1.0.0 isn't a problem: i've
> created
> > a set of tools that I am happy to share once they are fully ready. But
> > maintaining that BOM in the long run is more of a challenge (because we
> > can't force the sub-projects to use those versions so we have to monitor
> > what happens in each sub-project to take the appropriate version at the
> > product level).
> >
> > Thanks again for the feedback!
> >
> > S.
> >
> >
> > >
> > > /Anders
> > >
> > >
> > > >
> > > >
> > > > >
> > > > >
> > > > > There is also the possibility of creating a "grouping pom", which
> > lists
> > > > all
> > > > > artifacts as dependencies. You would then declare a dependency to
> > that
> > > > > grouping pom and get all deps magically sucked in. However, this is
> > not
> > > > > really the "Maven way" in my opinion as you wouldn't specify your
> > > direct
> > > > > deps bu sort of relly on transitive deps. There are some fans of
> this
> > > > > approach though here on this list.
> > > > >
> > > > >
> > > > > > 2. Build configs that *force* each sub-project to run with the
> list
> > > of
> > > > > > dependencies for the project (to ensure all tests pass, etc).
> This
> > is
> > > > to
> > > > > be
> > > > > > used alongside the regular build job for validation purpose
> > > > > >
> > > > >
> > > > > Maybe some enforcer rule?
> > > > >
> > > >
> > > > Like I said, this is to be used alongside the regular build job. So
> my
> > > SP4
> > > > 1.2.0-SNAPSHOT is building with a set of dependencies on its own and
> I
> > > want
> > > > to validate that with the dependencies of the target release for P,
> it
> > is
> > > > also working just fine. It may just be the same ideally or slightly
> > > > different (or not slightly at all which requires an explicit
> > validation).
> > > >
> > > > So I need to be able to swap those versions for validation purposes
> and
> > > run
> > > > the build with that.
> > > >
> > > > S.
> > > >
> > > >
> > > >
> > > > >
> > > > > /Anders
> > > > >
> > > > >
> > > > > >
> > > > > > I started to look at this and my first trial was to generate a
> > report
> > > > > with
> > > > > > all the dependencies of each project and build a consolidated
> > report
> > > > > that I
> > > > > > can match against the candidates. This would help manage the
> first
> > > goal
> > > > > as
> > > > > > if a dependency gets added, removed or updated, the global
> > > > > > dependencyManagement has to be impacted manually (do we upgrade
> or
> > > not,
> > > > > > etc).
> > > > > >
> > > > > > For the second part, it's not easy to force a dependency change
> in
> > > > Maven,
> > > > > > especially if the version has been specified at the project
> level.
> > > > > >
> > > > > > Thanks for reading that far. If you have any idea or know any
> > > > > organisation
> > > > > > that tried to implement that, I'd be interested
> > > > > >
> > > > > > Thanks!
> > > > > > S.
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: dependency management across projects
Posted by Stephane Nicoll <st...@gmail.com>.
On Mon, Feb 3, 2014 at 8:21 AM, Anders Hammar <an...@hammar.net> wrote:
> I believe so, yes. The key thing in my "solution" is the BOM. And the BOM
> will keep the appropriate version of the the sub-products together.
>
Right!
>
> There is no automatic solution for this that I know of. I suppose that
> tolls could be created, but keep in mind that in the end, "for backward
> compatibility reason SP2 has to use 1.8.0" is normally a human decision.
>
It is. The tool I aim to create is more report-based. Something that would
gather the dependencies on each sub-projects and report inconsistencies: a
dependency unknown to the product has been added or removed, etc.
Thanks for brainstorming this with me. It feels that we're indeed lacking
for a standard solution for this in the Maven land. Will look into that and
report here my progress
S.
>
> /Anders
>
>
> >
> > Creating manually the first BOM for P v1.0.0 isn't a problem: i've
> created
> > a set of tools that I am happy to share once they are fully ready. But
> > maintaining that BOM in the long run is more of a challenge (because we
> > can't force the sub-projects to use those versions so we have to monitor
> > what happens in each sub-project to take the appropriate version at the
> > product level).
> >
> > Thanks again for the feedback!
> >
> > S.
> >
> >
> > >
> > > /Anders
> > >
> > >
> > > >
> > > >
> > > > >
> > > > >
> > > > > There is also the possibility of creating a "grouping pom", which
> > lists
> > > > all
> > > > > artifacts as dependencies. You would then declare a dependency to
> > that
> > > > > grouping pom and get all deps magically sucked in. However, this is
> > not
> > > > > really the "Maven way" in my opinion as you wouldn't specify your
> > > direct
> > > > > deps bu sort of relly on transitive deps. There are some fans of
> this
> > > > > approach though here on this list.
> > > > >
> > > > >
> > > > > > 2. Build configs that *force* each sub-project to run with the
> list
> > > of
> > > > > > dependencies for the project (to ensure all tests pass, etc).
> This
> > is
> > > > to
> > > > > be
> > > > > > used alongside the regular build job for validation purpose
> > > > > >
> > > > >
> > > > > Maybe some enforcer rule?
> > > > >
> > > >
> > > > Like I said, this is to be used alongside the regular build job. So
> my
> > > SP4
> > > > 1.2.0-SNAPSHOT is building with a set of dependencies on its own and
> I
> > > want
> > > > to validate that with the dependencies of the target release for P,
> it
> > is
> > > > also working just fine. It may just be the same ideally or slightly
> > > > different (or not slightly at all which requires an explicit
> > validation).
> > > >
> > > > So I need to be able to swap those versions for validation purposes
> and
> > > run
> > > > the build with that.
> > > >
> > > > S.
> > > >
> > > >
> > > >
> > > > >
> > > > > /Anders
> > > > >
> > > > >
> > > > > >
> > > > > > I started to look at this and my first trial was to generate a
> > report
> > > > > with
> > > > > > all the dependencies of each project and build a consolidated
> > report
> > > > > that I
> > > > > > can match against the candidates. This would help manage the
> first
> > > goal
> > > > > as
> > > > > > if a dependency gets added, removed or updated, the global
> > > > > > dependencyManagement has to be impacted manually (do we upgrade
> or
> > > not,
> > > > > > etc).
> > > > > >
> > > > > > For the second part, it's not easy to force a dependency change
> in
> > > > Maven,
> > > > > > especially if the version has been specified at the project
> level.
> > > > > >
> > > > > > Thanks for reading that far. If you have any idea or know any
> > > > > organisation
> > > > > > that tried to implement that, I'd be interested
> > > > > >
> > > > > > Thanks!
> > > > > > S.
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: dependency management across projects
Posted by Anders Hammar <an...@hammar.net>.
>
> > The release of the BOM would be that release of "a single coherent unit"
> > then. It would specify the (marketing) version of the "platform" P.
> > For example, P v1.0.0 will include v1.2.3 of SP1 (sub-product 1), v1.4.3
> of
> > SP2, etc.
>
> Isn't it what I just write in my original post? (without mentioning the
> BOM)
>
I believe so, yes. The key thing in my "solution" is the BOM. And the BOM
will keep the appropriate version of the the sub-products together.
> The very point I am trying to make here is
> "how do you manage that manual BOM on a daily basis". Each sub-project has
> its own release cycle and we cannot force the versions it has to use for a
> specific branch. For instance, the product might state that the dependency
> D should be 2.2.0 (because that's the latest or the one that people
> generally use) but for backward compatibility reason SP2 has to use 1.8.0.
>
There is no automatic solution for this that I know of. I suppose that
tolls could be created, but keep in mind that in the end, "for backward
compatibility reason SP2 has to use 1.8.0" is normally a human decision.
/Anders
>
> Creating manually the first BOM for P v1.0.0 isn't a problem: i've created
> a set of tools that I am happy to share once they are fully ready. But
> maintaining that BOM in the long run is more of a challenge (because we
> can't force the sub-projects to use those versions so we have to monitor
> what happens in each sub-project to take the appropriate version at the
> product level).
>
> Thanks again for the feedback!
>
> S.
>
>
> >
> > /Anders
> >
> >
> > >
> > >
> > > >
> > > >
> > > > There is also the possibility of creating a "grouping pom", which
> lists
> > > all
> > > > artifacts as dependencies. You would then declare a dependency to
> that
> > > > grouping pom and get all deps magically sucked in. However, this is
> not
> > > > really the "Maven way" in my opinion as you wouldn't specify your
> > direct
> > > > deps bu sort of relly on transitive deps. There are some fans of this
> > > > approach though here on this list.
> > > >
> > > >
> > > > > 2. Build configs that *force* each sub-project to run with the list
> > of
> > > > > dependencies for the project (to ensure all tests pass, etc). This
> is
> > > to
> > > > be
> > > > > used alongside the regular build job for validation purpose
> > > > >
> > > >
> > > > Maybe some enforcer rule?
> > > >
> > >
> > > Like I said, this is to be used alongside the regular build job. So my
> > SP4
> > > 1.2.0-SNAPSHOT is building with a set of dependencies on its own and I
> > want
> > > to validate that with the dependencies of the target release for P, it
> is
> > > also working just fine. It may just be the same ideally or slightly
> > > different (or not slightly at all which requires an explicit
> validation).
> > >
> > > So I need to be able to swap those versions for validation purposes and
> > run
> > > the build with that.
> > >
> > > S.
> > >
> > >
> > >
> > > >
> > > > /Anders
> > > >
> > > >
> > > > >
> > > > > I started to look at this and my first trial was to generate a
> report
> > > > with
> > > > > all the dependencies of each project and build a consolidated
> report
> > > > that I
> > > > > can match against the candidates. This would help manage the first
> > goal
> > > > as
> > > > > if a dependency gets added, removed or updated, the global
> > > > > dependencyManagement has to be impacted manually (do we upgrade or
> > not,
> > > > > etc).
> > > > >
> > > > > For the second part, it's not easy to force a dependency change in
> > > Maven,
> > > > > especially if the version has been specified at the project level.
> > > > >
> > > > > Thanks for reading that far. If you have any idea or know any
> > > > organisation
> > > > > that tried to implement that, I'd be interested
> > > > >
> > > > > Thanks!
> > > > > S.
> > > > >
> > > >
> > >
> >
>
Re: dependency management across projects
Posted by Stephane Nicoll <st...@gmail.com>.
On Fri, Jan 31, 2014 at 1:16 PM, Anders Hammar <an...@hammar.net> wrote:
> The release of the BOM would be that release of "a single coherent unit"
> then. It would specify the (marketing) version of the "platform" P.
> For example, P v1.0.0 will include v1.2.3 of SP1 (sub-product 1), v1.4.3 of
> SP2, etc.
>
Isn't it what I just write in my original post? (without mentioning the BOM)
>
> Creating the BOM would be a manual work I think, as you want to make sure
> that exactly the correct versions are included (might not be the latest
> releases).
>
Yes, this is already what I do. The very point I am trying to make here is
"how do you manage that manual BOM on a daily basis". Each sub-project has
its own release cycle and we cannot force the versions it has to use for a
specific branch. For instance, the product might state that the dependency
D should be 2.2.0 (because that's the latest or the one that people
generally use) but for backward compatibility reason SP2 has to use 1.8.0.
Creating manually the first BOM for P v1.0.0 isn't a problem: i've created
a set of tools that I am happy to share once they are fully ready. But
maintaining that BOM in the long run is more of a challenge (because we
can't force the sub-projects to use those versions so we have to monitor
what happens in each sub-project to take the appropriate version at the
product level).
Thanks again for the feedback!
S.
>
> /Anders
>
>
> >
> >
> > >
> > >
> > > There is also the possibility of creating a "grouping pom", which lists
> > all
> > > artifacts as dependencies. You would then declare a dependency to that
> > > grouping pom and get all deps magically sucked in. However, this is not
> > > really the "Maven way" in my opinion as you wouldn't specify your
> direct
> > > deps bu sort of relly on transitive deps. There are some fans of this
> > > approach though here on this list.
> > >
> > >
> > > > 2. Build configs that *force* each sub-project to run with the list
> of
> > > > dependencies for the project (to ensure all tests pass, etc). This is
> > to
> > > be
> > > > used alongside the regular build job for validation purpose
> > > >
> > >
> > > Maybe some enforcer rule?
> > >
> >
> > Like I said, this is to be used alongside the regular build job. So my
> SP4
> > 1.2.0-SNAPSHOT is building with a set of dependencies on its own and I
> want
> > to validate that with the dependencies of the target release for P, it is
> > also working just fine. It may just be the same ideally or slightly
> > different (or not slightly at all which requires an explicit validation).
> >
> > So I need to be able to swap those versions for validation purposes and
> run
> > the build with that.
> >
> > S.
> >
> >
> >
> > >
> > > /Anders
> > >
> > >
> > > >
> > > > I started to look at this and my first trial was to generate a report
> > > with
> > > > all the dependencies of each project and build a consolidated report
> > > that I
> > > > can match against the candidates. This would help manage the first
> goal
> > > as
> > > > if a dependency gets added, removed or updated, the global
> > > > dependencyManagement has to be impacted manually (do we upgrade or
> not,
> > > > etc).
> > > >
> > > > For the second part, it's not easy to force a dependency change in
> > Maven,
> > > > especially if the version has been specified at the project level.
> > > >
> > > > Thanks for reading that far. If you have any idea or know any
> > > organisation
> > > > that tried to implement that, I'd be interested
> > > >
> > > > Thanks!
> > > > S.
> > > >
> > >
> >
>
Re: dependency management across projects
Posted by Anders Hammar <an...@hammar.net>.
> Some confusion here. You're talking about depMgmt and then say that the
> > required deps should be pulled in autoamtically. That will not happen.
> You
> > still need to specify the deps you have.
>
> Yes, but not the versions. The users will have to add the dependency they
> want to use, yes. The point here is that they don't have to care about the
> versions (but I do, which is why I need some tooling)
> > In any case, this is ok if you want to stay on the "Maven way". This
> > depMgmt could be handled/provided in a separate pom which is then
> included
> > in those projects that want to use the platform/product. Sometimes this
> is
> > called a BOM (Bill of Material). JBoss provides such a BOM for their app
> > server for example. A quick Google gave this page talking about that:
> > http://www.mastertheboss.com/jboss-maven/maven-and-jboss-how-to-use-boms
>
> BOM is exactly what I want to do, yes. What may not be clear in my original
> post is that I am looking at this problem as the "release manager" of P who
> wants to release P 1.0.0 with SP1 1.2.3, SP2 1.4.3, SP3 4.3.0, etc. I want
> to have an overview of where the different sub-projects stands with regards
> to the target for P.
>
> Assume that each sub-project has its own release cycle (and is released as
> a project on its own actually) and at some point those different projects
> have to be released as a single coherent unit.
>
The release of the BOM would be that release of "a single coherent unit"
then. It would specify the (marketing) version of the "platform" P.
For example, P v1.0.0 will include v1.2.3 of SP1 (sub-product 1), v1.4.3 of
SP2, etc.
Creating the BOM would be a manual work I think, as you want to make sure
that exactly the correct versions are included (might not be the latest
releases).
/Anders
>
>
> >
> >
> > There is also the possibility of creating a "grouping pom", which lists
> all
> > artifacts as dependencies. You would then declare a dependency to that
> > grouping pom and get all deps magically sucked in. However, this is not
> > really the "Maven way" in my opinion as you wouldn't specify your direct
> > deps bu sort of relly on transitive deps. There are some fans of this
> > approach though here on this list.
> >
> >
> > > 2. Build configs that *force* each sub-project to run with the list of
> > > dependencies for the project (to ensure all tests pass, etc). This is
> to
> > be
> > > used alongside the regular build job for validation purpose
> > >
> >
> > Maybe some enforcer rule?
> >
>
> Like I said, this is to be used alongside the regular build job. So my SP4
> 1.2.0-SNAPSHOT is building with a set of dependencies on its own and I want
> to validate that with the dependencies of the target release for P, it is
> also working just fine. It may just be the same ideally or slightly
> different (or not slightly at all which requires an explicit validation).
>
> So I need to be able to swap those versions for validation purposes and run
> the build with that.
>
> S.
>
>
>
> >
> > /Anders
> >
> >
> > >
> > > I started to look at this and my first trial was to generate a report
> > with
> > > all the dependencies of each project and build a consolidated report
> > that I
> > > can match against the candidates. This would help manage the first goal
> > as
> > > if a dependency gets added, removed or updated, the global
> > > dependencyManagement has to be impacted manually (do we upgrade or not,
> > > etc).
> > >
> > > For the second part, it's not easy to force a dependency change in
> Maven,
> > > especially if the version has been specified at the project level.
> > >
> > > Thanks for reading that far. If you have any idea or know any
> > organisation
> > > that tried to implement that, I'd be interested
> > >
> > > Thanks!
> > > S.
> > >
> >
>
Re: dependency management across projects
Posted by Stephane Nicoll <st...@gmail.com>.
Thanks for the response.
On Fri, Jan 31, 2014 at 11:33 AM, Anders Hammar <an...@hammar.net> wrote:
> >
> > 1. Provide a single/coherent dependencyManagement section so that users
> > using P do not have to care about the different versions of the
> > sub-projects: they use P and all the required dependencies are pulled
> > automatically
> >
>
> Some confusion here. You're talking about depMgmt and then say that the
> required deps should be pulled in autoamtically. That will not happen. You
> still need to specify the deps you have.
>
Yes, but not the versions. The users will have to add the dependency they
want to use, yes. The point here is that they don't have to care about the
versions (but I do, which is why I need some tooling)
> In any case, this is ok if you want to stay on the "Maven way". This
> depMgmt could be handled/provided in a separate pom which is then included
> in those projects that want to use the platform/product. Sometimes this is
> called a BOM (Bill of Material). JBoss provides such a BOM for their app
> server for example. A quick Google gave this page talking about that:
> http://www.mastertheboss.com/jboss-maven/maven-and-jboss-how-to-use-boms
BOM is exactly what I want to do, yes. What may not be clear in my original
post is that I am looking at this problem as the "release manager" of P who
wants to release P 1.0.0 with SP1 1.2.3, SP2 1.4.3, SP3 4.3.0, etc. I want
to have an overview of where the different sub-projects stands with regards
to the target for P.
Assume that each sub-project has its own release cycle (and is released as
a project on its own actually) and at some point those different projects
have to be released as a single coherent unit.
>
>
> There is also the possibility of creating a "grouping pom", which lists all
> artifacts as dependencies. You would then declare a dependency to that
> grouping pom and get all deps magically sucked in. However, this is not
> really the "Maven way" in my opinion as you wouldn't specify your direct
> deps bu sort of relly on transitive deps. There are some fans of this
> approach though here on this list.
>
>
> > 2. Build configs that *force* each sub-project to run with the list of
> > dependencies for the project (to ensure all tests pass, etc). This is to
> be
> > used alongside the regular build job for validation purpose
> >
>
> Maybe some enforcer rule?
>
Like I said, this is to be used alongside the regular build job. So my SP4
1.2.0-SNAPSHOT is building with a set of dependencies on its own and I want
to validate that with the dependencies of the target release for P, it is
also working just fine. It may just be the same ideally or slightly
different (or not slightly at all which requires an explicit validation).
So I need to be able to swap those versions for validation purposes and run
the build with that.
S.
>
> /Anders
>
>
> >
> > I started to look at this and my first trial was to generate a report
> with
> > all the dependencies of each project and build a consolidated report
> that I
> > can match against the candidates. This would help manage the first goal
> as
> > if a dependency gets added, removed or updated, the global
> > dependencyManagement has to be impacted manually (do we upgrade or not,
> > etc).
> >
> > For the second part, it's not easy to force a dependency change in Maven,
> > especially if the version has been specified at the project level.
> >
> > Thanks for reading that far. If you have any idea or know any
> organisation
> > that tried to implement that, I'd be interested
> >
> > Thanks!
> > S.
> >
>
Re: dependency management across projects
Posted by Ron Wheeler <rw...@artifact-software.com>.
As one of the fans of a non-Maven way to deal with this, I will point
you to some documentation on the approach.
http://blog.artifact-software.com/tech/?tag=maven
This should get you 4 articles that discuss our approach to Maven.
http://blog.artifact-software.com/tech/?p=121
Specifically addresses the problem that you are working through.
It works well for us.
Our big project had over 30 independent modules that depended on more
than 70 third party modules - Tomcat, Jetspeed, Apache Commons, Spring,
Hibernate, Mysql JSF, CXF, Jasper Reports, etc.
We are currently actively developing 2 projects each with about 10
modules (Maven Projects) with the same technology stack (Jackrabbit
replacing MySQL and adding Spring-Data) and have continued the same
approach.
It works well and makes the war files very small and cuts the build time
dramatically.
It relieves the developer from dealing with versions and dependency
configurations.
The individual shared libraries have the "right" versions with the
required exclusions already fixed up so inter-library conflicts are
resolved.
The developer just have to make sure that their code works.
If they want to add a third party library that is not included in the
"corporate" utility libraries, this is supposed to be discussed at the
project level to see if it should be shared or not.
We don't make a big deal about this now that we have a really good core
stack settled.
At the beginning of each release cycle we review the "approved" versions
and upgrade as appropriate. This lets us review and test dependency
changes as a team.
We use the STS version of Eclipse so we have full support for Maven and
can easily see where dependencies are coming from and where potential
dependency conflicts occur between third party software so exclusions
can be adjusted.
Ron
On 31/01/2014 5:33 AM, Anders Hammar wrote:
>> 1. Provide a single/coherent dependencyManagement section so that users
>> using P do not have to care about the different versions of the
>> sub-projects: they use P and all the required dependencies are pulled
>> automatically
>>
> Some confusion here. You're talking about depMgmt and then say that the
> required deps should be pulled in autoamtically. That will not happen. You
> still need to specify the deps you have.
> In any case, this is ok if you want to stay on the "Maven way". This
> depMgmt could be handled/provided in a separate pom which is then included
> in those projects that want to use the platform/product. Sometimes this is
> called a BOM (Bill of Material). JBoss provides such a BOM for their app
> server for example. A quick Google gave this page talking about that:
> http://www.mastertheboss.com/jboss-maven/maven-and-jboss-how-to-use-boms
>
> There is also the possibility of creating a "grouping pom", which lists all
> artifacts as dependencies. You would then declare a dependency to that
> grouping pom and get all deps magically sucked in. However, this is not
> really the "Maven way" in my opinion as you wouldn't specify your direct
> deps bu sort of relly on transitive deps. There are some fans of this
> approach though here on this list.
>
>
>> 2. Build configs that *force* each sub-project to run with the list of
>> dependencies for the project (to ensure all tests pass, etc). This is to be
>> used alongside the regular build job for validation purpose
>>
> Maybe some enforcer rule?
>
> /Anders
>
>
>> I started to look at this and my first trial was to generate a report with
>> all the dependencies of each project and build a consolidated report that I
>> can match against the candidates. This would help manage the first goal as
>> if a dependency gets added, removed or updated, the global
>> dependencyManagement has to be impacted manually (do we upgrade or not,
>> etc).
>>
>> For the second part, it's not easy to force a dependency change in Maven,
>> especially if the version has been specified at the project level.
>>
>> Thanks for reading that far. If you have any idea or know any organisation
>> that tried to implement that, I'd be interested
>>
>> Thanks!
>> S.
>>
--
Ron Wheeler
President
Artifact Software Inc
email: rwheeler@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: dependency management across projects
Posted by Anders Hammar <an...@hammar.net>.
>
> 1. Provide a single/coherent dependencyManagement section so that users
> using P do not have to care about the different versions of the
> sub-projects: they use P and all the required dependencies are pulled
> automatically
>
Some confusion here. You're talking about depMgmt and then say that the
required deps should be pulled in autoamtically. That will not happen. You
still need to specify the deps you have.
In any case, this is ok if you want to stay on the "Maven way". This
depMgmt could be handled/provided in a separate pom which is then included
in those projects that want to use the platform/product. Sometimes this is
called a BOM (Bill of Material). JBoss provides such a BOM for their app
server for example. A quick Google gave this page talking about that:
http://www.mastertheboss.com/jboss-maven/maven-and-jboss-how-to-use-boms
There is also the possibility of creating a "grouping pom", which lists all
artifacts as dependencies. You would then declare a dependency to that
grouping pom and get all deps magically sucked in. However, this is not
really the "Maven way" in my opinion as you wouldn't specify your direct
deps bu sort of relly on transitive deps. There are some fans of this
approach though here on this list.
> 2. Build configs that *force* each sub-project to run with the list of
> dependencies for the project (to ensure all tests pass, etc). This is to be
> used alongside the regular build job for validation purpose
>
Maybe some enforcer rule?
/Anders
>
> I started to look at this and my first trial was to generate a report with
> all the dependencies of each project and build a consolidated report that I
> can match against the candidates. This would help manage the first goal as
> if a dependency gets added, removed or updated, the global
> dependencyManagement has to be impacted manually (do we upgrade or not,
> etc).
>
> For the second part, it's not easy to force a dependency change in Maven,
> especially if the version has been specified at the project level.
>
> Thanks for reading that far. If you have any idea or know any organisation
> that tried to implement that, I'd be interested
>
> Thanks!
> S.
>