You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Christian Schulte <cs...@schulte.it> on 2016/12/01 04:53:53 UTC
Re: Structuring bigger Maven projects
Am 11/30/16 um 19:18 schrieb Florian Sch�tz:
>> A library project to be
>> shared between multiple applications each having its own release-cycle
>> should not be part of a multi-module project used to build such an
>> application and should be an independent project with its own
>> release-cycle.
>
> While this is of course a good idea, especially when starting to create
> new applications, the library projects will grow with the applications
> and not independently of them. We cannot start by first investing months
> to create the perfect will-work-for-ten-years library and only then
> start coding the application that will, in the first months, only use
> 10% of all these features. So, realistically, during a development
> cycle, both the application and the library will grow... (which doesn't
> mean that we cannot separate them)
That's what I was referring to with "inheritance vs. aggregation". In a
multi-module project you get versioning "for free" by using the
release:prepare release:perform goals. Maintaining the build is not a
one time task. You adjust things over time until you get things
structured the way it fits your situation perfectly. To not be trapped
later, you need to carefully decide the artifact coordinates in use and
the way versions will be assigned. If a shared library has been part of
a multi-module project together with an application, extracting it later
should not have an impact on the rest of the build. It should carry it's
own version from day one. It should carry it's own coordinate namespace
from day one. Once in a multi-module project, you will soon start to
inherit groupId and version. That can lead to having to update each and
every application POM when extracting that library later. Starting some
application from scatch today, keep in mind you will re-structure your
build over time until things have been sorted out. If you know from day
one that a specific module/project will have it's own release-cycle
sooner or later, be prepared for that from day one.
Regards,
--
Christian
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Structuring bigger Maven projects
Posted by Ben Tatham <be...@nanometrics.ca>.
I am happy to see this discussion, and so many responses. We have
struggled with it for years, and never found a great solution. Sounds like
we do what many others are doing:
- We have many shared internal libraries amount multiple products.
- We divide each logical set of libraries into their own repos, often
multi-module builds themselves.
- Each product is in its own repo as well, also often a multi-module build
depending on the complexity of the product.
We aren't a huge team, despite the number 4-6 products we have, but
basically any team can make a change to a library as needed. We do have
Pull Requests that are visible to the whole dev team, and we try to
maintain some consistency at the library level, but that is a separate
discussion.
We then have a policy that whenever a PR is merged into a "library", we
"release" that "library". We just jgitflow
<https://bitbucket.org/atlassian/jgit-flow/wiki/Home> for releasing
because, well obviously, we use git flow.
It is up to the product teams to update their versions for internal
libraries. As others have mentioned, we use versions plugin for that. We
always keep our versions in properties so we can use mvn
versions:update-properties. We configured it to only update our "internal"
repositories by default so that we can quickly do that without worry of
bring in some 3rd party/OSS update without vetting it first. This is
done with shared corporate parent pom configuration
<includes>
<include>com.yourcompany*:*</include>
</includes>
<excludeProperties>jetty.version,servlet-api.version,${moreExcludeProperties></excludeProperties>
Note the extra ${moreExcludeProperties> at the end, which allows child poms
to add to the list through another property in their own pom as needed.
We also automatically build our products every night with the latest of all
the internal libraries. While a bit delayed, at least we get a
notification if another teams' update of a library will break a separate
product - either compile or test.
In the old days of subversion, we used to actually automatically update all
the poms in our repo whenever we did a release - without a CI server. That
was automated through a plugin that scanned all the pom's in the svn repo
for use of the released library and auto-updated and commited the new
version. If not familiar with svn, you can picture each top-level
directory of svn being converted to a git repo (in a svn "repo" meant
something different than git, at least practically speaking). This was
nice because the author of the library change was the "author" of all the
pom updates, and thus easy to "blame"/"praise" for a break (or fix) and
because it was an instantaneous update, so CI would run immediately on all
the dependent projects. That was possible because with SVN you could do
commits without having to actually clone a local copy. With git spread
across multiple repos it would require all our devs to have all git repos
checked out in a consistent fashion to make this scalable -- so not
something we've preserved when we switch to git years ago. Alternatively,
we would have to auto-clone all the known repos in our system -- again, not
really scalable.
Food for thought -- it's great to hear so many other ways of dealing with
this problem.
-Ben
On Thu, Dec 1, 2016 at 8:57 AM Christian Schulte <cs...@schulte.it> wrote:
> Am 12/01/16 um 10:02 schrieb João Cabrita:
>
> [...]
>
> > Each of the applications live in a separate repository
>
> [...]
>
> > The libraries also have a repository each
>
> [...]
>
> That's due to the SCM in use. Yes. This is something to keep in mind as
> well. What can be one (big) repository using subversion, may need to be
> split into multiple (tons) of repositories when switching to e.g. git.
> Basing something on a concept introduced by the SCM in use will make it
> hard to switch to another technology later. Subversion on the server and
> git-svn locally is working great here. Commits to the subversion server
> are really clean because developers could re-arrange theire commits
> (rebase -i, squash, etc.) locally before. Depending on the subversion
> revision for anything you'll not change to another SCM technology ever
> soon (why would you?).
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
> --
--
Ben Tatham
Software Architect
*Nano**metrics* *Inc.*
Ottawa * I* Calgary *I* Houston *I* Beijing
T: +1 613 505 5065 *I* bentatham@nanometrics.ca
www.nanometrics.ca *I *www.microseismicmonitoring.com
This message is intended exclusively for the individual or entity to which
it is addressed. This communication may contain information that is
proprietary, privileged, confidential or otherwise legally exempt from
disclosure. If you are not the named addressee, or have been inadvertently
and erroneously referenced in the address line, you are not authorized to
read, print, retain, copy or disseminate this message or any part of it. If
you have received this message in error, please notify the sender
immediately by e-mail and delete all copies of the message.
Re: Structuring bigger Maven projects
Posted by Christian Schulte <cs...@schulte.it>.
Am 12/01/16 um 10:02 schrieb Jo�o Cabrita:
[...]
> Each of the applications live in a separate repository
[...]
> The libraries also have a repository each
[...]
That's due to the SCM in use. Yes. This is something to keep in mind as
well. What can be one (big) repository using subversion, may need to be
split into multiple (tons) of repositories when switching to e.g. git.
Basing something on a concept introduced by the SCM in use will make it
hard to switch to another technology later. Subversion on the server and
git-svn locally is working great here. Commits to the subversion server
are really clean because developers could re-arrange theire commits
(rebase -i, squash, etc.) locally before. Depending on the subversion
revision for anything you'll not change to another SCM technology ever
soon (why would you?).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Structuring bigger Maven projects
Posted by João Cabrita <jo...@gmail.com>.
Florian,
the use case you're describing is similar to what I'm using at my current
job: we have 3 applications and a bunch of internal libraries that are used
by those applications and, in some cases, applications developed by other
teams in the company.
Each of the applications live in a separate repository but are mult-module
maven projects to make advancing versions in all modules easier (we use the
maven release plugin for this, which isn't all that bad).
The libraries also have a repository each and use the same release process
as applications but, of course, each has its own version.
The way we've found to ease the burden of updating is having update jobs
that run the update goals in the versions maven plugin: those jobs update
the dependency versions, run a SNAPSHOT build to verify that it still
passes and, if it does, pushes the result into an updates branch that is
then reviewed and manually merged.
We use this not only for internal dependencies but external as well and
found it to work well.
The only downside so far is that you have to maintain an exclusions list
because several projects make "fake" releases like RCs and others use
"strange" versioning schemes like "odd minor versions are always unstable",
but this can be managed in a parent POM.
Hope this helps.
João Cabrita
On 1 December 2016 at 04:53, Christian Schulte <cs...@schulte.it> wrote:
> Am 11/30/16 um 19:18 schrieb Florian Schätz:
> >> A library project to be
> >> shared between multiple applications each having its own release-cycle
> >> should not be part of a multi-module project used to build such an
> >> application and should be an independent project with its own
> >> release-cycle.
> >
> > While this is of course a good idea, especially when starting to create
> > new applications, the library projects will grow with the applications
> > and not independently of them. We cannot start by first investing months
> > to create the perfect will-work-for-ten-years library and only then
> > start coding the application that will, in the first months, only use
> > 10% of all these features. So, realistically, during a development
> > cycle, both the application and the library will grow... (which doesn't
> > mean that we cannot separate them)
>
> That's what I was referring to with "inheritance vs. aggregation". In a
> multi-module project you get versioning "for free" by using the
> release:prepare release:perform goals. Maintaining the build is not a
> one time task. You adjust things over time until you get things
> structured the way it fits your situation perfectly. To not be trapped
> later, you need to carefully decide the artifact coordinates in use and
> the way versions will be assigned. If a shared library has been part of
> a multi-module project together with an application, extracting it later
> should not have an impact on the rest of the build. It should carry it's
> own version from day one. It should carry it's own coordinate namespace
> from day one. Once in a multi-module project, you will soon start to
> inherit groupId and version. That can lead to having to update each and
> every application POM when extracting that library later. Starting some
> application from scatch today, keep in mind you will re-structure your
> build over time until things have been sorted out. If you know from day
> one that a specific module/project will have it's own release-cycle
> sooner or later, be prepared for that from day one.
>
> Regards,
> --
> Christian
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>