You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by "E. Pregzt" <pr...@gmail.com> on 2010/02/15 15:45:54 UTC
Maven2/CI Best Practices for Distributed Development
Hi Everyone,
I was wandering what are the Maven beast practices for distributed teams
working on the same code base.
Let me flash out the structure of the teams and the structure of the code
base that I've been dealing with. The code base is organized in such a way
there is a common platform that consists several artifacts and there are
specific products that use the platform components, but also add domain
specific logic, screens and so on. There is a several teams working
in parallel on both common platform and specific products. For the
simplicity lets assume there is the Platform and products P1 and P2 streams
and four teams T1 and T2 working on product P1 and the Platform and T3 and
T4 working on product P2 and the Platform as well.
Teams are following SCRUM method and are working in 2 weeks sprints. For
each sprint period each team branches of the code and just before the and of
the sprint there is a merge of changes to the trunk. It is important to
mention there is is a continuous integration server (Hudson) running unit
and integration tests on trunk code base. Each team has also its own CI
server that runs test on the team's branch.
The build process has been developed using Maven 2, but IMHO needs
improvement, because originally there was only single team and no branches
were made, so the development process was sequential. Currently the biggest
problem is that artifacts are not versioned as SNAPSHOTS for the development
time and each developer must check out all Platform code base, compile it
locally to be able to compile changes in specific product's code (artifacts
are not being deployed in the shared company repository at the moment). The
code of Platform has became big, so build and testing of platform is
expensive process that slows down development within the teams working on
specific products.
Here are the the ideas in regards to the process changes:
1. Introduce the SNAPSHOTS to the artifacts
2. Introduce Nexus as a company repository
3. Make Hudson to deploy artifacts into Nexus once all tests pass
3. Let each team to use its own Nexus snapshot repository
4. Make each team's Hudson to deploy snapshots into the teams private
repository once all test pass
By doing that, each team can work on their own, Hudson protects
the correctness of the artifacts being deployed into
Nexus. Distributed teams can work in parallel without interfering each other
by for instance overriding snapshots.
Could you please validate the ideas, expand on them or possibly propose the
alternative approaches?
Cheers,
E. Pregzt
Re: Maven2/CI Best Practices for Distributed Development
Posted by "E. Pregzt" <pr...@gmail.com>.
Hi Stephen,
Just a quick note. We do have CI for each parallel stream. I think I
explained that in the initial message to the list. There is one Hudson
build that runs for trunk and each team has its own Hudson build that
runs all the team branched from trunk and also all the net new stuff
they developed.
Cheers,
R. Pregzt
On Tue, Feb 16, 2010 at 8:33 AM, Stephen Connolly
<st...@gmail.com> wrote:
> On 16 February 2010 08:26, E. Pregzt <pr...@gmail.com> wrote:
>> Hi Ron,
>>
>> The merging is not a issue or I should say this is well managed issue
>> :) And yes we are using Subversion. The teams that are about to start
>> their iterations/sprints branch the code from trunk. During two week
>> cycle they develop in their branches. There is a schedule of merges
>> agreed among the team leaders, so merges are sequential. Wen it comes
>> to merge each team pulls everything up from trunk to branch and
>> stabilize on the branch (now including all the changes) and then merge
>> it down to trunk. Next team does the same, so we have a guarantee that
>> we are moving in small steps and 'absorbing' relatively small change
>> sets. Wen last time finishes the trunk Hudson kicks in and performs
>> tests, if green we release software.
>>
>
> What I don't like about that technique is you do not have CI tests
> running on the branches...
>
> We do things somewhat differently...
>
> we do not deploy -SNAPSHOTs at all!
>
> we have a couple of hudson builds, one for each group, and then a
> final hudson build that aggregates all the trunks and pre-run's
> "versions:update-properties -DallowSnaphots=true" so that we are
> integrating the latest -SNAPSHOT versions of everything. All our
> hudson builds just run the goal "verify"... that also ensures that
> when it comes time to release with the release plugin, our releases
> just work.
>
> if you always build with "install" then you can hide issues where
> people have a project which cannot be released with "release:prepare
> release:perform"
>
>> This is somehow off the main topic, because my main concern is not
>> merging, but how to get the best from the Maven to support the
>> distributed development.
>>
>> Cheers,
>> E. Pregzt
>>
>> On Mon, Feb 15, 2010 at 10:28 PM, Ron Wheeler
>> <rw...@artifact-software.com> wrote:
>>> I would rather address the divergence during development with branches and
>>> synchronization than be merging code after everyone has tested their
>>> artifacts and are convinced that they are correct. This makes it hard to get
>>> a release out or even know where you are in the release since someone's
>>> opinion that they have tested code ready to go is not necessarily true.
>>>
>>> With a good SCM such as Subversion, it is easy to tell if your changes to
>>> supporting libraries are in conflict with someone else. You have the choice
>>> to branch the source and build your own snapshots if you do not want to
>>> address the problem up front.
>>> At least you know that you have a problem if 2 branches are moving forward.
>>> With two identically named and versioned objects that may or may not be the
>>> same, you are never sure what your state is or how much work is required to
>>> finish.
>>>
>>>
>>> Ron
>>>
>>> E. Pregzt wrote:
>>>>
>>>> Hi Ron,
>>>>
>>>> The idea with exclusive SNAPSHOT repositories for teams is an answer
>>>> to the issue when multiple teams are working on the same version
>>>> (SNAPSHOT) of the same artifact. If there'd be one SNAPSHOT repository
>>>> teams might start overriding the artifacts in the repo. I saw this in
>>>> previous organization I worked for and that led to completely random
>>>> unexpected test failures and even compilation errors when API had
>>>> changed.
>>>>
>>>> Cheers,
>>>> E. Pregzt
>>>>
>>>> On Mon, Feb 15, 2010 at 3:17 PM, Ron Wheeler
>>>> <rw...@artifact-software.com> wrote:
>>>>
>>>>>
>>>>> You didn't mention a SCM explicitely but it appears that you have one.
>>>>> That
>>>>> is essential.
>>>>>
>>>>> We have just started using SNAPSHOTs and Nexus and it is a great idea.
>>>>> Whether you deploy SNAPSHOTs automatically or manually probably does not
>>>>> make much difference provided everyone follows the rules.
>>>>>
>>>>> I am a little unclear on the need to separate SNAPSHOT repositories. I
>>>>> would
>>>>> try to make sure that the libraries and projects are sufficiently
>>>>> granular
>>>>> so that
>>>>> teams can be perfectly clear about who is doing what to each artifact so
>>>>> that merging is done in the SCM as the code is developed rather than
>>>>> afterwards once each team completes the testing. Having 2
>>>>> mysharedartifact-1.8-SNAPSHOT jars and trying to figure out if they are
>>>>> the
>>>>> same from the repository side is only going to add to the confusion.
>>>>> If 2 teams are working on the same artifact, they should be synchronizing
>>>>> frequently or branching so that the status of each library is clear
>>>>> throughout the process and if one team is using 1.8.1-SNAPSHOT and the
>>>>> other
>>>>> team is using 1.9-SNAPSHOT, then there is going to be a step after all
>>>>> the
>>>>> tests are done to get the libraries merged. They can both use the same
>>>>> Nexus
>>>>> repository.
>>>>>
>>>>> I hope that this helps.
>>>>>
>>>>>
>>>>> E. Pregzt wrote:
>>>>>
>>>>>>
>>>>>> Hi Everyone,
>>>>>>
>>>>>> I was wandering what are the Maven beast practices for distributed teams
>>>>>> working on the same code base.
>>>>>>
>>>>>> Let me flash out the structure of the teams and the structure of the
>>>>>> code
>>>>>> base that I've been dealing with. The code base is organized in such a
>>>>>> way
>>>>>> there is a common platform that consists several artifacts and there are
>>>>>> specific products that use the platform components, but also add domain
>>>>>> specific logic, screens and so on. There is a several teams working
>>>>>> in parallel on both common platform and specific products. For the
>>>>>> simplicity lets assume there is the Platform and products P1 and P2
>>>>>> streams
>>>>>> and four teams T1 and T2 working on product P1 and the Platform and T3
>>>>>> and
>>>>>> T4 working on product P2 and the Platform as well.
>>>>>>
>>>>>> Teams are following SCRUM method and are working in 2 weeks sprints. For
>>>>>> each sprint period each team branches of the code and just before the
>>>>>> and
>>>>>> of
>>>>>> the sprint there is a merge of changes to the trunk. It is important to
>>>>>> mention there is is a continuous integration server (Hudson) running
>>>>>> unit
>>>>>> and integration tests on trunk code base. Each team has also its own CI
>>>>>> server that runs test on the team's branch.
>>>>>>
>>>>>> The build process has been developed using Maven 2, but IMHO needs
>>>>>> improvement, because originally there was only single team and no
>>>>>> branches
>>>>>> were made, so the development process was sequential. Currently the
>>>>>> biggest
>>>>>> problem is that artifacts are not versioned as SNAPSHOTS for the
>>>>>> development
>>>>>> time and each developer must check out all Platform code base, compile
>>>>>> it
>>>>>> locally to be able to compile changes in specific product's code
>>>>>> (artifacts
>>>>>> are not being deployed in the shared company repository at the moment).
>>>>>> The
>>>>>> code of Platform has became big, so build and testing of platform is
>>>>>> expensive process that slows down development within the teams working
>>>>>> on
>>>>>> specific products.
>>>>>>
>>>>>> Here are the the ideas in regards to the process changes:
>>>>>> 1. Introduce the SNAPSHOTS to the artifacts
>>>>>> 2. Introduce Nexus as a company repository
>>>>>> 3. Make Hudson to deploy artifacts into Nexus once all tests pass
>>>>>> 3. Let each team to use its own Nexus snapshot repository
>>>>>> 4. Make each team's Hudson to deploy snapshots into the teams private
>>>>>> repository once all test pass
>>>>>>
>>>>>> By doing that, each team can work on their own, Hudson protects
>>>>>> the correctness of the artifacts being deployed into
>>>>>> Nexus. Distributed teams can work in parallel without interfering each
>>>>>> other
>>>>>> by for instance overriding snapshots.
>>>>>>
>>>>>> Could you please validate the ideas, expand on them or possibly propose
>>>>>> the
>>>>>> alternative approaches?
>>>>>>
>>>>>> Cheers,
>>>>>> E. Pregzt
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>>>>> For additional commands, e-mail: users-help@maven.apache.org
>>>>>
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>>>> For additional commands, e-mail: users-help@maven.apache.org
>>>>
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: users-help@maven.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven2/CI Best Practices for Distributed Development
Posted by Stephen Connolly <st...@gmail.com>.
On 16 February 2010 08:26, E. Pregzt <pr...@gmail.com> wrote:
> Hi Ron,
>
> The merging is not a issue or I should say this is well managed issue
> :) And yes we are using Subversion. The teams that are about to start
> their iterations/sprints branch the code from trunk. During two week
> cycle they develop in their branches. There is a schedule of merges
> agreed among the team leaders, so merges are sequential. Wen it comes
> to merge each team pulls everything up from trunk to branch and
> stabilize on the branch (now including all the changes) and then merge
> it down to trunk. Next team does the same, so we have a guarantee that
> we are moving in small steps and 'absorbing' relatively small change
> sets. Wen last time finishes the trunk Hudson kicks in and performs
> tests, if green we release software.
>
What I don't like about that technique is you do not have CI tests
running on the branches...
We do things somewhat differently...
we do not deploy -SNAPSHOTs at all!
we have a couple of hudson builds, one for each group, and then a
final hudson build that aggregates all the trunks and pre-run's
"versions:update-properties -DallowSnaphots=true" so that we are
integrating the latest -SNAPSHOT versions of everything. All our
hudson builds just run the goal "verify"... that also ensures that
when it comes time to release with the release plugin, our releases
just work.
if you always build with "install" then you can hide issues where
people have a project which cannot be released with "release:prepare
release:perform"
> This is somehow off the main topic, because my main concern is not
> merging, but how to get the best from the Maven to support the
> distributed development.
>
> Cheers,
> E. Pregzt
>
> On Mon, Feb 15, 2010 at 10:28 PM, Ron Wheeler
> <rw...@artifact-software.com> wrote:
>> I would rather address the divergence during development with branches and
>> synchronization than be merging code after everyone has tested their
>> artifacts and are convinced that they are correct. This makes it hard to get
>> a release out or even know where you are in the release since someone's
>> opinion that they have tested code ready to go is not necessarily true.
>>
>> With a good SCM such as Subversion, it is easy to tell if your changes to
>> supporting libraries are in conflict with someone else. You have the choice
>> to branch the source and build your own snapshots if you do not want to
>> address the problem up front.
>> At least you know that you have a problem if 2 branches are moving forward.
>> With two identically named and versioned objects that may or may not be the
>> same, you are never sure what your state is or how much work is required to
>> finish.
>>
>>
>> Ron
>>
>> E. Pregzt wrote:
>>>
>>> Hi Ron,
>>>
>>> The idea with exclusive SNAPSHOT repositories for teams is an answer
>>> to the issue when multiple teams are working on the same version
>>> (SNAPSHOT) of the same artifact. If there'd be one SNAPSHOT repository
>>> teams might start overriding the artifacts in the repo. I saw this in
>>> previous organization I worked for and that led to completely random
>>> unexpected test failures and even compilation errors when API had
>>> changed.
>>>
>>> Cheers,
>>> E. Pregzt
>>>
>>> On Mon, Feb 15, 2010 at 3:17 PM, Ron Wheeler
>>> <rw...@artifact-software.com> wrote:
>>>
>>>>
>>>> You didn't mention a SCM explicitely but it appears that you have one.
>>>> That
>>>> is essential.
>>>>
>>>> We have just started using SNAPSHOTs and Nexus and it is a great idea.
>>>> Whether you deploy SNAPSHOTs automatically or manually probably does not
>>>> make much difference provided everyone follows the rules.
>>>>
>>>> I am a little unclear on the need to separate SNAPSHOT repositories. I
>>>> would
>>>> try to make sure that the libraries and projects are sufficiently
>>>> granular
>>>> so that
>>>> teams can be perfectly clear about who is doing what to each artifact so
>>>> that merging is done in the SCM as the code is developed rather than
>>>> afterwards once each team completes the testing. Having 2
>>>> mysharedartifact-1.8-SNAPSHOT jars and trying to figure out if they are
>>>> the
>>>> same from the repository side is only going to add to the confusion.
>>>> If 2 teams are working on the same artifact, they should be synchronizing
>>>> frequently or branching so that the status of each library is clear
>>>> throughout the process and if one team is using 1.8.1-SNAPSHOT and the
>>>> other
>>>> team is using 1.9-SNAPSHOT, then there is going to be a step after all
>>>> the
>>>> tests are done to get the libraries merged. They can both use the same
>>>> Nexus
>>>> repository.
>>>>
>>>> I hope that this helps.
>>>>
>>>>
>>>> E. Pregzt wrote:
>>>>
>>>>>
>>>>> Hi Everyone,
>>>>>
>>>>> I was wandering what are the Maven beast practices for distributed teams
>>>>> working on the same code base.
>>>>>
>>>>> Let me flash out the structure of the teams and the structure of the
>>>>> code
>>>>> base that I've been dealing with. The code base is organized in such a
>>>>> way
>>>>> there is a common platform that consists several artifacts and there are
>>>>> specific products that use the platform components, but also add domain
>>>>> specific logic, screens and so on. There is a several teams working
>>>>> in parallel on both common platform and specific products. For the
>>>>> simplicity lets assume there is the Platform and products P1 and P2
>>>>> streams
>>>>> and four teams T1 and T2 working on product P1 and the Platform and T3
>>>>> and
>>>>> T4 working on product P2 and the Platform as well.
>>>>>
>>>>> Teams are following SCRUM method and are working in 2 weeks sprints. For
>>>>> each sprint period each team branches of the code and just before the
>>>>> and
>>>>> of
>>>>> the sprint there is a merge of changes to the trunk. It is important to
>>>>> mention there is is a continuous integration server (Hudson) running
>>>>> unit
>>>>> and integration tests on trunk code base. Each team has also its own CI
>>>>> server that runs test on the team's branch.
>>>>>
>>>>> The build process has been developed using Maven 2, but IMHO needs
>>>>> improvement, because originally there was only single team and no
>>>>> branches
>>>>> were made, so the development process was sequential. Currently the
>>>>> biggest
>>>>> problem is that artifacts are not versioned as SNAPSHOTS for the
>>>>> development
>>>>> time and each developer must check out all Platform code base, compile
>>>>> it
>>>>> locally to be able to compile changes in specific product's code
>>>>> (artifacts
>>>>> are not being deployed in the shared company repository at the moment).
>>>>> The
>>>>> code of Platform has became big, so build and testing of platform is
>>>>> expensive process that slows down development within the teams working
>>>>> on
>>>>> specific products.
>>>>>
>>>>> Here are the the ideas in regards to the process changes:
>>>>> 1. Introduce the SNAPSHOTS to the artifacts
>>>>> 2. Introduce Nexus as a company repository
>>>>> 3. Make Hudson to deploy artifacts into Nexus once all tests pass
>>>>> 3. Let each team to use its own Nexus snapshot repository
>>>>> 4. Make each team's Hudson to deploy snapshots into the teams private
>>>>> repository once all test pass
>>>>>
>>>>> By doing that, each team can work on their own, Hudson protects
>>>>> the correctness of the artifacts being deployed into
>>>>> Nexus. Distributed teams can work in parallel without interfering each
>>>>> other
>>>>> by for instance overriding snapshots.
>>>>>
>>>>> Could you please validate the ideas, expand on them or possibly propose
>>>>> the
>>>>> alternative approaches?
>>>>>
>>>>> Cheers,
>>>>> E. Pregzt
>>>>>
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>>>> For additional commands, e-mail: users-help@maven.apache.org
>>>>
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: users-help@maven.apache.org
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven2/CI Best Practices for Distributed Development
Posted by "E. Pregzt" <pr...@gmail.com>.
Hi Ron,
The merging is not a issue or I should say this is well managed issue
:) And yes we are using Subversion. The teams that are about to start
their iterations/sprints branch the code from trunk. During two week
cycle they develop in their branches. There is a schedule of merges
agreed among the team leaders, so merges are sequential. Wen it comes
to merge each team pulls everything up from trunk to branch and
stabilize on the branch (now including all the changes) and then merge
it down to trunk. Next team does the same, so we have a guarantee that
we are moving in small steps and 'absorbing' relatively small change
sets. Wen last time finishes the trunk Hudson kicks in and performs
tests, if green we release software.
This is somehow off the main topic, because my main concern is not
merging, but how to get the best from the Maven to support the
distributed development.
Cheers,
E. Pregzt
On Mon, Feb 15, 2010 at 10:28 PM, Ron Wheeler
<rw...@artifact-software.com> wrote:
> I would rather address the divergence during development with branches and
> synchronization than be merging code after everyone has tested their
> artifacts and are convinced that they are correct. This makes it hard to get
> a release out or even know where you are in the release since someone's
> opinion that they have tested code ready to go is not necessarily true.
>
> With a good SCM such as Subversion, it is easy to tell if your changes to
> supporting libraries are in conflict with someone else. You have the choice
> to branch the source and build your own snapshots if you do not want to
> address the problem up front.
> At least you know that you have a problem if 2 branches are moving forward.
> With two identically named and versioned objects that may or may not be the
> same, you are never sure what your state is or how much work is required to
> finish.
>
>
> Ron
>
> E. Pregzt wrote:
>>
>> Hi Ron,
>>
>> The idea with exclusive SNAPSHOT repositories for teams is an answer
>> to the issue when multiple teams are working on the same version
>> (SNAPSHOT) of the same artifact. If there'd be one SNAPSHOT repository
>> teams might start overriding the artifacts in the repo. I saw this in
>> previous organization I worked for and that led to completely random
>> unexpected test failures and even compilation errors when API had
>> changed.
>>
>> Cheers,
>> E. Pregzt
>>
>> On Mon, Feb 15, 2010 at 3:17 PM, Ron Wheeler
>> <rw...@artifact-software.com> wrote:
>>
>>>
>>> You didn't mention a SCM explicitely but it appears that you have one.
>>> That
>>> is essential.
>>>
>>> We have just started using SNAPSHOTs and Nexus and it is a great idea.
>>> Whether you deploy SNAPSHOTs automatically or manually probably does not
>>> make much difference provided everyone follows the rules.
>>>
>>> I am a little unclear on the need to separate SNAPSHOT repositories. I
>>> would
>>> try to make sure that the libraries and projects are sufficiently
>>> granular
>>> so that
>>> teams can be perfectly clear about who is doing what to each artifact so
>>> that merging is done in the SCM as the code is developed rather than
>>> afterwards once each team completes the testing. Having 2
>>> mysharedartifact-1.8-SNAPSHOT jars and trying to figure out if they are
>>> the
>>> same from the repository side is only going to add to the confusion.
>>> If 2 teams are working on the same artifact, they should be synchronizing
>>> frequently or branching so that the status of each library is clear
>>> throughout the process and if one team is using 1.8.1-SNAPSHOT and the
>>> other
>>> team is using 1.9-SNAPSHOT, then there is going to be a step after all
>>> the
>>> tests are done to get the libraries merged. They can both use the same
>>> Nexus
>>> repository.
>>>
>>> I hope that this helps.
>>>
>>>
>>> E. Pregzt wrote:
>>>
>>>>
>>>> Hi Everyone,
>>>>
>>>> I was wandering what are the Maven beast practices for distributed teams
>>>> working on the same code base.
>>>>
>>>> Let me flash out the structure of the teams and the structure of the
>>>> code
>>>> base that I've been dealing with. The code base is organized in such a
>>>> way
>>>> there is a common platform that consists several artifacts and there are
>>>> specific products that use the platform components, but also add domain
>>>> specific logic, screens and so on. There is a several teams working
>>>> in parallel on both common platform and specific products. For the
>>>> simplicity lets assume there is the Platform and products P1 and P2
>>>> streams
>>>> and four teams T1 and T2 working on product P1 and the Platform and T3
>>>> and
>>>> T4 working on product P2 and the Platform as well.
>>>>
>>>> Teams are following SCRUM method and are working in 2 weeks sprints. For
>>>> each sprint period each team branches of the code and just before the
>>>> and
>>>> of
>>>> the sprint there is a merge of changes to the trunk. It is important to
>>>> mention there is is a continuous integration server (Hudson) running
>>>> unit
>>>> and integration tests on trunk code base. Each team has also its own CI
>>>> server that runs test on the team's branch.
>>>>
>>>> The build process has been developed using Maven 2, but IMHO needs
>>>> improvement, because originally there was only single team and no
>>>> branches
>>>> were made, so the development process was sequential. Currently the
>>>> biggest
>>>> problem is that artifacts are not versioned as SNAPSHOTS for the
>>>> development
>>>> time and each developer must check out all Platform code base, compile
>>>> it
>>>> locally to be able to compile changes in specific product's code
>>>> (artifacts
>>>> are not being deployed in the shared company repository at the moment).
>>>> The
>>>> code of Platform has became big, so build and testing of platform is
>>>> expensive process that slows down development within the teams working
>>>> on
>>>> specific products.
>>>>
>>>> Here are the the ideas in regards to the process changes:
>>>> 1. Introduce the SNAPSHOTS to the artifacts
>>>> 2. Introduce Nexus as a company repository
>>>> 3. Make Hudson to deploy artifacts into Nexus once all tests pass
>>>> 3. Let each team to use its own Nexus snapshot repository
>>>> 4. Make each team's Hudson to deploy snapshots into the teams private
>>>> repository once all test pass
>>>>
>>>> By doing that, each team can work on their own, Hudson protects
>>>> the correctness of the artifacts being deployed into
>>>> Nexus. Distributed teams can work in parallel without interfering each
>>>> other
>>>> by for instance overriding snapshots.
>>>>
>>>> Could you please validate the ideas, expand on them or possibly propose
>>>> the
>>>> alternative approaches?
>>>>
>>>> Cheers,
>>>> E. Pregzt
>>>>
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: users-help@maven.apache.org
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven2/CI Best Practices for Distributed Development
Posted by Ron Wheeler <rw...@artifact-software.com>.
I would rather address the divergence during development with branches
and synchronization than be merging code after everyone has tested their
artifacts and are convinced that they are correct. This makes it hard to
get a release out or even know where you are in the release since
someone's opinion that they have tested code ready to go is not
necessarily true.
With a good SCM such as Subversion, it is easy to tell if your changes
to supporting libraries are in conflict with someone else. You have the
choice to branch the source and build your own snapshots if you do not
want to address the problem up front.
At least you know that you have a problem if 2 branches are moving
forward. With two identically named and versioned objects that may or
may not be the same, you are never sure what your state is or how much
work is required to finish.
Ron
E. Pregzt wrote:
> Hi Ron,
>
> The idea with exclusive SNAPSHOT repositories for teams is an answer
> to the issue when multiple teams are working on the same version
> (SNAPSHOT) of the same artifact. If there'd be one SNAPSHOT repository
> teams might start overriding the artifacts in the repo. I saw this in
> previous organization I worked for and that led to completely random
> unexpected test failures and even compilation errors when API had
> changed.
>
> Cheers,
> E. Pregzt
>
> On Mon, Feb 15, 2010 at 3:17 PM, Ron Wheeler
> <rw...@artifact-software.com> wrote:
>
>> You didn't mention a SCM explicitely but it appears that you have one. That
>> is essential.
>>
>> We have just started using SNAPSHOTs and Nexus and it is a great idea.
>> Whether you deploy SNAPSHOTs automatically or manually probably does not
>> make much difference provided everyone follows the rules.
>>
>> I am a little unclear on the need to separate SNAPSHOT repositories. I would
>> try to make sure that the libraries and projects are sufficiently granular
>> so that
>> teams can be perfectly clear about who is doing what to each artifact so
>> that merging is done in the SCM as the code is developed rather than
>> afterwards once each team completes the testing. Having 2
>> mysharedartifact-1.8-SNAPSHOT jars and trying to figure out if they are the
>> same from the repository side is only going to add to the confusion.
>> If 2 teams are working on the same artifact, they should be synchronizing
>> frequently or branching so that the status of each library is clear
>> throughout the process and if one team is using 1.8.1-SNAPSHOT and the other
>> team is using 1.9-SNAPSHOT, then there is going to be a step after all the
>> tests are done to get the libraries merged. They can both use the same Nexus
>> repository.
>>
>> I hope that this helps.
>>
>>
>> E. Pregzt wrote:
>>
>>> Hi Everyone,
>>>
>>> I was wandering what are the Maven beast practices for distributed teams
>>> working on the same code base.
>>>
>>> Let me flash out the structure of the teams and the structure of the code
>>> base that I've been dealing with. The code base is organized in such a way
>>> there is a common platform that consists several artifacts and there are
>>> specific products that use the platform components, but also add domain
>>> specific logic, screens and so on. There is a several teams working
>>> in parallel on both common platform and specific products. For the
>>> simplicity lets assume there is the Platform and products P1 and P2
>>> streams
>>> and four teams T1 and T2 working on product P1 and the Platform and T3 and
>>> T4 working on product P2 and the Platform as well.
>>>
>>> Teams are following SCRUM method and are working in 2 weeks sprints. For
>>> each sprint period each team branches of the code and just before the and
>>> of
>>> the sprint there is a merge of changes to the trunk. It is important to
>>> mention there is is a continuous integration server (Hudson) running unit
>>> and integration tests on trunk code base. Each team has also its own CI
>>> server that runs test on the team's branch.
>>>
>>> The build process has been developed using Maven 2, but IMHO needs
>>> improvement, because originally there was only single team and no branches
>>> were made, so the development process was sequential. Currently the
>>> biggest
>>> problem is that artifacts are not versioned as SNAPSHOTS for the
>>> development
>>> time and each developer must check out all Platform code base, compile it
>>> locally to be able to compile changes in specific product's code
>>> (artifacts
>>> are not being deployed in the shared company repository at the moment).
>>> The
>>> code of Platform has became big, so build and testing of platform is
>>> expensive process that slows down development within the teams working on
>>> specific products.
>>>
>>> Here are the the ideas in regards to the process changes:
>>> 1. Introduce the SNAPSHOTS to the artifacts
>>> 2. Introduce Nexus as a company repository
>>> 3. Make Hudson to deploy artifacts into Nexus once all tests pass
>>> 3. Let each team to use its own Nexus snapshot repository
>>> 4. Make each team's Hudson to deploy snapshots into the teams private
>>> repository once all test pass
>>>
>>> By doing that, each team can work on their own, Hudson protects
>>> the correctness of the artifacts being deployed into
>>> Nexus. Distributed teams can work in parallel without interfering each
>>> other
>>> by for instance overriding snapshots.
>>>
>>> Could you please validate the ideas, expand on them or possibly propose
>>> the
>>> alternative approaches?
>>>
>>> Cheers,
>>> E. Pregzt
>>>
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven2/CI Best Practices for Distributed Development
Posted by "E. Pregzt" <pr...@gmail.com>.
Hi Ron,
The idea with exclusive SNAPSHOT repositories for teams is an answer
to the issue when multiple teams are working on the same version
(SNAPSHOT) of the same artifact. If there'd be one SNAPSHOT repository
teams might start overriding the artifacts in the repo. I saw this in
previous organization I worked for and that led to completely random
unexpected test failures and even compilation errors when API had
changed.
Cheers,
E. Pregzt
On Mon, Feb 15, 2010 at 3:17 PM, Ron Wheeler
<rw...@artifact-software.com> wrote:
>
> You didn't mention a SCM explicitely but it appears that you have one. That
> is essential.
>
> We have just started using SNAPSHOTs and Nexus and it is a great idea.
> Whether you deploy SNAPSHOTs automatically or manually probably does not
> make much difference provided everyone follows the rules.
>
> I am a little unclear on the need to separate SNAPSHOT repositories. I would
> try to make sure that the libraries and projects are sufficiently granular
> so that
> teams can be perfectly clear about who is doing what to each artifact so
> that merging is done in the SCM as the code is developed rather than
> afterwards once each team completes the testing. Having 2
> mysharedartifact-1.8-SNAPSHOT jars and trying to figure out if they are the
> same from the repository side is only going to add to the confusion.
> If 2 teams are working on the same artifact, they should be synchronizing
> frequently or branching so that the status of each library is clear
> throughout the process and if one team is using 1.8.1-SNAPSHOT and the other
> team is using 1.9-SNAPSHOT, then there is going to be a step after all the
> tests are done to get the libraries merged. They can both use the same Nexus
> repository.
>
> I hope that this helps.
>
>
> E. Pregzt wrote:
>>
>> Hi Everyone,
>>
>> I was wandering what are the Maven beast practices for distributed teams
>> working on the same code base.
>>
>> Let me flash out the structure of the teams and the structure of the code
>> base that I've been dealing with. The code base is organized in such a way
>> there is a common platform that consists several artifacts and there are
>> specific products that use the platform components, but also add domain
>> specific logic, screens and so on. There is a several teams working
>> in parallel on both common platform and specific products. For the
>> simplicity lets assume there is the Platform and products P1 and P2
>> streams
>> and four teams T1 and T2 working on product P1 and the Platform and T3 and
>> T4 working on product P2 and the Platform as well.
>>
>> Teams are following SCRUM method and are working in 2 weeks sprints. For
>> each sprint period each team branches of the code and just before the and
>> of
>> the sprint there is a merge of changes to the trunk. It is important to
>> mention there is is a continuous integration server (Hudson) running unit
>> and integration tests on trunk code base. Each team has also its own CI
>> server that runs test on the team's branch.
>>
>> The build process has been developed using Maven 2, but IMHO needs
>> improvement, because originally there was only single team and no branches
>> were made, so the development process was sequential. Currently the
>> biggest
>> problem is that artifacts are not versioned as SNAPSHOTS for the
>> development
>> time and each developer must check out all Platform code base, compile it
>> locally to be able to compile changes in specific product's code
>> (artifacts
>> are not being deployed in the shared company repository at the moment).
>> The
>> code of Platform has became big, so build and testing of platform is
>> expensive process that slows down development within the teams working on
>> specific products.
>>
>> Here are the the ideas in regards to the process changes:
>> 1. Introduce the SNAPSHOTS to the artifacts
>> 2. Introduce Nexus as a company repository
>> 3. Make Hudson to deploy artifacts into Nexus once all tests pass
>> 3. Let each team to use its own Nexus snapshot repository
>> 4. Make each team's Hudson to deploy snapshots into the teams private
>> repository once all test pass
>>
>> By doing that, each team can work on their own, Hudson protects
>> the correctness of the artifacts being deployed into
>> Nexus. Distributed teams can work in parallel without interfering each
>> other
>> by for instance overriding snapshots.
>>
>> Could you please validate the ideas, expand on them or possibly propose
>> the
>> alternative approaches?
>>
>> Cheers,
>> E. Pregzt
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven2/CI Best Practices for Distributed Development
Posted by Kalle Korhonen <ka...@gmail.com>.
On Mon, Feb 15, 2010 at 11:54 AM, E. Pregzt <pr...@gmail.com> wrote:
> Unfortunately we don't have a luxury to have a dedicated team or two
> for platform at this stage. That would be a good think but unrealistic
> I'm afraid. Furthermore the platform is quite big and comprehensive
> and in oppose to SDK-style library also contains domain-specific
> business components/modules used by different product streams, for
> high reusability. Due to this fact the teams are naturally working on
> the specific product code and also on parts of the platform code.
Ah, the complexities of real world and for-profit software development
:) Yes, been there - slowly, the platform will only become bigger and
more convoluted if no one is left to maintain the core until it's time
to start all over again. In the long run, it would be better if the
core is regularly and mercilessly refactored to not allow it to grow
uncontrollably while moving the domain-specific parts to smaller
supporting libraries (if not in one of the products). But the sad
reality is that for the product teams, the dollar earned today if
often more important than two dollars tomorrow.
> I must say that release plugin is something that I'm missing on my
> list and I have to investigate further how to use that and what would
> be the impact on the existing build process. But I'd definitely give
> it a go if that might help to improve the build process.
> So, you guys have been using similar approach and it worked and worked well?
Releases work. The cost for cutting releases every two weeks (after
every sprint) is typically too high with traditional build tools (so
you resort to using running buildnumber suffixes or timestamps), and
while still fundamentally high, they can be reasonable with Maven,
assuming you invest enough in automating the releases. Even with
snapshots, in a fast-paced free-for-all development environment, you
often run into those unexpected test failures, frustrating development
teams or team members (if the snapshots are per-team only) causing
lost development time. Even if you cannot afford a dedicated platform
team, insist that changes to the platform need to be scheduled and
scoped for just the same was as the product stories. And ensure that
there is some mechanism how the platform stories are prioritized and
that teams that don't know exactly what they expect from the platform
and/or cannot afford the resources to implement the changes to the
platform simply have to wait and stick with a previous version of the
platform until they do.
Kalle
> On Mon, Feb 15, 2010 at 5:13 PM, Kalle Korhonen
> <ka...@gmail.com> wrote:
>> SNAPSHOTs are essential (otherwise developers end up with multiple
>> different versions of platform libs in their local repositories) and
>> you should deploy them automatically if you want to avoid compiling
>> the platform code. @Ron; multiple repositories are needed to
>> distinguish the branched platform versions during the iterations (if
>> the platform branches are not versioned, say
>> platform-1.0-team1-sprint2-SNAPSHOT). However, you'd avoid the merging
>> issues and over-complicating things with multiple snapshot
>> repositories if you could stick with just the trunk version of the
>> platform code and perhaps make one team generally responsible for
>> making the platform changes, other teams acting as customers of the
>> platform team.
>>
>> In addition to doing what you suggested, you should automate making
>> the releases with the release plugin. It may be a bit painful to start
>> with as it's much more rigid process than just randomly hacking away
>> with no regards to versions but should start paying dividends fairly
>> quickly. I would almost insist that teams generally use only released
>> versions of the platform code, thereby reserving SNAPSHOTs only for
>> those times where a client team and the platform team need to
>> co-ordinate their changes and explore the options to get the API
>> right. If the release process is well automated, the platform team can
>> release platform several times even during the iteration and the
>> client teams can pick up new versions at their own pace. Disable
>> javadocs, source code packaging, dependencies repository discovery
>> reports for release to make it go faster - you want the release to go
>> as fast as possible.
>>
>> The reality in a big software company is seldom as pretty as you'd
>> hope it to be. If the platform code isn't very coherent and it's
>> tightly coupled with a specific product implementation, you often need
>> changes on both sides to make some new feature work. If a specific
>> part of platform is highly specific to a particular product or a a
>> group of product, consider mercilessly refactoring that part into its
>> own common module, out of the core platform so they are not force to
>> evolve together. The teams will selfishly choose the easiest path for
>> them, often adding something to the platform that shouldn't be there
>> if given a choice because they are under a constant pressure to
>> deliver. In the long run it's better if you can resist those kind of
>> changes and have somebody look after the evolution of the platform
>> itself. Each product team will likely complain about the sluggish pace
>> of the platform team (if there's a gatekeeper for the platform - and
>> it doesn't mean they have to write the code, just participate and know
>> what changes go into the platform code) but it's the price you have to
>> pay to keep the platform coherent.
>>
>> Kalle
>>
>>
>> On Mon, Feb 15, 2010 at 7:17 AM, Ron Wheeler
>> <rw...@artifact-software.com> wrote:
>>>
>>> You didn't mention a SCM explicitely but it appears that you have one. That
>>> is essential.
>>>
>>> We have just started using SNAPSHOTs and Nexus and it is a great idea.
>>> Whether you deploy SNAPSHOTs automatically or manually probably does not
>>> make much difference provided everyone follows the rules.
>>>
>>> I am a little unclear on the need to separate SNAPSHOT repositories. I would
>>> try to make sure that the libraries and projects are sufficiently granular
>>> so that
>>> teams can be perfectly clear about who is doing what to each artifact so
>>> that merging is done in the SCM as the code is developed rather than
>>> afterwards once each team completes the testing. Having 2
>>> mysharedartifact-1.8-SNAPSHOT jars and trying to figure out if they are the
>>> same from the repository side is only going to add to the confusion.
>>> If 2 teams are working on the same artifact, they should be synchronizing
>>> frequently or branching so that the status of each library is clear
>>> throughout the process and if one team is using 1.8.1-SNAPSHOT and the other
>>> team is using 1.9-SNAPSHOT, then there is going to be a step after all the
>>> tests are done to get the libraries merged. They can both use the same Nexus
>>> repository.
>>>
>>> I hope that this helps.
>>>
>>>
>>> E. Pregzt wrote:
>>>>
>>>> Hi Everyone,
>>>>
>>>> I was wandering what are the Maven beast practices for distributed teams
>>>> working on the same code base.
>>>>
>>>> Let me flash out the structure of the teams and the structure of the code
>>>> base that I've been dealing with. The code base is organized in such a way
>>>> there is a common platform that consists several artifacts and there are
>>>> specific products that use the platform components, but also add domain
>>>> specific logic, screens and so on. There is a several teams working
>>>> in parallel on both common platform and specific products. For the
>>>> simplicity lets assume there is the Platform and products P1 and P2
>>>> streams
>>>> and four teams T1 and T2 working on product P1 and the Platform and T3 and
>>>> T4 working on product P2 and the Platform as well.
>>>>
>>>> Teams are following SCRUM method and are working in 2 weeks sprints. For
>>>> each sprint period each team branches of the code and just before the and
>>>> of
>>>> the sprint there is a merge of changes to the trunk. It is important to
>>>> mention there is is a continuous integration server (Hudson) running unit
>>>> and integration tests on trunk code base. Each team has also its own CI
>>>> server that runs test on the team's branch.
>>>>
>>>> The build process has been developed using Maven 2, but IMHO needs
>>>> improvement, because originally there was only single team and no branches
>>>> were made, so the development process was sequential. Currently the
>>>> biggest
>>>> problem is that artifacts are not versioned as SNAPSHOTS for the
>>>> development
>>>> time and each developer must check out all Platform code base, compile it
>>>> locally to be able to compile changes in specific product's code
>>>> (artifacts
>>>> are not being deployed in the shared company repository at the moment).
>>>> The
>>>> code of Platform has became big, so build and testing of platform is
>>>> expensive process that slows down development within the teams working on
>>>> specific products.
>>>>
>>>> Here are the the ideas in regards to the process changes:
>>>> 1. Introduce the SNAPSHOTS to the artifacts
>>>> 2. Introduce Nexus as a company repository
>>>> 3. Make Hudson to deploy artifacts into Nexus once all tests pass
>>>> 3. Let each team to use its own Nexus snapshot repository
>>>> 4. Make each team's Hudson to deploy snapshots into the teams private
>>>> repository once all test pass
>>>>
>>>> By doing that, each team can work on their own, Hudson protects
>>>> the correctness of the artifacts being deployed into
>>>> Nexus. Distributed teams can work in parallel without interfering each
>>>> other
>>>> by for instance overriding snapshots.
>>>>
>>>> Could you please validate the ideas, expand on them or possibly propose
>>>> the
>>>> alternative approaches?
>>>>
>>>> Cheers,
>>>> E. Pregzt
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: users-help@maven.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven2/CI Best Practices for Distributed Development
Posted by "E. Pregzt" <pr...@gmail.com>.
Hi Kalle,
I agree on SNAPSHOTs. As you said SNAPSHOTS are essential to improve
the process. I think we still will go ahead with the exclusive Nexus
SNAPSHOT repositories for teams to avoid team specific versions for
artifacts. This is pretty easy to do and manage with Nexus as far as I
can tell.
Unfortunately we don't have a luxury to have a dedicated team or two
for platform at this stage. That would be a good think but unrealistic
I'm afraid. Furthermore the platform is quite big and comprehensive
and in oppose to SDK-style library also contains domain-specific
business components/modules used by different product streams, for
high reusability. Due to this fact the teams are naturally working on
the specific product code and also on parts of the platform code.
I must say that release plugin is something that I'm missing on my
list and I have to investigate further how to use that and what would
be the impact on the existing build process. But I'd definitely give
it a go if that might help to improve the build process.
So, you guys have been using similar approach and it worked and worked well?
Cheers,
E. Pregzt
On Mon, Feb 15, 2010 at 5:13 PM, Kalle Korhonen
<ka...@gmail.com> wrote:
> SNAPSHOTs are essential (otherwise developers end up with multiple
> different versions of platform libs in their local repositories) and
> you should deploy them automatically if you want to avoid compiling
> the platform code. @Ron; multiple repositories are needed to
> distinguish the branched platform versions during the iterations (if
> the platform branches are not versioned, say
> platform-1.0-team1-sprint2-SNAPSHOT). However, you'd avoid the merging
> issues and over-complicating things with multiple snapshot
> repositories if you could stick with just the trunk version of the
> platform code and perhaps make one team generally responsible for
> making the platform changes, other teams acting as customers of the
> platform team.
>
> In addition to doing what you suggested, you should automate making
> the releases with the release plugin. It may be a bit painful to start
> with as it's much more rigid process than just randomly hacking away
> with no regards to versions but should start paying dividends fairly
> quickly. I would almost insist that teams generally use only released
> versions of the platform code, thereby reserving SNAPSHOTs only for
> those times where a client team and the platform team need to
> co-ordinate their changes and explore the options to get the API
> right. If the release process is well automated, the platform team can
> release platform several times even during the iteration and the
> client teams can pick up new versions at their own pace. Disable
> javadocs, source code packaging, dependencies repository discovery
> reports for release to make it go faster - you want the release to go
> as fast as possible.
>
> The reality in a big software company is seldom as pretty as you'd
> hope it to be. If the platform code isn't very coherent and it's
> tightly coupled with a specific product implementation, you often need
> changes on both sides to make some new feature work. If a specific
> part of platform is highly specific to a particular product or a a
> group of product, consider mercilessly refactoring that part into its
> own common module, out of the core platform so they are not force to
> evolve together. The teams will selfishly choose the easiest path for
> them, often adding something to the platform that shouldn't be there
> if given a choice because they are under a constant pressure to
> deliver. In the long run it's better if you can resist those kind of
> changes and have somebody look after the evolution of the platform
> itself. Each product team will likely complain about the sluggish pace
> of the platform team (if there's a gatekeeper for the platform - and
> it doesn't mean they have to write the code, just participate and know
> what changes go into the platform code) but it's the price you have to
> pay to keep the platform coherent.
>
> Kalle
>
>
> On Mon, Feb 15, 2010 at 7:17 AM, Ron Wheeler
> <rw...@artifact-software.com> wrote:
>>
>> You didn't mention a SCM explicitely but it appears that you have one. That
>> is essential.
>>
>> We have just started using SNAPSHOTs and Nexus and it is a great idea.
>> Whether you deploy SNAPSHOTs automatically or manually probably does not
>> make much difference provided everyone follows the rules.
>>
>> I am a little unclear on the need to separate SNAPSHOT repositories. I would
>> try to make sure that the libraries and projects are sufficiently granular
>> so that
>> teams can be perfectly clear about who is doing what to each artifact so
>> that merging is done in the SCM as the code is developed rather than
>> afterwards once each team completes the testing. Having 2
>> mysharedartifact-1.8-SNAPSHOT jars and trying to figure out if they are the
>> same from the repository side is only going to add to the confusion.
>> If 2 teams are working on the same artifact, they should be synchronizing
>> frequently or branching so that the status of each library is clear
>> throughout the process and if one team is using 1.8.1-SNAPSHOT and the other
>> team is using 1.9-SNAPSHOT, then there is going to be a step after all the
>> tests are done to get the libraries merged. They can both use the same Nexus
>> repository.
>>
>> I hope that this helps.
>>
>>
>> E. Pregzt wrote:
>>>
>>> Hi Everyone,
>>>
>>> I was wandering what are the Maven beast practices for distributed teams
>>> working on the same code base.
>>>
>>> Let me flash out the structure of the teams and the structure of the code
>>> base that I've been dealing with. The code base is organized in such a way
>>> there is a common platform that consists several artifacts and there are
>>> specific products that use the platform components, but also add domain
>>> specific logic, screens and so on. There is a several teams working
>>> in parallel on both common platform and specific products. For the
>>> simplicity lets assume there is the Platform and products P1 and P2
>>> streams
>>> and four teams T1 and T2 working on product P1 and the Platform and T3 and
>>> T4 working on product P2 and the Platform as well.
>>>
>>> Teams are following SCRUM method and are working in 2 weeks sprints. For
>>> each sprint period each team branches of the code and just before the and
>>> of
>>> the sprint there is a merge of changes to the trunk. It is important to
>>> mention there is is a continuous integration server (Hudson) running unit
>>> and integration tests on trunk code base. Each team has also its own CI
>>> server that runs test on the team's branch.
>>>
>>> The build process has been developed using Maven 2, but IMHO needs
>>> improvement, because originally there was only single team and no branches
>>> were made, so the development process was sequential. Currently the
>>> biggest
>>> problem is that artifacts are not versioned as SNAPSHOTS for the
>>> development
>>> time and each developer must check out all Platform code base, compile it
>>> locally to be able to compile changes in specific product's code
>>> (artifacts
>>> are not being deployed in the shared company repository at the moment).
>>> The
>>> code of Platform has became big, so build and testing of platform is
>>> expensive process that slows down development within the teams working on
>>> specific products.
>>>
>>> Here are the the ideas in regards to the process changes:
>>> 1. Introduce the SNAPSHOTS to the artifacts
>>> 2. Introduce Nexus as a company repository
>>> 3. Make Hudson to deploy artifacts into Nexus once all tests pass
>>> 3. Let each team to use its own Nexus snapshot repository
>>> 4. Make each team's Hudson to deploy snapshots into the teams private
>>> repository once all test pass
>>>
>>> By doing that, each team can work on their own, Hudson protects
>>> the correctness of the artifacts being deployed into
>>> Nexus. Distributed teams can work in parallel without interfering each
>>> other
>>> by for instance overriding snapshots.
>>>
>>> Could you please validate the ideas, expand on them or possibly propose
>>> the
>>> alternative approaches?
>>>
>>> Cheers,
>>> E. Pregzt
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven2/CI Best Practices for Distributed Development
Posted by Kalle Korhonen <ka...@gmail.com>.
SNAPSHOTs are essential (otherwise developers end up with multiple
different versions of platform libs in their local repositories) and
you should deploy them automatically if you want to avoid compiling
the platform code. @Ron; multiple repositories are needed to
distinguish the branched platform versions during the iterations (if
the platform branches are not versioned, say
platform-1.0-team1-sprint2-SNAPSHOT). However, you'd avoid the merging
issues and over-complicating things with multiple snapshot
repositories if you could stick with just the trunk version of the
platform code and perhaps make one team generally responsible for
making the platform changes, other teams acting as customers of the
platform team.
In addition to doing what you suggested, you should automate making
the releases with the release plugin. It may be a bit painful to start
with as it's much more rigid process than just randomly hacking away
with no regards to versions but should start paying dividends fairly
quickly. I would almost insist that teams generally use only released
versions of the platform code, thereby reserving SNAPSHOTs only for
those times where a client team and the platform team need to
co-ordinate their changes and explore the options to get the API
right. If the release process is well automated, the platform team can
release platform several times even during the iteration and the
client teams can pick up new versions at their own pace. Disable
javadocs, source code packaging, dependencies repository discovery
reports for release to make it go faster - you want the release to go
as fast as possible.
The reality in a big software company is seldom as pretty as you'd
hope it to be. If the platform code isn't very coherent and it's
tightly coupled with a specific product implementation, you often need
changes on both sides to make some new feature work. If a specific
part of platform is highly specific to a particular product or a a
group of product, consider mercilessly refactoring that part into its
own common module, out of the core platform so they are not force to
evolve together. The teams will selfishly choose the easiest path for
them, often adding something to the platform that shouldn't be there
if given a choice because they are under a constant pressure to
deliver. In the long run it's better if you can resist those kind of
changes and have somebody look after the evolution of the platform
itself. Each product team will likely complain about the sluggish pace
of the platform team (if there's a gatekeeper for the platform - and
it doesn't mean they have to write the code, just participate and know
what changes go into the platform code) but it's the price you have to
pay to keep the platform coherent.
Kalle
On Mon, Feb 15, 2010 at 7:17 AM, Ron Wheeler
<rw...@artifact-software.com> wrote:
>
> You didn't mention a SCM explicitely but it appears that you have one. That
> is essential.
>
> We have just started using SNAPSHOTs and Nexus and it is a great idea.
> Whether you deploy SNAPSHOTs automatically or manually probably does not
> make much difference provided everyone follows the rules.
>
> I am a little unclear on the need to separate SNAPSHOT repositories. I would
> try to make sure that the libraries and projects are sufficiently granular
> so that
> teams can be perfectly clear about who is doing what to each artifact so
> that merging is done in the SCM as the code is developed rather than
> afterwards once each team completes the testing. Having 2
> mysharedartifact-1.8-SNAPSHOT jars and trying to figure out if they are the
> same from the repository side is only going to add to the confusion.
> If 2 teams are working on the same artifact, they should be synchronizing
> frequently or branching so that the status of each library is clear
> throughout the process and if one team is using 1.8.1-SNAPSHOT and the other
> team is using 1.9-SNAPSHOT, then there is going to be a step after all the
> tests are done to get the libraries merged. They can both use the same Nexus
> repository.
>
> I hope that this helps.
>
>
> E. Pregzt wrote:
>>
>> Hi Everyone,
>>
>> I was wandering what are the Maven beast practices for distributed teams
>> working on the same code base.
>>
>> Let me flash out the structure of the teams and the structure of the code
>> base that I've been dealing with. The code base is organized in such a way
>> there is a common platform that consists several artifacts and there are
>> specific products that use the platform components, but also add domain
>> specific logic, screens and so on. There is a several teams working
>> in parallel on both common platform and specific products. For the
>> simplicity lets assume there is the Platform and products P1 and P2
>> streams
>> and four teams T1 and T2 working on product P1 and the Platform and T3 and
>> T4 working on product P2 and the Platform as well.
>>
>> Teams are following SCRUM method and are working in 2 weeks sprints. For
>> each sprint period each team branches of the code and just before the and
>> of
>> the sprint there is a merge of changes to the trunk. It is important to
>> mention there is is a continuous integration server (Hudson) running unit
>> and integration tests on trunk code base. Each team has also its own CI
>> server that runs test on the team's branch.
>>
>> The build process has been developed using Maven 2, but IMHO needs
>> improvement, because originally there was only single team and no branches
>> were made, so the development process was sequential. Currently the
>> biggest
>> problem is that artifacts are not versioned as SNAPSHOTS for the
>> development
>> time and each developer must check out all Platform code base, compile it
>> locally to be able to compile changes in specific product's code
>> (artifacts
>> are not being deployed in the shared company repository at the moment).
>> The
>> code of Platform has became big, so build and testing of platform is
>> expensive process that slows down development within the teams working on
>> specific products.
>>
>> Here are the the ideas in regards to the process changes:
>> 1. Introduce the SNAPSHOTS to the artifacts
>> 2. Introduce Nexus as a company repository
>> 3. Make Hudson to deploy artifacts into Nexus once all tests pass
>> 3. Let each team to use its own Nexus snapshot repository
>> 4. Make each team's Hudson to deploy snapshots into the teams private
>> repository once all test pass
>>
>> By doing that, each team can work on their own, Hudson protects
>> the correctness of the artifacts being deployed into
>> Nexus. Distributed teams can work in parallel without interfering each
>> other
>> by for instance overriding snapshots.
>>
>> Could you please validate the ideas, expand on them or possibly propose
>> the
>> alternative approaches?
>>
>> Cheers,
>> E. Pregzt
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Maven2/CI Best Practices for Distributed Development
Posted by Ron Wheeler <rw...@artifact-software.com>.
You didn't mention a SCM explicitely but it appears that you have one.
That is essential.
We have just started using SNAPSHOTs and Nexus and it is a great idea.
Whether you deploy SNAPSHOTs automatically or manually probably does not
make much difference provided everyone follows the rules.
I am a little unclear on the need to separate SNAPSHOT repositories. I
would try to make sure that the libraries and projects are sufficiently
granular so that
teams can be perfectly clear about who is doing what to each artifact so
that merging is done in the SCM as the code is developed rather than
afterwards once each team completes the testing. Having 2
mysharedartifact-1.8-SNAPSHOT jars and trying to figure out if they are
the same from the repository side is only going to add to the confusion.
If 2 teams are working on the same artifact, they should be
synchronizing frequently or branching so that the status of each library
is clear throughout the process and if one team is using 1.8.1-SNAPSHOT
and the other team is using 1.9-SNAPSHOT, then there is going to be a
step after all the tests are done to get the libraries merged. They can
both use the same Nexus repository.
I hope that this helps.
E. Pregzt wrote:
> Hi Everyone,
>
> I was wandering what are the Maven beast practices for distributed teams
> working on the same code base.
>
> Let me flash out the structure of the teams and the structure of the code
> base that I've been dealing with. The code base is organized in such a way
> there is a common platform that consists several artifacts and there are
> specific products that use the platform components, but also add domain
> specific logic, screens and so on. There is a several teams working
> in parallel on both common platform and specific products. For the
> simplicity lets assume there is the Platform and products P1 and P2 streams
> and four teams T1 and T2 working on product P1 and the Platform and T3 and
> T4 working on product P2 and the Platform as well.
>
> Teams are following SCRUM method and are working in 2 weeks sprints. For
> each sprint period each team branches of the code and just before the and of
> the sprint there is a merge of changes to the trunk. It is important to
> mention there is is a continuous integration server (Hudson) running unit
> and integration tests on trunk code base. Each team has also its own CI
> server that runs test on the team's branch.
>
> The build process has been developed using Maven 2, but IMHO needs
> improvement, because originally there was only single team and no branches
> were made, so the development process was sequential. Currently the biggest
> problem is that artifacts are not versioned as SNAPSHOTS for the development
> time and each developer must check out all Platform code base, compile it
> locally to be able to compile changes in specific product's code (artifacts
> are not being deployed in the shared company repository at the moment). The
> code of Platform has became big, so build and testing of platform is
> expensive process that slows down development within the teams working on
> specific products.
>
> Here are the the ideas in regards to the process changes:
> 1. Introduce the SNAPSHOTS to the artifacts
> 2. Introduce Nexus as a company repository
> 3. Make Hudson to deploy artifacts into Nexus once all tests pass
> 3. Let each team to use its own Nexus snapshot repository
> 4. Make each team's Hudson to deploy snapshots into the teams private
> repository once all test pass
>
> By doing that, each team can work on their own, Hudson protects
> the correctness of the artifacts being deployed into
> Nexus. Distributed teams can work in parallel without interfering each other
> by for instance overriding snapshots.
>
> Could you please validate the ideas, expand on them or possibly propose the
> alternative approaches?
>
> Cheers,
> E. Pregzt
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org