Re: Multi-module builds in Continuum

[Emmanuel Venisse <em...@gmail.com>]

I think it would be better to use only one working copy for a whole
multi-module project. With this, we'll save lot of space on the disk and for
each multi-module project we'd can do a single build with the recursive
mode.

An other point is we won't have a build in the middle and we'll can support
correctly multi-module projects with flat structure.

I don't know yet if we can implement this in 1.X or if it would be better to
wait 2.0 because it is a big change.

Emmanuel

On Wed, Apr 30, 2008 at 10:01 AM, Marica Tan <ct...@exist.com> wrote:

> On Fri, Apr 18, 2008 at 12:26 PM, Marica Tan <ct...@exist.com> wrote:
>
> > Hi everyone,
> >
> > Current Scenario:
> > When changes/updates occured in between builds of a multi-module project,
> > some modules failed to build because of those changes not being
> reflected.
> >
> > Let say we have a scheduled build of a multi-module project with
> > sub-project A, B and C, where C is dependent on A.
> > B is currently building when changes was made on A and C. So when C gets
> > to build it fails because changes on A wasn't reflected.
> >
> >
> > Our group here in Exist had some discussions on how to approach this
> > problem, which somehow also addresses
> > http://jira.codehaus.org/browse/CONTINUUM-798.
> > So far we've came up with a list of things that we would like to propose.
> >
> > 1. This is the current scenario when building a multi-module project: for
> > every module, the steps performed are checkout/update then build. What
> we're
> > proposing is to checkout/update group atomically before builds, meaning
> > there will only be one checkout/update and this would be at the group
> level.
> > The build order is still like the current process (by dependency order),
> the
> > only difference is that the checkout/update will be performed only once
> at
> > the group level and at the start of the group build.
> >
> > 2. Notification will be moved at the end of the group build.
> >
> > 3. Reflect the changes in UI. Maybe a tree view.
> >
> > 4. Add features related to building
>
>
> > 5. Aggregated build result/in progress
>
>
> This is actually under number 4.  There will only be one build result per
> group instead of having one per project.
>
> >
> > 6. Need skip/don't build state that retains previous state for
> statistics.
> >
> > 7. Cancel build is still per project. When a build is cancelled, it will
> > trigger a skip.
> >    - can set a flag that the build was cancelled / skipped, but the state
> > is still the previous state.
> >
> > 8. Add group cancel
> >
> > 9. Add transient state
> >
>
> I'll create another thread for this.
>
> >
> > 10. Decision to build is still per project
> >
>
> This is just a note.
>
> >
> > 11. Track hierarchy in config
> >
>
> A project should be aware of its parent and its children or sub projects to
> handle building of project group that has more than one pom with modules.
>
> A
> |__B
> |    |__C
> |    |__D
> |__E
>
> From the figure above, after D finishes building, I could return to A and
> build E.
>
>
> >
> > Comments, suggestions, violent reactions are welcome :)
> >
> > Thanks,
> > Marica
> >
>

Re: Multi-module builds in Continuum

[Brett Porter <br...@apache.org>]

On 16/05/2008, at 6:06 AM, Emmanuel Venisse wrote:

> I think it would be better to use only one working copy for a whole
> multi-module project. With this, we'll save lot of space on the disk  
> and for
> each multi-module project we'd can do a single build with the  
> recursive
> mode.

Agreed - though I think the "recursive" mode should remain optional.

>
>
> An other point is we won't have a build in the middle and we'll can  
> support
> correctly multi-module projects with flat structure.

I still like the module-by-module build - just having one checkout  
means that you only build which modules changes (or their dependencies  
did), rather than rebuilding the parent and everything every time.

> I don't know yet if we can implement this in 1.X or if it would be  
> better to
> wait 2.0 because it is a big change.

This is one of my goals of the SCM changes :) Since it's mostly  
internal, I think it can be done in 1.x.

Cheers,
Brett

--
Brett Porter
brett@apache.org
http://blogs.exist.com/bporter/