Gump *incremental* integration builds...

["Adam R. B. Jack" <aj...@trysybase.com>]

Nicola started an interested conversation on how we'd implement optimised
Gumps to run based off "only the changes in source control". I think this is
an awesome idea, I feel we ought nightly do the full shebang [just in case]
and nag, but daily do incremental/optimised gumps.

I think we can do this easily (with some tweaks) and, I'd like to explain
current internals/implementation to make the point...

The way Python Gump is implemented it creates a
classpath/environment/commandline for each project and then calls launches
the build. Nothing (other than expectation of stuff on disk) is actually
dependent upon past or future builds in the 'build dependency tree'. So long
as jars/whatever remain available (from a previous build) we can build one
project or it's stack w/o issue.

We have a 'GumpRun' object which contains GumpOptions (switches, e.g.
'optimized run') and GumpSet -- the work to do. GumpSet (currently) contains
:

0) Input project expression [or list] e.g. "ws-*" or "*" = "all".
1) The list of projects requested -- i.e. the above expression expanded
against the current workspace's list of projects.
2) list of projects (full stack) needed to be built, i.e. the above plus all
the dependencies.
3) The modules that contain the projects above.

The Run is the context, the Options are how, and the Set is what.

Currently (a week or so now) we have a flag on the module object for that
gives a boolean "isUpdated()". If a CVS or SCM checkout/update succeeds but
is not 'silent' (we run both with --quiet) we mark it as updated/modified.
Run now (and this I don't like) if a module has changed we assume that a
project has changed [see later]. This is all a bit of a hack, but 'works' to
a point.

We could easily have a --optimized switch on integrate.py that doesn't build
all, it only builds projects that are marked as modified (or never been
built -- which is actually the same). The output

We could (on a dedicated Gump machine) have a loop:

    continuous_loop:
        if a new day and not one done:
            do a full gump run
        else:
            do a gump --optimise

        if nothing updated/done in last build:
            sleep for a while

        goto continuous_loop

The only thing I'm not sure about with all this is "when is a project
updated"? Using module update to assume all projects is a tad brute
force/ugly, but might be the best/safest we have available. Some projects
don't have source code, per se -- e.g. test projects, and we don't really
(in the metadata) tell Gump if a project has source code, e.g. some
sub-directory. We can't do a "did any file get updated in here" (via
timestamp) without such a thing. I can't think of anything better than I
have, can anybody else?

Thoughts?

regards,

Adam
--
Experience the Unwired Enterprise:
http://www.sybase.com/unwiredenterprise
Try Sybase: http://www.try.sybase.com