published module storage

[Shawn Castrianni <Sh...@halliburton.com>]

We have a debate going on about where to store our published modules.  Some say it should be on a filesystem so that we can delete older builds to reclaim disk space since we should always be able to repeat a build from the past if needed.  Some say they all should be checked into an SCM repository so that we can easily get old dependencies when repeating a build and since some SCM repos like subversion stores binaries as diff, saving disk space.

At first I was in favor of option 1, but I am starting to lean towards option 2.  If I want to repeat a build from 2 years ago, I don't want to have to repeat all of its dependent builds as well since those would most likely be deleted from the file server.  With everything checked into an SCM tool, I can always retrieve any past dependent build to be used a dependency to repeat an old build.  However, how would I configure ivy to get dependencies from an SCM repo?  I don't want to have to implement my own resolver.   What do other people do to be able to support rebuilding something from the past with dependencies?

---
Shawn Castrianni
CM Lead Architect
Landmark
Halliburton Drilling, Evaluation and Digital Solutions Building 2
2101 City West Blvd.
Houston, TX  77042
Work:  713-839-3086
Cell:  832-654-0888
Fax:  713-839-2758

----------------------------------------------------------------------
This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient.  Any review, use, distribution, or disclosure by others is strictly prohibited.  If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message.

Re: published module storage

[Matthias Kilian <ki...@outback.escape.de>]

On Mon, Feb 11, 2008 at 11:49:29PM -0600, Shawn Castrianni wrote:
> We have a debate going on about where to store our published
> modules.  Some say it should be on a filesystem so that we can
> delete older builds to reclaim disk space since we should always
> be able to repeat a build from the past if needed.  Some say they
> all should be checked into an SCM repository so that we can easily
> get old dependencies when repeating a build and since some SCM
> repos like subversion stores binaries as diff, saving disk space.

IMHO, binaries don't belong into an *S*CM. And for binary diffs, I
don't think you'll save much disk space, unless most of the .class
files included in different versions of you artifacts (assuming
.jar files) don't change at all between builds. Worse, if you artifacts
are .tar.gz files, or if they are .war or .ear files containing
freshly built .jar files, there'll be almost no block of data common
to the previously built version.

> At first I was in favor of option 1, but I am starting to lean
> towards option 2.  If I want to repeat a build from 2 years ago,
> I don't want to have to repeat all of its dependent builds as
> well since those would most likely be deleted from the file server.
> With everything checked into an SCM tool, I can always retrieve
> any past dependent build to be used a dependency to repeat an old
> build.  However, how would I configure ivy to get dependencies
> from an SCM repo?  I don't want to have to implement my own
> resolver.   What do other people do to be able to support rebuilding
> something from the past with dependencies?

Keep the complete modules in the ivy repository; disk space is cheap
these days. In addition, check in and tag the delivered ivy file
of a project whenever you run a release build (to be sure you *can*
rebuild everything from scratch should your ivy repository ever get
damaged).

If you've one single ivy repository (i.e. not several separate
repositories for different developement groups depending on each
other), it may be feasible to scan the repository every now and
then and search for old versions of modules not used by any other
module and delete the unused versions.

Ciao,
	Kili

-- 
GDB has a 'break' feature; why doesn't it have 'fix' too?

Re: published module storage

[Stephen Nesbitt <st...@alumni.cmc.edu>]

On Monday 11 February 2008 09:49:29 pm Shawn Castrianni wrote:
> We have a debate going on about where to store our published modules.  Some
> say it should be on a filesystem so that we can delete older builds to
> reclaim disk space since we should always be able to repeat a build from
> the past if needed.  Some say they all should be checked into an SCM
> repository so that we can easily get old dependencies when repeating a
> build and since some SCM repos like subversion stores binaries as diff,
> saving disk space.
By "published" do you mean "formally released". If so my preference would be - 
in contrast to Matthias - to put them in the SCM system. My reasons for doing 
is the assumption that releases are relatively infrequent and therefore 
important to retain for a relatively long period of time. Storing them in the 
SCM allows one to associate metadata with them, provides ready access to any 
machine or user who has access to the SCM (no need to maintain a separate 
repository) and is likely to be backed up.

Both options are valid. I just prefer the consistency and power of placing 
them in the SCM. 
>
> At first I was in favor of option 1, but I am starting to lean towards
> option 2.  If I want to repeat a build from 2 years ago, I don't want to
> have to repeat all of its dependent builds as well since those would most
> likely be deleted from the file server.  With everything checked into an
> SCM tool, I can always retrieve any past dependent build to be used a
> dependency to repeat an old build.  However, how would I configure ivy to
> get dependencies from an SCM repo?  I don't want to have to implement my
> own resolver.   What do other people do to be able to support rebuilding
> something from the past with dependencies?
I'm not a huge fan of depending on rebuilds to recreate important releases. 
There is too much that can go wrong, too many dependencies that are known but 
not readily available on a build machine (db version, etc.), and too much 
time and effort involved.

-steve
CM Architect <former>