Good practice to store binary builds in the src repo?

[Matt England <me...@mengland.net>]

Summary:

Does the Subversion community have a recommend practice for controlling 
"ongoing" binary builds in the repo?

My project currently stores, automatically, the binary build output 
("globbed" together in a standard format in a .tar.gz file for each 
"client" or "server" distribution on a per-platform--eg, mingw, 
debian3--basis) in the same repo as our source code.  This seems to work 
quite well for us save for possibly one thing: it makes my repo directory 
on my server grow to be very large (the tar balls run from 3MB to more the 
10MB).

Granted, we don't check in the binaries that we build all the time (we have 
separate "trunk/src" and "dist/<platform>" directories to help this 
separation), but still my repo server tends to balloon a bit 
nonetheless...although this is not necessarily an issue, and checkouts 
don't have to include our tarball build output in order to build a new tarball.

The big question I have: is there some other reason why we should not be 
doing this, something we may not have stumbled upon yet?

Maybe we want builds stored in a different repo, or at least a different 
place?  We find it quite convenient to have it all in one spot.

An aside: we do control all of our "input"/vendor/external 
libraries/binaries that we embed in our builds or at least depend upon to 
run (thinks like OpenSSL, bzip2, PostgreSQL stuff) in our repo (in 
trunk/dist/external), and we keep the source for these "external" projects 
in a separate repo, and carefully control the copy from one repo to 
another.  (Let me know if this doesn't make sense, and I can provide a 
more-detailed repo organization/layout.)

I realize that my question/message may not be that organized or coherent, 
but I've waited too long to put forth this question to the community, and I 
feel like I at least need to get the conversation going.

For what it's worth, I find this question related to the following note 
that I just posted:

   <http://svn.haxx.se/users/archive-2006-02/0878.shtml>

...but different enough that I thought it warranted a separate conversation 
thread.

-Matt 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Good practice to store binary builds in the src repo?

[John Waycott <ja...@cox.net>]

Matt England wrote:
>
> ...and a separate "build output" repo seems like the most attractive 
> alternative at the moment.  Alas, we still need to implement a better 
> release-control (that includes more-detailed file naming of the output 
> tarballs, rpms, and other packages) on top of the whatever mechanism 
> we use to save/archive the release sets.  I'm also looking for 
> suggestions on methods/tools/paradigms for these processes, too.  I've 
> built and help built processes like these for other software projects, 
> but I need to make more-robust things then I have in the past.
>
> I'm looking forward to more comments.
>
> I'm also interested to know if 'svn obliterate' may come to fruition 
> in the near future, in the event we may want to "thoroughly clean" 
> some files and their history from the existing repo; however, this is 
> not a prime concern.
>
We are about to do this exact thing. We call it the "engineering 
releases" repository. We won't put every build there, only builds that 
need to be tested by the test group. We also will use it to store 
outputs from the other groups that end up in the finished product, such 
as finished documentation. A good document control system might be a 
better solution, but this is what we have.

Our release repository is structured along the lines of 
product/version/build. The build folder name is identical to the source 
tag name. In the version folder, we also put output from other groups 
such as documentation and on-line help files.

Once the test group approves a release, they copy the <build> folder to 
'final'. For example, widget/2.0/2.0.0.7 gets copied to 
widget/2.0/2.0-final. They then create the release notes and upload 
those. The CM group creates the final package with the release notes, 
on-line docs, etc. and uploads that to our PLM system in Oracle. We have 
lots of different types of products, so the details are going to be 
different for the different product types.

The advantage I see of using Subversion is that you have a central 
mechanism to move files between different departments involved in the 
final release. There's no more 'where's the build?', 'where do I put the 
demoshield output?' type questions, no more huge email attachments, and 
the security is better than using a shared file system.

-- John



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Good practice to store binary builds in the src repo?

[Matt England <me...@mengland.net>]

At 2/18/2006 08:38 PM, eg wrote:
>I generally do not store ongoing binary builds in repositories.
>As part of a configuration management procedure you theoretically need to able
>to get back to any build at a previous timepoint (however infrequent).
>You can do this through a combination of careful tagging/labelling, build
>procedures, libraries and toolsets.

...which certainly needs to be done regardless.

I also like the benefit of being able to get back to the exact binary that 
was built, in case the exact build environment could not have been easily 
duplicated...

>Given that we do the above, simple archiving (outside of repository) of 
>labelled
>copies of binary builds has sufficed for our needs. YMMV.
>
>If I did, I would consider using a separate repository.

...and a separate "build output" repo seems like the most attractive 
alternative at the moment.  Alas, we still need to implement a better 
release-control (that includes more-detailed file naming of the output 
tarballs, rpms, and other packages) on top of the whatever mechanism we use 
to save/archive the release sets.  I'm also looking for suggestions on 
methods/tools/paradigms for these processes, too.  I've built and help 
built processes like these for other software projects, but I need to make 
more-robust things then I have in the past.

I'm looking forward to more comments.

I'm also interested to know if 'svn obliterate' may come to fruition in the 
near future, in the event we may want to "thoroughly clean" some files and 
their history from the existing repo; however, this is not a prime concern.

-Matt 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Good practice to store binary builds in the src repo?

[eg <eg...@gmail.com>]

Matt England <mengland <at> mengland.net> writes:

> 
> Summary:
> 
> Does the Subversion community have a recommend practice for controlling 
> "ongoing" binary builds in the repo?
> 

I dont speak from a Subversion or an open-source perspective, but rather as a
user of revision control systems in software development over the past 17 years.

The answer for you will depend, at least in part, on how you and your users
intend to use these ongoing binary builds.

I generally do not store ongoing binary builds in repositories. 
As part of a configuration management procedure you theoretically need to able
to get back to any build at a previous timepoint (however infrequent). 
You can do this through a combination of careful tagging/labelling, build
procedures, libraries and toolsets. 
Given that we do the above, simple archiving (outside of repository) of labelled
copies of binary builds has sufficed for our needs. YMMV.

If I did, I would consider using a separate repository. Storing "large" amounts
of binary files in a repository tends to have some kind of cost in most version
control systems, at least during certain operations. 

Others will no doubt share differing viewpoints and experience.




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org