You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@subversion.apache.org by David Aldrich <Da...@EU.NEC.COM> on 2011/02/08 10:16:21 UTC

Request for comments on our vendor library structure

We are developing a fairly large application that uses open source libraries. We have one svn repo per application. Currently, the open source libraries are all stored with the application. This is a bad idea because the working copy and repo are now very large. Also, every branch gets a copy of all the open source code.

So I propose to move the open source libraries into a separate repository. We can then reference that code either using externals or, more likely, environment variables in the build files. I prefer the latter because then the open source code will never appear in the project's working copy and we can easily exclude it from branches if it is not required.

So how should I structure the open source (vendor) library repo? I was thinking of the following:

That way, I can tag each library individually.

We then check out OpenSourceLibs beside the application's working copy, and specify in the makefiles which libraries to use by referencing the tags.

My concern is that each developer must correctly checkout and maintain his/her OpenSourceLibs working copy. It would be best to do this as a sparse working copy so they only get what is needed, but that is not easy for the developer to do as I have a repo of projects of tags i.e. several layers for him to manage. Or they could just checkout and update the entire OpenSourceLibs folder, but that would be very large.

Would anyone like to suggest how best to handle this please?

Best regards

David

Sample disclaimer text

RE: Request for comments on our vendor library structure

Posted by David Aldrich <Da...@EU.NEC.COM>.

Hi Bob

> Personally, hard drive space is cheap. I would rather have all the
> dependencies for each project in my working copy. Sure, store it once in the
> repository and use externals in your projects.

Thanks very much for your comments. I prefer your approach and have seriously considered using externals as you suggest. Our concern is not so much disk space but svn checkout and update times. We use http and we find that checkouts can take a long time. This even discourages people from using branches. Perhaps things will be better with http2.

Best regards

David

RE: Request for comments on our vendor library structure

Posted by Bob Archer <Bo...@amsi.com>.

> We are developing a fairly large application that uses open source
> libraries. We have one svn repo per application. Currently, the
> open source libraries are all stored with the application. This is
> a bad idea because the working copy and repo are now very large.
> Also, every branch gets a copy of all the open source code.
> 
> So I propose to move the open source libraries into a separate
> repository. We can then reference that code either using externals
> or, more likely, environment variables in the build files. I prefer
> the latter because then the open source code will never appear in
> the project's working copy and we can easily exclude it from
> branches if it is not required.
> 
> So how should I structure the open source (vendor) library repo?  I
> was thinking of the following:
> 
> OpenSourceLibs
>        |
>        |---- Boost
>        |        |---- tags
>        |        |---- trunk
>        |
>        |---- wxWidgets
>        |        |---- tags
>        |        |---- trunk
> etc.
> 
> That way, I can tag each library individually.
> 
> We then check out OpenSourceLibs beside the application's working
> copy, and specify in the makefiles which libraries to use by
> referencing the tags.
> 
> My concern is that each developer must correctly checkout and
> maintain his/her OpenSourceLibs working copy. It would be best to
> do this as a sparse working copy so they only get what is needed,
> but that is not easy for the developer to do as I have a repo of
> projects of tags i.e. several layers for him to manage. Or they
> could just checkout and update the entire OpenSourceLibs folder,
> but that would be very large.
> 
> Would anyone like to suggest how best to handle this please?
> 

Many people like to make their projects atomic. That is, when you check out the project folder you have everything you need to build it, including build tools and compilers. Some scale that back and require that the compilers be on the machine... some ever further requiring that the compilers and build tools be on the machine. You want to go one step further and require that common libs/code be on the machine. 

There is really no right or wrong way... it is just what works best for you. Personally, hard drive space is cheap. I would rather have all the dependencies for each project in my working copy. Sure, store it once in the repository and use externals in your projects. 

Once again, this is all preference, and I just gave you mine. Only you can decide what is right for you.

BOb

Re: Request for comments on our vendor library structure

Posted by Mark Parker <go...@gmail.com>.

On 2/8/2011 2:16 AM, David Aldrich wrote:
> Hi
>
> We are developing a fairly large application that uses open source
> libraries. We have one svn repo per application. Currently, the open
> source libraries are all stored with the application. This is a bad
> idea because the working copy and repo are now very large. Also,
> every branch gets a copy of all the open source code.
>
> So I propose to move the open source libraries into a separate
> repository. We can then reference that code either using externals
> or, more likely, environment variables in the build files. I prefer
> the latter because then the open source code will never appear in the
> project's working copy and we can easily exclude it from branches if
> it is not required.
>
> So how should I structure the open source (vendor) library repo?  I
> was thinking of the following:
>
> OpenSourceLibs
>        |
>        |---- Boost
>        |        |---- tags
>        |        |---- trunk
>        |
>        |---- wxWidgets
>        |        |---- tags
>        |        |---- trunk
> etc.
>
> That way, I can tag each library individually.
>
> We then check out OpenSourceLibs beside the application's working
> copy, and specify in the makefiles which libraries to use by
> referencing the tags.
>
> My concern is that each developer must correctly checkout and
> maintain his/her OpenSourceLibs working copy. It would be best to do
> this as a sparse working copy so they only get what is needed, but
> that is not easy for the developer to do as I have a repo of projects
> of tags i.e. several layers for him to manage. Or they could just
> checkout and update the entire OpenSourceLibs folder, but that would
> be very large.
>
> Would anyone like to suggest how best to handle this please?
>
> Best regards
>
> David
>
> Sample disclaimer text
>

We use a structure similar to yours, in the same repository as our other 
projects, and we use externals in our individual projects to pull in the 
necessary libraries for each project. This has served us very well for 
some time.

Mark