You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Cory Riddell <co...@codeware.com> on 2011/09/23 22:37:51 UTC
Does Subversion get slower over time?
I'm thinking about committing all the third party libraries that our
application depends on to make it easier to checkout a buildable set of
files from the past.
Some of the libraries are around a gigabyte each. Right now our
repository has around a decade of history and is 5 GB. If we start
stuffing third party libraries into it, the repository could grow to 50
GB in a few years. Is a 50 GB repository going to be significantly
slower than our current 5 GB one?
Cory
Re: Does Subversion get slower over time?
Posted by Mark Phippard <ma...@gmail.com>.
On Fri, Sep 23, 2011 at 4:37 PM, Cory Riddell <co...@codeware.com> wrote:
> I'm thinking about committing all the third party libraries that our
> application depends on to make it easier to checkout a buildable set of
> files from the past.
>
> Some of the libraries are around a gigabyte each. Right now our
> repository has around a decade of history and is 5 GB. If we start
> stuffing third party libraries into it, the repository could grow to 50
> GB in a few years. Is a 50 GB repository going to be significantly
> slower than our current 5 GB one?
I do not think so, but it will obviously make things like dump/load
and backups slower. Are you aware of the svn:externals feature? You
could just create a separate repository for your libraries and use
svn:externals to pull them in. That gives you the best of both
worlds.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: AW: Does Subversion get slower over time?
Posted by Cory Riddell <co...@codeware.com>.
Markus,
On 9/26/2011 1:45 AM, Markus Schaber wrote:
> Just a suggestion: Set up a second repository just aside your primary
> one, and put the 3rd party stuff there. You can then use svn:externals
> to include the libraries in your regular project trees.
Using externals seems to be the way to go. Last night I moved our
repository from C:\SvnRepo to C:\CompanyName\SvnRepo and had everybody
do a relocate this morning. That seems to have gone fine. Step 2 is
adding more repositories under C:\CompanyName that will be referenced
via externals.
Thanks for the advice.
Cory
AW: Does Subversion get slower over time?
Posted by Markus Schaber <m....@3s-software.com>.
Hi, Cory,
Von: Cory Riddell [mailto:cory@codeware.com]
>
> I'm thinking about committing all the third party libraries that our
> application depends on to make it easier to checkout a buildable set
of
> files from the past.
>
> Some of the libraries are around a gigabyte each. Right now our
repository
> has around a decade of history and is 5 GB. If we start stuffing third
> party libraries into it, the repository could grow to 50 GB in a few
years.
> Is a 50 GB repository going to be significantly slower than our
current 5
> GB one?
Just a suggestion: Set up a second repository just aside your primary
one, and put the 3rd party stuff there. You can then use svn:externals
to include the libraries in your regular project trees.
Best regards
Markus Schaber
--
___________________________
We software Automation.
3S-Smart Software Solutions GmbH
Markus Schaber | Developer
Memminger Str. 151 | 87439 Kempten | Germany | Tel. +49-831-54031-0 |
Fax +49-831-54031-50
Email: m.schaber@3s-software.com | Web: http://www.3s-software.com
CoDeSys internet forum: http://forum.3s-software.com
Download CoDeSys sample projects:
http://www.3s-software.com/index.shtml?sample_projects
Managing Directors: Dipl.Inf. Dieter Hess, Dipl.Inf. Manfred Werner |
Trade register: Kempten HRB 6186 | Tax ID No.: DE 167014915
Re: Does Subversion get slower over time?
Posted by Mark Phippard <ma...@gmail.com>.
On Fri, Sep 23, 2011 at 5:16 PM, Cory Riddell <co...@codeware.com> wrote:
> On 9/23/2011 3:59 PM, Les Mikesell wrote:
>> On Fri, Sep 23, 2011 at 3:37 PM, Cory Riddell <co...@codeware.com> wrote:
>>> I'm thinking about committing all the third party libraries that our
>>> application depends on to make it easier to checkout a buildable set of
>>> files from the past.
>>>
>>> Some of the libraries are around a gigabyte each. Right now our
>>> repository has around a decade of history and is 5 GB. If we start
>>> stuffing third party libraries into it, the repository could grow to 50
>>> GB in a few years. Is a 50 GB repository going to be significantly
>>> slower than our current 5 GB one?
>> I'd be more concerned about future maintenance issues than operational
>> speed. A 50 GB repo is going to be substantially harder to deal with
>> if you ever need to dump/filter/load to pull something out. Why not
>> use separate repositories and reference the components with externals?
>>
>
> Both you and Mark suggested I use externals. I've avoided using
> externals because I've read more than one article advising against them.
> I'm going to have to rethink this I guess. Can you use them without
> abusing them?
For the needs you outlined, I do not think the feature could be much
better. So I would definitely recommend it. There are other
scenarios where the feature is still useful but starts to have some
drawbacks. I think the benefits tend to outweigh those negatives.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Does Subversion get slower over time?
Posted by Les Mikesell <le...@gmail.com>.
On Fri, Sep 23, 2011 at 4:25 PM, Mike Dixon
<mi...@denovosoftware.com> wrote:
>
> From personal experience back in the 1.5 days, externals were horribly slow
> and generally unwieldy to work with. Subversion had to open a fresh
> connection to the repository for every external, you have to set peg
> revisions on all your external definitions when tagging, etc etc.
How many were you trying to use at once? And pegging to a tag is an
advantage, not a problem, since it lets you divorce the development of
components from the projects using them.
> It's quite possible that they've improved in the last few years, but
> personally I'm pretty unlikely to mess with them again. It's easy enough
> just to copy everything into the main repository at the appropriate place
> and just not have to worry about it again.
There are likely to be many places where you'd need to include a
component library, and you may want it to be at different revisions in
each place. With external references (whether or not they point back
to the same repo) this is easy to manage and it especially makes
things easier when the components are developed by different groups on
different release schedules.
--
Les Mikesell
lesmikesell@gmail.com
Re: Does Subversion get slower over time?
Posted by Mike Dixon <mi...@denovosoftware.com>.
On 9/23/2011 2:18 PM, Ryan Schmidt wrote:
>
> On Sep 23, 2011, at 16:16, Cory Riddell wrote:
>
>> Both you and Mark suggested I use externals. I've avoided using
>> externals because I've read more than one article advising against them.
>> I'm going to have to rethink this I guess. Can you use them without
>> abusing them?
>
> What article was that? I'd say there are perfectly fine ways to use externals. Obviously the developers thought it was a useful feature when they implemented it.
>
> There have been some problems with the new file-level externals feature, so I would stay away from that, but the old directory-level externals feature has always worked just fine for me, as long as you understand how it works and what you can and can't do with it.
>
>
From personal experience back in the 1.5 days, externals were horribly
slow and generally unwieldy to work with. Subversion had to open a fresh
connection to the repository for every external, you have to set peg
revisions on all your external definitions when tagging, etc etc.
It's quite possible that they've improved in the last few years, but
personally I'm pretty unlikely to mess with them again. It's easy enough
just to copy everything into the main repository at the appropriate
place and just not have to worry about it again.
-Mike
Re: Does Subversion get slower over time?
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Sep 23, 2011, at 16:16, Cory Riddell wrote:
> Both you and Mark suggested I use externals. I've avoided using
> externals because I've read more than one article advising against them.
> I'm going to have to rethink this I guess. Can you use them without
> abusing them?
What article was that? I'd say there are perfectly fine ways to use externals. Obviously the developers thought it was a useful feature when they implemented it.
There have been some problems with the new file-level externals feature, so I would stay away from that, but the old directory-level externals feature has always worked just fine for me, as long as you understand how it works and what you can and can't do with it.
Re: Does Subversion get slower over time?
Posted by Cory Riddell <co...@codeware.com>.
On 9/23/2011 3:59 PM, Les Mikesell wrote:
> On Fri, Sep 23, 2011 at 3:37 PM, Cory Riddell <co...@codeware.com> wrote:
>> I'm thinking about committing all the third party libraries that our
>> application depends on to make it easier to checkout a buildable set of
>> files from the past.
>>
>> Some of the libraries are around a gigabyte each. Right now our
>> repository has around a decade of history and is 5 GB. If we start
>> stuffing third party libraries into it, the repository could grow to 50
>> GB in a few years. Is a 50 GB repository going to be significantly
>> slower than our current 5 GB one?
> I'd be more concerned about future maintenance issues than operational
> speed. A 50 GB repo is going to be substantially harder to deal with
> if you ever need to dump/filter/load to pull something out. Why not
> use separate repositories and reference the components with externals?
>
Both you and Mark suggested I use externals. I've avoided using
externals because I've read more than one article advising against them.
I'm going to have to rethink this I guess. Can you use them without
abusing them?
cory
Re: Does Subversion get slower over time?
Posted by Les Mikesell <le...@gmail.com>.
On Fri, Sep 23, 2011 at 3:37 PM, Cory Riddell <co...@codeware.com> wrote:
> I'm thinking about committing all the third party libraries that our
> application depends on to make it easier to checkout a buildable set of
> files from the past.
>
> Some of the libraries are around a gigabyte each. Right now our
> repository has around a decade of history and is 5 GB. If we start
> stuffing third party libraries into it, the repository could grow to 50
> GB in a few years. Is a 50 GB repository going to be significantly
> slower than our current 5 GB one?
I'd be more concerned about future maintenance issues than operational
speed. A 50 GB repo is going to be substantially harder to deal with
if you ever need to dump/filter/load to pull something out. Why not
use separate repositories and reference the components with externals?
--
Les Mikesell
lesmikesell@gmail.com