You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Brian Krusic <br...@krusic.com> on 2007/07/21 00:41:21 UTC
how does tigris do server updates?
Hello once again;
When you guys at Tigris do your back end SVN updates from say repo
version 1.3 to 1.4, etc....
... how do you go about it in terms of minimizing down time?
I'm asking as we have about 5 repos each at about 40-60GB and based
on our tests, the process of converting takes about 1.5 days per repo
on a P3. We have serveral Opterons and Core Duos which we can spread
the conversion process on that would speed this up.
We figure 1 week total for implementation/debug. We are testing now
but will most liely wait till 1,5 is out as the sync/proxy feature is
shweeeet!
I know that I am asking systems methodology advice, but since you
guys do it all day long, it doesn't hurt to ask.
- Brian
Re: how does tigris do server updates?
Posted by Blair Zajac <bl...@orcaware.com>.
OK. You're mirroring the repository itself. Yes, I would want a
smaller repository.
Have you looked at WANdisco's products for multi-master commits?
Regards,
Blair
Brian Krusic wrote:
> While client traffic is not effected, server to sever traffic will be.
> Our needs aren't limited to simple client traffic.
>
> I'm sure smaller repos will effect sync time and our locations in Europe
> and Canada will have slave repos when the time comes (v1.5).
>
> They already have there own repos that we in the US update from as we've
> done a sort of manual load balancing approach where repo location is
> according to traffic patterns.
>
> -Brian
> On Jul 21, 2007, at 12:27 PM, Blair Zajac wrote:
>
>> Brian Krusic wrote:
>>> Hi Mark,
>>> While u r correct about serving dbs regardless version as thats what
>>> we do on some of our repos now, there are benefits to upgrading so I
>>> disagree with you.
>>> A repo we had in 1.2.3 was 50GB at the time. We converted to 1.4 and
>>> it shrank to 38GB.
>>> So yes, we do need to convert.
>>> With offices in geographically diverse regions, the leaner and meaner
>>> our dbs are, the better.
>>
>> Shrinking the repository size won't have an impact on the network
>> traffic to your clients, even if they are geographically dispersed.
>> So while it's nice to dump and load, it's not necessary.
>>
>> Regards,
>> Blair
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: how does tigris do server updates?
Posted by Brian Krusic <br...@krusic.com>.
While client traffic is not effected, server to sever traffic will
be. Our needs aren't limited to simple client traffic.
I'm sure smaller repos will effect sync time and our locations in
Europe and Canada will have slave repos when the time comes (v1.5).
They already have there own repos that we in the US update from as
we've done a sort of manual load balancing approach where repo
location is according to traffic patterns.
-Brian
On Jul 21, 2007, at 12:27 PM, Blair Zajac wrote:
> Brian Krusic wrote:
>> Hi Mark,
>> While u r correct about serving dbs regardless version as thats
>> what we do on some of our repos now, there are benefits to
>> upgrading so I disagree with you.
>> A repo we had in 1.2.3 was 50GB at the time. We converted to 1.4
>> and it shrank to 38GB.
>> So yes, we do need to convert.
>> With offices in geographically diverse regions, the leaner and
>> meaner our dbs are, the better.
>
> Shrinking the repository size won't have an impact on the network
> traffic to your clients, even if they are geographically
> dispersed. So while it's nice to dump and load, it's not necessary.
>
> Regards,
> Blair
>
> --
> Blair Zajac, Ph.D.
> CTO, OrcaWare Technologies
> <bl...@orcaware.com>
> Subversion training, consulting and support
> http://www.orcaware.com/svn/
>
Re: how does tigris do server updates?
Posted by Blair Zajac <bl...@orcaware.com>.
Blair Zajac wrote:
> Brian Krusic wrote:
>> Hi Mark,
>>
>> While u r correct about serving dbs regardless version as thats what
>> we do on some of our repos now, there are benefits to upgrading so I
>> disagree with you.
>>
>> A repo we had in 1.2.3 was 50GB at the time. We converted to 1.4 and
>> it shrank to 38GB.
>>
>> So yes, we do need to convert.
>>
>> With offices in geographically diverse regions, the leaner and meaner
>> our dbs are, the better.
>
> Shrinking the repository size won't have an impact on the network
> traffic to your clients, even if they are geographically dispersed. So
> while it's nice to dump and load, it's not necessary.
Some more thoughts. Saving 12 Gbytes in repository size is nice.
What I would do is
1) Dump and load into a fresh repository while the older repository is
still live.
2) If the old repository has had commits, then dump and load those in to
the new repository.
3) Copy any hook scripts from the old repository into the new repository.
4) Shut down the svn server.
5) Perform step 2 once more to pick up any last commits.
6) Rename the old repository and rename the new repository into its place.
7) Start the svn server.
You're looking at a couple of minutes of downtime here.
Regards,
Blair
--
Blair Zajac, Ph.D.
<bl...@orcaware.com>
Subversion training, consulting and support
http://www.orcaware.com/svn/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: how does tigris do server updates?
Posted by Blair Zajac <bl...@orcaware.com>.
Brian Krusic wrote:
> Hi Mark,
>
> While u r correct about serving dbs regardless version as thats what we
> do on some of our repos now, there are benefits to upgrading so I
> disagree with you.
>
> A repo we had in 1.2.3 was 50GB at the time. We converted to 1.4 and it
> shrank to 38GB.
>
> So yes, we do need to convert.
>
> With offices in geographically diverse regions, the leaner and meaner
> our dbs are, the better.
Shrinking the repository size won't have an impact on the network
traffic to your clients, even if they are geographically dispersed. So
while it's nice to dump and load, it's not necessary.
Regards,
Blair
--
Blair Zajac, Ph.D.
CTO, OrcaWare Technologies
<bl...@orcaware.com>
Subversion training, consulting and support
http://www.orcaware.com/svn/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: how does tigris do server updates?
Posted by Mark Phippard <ma...@gmail.com>.
On 7/20/07, Brian Krusic <br...@krusic.com> wrote:
> I thought you said that svnsync was a feature as of 1.4.
>
> That accounts for 1 repo. We still have 24 more at v1.2.3 which doesn't
> have svnsync does it?
But you can run a 1.4 server with any repository. I'd assume you are
already using the 1.4 server code with these repositories. If not,
you should be.
svnsync talks to the server, not the repository.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: how does tigris do server updates?
Posted by Brian Krusic <br...@krusic.com>.
I thought you said that svnsync was a feature as of 1.4.
That accounts for 1 repo. We still have 24 more at v1.2.3 which
doesn't have svnsync does it?
-Brian
On Jul 20, 2007, at 6:24 PM, Mark Phippard wrote:
> On 7/20/07, Brian Krusic <br...@krusic.com> wrote:
>> Most of our repos are still at 1.2.3 as we only did 1 conversion
>> to 1.4.
>>
>> Also, I was wrong about our repo #s as I just counted and we have
>> a total of
>> 25 with sizes at 175MB-55GB.
>>
>> I hope that you now see why I am asking for some advice.
>
> Didn't I just give some?
>
> dump/load of a big repository is going to take a long time. What are
> you looking for? I gave you svnsync as an option for avoiding the
> downtime.
>
> --
> Thanks
>
> Mark Phippard
> http://markphip.blogspot.com/
>
Re: how does tigris do server updates?
Posted by Mark Phippard <ma...@gmail.com>.
On 7/20/07, Brian Krusic <br...@krusic.com> wrote:
> Most of our repos are still at 1.2.3 as we only did 1 conversion to 1.4.
>
> Also, I was wrong about our repo #s as I just counted and we have a total of
> 25 with sizes at 175MB-55GB.
>
> I hope that you now see why I am asking for some advice.
Didn't I just give some?
dump/load of a big repository is going to take a long time. What are
you looking for? I gave you svnsync as an option for avoiding the
downtime.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: how does tigris do server updates?
Posted by Brian Krusic <br...@krusic.com>.
Hi Mark,
Most of our repos are still at 1.2.3 as we only did 1 conversion to 1.4.
Also, I was wrong about our repo #s as I just counted and we have a
total of 25 with sizes at 175MB-55GB.
I hope that you now see why I am asking for some advice.
- Brian
On Jul 20, 2007, at 6:14 PM, Mark Phippard wrote:
> On 7/20/07, Brian Krusic <br...@krusic.com> wrote:
>
>> While u r correct about serving dbs regardless version as thats what
>> we do on some of our repos now, there are benefits to upgrading so I
>> disagree with you.
>>
>> A repo we had in 1.2.3 was 50GB at the time. We converted to 1.4 and
>> it shrank to 38GB.
>>
>> So yes, we do need to convert.
>>
>> With offices in geographically diverse regions, the leaner and meaner
>> our dbs are, the better.
>
> You missed my point, as I did mention that in my message.
>
> 1. Not every release has made changes in the repository. For
> example, 1.5 changes the repository but there is no need to dump/load,
> the changes will happen automatically.
>
> 2. When there is something like a space reduction (which I cannot
> imagine happening again), you still do not need to base your upgrade
> decision on this issue. You could have upgraded the repository to 1.4
> and then re-loaded the repository when you had an acceptable window of
> downtime.
>
> The bottom line is that tigris handles this issue by trying to
> avoid it.
>
> Something else to keep in mind is that as of 1.4, we now have the
> svnsync tool. This is great for doing a migration. You can create a
> new fsfs repository using the new release, and then use svnsync to
> migrate from the old live repository to this one. Once the sync is
> finished, just want for your next downtime window, do one final sync,
> then shutdown the server and move in the new repository and restart.
> Or better, would be to just change the server to be serving from the
> new location. If you use this approach just make sure the new
> repository you create has the same UUID as the master.
>
> --
> Thanks
>
> Mark Phippard
> http://markphip.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
>
Re: how does tigris do server updates?
Posted by Mark Phippard <ma...@gmail.com>.
On 7/20/07, Brian Krusic <br...@krusic.com> wrote:
> While u r correct about serving dbs regardless version as thats what
> we do on some of our repos now, there are benefits to upgrading so I
> disagree with you.
>
> A repo we had in 1.2.3 was 50GB at the time. We converted to 1.4 and
> it shrank to 38GB.
>
> So yes, we do need to convert.
>
> With offices in geographically diverse regions, the leaner and meaner
> our dbs are, the better.
You missed my point, as I did mention that in my message.
1. Not every release has made changes in the repository. For
example, 1.5 changes the repository but there is no need to dump/load,
the changes will happen automatically.
2. When there is something like a space reduction (which I cannot
imagine happening again), you still do not need to base your upgrade
decision on this issue. You could have upgraded the repository to 1.4
and then re-loaded the repository when you had an acceptable window of
downtime.
The bottom line is that tigris handles this issue by trying to avoid it.
Something else to keep in mind is that as of 1.4, we now have the
svnsync tool. This is great for doing a migration. You can create a
new fsfs repository using the new release, and then use svnsync to
migrate from the old live repository to this one. Once the sync is
finished, just want for your next downtime window, do one final sync,
then shutdown the server and move in the new repository and restart.
Or better, would be to just change the server to be serving from the
new location. If you use this approach just make sure the new
repository you create has the same UUID as the master.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: how does tigris do server updates?
Posted by Brian Krusic <br...@krusic.com>.
Hi Mark,
While u r correct about serving dbs regardless version as thats what
we do on some of our repos now, there are benefits to upgrading so I
disagree with you.
A repo we had in 1.2.3 was 50GB at the time. We converted to 1.4 and
it shrank to 38GB.
So yes, we do need to convert.
With offices in geographically diverse regions, the leaner and meaner
our dbs are, the better.
-Brian
On Jul 20, 2007, at 5:45 PM, Mark Phippard wrote:
> On 7/20/07, Brian Krusic <br...@krusic.com> wrote:
>> Hello once again;
>>
>> When you guys at Tigris do your back end SVN updates from say repo
>> version
>> 1.3 to 1.4, etc....
>>
>> ... how do you go about it in terms of minimizing down time?
>>
>> I'm asking as we have about 5 repos each at about 40-60GB and
>> based on our
>> tests, the process of converting takes about 1.5 days per repo on
>> a P3. We
>> have serveral Opterons and Core Duos which we can spread the
>> conversion
>> process on that would speed this up.
>>
>> We figure 1 week total for implementation/debug. We are testing
>> now but
>> will most liely wait till 1,5 is out as the sync/proxy feature is
>> shweeeet!
>>
>> I know that I am asking systems methodology advice, but since you
>> guys do it
>> all day long, it doesn't hurt to ask.
>
> Did you know that you do not need to do this? You could have created
> repositories with SVN 1.0 and still serve them with 1.4 as well as the
> upcoming 1.5. I think there have been two releases where there were
> some minor improvements you could get by doing a dump/load, but you
> have never had to. The next release will add small a SQLite database
> to the repository, as an example, and it will just get created the
> first time it is needed.
>
> --
> Thanks
>
> Mark Phippard
> http://markphip.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: how does tigris do server updates?
Posted by Mark Phippard <ma...@gmail.com>.
On 7/20/07, Brian Krusic <br...@krusic.com> wrote:
> Hello once again;
>
> When you guys at Tigris do your back end SVN updates from say repo version
> 1.3 to 1.4, etc....
>
> ... how do you go about it in terms of minimizing down time?
>
> I'm asking as we have about 5 repos each at about 40-60GB and based on our
> tests, the process of converting takes about 1.5 days per repo on a P3. We
> have serveral Opterons and Core Duos which we can spread the conversion
> process on that would speed this up.
>
> We figure 1 week total for implementation/debug. We are testing now but
> will most liely wait till 1,5 is out as the sync/proxy feature is shweeeet!
>
> I know that I am asking systems methodology advice, but since you guys do it
> all day long, it doesn't hurt to ask.
Did you know that you do not need to do this? You could have created
repositories with SVN 1.0 and still serve them with 1.4 as well as the
upcoming 1.5. I think there have been two releases where there were
some minor improvements you could get by doing a dump/load, but you
have never had to. The next release will add small a SQLite database
to the repository, as an example, and it will just get created the
first time it is needed.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org