svn update or rsync - which is best to update live files

[Christopher D Haakinson <cd...@us.ibm.com>]

Hi, I have subversion up and running quite well now(thanks to everyone
here :-))

Now I've come to the point in my development where I need to figure out
what's the best method for transferring my svn files to my live site.

I've read that I should use a post-commit hook, which is fine I'm using a
pre-commit hook already so I'm aware of how they work.

My main question is:  Which method is best for updating my live files?
Should I use rsync in my post hook or should I use  svn update?

http://subversion.apache.org/faq.html#website-auto-update   this page
recommends using svn update, but I wanted to get some input from the
community too.

Thanks in advance!

Re: svn update or rsync - which is best to update live files

[Campbell Allan <ca...@sword-ciboodle.com>]

On Sunday 27 Feb 2011, Chris Albertson wrote:
> OK,   There is one way that works.  I'd suggest using it.
>
> Most server OSes now have some way to make a virtual hoist.  Solaris
> has zones, Linux has xen and there is VMware.  The idea is that you
> build the new updated web site on a new virtual host.  and then you
> test it.  Once you like the tst results you swap IP addresses.     You
> don't really want to go live with changes that are untested.  The IP
> address is "atomic" in that the entire site goes live at once.
>
> BTW I've always liked the way Solaris does this in that you can have
> many "zones" each with their own IP address and file system but there
> is only one instance of the operating system running on the hardware.
>  I always thought it a waste to have multiple copies of the same
> software in RAM, Sun solved that problem well.

I agree with your sentiment of not running untested configurations. The 
deployment/environment setup should also be automated but with manual 
oversight. That usually ensures all the steps to setup the environment are 
followed but any issues can be handled at the time and not found out the next 
morning. I guess it depends on how much business value the website brings 
whether this is worth it :)

The ip address change might be atomic but this only works if there is no state 
stored in the webserver/appserver. Any stateful sessions will be lost. Still, 
for single physical server sites it's a nice approach. If there is more than 
one physical machine involved then switching the loadbalancer policy would do 
the same trick. This then relies on having enough spare server capacity to 
support the two environments running at the same time though this could be 
mitigated by doing the switchover at a quiet time. Sometimes it just isn't 
possible to do an atomic switchover. For example, if a backend service also 
needs to be updated or a database schema needs changed.

-- 

__________________________________________________________________________________
Sword Ciboodle is the trading name of ciboodle Limited (a company 
registered in Scotland with registered number SC143434 and whose 
registered office is at India of Inchinnan, Renfrewshire, UK, 
PA4 9LH) which is part of the Sword Group of companies.

This email (and any attachments) is intended for the named
recipient(s) and is private and confidential. If it is not for you, 
please inform us and then delete it. If you are not the intended 
recipient(s), the use, disclosure, copying or distribution of any 
information contained within this email is prohibited. Messages to 
and from us may be monitored. If the content is not about the 
business of the Sword Group then the message is neither from nor 
sanctioned by us.

Internet communications are not secure. You should scan this
message and any attachments for viruses. Under no circumstances
do we accept liability for any loss or damage which may result from
your receipt of this email or any attachment.
__________________________________________________________________________________

Re: svn update or rsync - which is best to update live files

[Chris Albertson <al...@gmail.com>]

OK,   There is one way that works.  I'd suggest using it.

Most server OSes now have some way to make a virtual hoist.  Solaris
has zones, Linux has xen and there is VMware.  The idea is that you
build the new updated web site on a new virtual host.  and then you
test it.  Once you like the tst results you swap IP addresses.     You
don't really want to go live with changes that are untested.  The IP
address is "atomic" in that the entire site goes live at once.

BTW I've always liked the way Solaris does this in that you can have
many "zones" each with their own IP address and file system but there
is only one instance of the operating system running on the hardware.
 I always thought it a waste to have multiple copies of the same
software in RAM, Sun solved that problem well.

-- 
=====
Chris Albertson
Redondo Beach, California

Re: svn update or rsync - which is best to update live files

[Ryan Schmidt <su...@ryandesign.com>]

On Feb 26, 2011, at 17:26, Blair Zajac wrote:

> On Feb 26, 2011, at 2:47 PM, Ryan Schmidt wrote:
> 
>> On Feb 25, 2011, at 13:53, Blair Zajac wrote:
>> 
>>> First, both svn and rsync do atomic replaces of the file, so you don't have to worry about the live files being observed in some odd state.
>> 
>> Of course you do. The individual files may be atomically replaced, but the entire set of files won't be. If, like most web sites, your site is made with a scripting language, and file A includes file B, and both files have been changed, there will probably be a time during the update when A has been updated and B has not, or vice versa.
> 
> Well, you're going past the original question.  Neither rsync nor svn help with that case, as there aren't atomic updates to sets of files, which you state in the thread linked below.
> 
>> Here is a previous discussion with recommendations. See especially the paragraph that starts "For finally deploying a site to the production server" and the example that follows:
>> 
>> http://svn.haxx.se/users/archive-2009-04/0083.shtml
> 
> Right, the symlink approach gets you 90% there.
> 
> But even this doesn't get you 100% of the way there, if the browser downloads C and D and in between the symlink is changed (not talking about when A includes B that is a single GET).

Heck, there's even the problem of the web app changing between requests. What if you log into the web app when it is at version A, and then update to version B which introduces changes in the login procedure which cause additional data to be stored in the session with the login? Users who already logged into the app with version A will now be running around version B of the app without this additional information in their session, until they log out and back in.

There's tons of "what if" scenarios that should be considered. In our case, we decided that we did not want the app's files on the server in an inconsistent state for the possibly minutes it could take to run an "svn up". We decided that the millisecond it takes to delete a symlink and create a new one was a much more acceptable situation for us. YMMV.

Re: svn update or rsync - which is best to update live files

[Blair Zajac <bl...@orcaware.com>]

On Feb 26, 2011, at 2:47 PM, Ryan Schmidt wrote:

> On Feb 25, 2011, at 13:53, Blair Zajac wrote:
>
>> On 2/25/11 11:39 AM, Christopher D Haakinson wrote:
>>>
>>
>>> Now I've come to the point in my development where I need to  
>>> figure out what's
>>> the best method for transferring my svn files to my live site.
>>>
>>> I've read that I should use a post-commit hook, which is fine I'm  
>>> using a
>>> pre-commit hook already so I'm aware of how they work.
>>>
>>> My main question is: Which method is best for updating my live  
>>> files? Should I
>>> use rsync in my post hook or should I use svn update?
>>>
>>> http://subversion.apache.org/faq.html#website-auto-update this  
>>> page recommends
>>> using svn update, but I wanted to get some input from the  
>>> community too.
>>>
>>> Thanks in advance!
>>
>> First, both svn and rsync do atomic replaces of the file, so you  
>> don't have to worry about the live files being observed in some odd  
>> state.
>
> Of course you do. The individual files may be atomically replaced,  
> but the entire set of files won't be. If, like most web sites, your  
> site is made with a scripting language, and file A includes file B,  
> and both files have been changed, there will probably be a time  
> during the update when A has been updated and B has not, or vice  
> versa.

Well, you're going past the original question.  Neither rsync nor svn  
help with that case, as there aren't atomic updates to sets of files,  
which you state in the thread linked below.

> Here is a previous discussion with recommendations. See especially  
> the paragraph that starts "For finally deploying a site to the  
> production server" and the example that follows:
>
> http://svn.haxx.se/users/archive-2009-04/0083.shtml

Right, the symlink approach gets you 90% there.

But even this doesn't get you 100% of the way there, if the browser  
downloads C and D and in between the symlink is changed (not talking  
about when A includes B that is a single GET).

Blair

Re: svn update or rsync - which is best to update live files

[Ryan Schmidt <su...@ryandesign.com>]

On Feb 25, 2011, at 13:53, Blair Zajac wrote:

> On 2/25/11 11:39 AM, Christopher D Haakinson wrote:
>> 
> 
>> Now I've come to the point in my development where I need to figure out what's
>> the best method for transferring my svn files to my live site.
>> 
>> I've read that I should use a post-commit hook, which is fine I'm using a
>> pre-commit hook already so I'm aware of how they work.
>> 
>> My main question is: Which method is best for updating my live files? Should I
>> use rsync in my post hook or should I use svn update?
>> 
>> http://subversion.apache.org/faq.html#website-auto-update this page recommends
>> using svn update, but I wanted to get some input from the community too.
>> 
>> Thanks in advance!
> 
> First, both svn and rsync do atomic replaces of the file, so you don't have to worry about the live files being observed in some odd state.

Of course you do. The individual files may be atomically replaced, but the entire set of files won't be. If, like most web sites, your site is made with a scripting language, and file A includes file B, and both files have been changed, there will probably be a time during the update when A has been updated and B has not, or vice versa.

Here is a previous discussion with recommendations. See especially the paragraph that starts "For finally deploying a site to the production server" and the example that follows:

http://svn.haxx.se/users/archive-2009-04/0083.shtml

We manually ran a script on the production server when we wanted to deploy a tag there. If you want to do automatic deploys in the post-commit hook of the repository server, for example in response to creating a tag with a particular naming format, then the SVN::Notify::Mirror perl module will help you do that. Its author has explained its use on this mailing list many times, for example here:

http://svn.haxx.se/users/archive-2006-02/1148.shtml

Re: svn update or rsync - which is best to update live files

[Blair Zajac <bl...@orcaware.com>]

On 2/25/11 11:39 AM, Christopher D Haakinson wrote:
> Hi, I have subversion up and running quite well now(thanks to everyone here :-))
>
> Now I've come to the point in my development where I need to figure out what's
> the best method for transferring my svn files to my live site.
>
> I've read that I should use a post-commit hook, which is fine I'm using a
> pre-commit hook already so I'm aware of how they work.
>
> My main question is: Which method is best for updating my live files? Should I
> use rsync in my post hook or should I use svn update?
>
> http://subversion.apache.org/faq.html#website-auto-update this page recommends
> using svn update, but I wanted to get some input from the community too.
>
> Thanks in advance!

First, both svn and rsync do atomic replaces of the file, so you don't have to 
worry about the live files being observed in some odd state.

There are tradeoffs.

- rsync will need to stat() each file in your working copy
- svn will only update the files that are newer; however
- svn currently requires a lock on each subdirectory

rsync will have two times the stat()'s, one for the source and one for the 
destination.

Probably for speed, your best bet is to time either.

In 1.7, I would definitely use svn, as the locking issue goes away.

Blair

Re: svn update or rsync - which is best to update live files

[Thorsten Schöning <ts...@am-soft.de>]

Guten Tag Christopher D Haakinson,
am Freitag, 25. Februar 2011 um 20:39 schrieben Sie:

> My main question is:  Which method is best for updating my live files?
> Should I use rsync in my post hook or should I use  svn update?

I would always prefer svn update and use tags for production software,
this way your are able to version configurations, quickly needed fixes
for errors etc.

Mit freundlichen Grüßen,

Thorsten Schöning

-- 
Thorsten Schöning
AM-SoFT IT-Systeme - Hameln | Potsdam | Leipzig
 
Telefon: Potsdam: 0331-743881-0
E-Mail:  tschoening@am-soft.de
Web:     http://www.am-soft.de

AM-SoFT GmbH IT-Systeme, Konsumhof 1-5, 14482 Potsdam
Amtsgericht Potsdam HRB 21278 P, Geschäftsführer: Andreas Muchow