Planning a SVN upgrade

[Maureen Barger <mo...@gmail.com>]

Hi -
I am currently planning an upgrade from SVN 1.5 (using svnserve and
ssh tunnel) to SVN 1.8.1 fronted with Apache and webdav using AD for
authNz.
We have about 50 repos. I'll be moving from an older Ubuntu 8 install
to Centos 6 x64.

My thought was I could upgrade the SVN installation in place, bringing
the repo up to 1.8 and then dump those repos and bring them online in
the new environment.

We currently use Eclipse as our IDE and Jenkins as our CI tool with
Nexus as the object repo. I was thinking to leave the upgrade of
Eclipse client and svnkit to the indiviidual so they can decide what
direction to take with their working copies et al. I do not foresee
any changes I would need to make to Jenkins or Nexus.

Has anyone made a jump this large before? Any comments about my upgrade plan?

Thanks!

Re: Planning a SVN upgrade

[Thomas Harold <th...@nybeta.com>]

On 8/23/2013 4:09 PM, Maureen Barger wrote:
> Has anyone made a jump this large before? Any comments about my upgrade plan?

We jumped from 1.5 to 1.8 on our server-side repositories.  Our clients 
were using a mix of 1.6 and 1.7 (most of the "important to us" 
improvements in SVN 1.6 and 1.7 were client-side).

If you dig back a week in the mailing list, you can see Geoff's and 
myself's conversation on the upgrade process.  ("How Big A Dump File Can 
Be Handled?")

We did our upgrade in-place, but had a second set of spindles (an NFS 
file system on another server) that we used to dump/load from.  Locking 
each repo down to read-only while the script processed it, then opening 
it back up.  Running "svnadmin verify" before each dump and again after 
the load.

To echo what Mark said, I suggest upgrading the box first (moving to 
CentOS 6), installing WANdisco SVN 1.8 right off the bat, but just 
bringing over the SVN repositories "as-is" (in 1.4/1.5/etc format).  Get 
it up and running, then worry about the upgrade to 1.8 repo format.

(WANdisco is currently the easiest way to get 1.8.1 installed on CentOS 
6.  You download an install script, it adds a repo and installs SVN 1.8.1.)

RE: Planning a SVN upgrade

[Geoff Field <Ge...@aapl.com.au>]

________________________________
From: Mark Phippard
Sent: Saturday, 24 August 2013 6:35 AM
On Fri, Aug 23, 2013 at 4:09 PM, Maureen Barger wrote:

I am currently planning an upgrade from SVN 1.5 (using svnserve and
ssh tunnel) to SVN 1.8.1 fronted with Apache and webdav using AD for
authNz.
We have about 50 repos. I'll be moving from an older Ubuntu 8 install
to Centos 6 x64.

We have just upgraded our server from 1.2.3 to 1.8.1, including a move from Apache 2.0 to Apache2.2.
It took me some time, but it was done as a bit of a background task.  There were 74 BDB repositories that had to be dumped and loaded to FSFS format.


My thought was I could upgrade the SVN installation in place, bringing
the repo up to 1.8 and then dump those repos and bring them online in
the new environment.

If it were me, I would not upgrade the repositories.  SVN 1.8 can just serve the old repositories.  I would do the upgrade and only after I was using it for a while would I then consider to start doing a dump/load on the repositories.  You could then do them one by one as desired.  The main benefit in upgrading the repository is to use less disk space.
I also would not upgrade existing repositories just for the sake of it.  If there's a feature you feel would be useful that's only available with the 1.8 repo format, then I would do it, but ONLY for the active repos.  The only reason I upgraded the BDB repositories was because I could no longer access them with the new server software.  Even then, I left them in the 1.2 format in case we had to go back to the original server for some reason.

If you're going to do an upgrade on the repositories, make sure you back them up first.  Then the disk space issue becomes moot, because the backups take space as well.
We currently use Eclipse as our IDE and Jenkins as our CI tool with
Nexus as the object repo. I was thinking to leave the upgrade of
Eclipse client and svnkit to the indiviidual so they can decide what
direction to take with their working copies et al.

Yes, your clients can already be using 1.8 if they want to.  There is no need to upgrade the client either before or after the server.  Let the clients manage it.  Only exception is if there are specific new features you want to implement across the board.  If you do a lot of branching and merging, it would be a good idea for the people that do merge to all be using the same version.  Likewise, there are other features that might be like this.

I concur.  (Of course, Mark has a lot more SVN experience and in-depth knowledge than I do.)   Leave it to the individual to decide whether to upgrade.  There are very few cases where the server and client software are incompatible between versions.  Mind you, I did have to do our upgrade to the server because version 1.8.x of the client doesn't play nicely with version 1.2.x of the server, in terms of adding new files and displaying logs.  That's how I came to join the mailing lists in the first place.

I do not foresee
any changes I would need to make to Jenkins or Nexus.

Just the URL to access the repository will change.

Even that doesn't have to change.  We're using the same URLs.

Has anyone made a jump this large before? Any comments about my upgrade plan?

There is nothing unusual about this.  People have jumped from 1.1 to 1.8.

In my case, 1.2 to 1.8.  By comparison, 1.5 to 1.8 should be easy.

Geoff


--
Apologies for the auto-generated legal boilerplate added by our IT department:


- The contents of this email, and any attachments, are strictly private
and confidential.
- It may contain legally privileged or sensitive information and is intended
solely for the individual or entity to which it is addressed.
- Only the intended recipient may review, reproduce, retransmit, disclose,
disseminate or otherwise use or take action in reliance upon the information
contained in this email and any attachments, with the permission of
Australian Arrow Pty. Ltd.
- If you have received this communication in error, please reply to the sender
immediately and promptly delete the email and attachments, together with
any copies, from all computers.
- It is your responsibility to scan this communication and any attached files
for computer viruses and other defects and we recommend that it be
subjected to your virus checking procedures prior to use.
- Australian Arrow Pty. Ltd. does not accept liability for any loss or damage
of any nature, howsoever caused, which may result
directly or indirectly from this communication or any attached files. 

Re: Planning a SVN upgrade

[Mark Phippard <ma...@gmail.com>]

On Fri, Aug 23, 2013 at 4:09 PM, Maureen Barger <mo...@gmail.com> wrote:


> I am currently planning an upgrade from SVN 1.5 (using svnserve and
> ssh tunnel) to SVN 1.8.1 fronted with Apache and webdav using AD for
> authNz.
> We have about 50 repos. I'll be moving from an older Ubuntu 8 install
> to Centos 6 x64.
>
> My thought was I could upgrade the SVN installation in place, bringing
> the repo up to 1.8 and then dump those repos and bring them online in
> the new environment.
>

If it were me, I would not upgrade the repositories.  SVN 1.8 can just
serve the old repositories.  I would do the upgrade and only after I was
using it for a while would I then consider to start doing a dump/load on
the repositories.  You could then do them one by one as desired.  The main
benefit in upgrading the repository is to use less disk space.


> We currently use Eclipse as our IDE and Jenkins as our CI tool with
> Nexus as the object repo. I was thinking to leave the upgrade of
> Eclipse client and svnkit to the indiviidual so they can decide what
> direction to take with their working copies et al.


Yes, your clients can already be using 1.8 if they want to.  There is no
need to upgrade the client either before or after the server.  Let the
clients manage it.  Only exception is if there are specific new features
you want to implement across the board.  If you do a lot of branching and
merging, it would be a good idea for the people that do merge to all be
using the same version.  Likewise, there are other features that might be
like this.


> I do not foresee
> any changes I would need to make to Jenkins or Nexus.
>

Just the URL to access the repository will change.

>
> Has anyone made a jump this large before? Any comments about my upgrade
> plan?
>

There is nothing unusual about this.  People have jumped from 1.1 to 1.8.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: Planning a SVN upgrade

[Nico Kadel-Garcia <nk...@gmail.com>]

On Thu, Sep 5, 2013 at 10:46 AM, Maureen Barger <mo...@gmail.com> wrote:

> That is a great idea, Giulio. How do you then make the mirrored repo
> writable?
>
> You don't make mirrors writable. You can make them read-only, and do
pass-through for write operations to the main repository for low grade
failover behavior, but mirrors are always, always, always at risk of
split-brain problems for write operations *unless* you invest enormous
sophistication in keeping them synced with the master, *or* unless you just
inveest some cash up front in the more sophisticated commercial tools our
friends and developers over at Wandisco provide their commercial grade
"Multi-Site" setups.

Doing manual switchovers of clients to an aalternative upstream mirrored
site is awkward and painful enough that, for commercial use, I'd go to
Multi-Site based tools in a heartbeat..

Re: Strange libtool error

[Ryan Schmidt <su...@ryandesign.com>]

On Sep 6, 2013, at 15:05, Scott R. Keszler <ke...@srkconsulting.com> wrote:

> /usr/local/src/subversion/libtool: line 865: X--tag=CC: command not found

googling for this error yields a number of hits.

https://www.google.com/search?rls=en&q=%22X--tag=CC:+command+not+found%22&ie=UTF-8&oe=UTF-8

Do you already have libtool installed, and if so is it an older version?

One result suggests doing this before running configure:

export echo=echo

If that's not it, read some of the other results.

Strange libtool error

["Scott R. Keszler" <ke...@srkconsulting.com>]

Compiling all components on RHEL 5.9: apr-1.4.8, apr-iconv-1.2.1, apr-util-1.5.2, httpd-2.2.24, serf-1.2.1, zlib-1.2.8, and subversion-1.8.3.

./configure --enable-optimize --enable-mod-activation --with-apr=/usr/local/apr/bin/apr-1-config --with-apr-util=/usr/local/apr/bin/apu-1-config --with-serf=/usr/local --with-apxs=/usr/local/sbin/apxs --with-apache-libexecdir=/usr/local/lib/apache --with-expat=/usr/include:/usr/lib64:expat --without-berkeley-db --with-sasl=/usr/lib64 --with-libmagic=/usr/lib64 --with-editor=/usr/bin/vim --with-zlib=/usr/local/lib
configure: Configuring Subversion 1.8.3
[...]
configure: creating ./config.status
config.status: creating Makefile
config.status: creating tools/backup/hot-backup.py
config.status: creating tools/hook-scripts/commit-access-control.pl
config.status: creating subversion/bindings/swig/perl/native/Makefile.PL
config.status: creating subversion/svn_private_config.h.tmp
config.status: executing libtool commands
config.status: executing svn_private_config.h.tmp commands

/usr/bin/make
/bin/sh /usr/local/src/subversion/libtool --tag=CC --silent --mode=compile gcc -std=c89  -DLINUX -D_REENTRANT -D_GNU_SOURCE   -fwhole-program -O3 -g -g -pthread   -I./subversion/include -I./subversion -I/usr/local/apr/include/apr-1   -I/usr/local/apr/include/apr-1 -I/usr/local/src/apr-util/../apr-iconv/include                            -I/usr/include -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/gnome-keyring-1 -I/usr/lib64/include -I/usr/lib64/include -I/usr/local/include/serf-1 -I/usr/local/src/subversion/sqlite-amalgamation -I/usr/include -I/usr/local/lib/include  -o subversion/libsvn_delta/cancel.lo -c subversion/libsvn_delta/cancel.c
/usr/local/src/subversion/libtool: line 865: X--tag=CC: command not found
/usr/local/src/subversion/libtool: line 898: libtool: ignoring unknown tag : command not found
/usr/local/src/subversion/libtool: line 865: X--mode=compile: command not found
/usr/local/src/subversion/libtool: line 1031: *** Warning: inferring the mode of operation is deprecated.: command not found
/usr/local/src/subversion/libtool: line 1032: *** Future versions of Libtool will require --mode=MODE be specified.: command not found
/usr/local/src/subversion/libtool: line 1175: Xgcc: command not found
/usr/local/src/subversion/libtool: line 1175: X-std=c89: command not found
/usr/local/src/subversion/libtool: line 1175: X-DLINUX: command not found
/usr/local/src/subversion/libtool: line 1175: X-D_REENTRANT: command not found
/usr/local/src/subversion/libtool: line 1175: X-D_GNU_SOURCE: command not found
/usr/local/src/subversion/libtool: line 1175: X-fwhole-program: command not found
/usr/local/src/subversion/libtool: line 1175: X-O3: command not found
/usr/local/src/subversion/libtool: line 1175: X-g: command not found
/usr/local/src/subversion/libtool: line 1175: X-g: command not found
/usr/local/src/subversion/libtool: line 1175: X-pthread: command not found
/usr/local/src/subversion/libtool: line 1175: X-I./subversion/include: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I./subversion: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I/usr/local/apr/include/apr-1: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I/usr/local/apr/include/apr-1: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I/usr/local/src/apr-util/../apr-iconv/include: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I/usr/include: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I/usr/include/glib-2.0: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I/usr/lib64/glib-2.0/include: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I/usr/include/gnome-keyring-1: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I/usr/lib64/include: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I/usr/lib64/include: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I/usr/local/include/serf-1: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I/usr/local/src/subversion/sqlite-amalgamation: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I/usr/include: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-I/usr/local/lib/include: No such file or directory
/usr/local/src/subversion/libtool: line 1175: X-c: command not found
/usr/local/src/subversion/libtool: line 1227: Xsubversion/libsvn_delta/cancel.lo: No such file or directory
/usr/local/src/subversion/libtool: line 1232: libtool: compile: cannot determine name of library object from `': command not found
make: *** [subversion/libsvn_delta/cancel.lo] Error 1

Re: Planning a SVN upgrade

[Giulio Troccoli <gi...@mediatelgroup.co.uk>]

On 05/09/13 15:46, Maureen Barger wrote:
> That is a great idea, Giulio. How do you then make the mirrored repo writable?
>
> On Mon, Sep 2, 2013 at 4:47 AM, Giulio Troccoli
> <gi...@mediatelgroup.co.uk> wrote:
>> On 23/08/13 21:09, Maureen Barger wrote:
>>> Hi -
>>> I am currently planning an upgrade from SVN 1.5 (using svnserve and
>>> ssh tunnel) to SVN 1.8.1 fronted with Apache and webdav using AD for
>>> authNz.
>>> We have about 50 repos. I'll be moving from an older Ubuntu 8 install
>>> to Centos 6 x64.
>>>
>>> My thought was I could upgrade the SVN installation in place, bringing
>>> the repo up to 1.8 and then dump those repos and bring them online in
>>> the new environment.
>>>
>>> We currently use Eclipse as our IDE and Jenkins as our CI tool with
>>> Nexus as the object repo. I was thinking to leave the upgrade of
>>> Eclipse client and svnkit to the indiviidual so they can decide what
>>> direction to take with their working copies et al. I do not foresee
>>> any changes I would need to make to Jenkins or Nexus.
>>>
>>> Has anyone made a jump this large before? Any comments about my upgrade
>>> plan?
>>>
>>> Thanks!
>> Being a totally new server, may I suggest using svnsync instead of a
>> dump/load cycle? It's very easy to set up, you can still use the old
>> repositories while syncing and if you take care of using the same UUID on
>> the new repository you might even be able to make the switch completely
>> transparent to the clients.
>>
>> I did an upgrade about three years ago, I think from 1.4 to 1.6, and I used
>> svnsync. It worked very well.
>>
>> I don't share others' concerns about not upgrading the repository (which
>> will happen if you use svnsync). I don't see why now. Besides, using
>> svnsync, you don't touch the old repositories at all so you still have the
>> old format repos if you need them.
>>
>> Just my 2p

Bear in mind this was about 4 years ago and I moved company so I don't 
have my notes.

I actually started using svnsync for our DR, but then I realised it's an 
excellent tool for upgrading too.

What I did was roughly something like:
- get the UUID for the old repo
- svnadmin create
- set up appropriate hooks - you need to do something with the hooks, 
the book should be able to tell you exactly what
- svnadmin setuuid
- svnsync init
- svnsync sync  - this should be done few times because the first time 
it will take a long time and by the time it finished there were quite a 
few new revisions. Basically you need to reach a point when the next 
svnsycn sync will take very little
- stop commits to the old repo - use the start-commit hook
- svnsync sync - one final sync
- remove a couple of svn properties on revision 0 on the new repo - I 
don't remember what they are called, but they are used by svnsync to 
keep track of what it's done.
- remove all hooks for the new repo and copy over all hooks from the old 
one - this was a bit tricky for me as one hook stopped working because 
of the new environment, so take care (of course if you test everything 
before doing the real thing you will be fine when you do it for real)
- change the dns so the the repo's URL points to the new server
- done - you may want to tell the users. Becuase the two repos ahve the 
same UUID and the URL hasn't change the users don't have to do any svn 
switch at all.

Don't forget to take care of authentication and authorisation. I think I 
had only one user allowed to write to the new repo, which was the 
special user I used for svnsync (with the --sync-username). After the 
last svnsync sync I set up all correct authentication and authorisation.

Sorry for beign a bit too sketchy, but as I said it was about 4 years 
ago and I don't have any notes.

Good luck

Giulio

Re: Planning a SVN upgrade

[Maureen Barger <mo...@gmail.com>]

That is a great idea, Giulio. How do you then make the mirrored repo writable?

On Mon, Sep 2, 2013 at 4:47 AM, Giulio Troccoli
<gi...@mediatelgroup.co.uk> wrote:
>
> On 23/08/13 21:09, Maureen Barger wrote:
>>
>> Hi -
>> I am currently planning an upgrade from SVN 1.5 (using svnserve and
>> ssh tunnel) to SVN 1.8.1 fronted with Apache and webdav using AD for
>> authNz.
>> We have about 50 repos. I'll be moving from an older Ubuntu 8 install
>> to Centos 6 x64.
>>
>> My thought was I could upgrade the SVN installation in place, bringing
>> the repo up to 1.8 and then dump those repos and bring them online in
>> the new environment.
>>
>> We currently use Eclipse as our IDE and Jenkins as our CI tool with
>> Nexus as the object repo. I was thinking to leave the upgrade of
>> Eclipse client and svnkit to the indiviidual so they can decide what
>> direction to take with their working copies et al. I do not foresee
>> any changes I would need to make to Jenkins or Nexus.
>>
>> Has anyone made a jump this large before? Any comments about my upgrade
>> plan?
>>
>> Thanks!
>
> Being a totally new server, may I suggest using svnsync instead of a
> dump/load cycle? It's very easy to set up, you can still use the old
> repositories while syncing and if you take care of using the same UUID on
> the new repository you might even be able to make the switch completely
> transparent to the clients.
>
> I did an upgrade about three years ago, I think from 1.4 to 1.6, and I used
> svnsync. It worked very well.
>
> I don't share others' concerns about not upgrading the repository (which
> will happen if you use svnsync). I don't see why now. Besides, using
> svnsync, you don't touch the old repositories at all so you still have the
> old format repos if you need them.
>
> Just my 2p

Re: Planning a SVN upgrade

[Giulio Troccoli <gi...@mediatelgroup.co.uk>]

On 23/08/13 21:09, Maureen Barger wrote:
> Hi -
> I am currently planning an upgrade from SVN 1.5 (using svnserve and
> ssh tunnel) to SVN 1.8.1 fronted with Apache and webdav using AD for
> authNz.
> We have about 50 repos. I'll be moving from an older Ubuntu 8 install
> to Centos 6 x64.
>
> My thought was I could upgrade the SVN installation in place, bringing
> the repo up to 1.8 and then dump those repos and bring them online in
> the new environment.
>
> We currently use Eclipse as our IDE and Jenkins as our CI tool with
> Nexus as the object repo. I was thinking to leave the upgrade of
> Eclipse client and svnkit to the indiviidual so they can decide what
> direction to take with their working copies et al. I do not foresee
> any changes I would need to make to Jenkins or Nexus.
>
> Has anyone made a jump this large before? Any comments about my upgrade plan?
>
> Thanks!
Being a totally new server, may I suggest using svnsync instead of a 
dump/load cycle? It's very easy to set up, you can still use the old 
repositories while syncing and if you take care of using the same UUID 
on the new repository you might even be able to make the switch 
completely transparent to the clients.

I did an upgrade about three years ago, I think from 1.4 to 1.6, and I 
used svnsync. It worked very well.

I don't share others' concerns about not upgrading the repository (which 
will happen if you use svnsync). I don't see why now. Besides, using 
svnsync, you don't touch the old repositories at all so you still have 
the old format repos if you need them.

Just my 2p