Move to a new repo and keep the history, Part 2

[K F <cm...@yahoo.com>]

Recap – I would like to move some directories from one repository to another while keeping the history.
 
    Repo1/tags/tag1
 
and move tag1 to Repo2 
 
    Repo2/tags/tag1
 
I’ve run 
    svnadmin dump /svnrepo/ Repo1/ > repo1/ repo1_dump

 and the resulting file appears to be correct (get Dumped revision). 

I’ve then run 
    svndumpfilter include tags < repo1_dump > stags

and the resulting file appears to be correct (get Dropped nodes for branches and trunk). 

I then tried doing another dumpfilter on stags,
    svndumpfilter include tag1 < stags > tag1

to get just the tag1 folder but it doesn't seem to work, it drops all nodes under it. Am I doing something incorrectly or is it impossible to do a 'double' filter? I do not want all the tags under the tag directory, just specific ones (all at once preferably, but I would guess would have to be done one at a time).

Thanks,
Rich

Re: Move to a new repo and keep the history, Part 2

[Thomas Harold <th...@nybeta.com>]

On 7/14/2011 12:29 PM, K F wrote:
> Recap – I would like to move some directories from one repository to
> another while keeping the history.
>

I went through this a few months ago (and maybe this will help).  We 
were using a big monolithic repository for all of our jobs. Our 
repository was arranged as:

("jobs" repository)
/A/ABClient/ABJob1/...
/A/AXClient/AXJob1/...
/A/AXClient/AXJob2/...
/B/BQClient/BQJob1/...
/C/CAClient/CAJob1/...
/C/CMClient/CMJob1/...
/C/CMClient/CMJob2/...

We wanted to split each client out to a different repository.  The 
output repositories would look like:

("jobs-ab")
/ABJob1/...

("jobs-ax")
/AXJob1/...
/AXJob2/...

("jobs-bq")
/BQJob1/...

("jobs-ca")
/CAJob1/...

("jobs-cm")
/CMJob1/...
/CMJob2/...

Which made all the URLs a lot shorter because the "/A/ABClient" was 
often something lengthy like "/A/AB_Acme_Border_Wings_Inc".

1) A shell script to split out a specific directory.  We had to edit the 
CLCODE and CLPATH lines for each run (took 30-40 minutes to parse the 
monolithic jobs repository and split out a particular client's tree).

Each time I did a new client, I had to make sure that everyone was ready 
for that client's project tree to move.  Alternately, I could have made 
the entire "jobs" tree read-only for a week...

(Apologies if there are errors in this as I had to quickly edit out some 
client/company specific paths.  I always executed the script as "bash -x 
scriptname" so I could spot errors.  The "date" lines are just there so 
I could keep track of how long it took.)

#!/bin/bash

DESTDIR=/var/svn/
DESTPFX=svn-raw-jobs-
DESTSFX=10xx.dump.gz

CLCODE=bq
CLPATH=B/BQClient

SDFOPTS='--drop-empty-revs  --renumber-revs'

date

echo ${DESTDIR}${DESTPFX}${CLCODE}${DESTSFX}

svnadmin dump --quiet /var/svn/jobs | \
     svndumpfilter include --quiet $SDFOPTS $CLPATH | gzip > \
     ${DESTDIR}${DESTPFX}${CLCODE}${DESTSFX}

date

2) Created the new repository (such as "jobs-bq" for the BQClient). 
Although you could probably roll that into the import script.

# svnadmin create jobs-bq

3) Edited the following import script for each new run.  Loading it up 
was a fairly quick process.  Note that we update the UUID of the new 
repository to make sure that nobody commits outdated stuff.

I gave up on trying to re-base on the fly using "sed" and simply moved 
all of the individual job folders into the root of the new repository 
and then cleaned up the left-over folder.

Our script also had to create the "letter" level in the new repository. 
  Otherwise the import had no place to hang itself off of.

You may want to drop the chmod/chgrp lines towards the end.  For our 
server, we only use svn+ssh authentication.  Each users has their own 
local account on the server and they belong to a "svn-jobs" group which 
gives them read/write access to the entire repository.

#!/bin/bash

SRCDIR=/var/svn/
SRCPFX=svn-raw-jobs-
SRCSFX=10xx.dump.gz

DESTDIR=/var/svn/
DESTPFX=svn-newbase-jobs-
DESTSFX=10xx.dump.gz

CLPARENT=B
CLCODE=bq

date

svn mkdir -m "Import from jobs" \
     file:///var/svn/jobs-${CLCODE}/${CLPARENT}

gunzip -c ${SRCDIR}${SRCPFX}${CLCODE}${SRCSFX} | \
     svnadmin load --quiet /var/svn/jobs-${CLCODE}

svnlook uuid /var/svn/jobs-${CLCODE}
svnadmin setuuid /var/svn/jobs-${CLCODE}
svnlook uuid /var/svn/jobs-${CLCODE}
svnadmin pack /var/svn/jobs-${CLCODE}

chmod -R 775 /var/svn/jobs-${CLCODE}
chmod -R g+s /var/svn/jobs-${CLCODE}/db
chgrp -R svn-jobs /var/svn/jobs-${CLCODE}

date

4) After the load into the new repository, I would access the repository 
through TortoiseSVN's Repository Browser and drag all of the individual 
jobs folders from being under "/A/ABClient/" up to the root of the new 
repository.  Then I deleted the "/A/ABClient/" tree.

If I could have figured out how to remap on the fly via "sed", I could 
have avoided step #4.  But it was good enough for our purposes and a few 
historical entries in the SVN log showing the movement of the folders 
wasn't a big deal.

5) After we finished the splits, we chmod'd the old repository to be 
read-only and took it completely offline a month or two later.