Running svnadmin pack on a live repository

[Magnus Torfason <zu...@gmail.com>]

I am wondering what happens if one runs svnadmin pack on a
repository at the same time as a client is performing a
checkout (for example). Is this supported, or is the
onus on the administrator to prevent access to the repository
while it is packed? I didn't see this mentioned in the release
notes or in the book.

I realize that this may be more of a usage question, but the
motivation is to try to specify the requirements for performing
an absolute obliteration on a live repository. If packing
requires repository access to be shut down, I don't think an
absolute obliteration operation should aim for more, agreed?

If svnadmin pack handles live repositories gracefully I would
be interested in determining whether the same method could be used
for obliteration, since it requires a complete rewrite of the
repository in the same way. If not, I would want the specification
to reflect that, since I would like the specification to be
reasonably implementable.

Best,
Magnus

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=1234935

Re: Running svnadmin pack on a live repository

[Magnus Torfason <zu...@gmail.com>]

Hi, thanks for the comments, just a few minor notes below.

Hyrum K. Wright wrote:

> 'svnadmin pack' can be run on live repositories.  I didn't even know the 
> book mentioned packing, but we probably should do I better job of 
> describing those usage scenarios in the release notes.

The book *doesn't* mention packing, which is of course the reason I
found nothing on packing live repositories there :-)

> Agreed.  If an admin wanted to take the repo offline for an obliterate, 
> they'd just use svndumpfilter.  One of the major selling points of 
> 'svnadmin obliterate' is that it should be able to be done while the 
> repository is live.

Well, almost, but not quite. Even a completely offline obliterate
command brings some benefits over and above svndumpfilter.
Most important: Copying from an obliterated path breaks svndumpfilter.
Also important: It is very limited in choosing what should be removed.

> Obliteration is a little more indepth than packing.  Packing just 
> squishes all the files together, so their internal offsets and such are 
> still valid, whereas obliterate removes actual data, potentially 
> invalidating these offsets (unless you just backfill the removed data 
> with zeros, but then you miss the space-saving benefits).  With packing 
> enabled, we simply insert a shim into the code which interprets those 
> offsets so that it can work within pack files.  No mess, no fuss.  
> Because you're actually rewriting history, obliterate may be a bit more 
> complex.

Judging from this, and some FSFS reading I did just now, a rev file will
contain pointers to specific byte positions within older rev files. I
hadn't realized that. (I thought the pointers were to logical positions
within older rev files). This surely complicates any attempt to
implement online absolute obliteration.

> By the way, I'm interested in your work on 'svnadmin obliterate' and 
> when I get a chance will review some of the documentation you're 
> written.  Of course, we want to make sure as much of it as possible ends 
> up in http://svn.collab.net/repos/svn/trunk/notes/obliterate/  That 
> archive your thoughts for posterity and allow others to iterate on your 
> design work.

Thanks, I'll make sure that it does.

> -Hyrum

Best,
Magnus

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=1239189

Re: Running svnadmin pack on a live repository

["Hyrum K. Wright" <hy...@mail.utexas.edu>]

On Feb 26, 2009, at 3:17 PM, Magnus Torfason wrote:

> I am wondering what happens if one runs svnadmin pack on a
> repository at the same time as a client is performing a
> checkout (for example). Is this supported, or is the
> onus on the administrator to prevent access to the repository
> while it is packed? I didn't see this mentioned in the release
> notes or in the book.

'svnadmin pack' can be run on live repositories.  I didn't even know  
the book mentioned packing, but we probably should do I better job of  
describing those usage scenarios in the release notes.

> I realize that this may be more of a usage question, but the
> motivation is to try to specify the requirements for performing
> an absolute obliteration on a live repository. If packing
> requires repository access to be shut down, I don't think an
> absolute obliteration operation should aim for more, agreed?

Agreed.  If an admin wanted to take the repo offline for an  
obliterate, they'd just use svndumpfilter.  One of the major selling  
points of 'svnadmin obliterate' is that it should be able to be done  
while the repository is live.

> If svnadmin pack handles live repositories gracefully I would
> be interested in determining whether the same method could be used
> for obliteration, since it requires a complete rewrite of the
> repository in the same way. If not, I would want the specification
> to reflect that, since I would like the specification to be
> reasonably implementable.


Obliteration is a little more indepth than packing.  Packing just  
squishes all the files together, so their internal offsets and such  
are still valid, whereas obliterate removes actual data, potentially  
invalidating these offsets (unless you just backfill the removed data  
with zeros, but then you miss the space-saving benefits).  With  
packing enabled, we simply insert a shim into the code which  
interprets those offsets so that it can work within pack files.  No  
mess, no fuss.  Because you're actually rewriting history, obliterate  
may be a bit more complex.

By the way, I'm interested in your work on 'svnadmin obliterate' and  
when I get a chance will review some of the documentation you're  
written.  Of course, we want to make sure as much of it as possible  
ends up in http://svn.collab.net/repos/svn/trunk/notes/obliterate/   
That archive your thoughts for posterity and allow others to iterate  
on your design work.

-Hyrum

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=1235059