What to do when disk is nearly full

["Sinang, Danny" <D....@spi-bpo.com>]

Dear All,
 
What should I do when the disk that SVN is on is nearly full ?
 
I know I can migrate my repositories to a bigger disk. But is it also
possible to :
 
1. Move some repositories to NFS partitions
2. Obliterate old file copies from some repositories ?
 
Regards,
Danny
 
 

Re: What to do when disk is nearly full

[Nico Kadel-Garcia <nk...@comcast.net>]

Ryan Schmidt wrote:
> On May 16, 2006, at 18:07, Nico Kadel-Garcia wrote:
>
>>> What should I do when the disk that SVN is on is nearly full ?
>>>
>>> I know I can migrate my repositories to a bigger disk. But is it
>>> also possible to :
>>>
>>> 1. Move some repositories to NFS partitions
>>> 2. Obliterate old file copies from some repositories ?
>>
>> It's often possible to throw out a lot of old debris, by creating a
>> duplicate of the repository that is much smaller. But for various
>> reasons, NFS is a bad idea for Subversion repositories:
>>
>> Look into the "svndumpfilter" tool for doing duplications of a
>> repositories contents, with careful use of the "exclude" option.
>> But "old file copies" is usually a misnomer: if the fle exists in
>> *any* active branch, or is a base file for difference-recorded
>> copies of the file, you don't gain much by trying to eliminate
>> other copies of the file because Subversion uses a database, not
>> merely copies of the file.
>
> If your repository is FSFS (which IMHO it should be) you can move old
> revision files off to a second drive and symlink them back into place
> if you like.
>
> As far as I know, NFS breaks BDB repositories but isn't a problem for
> FSFS repositories.
>
> If you need to convert your repository from BDB to FSFS, instructions
> are in the FAQ on the web site.

If he's running out of disk space as it is, he may have some difficulty 
finding a place to migrate it to. And there's a potentially big performance 
hit for using something like NFS instead of local disk. Given the low cost 
of modern SATA nad high-speed IDE drives, there's little reason to use NFS 
for something like thus unless you need some sort of weird 
high-avaialability network appliance setup. 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: What to do when disk is nearly full

[Ryan Schmidt <su...@ryandesign.com>]

On May 16, 2006, at 18:07, Nico Kadel-Garcia wrote:

>> What should I do when the disk that SVN is on is nearly full ?
>>
>> I know I can migrate my repositories to a bigger disk. But is it  
>> also possible to :
>>
>> 1. Move some repositories to NFS partitions
>> 2. Obliterate old file copies from some repositories ?
>
> It's often possible to throw out a lot of old debris, by creating a  
> duplicate of the repository that is much smaller. But for various  
> reasons, NFS is a bad idea for Subversion repositories:
>
> Look into the "svndumpfilter" tool for doing duplications of a  
> repositories contents, with careful use of the "exclude" option.  
> But "old file copies" is usually a misnomer: if the fle exists in  
> *any* active branch, or is a base file for difference-recorded  
> copies of the file, you don't gain much by trying to eliminate  
> other copies of the file because Subversion uses a database, not  
> merely copies of the file.

If your repository is FSFS (which IMHO it should be) you can move old  
revision files off to a second drive and symlink them back into place  
if you like.

As far as I know, NFS breaks BDB repositories but isn't a problem for  
FSFS repositories.

If you need to convert your repository from BDB to FSFS, instructions  
are in the FAQ on the web site.



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: What to do when disk is nearly full

[Nico Kadel-Garcia <nk...@comcast.net>]

It's often possible to throw out a lot of old debris, by creating a duplicate of the repository that is much smaller. But for various reasons, NFS is a bad idea for Subversion repositories: 

Look into the "svndumpfilter" tool for doing duplications of a repositories contents, with careful use of the "exclude" option. But "old file copies" is usually a misnomer: if the fle exists in *any* active branch, or is a base file for difference-recorded copies of the file, you don't gain much by trying to eliminate other copies of the file because Subversion uses a database, not merely copies of the file.
  ----- Original Message ----- 
  From: Sinang, Danny 
  To: users@subversion.tigris.org 
  Sent: Tuesday, May 16, 2006 10:48 AM
  Subject: What to do when disk is nearly full


  Dear All,

  What should I do when the disk that SVN is on is nearly full ?

  I know I can migrate my repositories to a bigger disk. But is it also possible to :

  1. Move some repositories to NFS partitions
  2. Obliterate old file copies from some repositories ?

  Regards,
  Danny

Re: What to do when disk is nearly full

[Nick Thompson <ni...@agere.com>]

On Tuesday 16 May 2006 19:53, Jared Hardy wrote:
> On 5/16/06, Sinang, Danny <D....@spi-bpo.com> wrote:
> > 1. Move some repositories to NFS partitions
>
> As I understand it, this should be possible using FSFS, but may
> have some major performance issues.

We use Subversion with all of our FSFS repos mounted over NFS. It 
works fine and is fast as long as NFS caching is enabled. OTOH, we 
have gigabit Ethernet between the server and a NetApps filer which 
has several gig of disk cache. Our remote NFS accesses are faster 
than typical local disk accesses, though a local RAID array might do 
better with good hardware.

I think NFS caching is reasonable to use in our setup where only a 
single server accesses the repos. But it might not work so well if 
more than one server is used or if clients ever try to access the 
repo directly. With NFS cache turned off, Subversion client 
operations are many (~10) times slower.

Also make sure the NFS file locking is enabled and working. This is 
vital. Not all NFS servers support this, so please check carefully.

Bye,
-- 
> Nick Thompson

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: What to do when disk is nearly full

[Jared Hardy <ja...@gmail.com>]

On 5/16/06, Sinang, Danny <D....@spi-bpo.com> wrote:
> 1. Move some repositories to NFS partitions

As I understand it, this should be possible using FSFS, but may have
some major performance issues.

> 2. Obliterate old file copies from some repositories ?

My recommendation would be to use svndumpfilter to create two
repositories, split at some revision # that makes sense. Choose a
revision X that corresponds to a date before which most or all
repository data is too "old" to be useful, and make an "old" dump
containing revisions 0-X, and a "new" dump starting at version X and
keeping history through to HEAD. Burn the old dump half to DVDs, or
something suitable for archiving, and start using the new dump repo
exclusively. I personally plan to do something like this soon, except
that the old dump half will stay online (read only) via a slower
server/disk, and the new half will stay on the fast server with RAID
storage.
    Note that you may not save much space with this unless there are a
lot of deltas between 0-X that are not actually reflected in the data
of version X. You may be better off splitting your hierarchy into
separate repositories in some logical way, so you can maintain them on
separate storage volumes.

I have also considered experimenting with copying and simlinking old
revision number directories in the FSFS store to a different, slower
(cheaper) storage volume, but maintaining the same path on in the
original FSFS volume. Old revision data never changes, so I'm thinking
it will be accessed far less often, and write access could be safely
limited. I may try using an NFS volume as the slow volume. This is
playing with internals of FSFS data though, so it is probably not safe
to try at home unless you know a lot about the FSFS internal workings.
Definitely don't try this on a production repo without a lot of
experimentation on a backup repo first. Any Devs care to comment on
this?

    Jared

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org