Re: File capacity limitation in theory in SVN

[Mark Phippard <ma...@gmail.com>]

Redirecting this to the users@ list where it is more appropriate.

On Wed, Mar 20, 2019 at 7:33 AM sosogh <so...@126.com> wrote:

> Hi list
>
> There are about 1.65T , 2.72 million files ，274 thousand folders in Samba.
> It consist of any file types : txt , bin , pic , audio , video and so on .
> We are considering moving it from samba to SVN.
> And the data may grow larger and larger .
> We wonder that is there file capacity limitation in theory in SVN?
> or if the data is too large , will it cause any downgrade performance ?
> Thank you !
>
>
There are no size limits.  That said ...

*  Subversion is not a good choice to use as a file server for the simple
fact that you can not ever really delete anything.  Do not use Subversion
unless you are doing this to have version control and to store the history
of all files forever. That is what Subversion exists to provide.

* While there are no limits on number of files or total size, when you
start adding these files and folders to your repository I would strongly
recommend that you break it up into a lot of commits and not just one
really big commit.  If you do one large commit then you get a single giant
revision in the history and whenever you run commands like svn log that
look at the history it has to return this massive commit that can make
those commands not as nice to run as they are under normal circumstances.


-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: File capacity limitation in theory in SVN

[Daniel Shahaf <d....@daniel.shahaf.name>]

Mark Phippard wrote on Wed, 20 Mar 2019 11:53 +00:00:
> * While there are no limits on number of files or total size,

Not to nitpick, but since there's no such thing as "unlimited capacity",
let me be one level more accurate:

Internally Subversion stores file size using signed 64-bit integers, so
the theoretical limit on the size of any one file or one directory
representation is 2**63-1 bytes.

The size of a directory representation is proportional to the sum of the
lengths of filenames within it.

Re: File capacity limitation in theory in SVN

[Paul Hammant <pa...@hammant.org>]

Soso,

I've tested Subversion up to 4TB myself (one repo), with the intention of
using https://github.com/subsyncit/subsyncit with it for corporate file
sync. That is still a work in progress, as it happens.

- Paul

On Wed, Mar 20, 2019 at 11:53 AM Mark Phippard <ma...@gmail.com> wrote:

> Redirecting this to the users@ list where it is more appropriate.
>
> On Wed, Mar 20, 2019 at 7:33 AM sosogh <so...@126.com> wrote:
>
>> Hi list
>>
>> There are about 1.65T , 2.72 million files ，274 thousand folders in
>> Samba.
>> It consist of any file types : txt , bin , pic , audio , video and so on .
>> We are considering moving it from samba to SVN.
>> And the data may grow larger and larger .
>> We wonder that is there file capacity limitation in theory in SVN?
>> or if the data is too large , will it cause any downgrade performance ?
>> Thank you !
>>
>>
> There are no size limits.  That said ...
>
> *  Subversion is not a good choice to use as a file server for the simple
> fact that you can not ever really delete anything.  Do not use Subversion
> unless you are doing this to have version control and to store the history
> of all files forever. That is what Subversion exists to provide.
>
> * While there are no limits on number of files or total size, when you
> start adding these files and folders to your repository I would strongly
> recommend that you break it up into a lot of commits and not just one
> really big commit.  If you do one large commit then you get a single giant
> revision in the history and whenever you run commands like svn log that
> look at the history it has to return this massive commit that can make
> those commands not as nice to run as they are under normal circumstances.
>
>
> --
> Thanks
>
> Mark Phippard
> http://markphip.blogspot.com/
>

Re: File capacity limitation in theory in SVN

[Nico Kadel-Garcia <nk...@gmail.com>]

On Wed, Mar 20, 2019 at 7:53 AM Mark Phippard <ma...@gmail.com> wrote:
>
> Redirecting this to the users@ list where it is more appropriate.
>
> On Wed, Mar 20, 2019 at 7:33 AM sosogh <so...@126.com> wrote:
>>
>> Hi list
>>
>> There are about 1.65T , 2.72 million files ，274 thousand folders in Samba.
>> It consist of any file types : txt , bin , pic , audio , video and so on .
>> We are considering moving it from samba to SVN.
>> And the data may grow larger and larger .
>> We wonder that is there file capacity limitation in theory in SVN?
>> or if the data is too large , will it cause any downgrade performance ?
>> Thank you !
>>
>
> There are no size limits.  That said ...
>
> *  Subversion is not a good choice to use as a file server for the simple fact that you can not ever really delete anything.  Do not use Subversion unless you are doing this to have version control and to store the history of all files forever. That is what Subversion exists to provide.

Amen. This...... is unlikely to work well. Be prepared to fracture it
into a whole lot of distinct Subversion repositories if you want it to
perform well at all. *Nothing* will work well for a single repository
of 2.72 million files with 274 thousand folders, which hints that each
folder contains more than 1000 files and certain folders will contain
far more.

> * While there are no limits on number of files or total size, when you start adding these files and folders to your repository I would strongly recommend that you break it up into a lot of commits and not just one really big commit.  If you do one large commit then you get a single giant revision in the history and whenever you run commands like svn log that look at the history it has to return this massive commit that can make those commands not as nice to run as they are under normal circumstances.

Also, amen.