You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by svnuser <jz...@gmail.com> on 2009/03/27 01:45:21 UTC
Repo packing with subversion 1.6 results
I have sharded repo using 1.5.6 with 22K revisions, size 2.9G
I loaded its dumps into a new repo using 1.5.1 (I meant to use 1.5.6 but
this should do)
My new repo has the same size as my original (expected).
I run upgrade/packing on it with 1.6 and all I save is 100M.
I load the dumps again but this time I use 1.6 and my repo is now 2G. I do
the packing and my repo shrinks by 100M.
Are these expected results? Why such diff in size loading with 1.6?
btw my repo is mostly source code but I do have some binaries.
cheers
--
View this message in context: http://www.nabble.com/Repo-packing-with-subversion-1.6-results-tp22734691p22734691.html
Sent from the Subversion Users mailing list archive at Nabble.com.
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=1438290
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Repo packing with subversion 1.6 results
Posted by "Hyrum K. Wright" <hy...@mail.utexas.edu>.
On Mar 27, 2009, at 7:40 PM, svnuser wrote:
> Thanks for the info guys.
> Just for completeness: the OS I used for testing was ubuntu 8.10.
>
> Just as a side note: I found 1.6 a pain to build from source due to
> reliance
> on newer versions of some components. I had to hack configure to
> avoid some
> configuration errors as I was not interested in building all the
> access
> modules. Perhaps configure can be improved going forward to make it
> a bit
> more user friendly.
Sorry the build process was a little painful for you. However, it is
fair when manually compiling new versions of Subversion to be prepared
to rebuild dependencies.
You mention you had problems with configure, but you don't give
specific output or system configuration which would have caused these
problems. If you are interested in getting these addressed, more
information would be useful.
Thanks,
-Hyrum
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=1460621
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Repo packing with subversion 1.6 results
Posted by svnuser <jz...@gmail.com>.
Thanks for the info guys.
Just for completeness: the OS I used for testing was ubuntu 8.10.
Just as a side note: I found 1.6 a pain to build from source due to reliance
on newer versions of some components. I had to hack configure to avoid some
configuration errors as I was not interested in building all the access
modules. Perhaps configure can be improved going forward to make it a bit
more user friendly.
Cheers
Jarek
Hyrum K. Wright-3 wrote:
>
> On Mar 27, 2009, at 12:54 AM, B Smith-Mannschott wrote:
>
>> On Fri, Mar 27, 2009 at 02:45, svnuser <jz...@gmail.com> wrote:
>>> I have sharded repo using 1.5.6 with 22K revisions, size 2.9G
>>> I loaded its dumps into a new repo using 1.5.1 (I meant to use
>>> 1.5.6 but
>>> this should do)
>>> My new repo has the same size as my original (expected).
>>> I run upgrade/packing on it with 1.6 and all I save is 100M.
>>>
>>> I load the dumps again but this time I use 1.6 and my repo is now
>>> 2G. I do
>>> the packing and my repo shrinks by 100M.
>>>
>>> Are these expected results?
>>
>> Yes
>
> Depending on the size of the revisions (the amount of "stuff" in them)
> and the underlying OS filesystem, packing will have varying amounts of
> benefit. Generally, the space savings will be proportional to the
> number of versions in the repository.
>
>>> Why such diff in size loading with 1.6?
>>> btw my repo is mostly source code but I do have some binaries.
>>
>> <http://subversion.tigris.org/svn_1.6_releasenotes.html#filesystem-improvements
>> >
>>
>> # Sharing multiple common representations (issue 2286, server)
>> # ------------------------------------------------------------
>> #
>> # When using many branches and merging between them often, it is
>> common
>> # to have files with similar lines of history which contain the exact
>> # same content. In the past, Subversion has stored these files as
>> deltas
>> # against previous versions of the file. Subversion 1.6 will now use
>> # existing representations in the filesystem for duplicate
>> # storage. Depending on the size of the repository, and the degree of
>> # branching and merging, this can cause an up to 20% space reduction
>> for
>> # Berkeley DB repositories and a 15% reduction for FSFS repositories.
>>
>> ------------------------------------------------------
>> http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=1440247
>
> The major decreases you are seeing are from rep-sharing, as noted
> above. For rep-sharing to work, the rep-cache has to be populated,
> which happens when you load the dumpfile on the 1.6 repository. Doing
> an 'svnadmin upgrade' to 1.6-format will enable rep-sharing going
> forward, but it does not populate the cache with existing revisions,
> nor rewrite old history to use rep-sharing.
>
> -Hyrum
>
> ------------------------------------------------------
> http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=1443571
>
> To unsubscribe from this discussion, e-mail:
> [users-unsubscribe@subversion.tigris.org].
>
>
--
View this message in context: http://www.nabble.com/Repo-packing-with-subversion-1.6-results-tp22734691p22752408.html
Sent from the Subversion Users mailing list archive at Nabble.com.
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=1450375
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Repo packing with subversion 1.6 results
Posted by "Hyrum K. Wright" <hy...@mail.utexas.edu>.
On Mar 27, 2009, at 12:54 AM, B Smith-Mannschott wrote:
> On Fri, Mar 27, 2009 at 02:45, svnuser <jz...@gmail.com> wrote:
>> I have sharded repo using 1.5.6 with 22K revisions, size 2.9G
>> I loaded its dumps into a new repo using 1.5.1 (I meant to use
>> 1.5.6 but
>> this should do)
>> My new repo has the same size as my original (expected).
>> I run upgrade/packing on it with 1.6 and all I save is 100M.
>>
>> I load the dumps again but this time I use 1.6 and my repo is now
>> 2G. I do
>> the packing and my repo shrinks by 100M.
>>
>> Are these expected results?
>
> Yes
Depending on the size of the revisions (the amount of "stuff" in them)
and the underlying OS filesystem, packing will have varying amounts of
benefit. Generally, the space savings will be proportional to the
number of versions in the repository.
>> Why such diff in size loading with 1.6?
>> btw my repo is mostly source code but I do have some binaries.
>
> <http://subversion.tigris.org/svn_1.6_releasenotes.html#filesystem-improvements
> >
>
> # Sharing multiple common representations (issue 2286, server)
> # ------------------------------------------------------------
> #
> # When using many branches and merging between them often, it is
> common
> # to have files with similar lines of history which contain the exact
> # same content. In the past, Subversion has stored these files as
> deltas
> # against previous versions of the file. Subversion 1.6 will now use
> # existing representations in the filesystem for duplicate
> # storage. Depending on the size of the repository, and the degree of
> # branching and merging, this can cause an up to 20% space reduction
> for
> # Berkeley DB repositories and a 15% reduction for FSFS repositories.
>
> ------------------------------------------------------
> http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=1440247
The major decreases you are seeing are from rep-sharing, as noted
above. For rep-sharing to work, the rep-cache has to be populated,
which happens when you load the dumpfile on the 1.6 repository. Doing
an 'svnadmin upgrade' to 1.6-format will enable rep-sharing going
forward, but it does not populate the cache with existing revisions,
nor rewrite old history to use rep-sharing.
-Hyrum
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=1443571
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Repo packing with subversion 1.6 results
Posted by B Smith-Mannschott <bs...@gmail.com>.
On Fri, Mar 27, 2009 at 02:45, svnuser <jz...@gmail.com> wrote:
> I have sharded repo using 1.5.6 with 22K revisions, size 2.9G
> I loaded its dumps into a new repo using 1.5.1 (I meant to use 1.5.6 but
> this should do)
> My new repo has the same size as my original (expected).
> I run upgrade/packing on it with 1.6 and all I save is 100M.
>
> I load the dumps again but this time I use 1.6 and my repo is now 2G. I do
> the packing and my repo shrinks by 100M.
>
> Are these expected results?
Yes
> Why such diff in size loading with 1.6?
> btw my repo is mostly source code but I do have some binaries.
<http://subversion.tigris.org/svn_1.6_releasenotes.html#filesystem-improvements>
# Sharing multiple common representations (issue 2286, server)
# ------------------------------------------------------------
#
# When using many branches and merging between them often, it is common
# to have files with similar lines of history which contain the exact
# same content. In the past, Subversion has stored these files as deltas
# against previous versions of the file. Subversion 1.6 will now use
# existing representations in the filesystem for duplicate
# storage. Depending on the size of the repository, and the degree of
# branching and merging, this can cause an up to 20% space reduction for
# Berkeley DB repositories and a 15% reduction for FSFS repositories.
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=1440247
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].