You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Kynn Jones <ky...@gmail.com> on 2008/12/02 19:09:11 UTC
Frequent revisions to large compressed files
Hi. I would like to put some large compressed files under version control
using Subversion. The largest of these files is about 200MB and the
smallest is about 40MB, and they are all compressed using gzip, and I expect
that I will need to commit new revisions of these files at the rate of once
a day.
Does Subversion keep track of changes incrementally? And, if so, would
small incremental changes in the *uncompressed* versions of these files
translate to small incremental changes in the depository?
I suspect that the answer to the last question is "no", because (I guess)
that, for it to be "yes", Subversion would have to be smart enough to
realize that it is dealing with a compressed file and treat the contents
accordingly. This seems like too much to expect...
My concern here is that as new revisions of these files are committed to the
depository, its disk footprint will balloon rapidly, because, even though
the uncompressed revisions are minor, the compressed file will change
drastically from one revision to the next, so that even an incremental
update will require having almost the entire file saved each time.
(I hope the above made sense!)
Any words of wisdom on frequently committing large compressed files to
Subversion would be much appreciated!
TIA!
Kynn
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=804046
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Frequent revisions to large compressed files
Posted by Kynn Jones <ky...@gmail.com>.
Thanks!
Kynn
On Tue, Dec 2, 2008 at 3:21 PM, Ryan Schmidt <
subversion-2008c@ryandesign.com> wrote:
>
> On Dec 2, 2008, at 13:09, Kynn Jones wrote:
>
> Hi. I would like to put some large compressed files under version control
>> using Subversion. The largest of these files is about 200MB and the
>> smallest is about 40MB, and they are all compressed using gzip, and I expect
>> that I will need to commit new revisions of these files at the rate of once
>> a day.
>>
>> Does Subversion keep track of changes incrementally? And, if so, would
>> small incremental changes in the *uncompressed* versions of these files
>> translate to small incremental changes in the depository?
>>
>> I suspect that the answer to the last question is "no", because (I guess)
>> that, for it to be "yes", Subversion would have to be smart enough to
>> realize that it is dealing with a compressed file and treat the contents
>> accordingly. This seems like too much to expect...
>>
>> My concern here is that as new revisions of these files are committed to
>> the depository, its disk footprint will balloon rapidly, because, even
>> though the uncompressed revisions are minor, the compressed file will change
>> drastically from one revision to the next, so that even an incremental
>> update will require having almost the entire file saved each time.
>>
>> (I hope the above made sense!)
>>
>
> You're correct in all you say. It in fact depends on the compression
> algorithm; if it causes the old and new files to look totally different,
> then Subversion has no choice but to store all of that difference.
> Subversion does not examine your binary files or attempt to decompress them.
>
> Any words of wisdom on frequently committing large compressed files to
>> Subversion would be much appreciated!
>>
>
> Either store the uncompressed files in the repository (yes the initial
> revision will be larger but subsequent revisions will be only as big as the
> changes so this should be better over time than regular compressed files),
> or compress the files in a way that causes only the differences in the
> original files to show through in the compressed files. For example, for
> gzip, you can use the --rsyncable flag. gzip requires a patch to implement
> this flag; you can get it here:
>
> http://trac.macports.org/ticket/16446
>
>
>
>
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=961128
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Frequent revisions to large compressed files
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Dec 2, 2008, at 13:09, Kynn Jones wrote:
> Hi. I would like to put some large compressed files under version
> control using Subversion. The largest of these files is about
> 200MB and the smallest is about 40MB, and they are all compressed
> using gzip, and I expect that I will need to commit new revisions
> of these files at the rate of once a day.
>
> Does Subversion keep track of changes incrementally? And, if so,
> would small incremental changes in the *uncompressed* versions of
> these files translate to small incremental changes in the depository?
>
> I suspect that the answer to the last question is "no", because (I
> guess) that, for it to be "yes", Subversion would have to be smart
> enough to realize that it is dealing with a compressed file and
> treat the contents accordingly. This seems like too much to expect...
>
> My concern here is that as new revisions of these files are
> committed to the depository, its disk footprint will balloon
> rapidly, because, even though the uncompressed revisions are minor,
> the compressed file will change drastically from one revision to
> the next, so that even an incremental update will require having
> almost the entire file saved each time.
>
> (I hope the above made sense!)
You're correct in all you say. It in fact depends on the compression
algorithm; if it causes the old and new files to look totally
different, then Subversion has no choice but to store all of that
difference. Subversion does not examine your binary files or attempt
to decompress them.
> Any words of wisdom on frequently committing large compressed files
> to Subversion would be much appreciated!
Either store the uncompressed files in the repository (yes the
initial revision will be larger but subsequent revisions will be only
as big as the changes so this should be better over time than regular
compressed files), or compress the files in a way that causes only
the differences in the original files to show through in the
compressed files. For example, for gzip, you can use the --rsyncable
flag. gzip requires a patch to implement this flag; you can get it here:
http://trac.macports.org/ticket/16446
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=912494
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].