You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Andreas Hasenack <an...@mandriva.com.br> on 2007/07/02 16:49:02 UTC
best way to update large blobs in svn
I have a situation where I will probably have to store in svn a large
blob, like 230Mb. It's a tarball that only contains source code, but I
need it stored as the tarball.
I'm wondering if there is any best procedure about updating this tarball
considering the amount of space that will be used.
For example, say tarball-1.0.tar.bz2 is 230Mb and tarball-1.1.tar.bz2 is
about the same size. It has a few differences, though (it's a new
version). Usually, one would:
- svn add tarball-1.1.tar.bz2
- svn del tarball-1.0.tar.bz2
- svn commit
If I did a rename before to 1.1 and then updated the tarball, would it
be better? Like:
- svn rename tarball-1.0.tar.bz2 tarball-1.1.tar.bz2
- cp ~/tarball-1.1.tar.bz2 .
- svn commit
Or even:
- svn rename tarball-1.0.tar.bz2 tarball-1.1.tar.bz2
- svn commit
- cp ~/tarball-1.1.tar.bz2 .
- svn commit
I tried it here, but saw no noticeable difference in the repository
size. The end result was sizeof(1.0)+sizeof(1.1) in all cases. Is there
some other way or is that it?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: best way to update large blobs in svn
Posted by Greg Thomas <th...@omc.bt.co.uk>.
On Wed, 4 Jul 2007 09:19:29 -0300, Andreas Hasenack
<an...@mandriva.com.br> wrote:
>In all the cases I showed? "svn del old + svn add new" and
>"svn rename old2new + overwrite with new"? That's my main concern,
>because currently I'm doing the former (del old + add new).
I'd missed that. TBH, if you're doing a del + add, I don't see
anything that will help.
Greg
--
This post represents the views of the author and does
not necessarily accurately represent the views of BT.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: best way to update large blobs in svn
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Wed, Jul 04, 2007 at 09:49:01AM +0100, Greg Thomas wrote:
> On Tue, 3 Jul 2007 09:53:15 -0300, Andreas Hasenack
> <an...@mandriva.com.br> wrote:
>
> >On Tue, Jul 03, 2007 at 08:31:19AM +0100, Greg Thomas wrote:
> >> On Mon, 2 Jul 2007 22:31:38 -0500, "Matt Sickler"
> >> <cr...@gmail.com> wrote:
> >>
> >> >Maybe its best to rethink that "must be stored as a tar/bz2" requirement.
> >>
> >> It might be better to store it as it's component files and a script to
> >> generate the tar file.
> >
> >But my original question remains: is there a, let's say, "less worse"
> >way to update binary blobs?
>
> svn will only transfer differences. If your binary blob is similar to
> the previous version (e.g. a .gz instead of a .bz2) you'll get a
> better effect (less bandwidth & diskspace used).
In all the cases I showed? "svn del old + svn add new" and
"svn rename old2new + overwrite with new"? That's my main concern,
because currently I'm doing the former (del old + add new).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: best way to update large blobs in svn
Posted by Greg Thomas <th...@omc.bt.co.uk>.
On Tue, 3 Jul 2007 09:53:15 -0300, Andreas Hasenack
<an...@mandriva.com.br> wrote:
>On Tue, Jul 03, 2007 at 08:31:19AM +0100, Greg Thomas wrote:
>> On Mon, 2 Jul 2007 22:31:38 -0500, "Matt Sickler"
>> <cr...@gmail.com> wrote:
>>
>> >Maybe its best to rethink that "must be stored as a tar/bz2" requirement.
>>
>> It might be better to store it as it's component files and a script to
>> generate the tar file.
>
>But my original question remains: is there a, let's say, "less worse"
>way to update binary blobs?
svn will only transfer differences. If your binary blob is similar to
the previous version (e.g. a .gz instead of a .bz2) you'll get a
better effect (less bandwidth & diskspace used).
Greg
--
This post represents the views of the author and does
not necessarily accurately represent the views of BT.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: best way to update large blobs in svn
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Tue, Jul 03, 2007 at 08:31:19AM +0100, Greg Thomas wrote:
> On Mon, 2 Jul 2007 22:31:38 -0500, "Matt Sickler"
> <cr...@gmail.com> wrote:
>
> >Maybe its best to rethink that "must be stored as a tar/bz2" requirement.
>
> It might be better to store it as it's component files and a script to
> generate the tar file.
But my original question remains: is there a, let's say, "less worse"
way to update binary blobs?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: best way to update large blobs in svn
Posted by Greg Thomas <th...@omc.bt.co.uk>.
On Mon, 2 Jul 2007 22:31:38 -0500, "Matt Sickler"
<cr...@gmail.com> wrote:
>Maybe its best to rethink that "must be stored as a tar/bz2" requirement.
It might be better to store it as it's component files and a script to
generate the tar file.
Greg
--
This post represents the views of the author and does
not necessarily accurately represent the views of BT.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: best way to update large blobs in svn
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Mon, Jul 02, 2007 at 10:31:38PM -0500, Matt Sickler wrote:
> Maybe its best to rethink that "must be stored as a tar/bz2" requirement.
> What is that requirement anyway?
We store full .src.rpm packages in svn. Open, i.e., the SOURCES and
SPECS structure. This has enormous benefits as we can recreate the
.src.rpm of any version we ever released.
This package in particular is openoffice, which is not your regular
package because of its size :)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: best way to update large blobs in svn
Posted by Matt Sickler <cr...@gmail.com>.
Maybe its best to rethink that "must be stored as a tar/bz2" requirement.
What is that requirement anyway?
On 7/2/07, Talden <ta...@gmail.com> wrote:
> Since the tarball is compressed (bz2) expecting Subversions diffing to save
> you any space at all is probably a lost cause.
>
> If you're going to expect many revisions of that tarball you're going to
> need to consider at the least storing it without the compression - though
> that will create it's own headaches in terms of working copy storage and
> transmission (at least once) I expect.
>
> --
> Talden
>
>
> On 7/3/07, Andreas Hasenack <an...@mandriva.com.br> wrote:
> > I have a situation where I will probably have to store in svn a large
> > blob, like 230Mb. It's a tarball that only contains source code, but I
> > need it stored as the tarball.
> >
> > I'm wondering if there is any best procedure about updating this tarball
> > considering the amount of space that will be used.
> >
> > For example, say tarball-1.0.tar.bz2 is 230Mb and tarball-1.1.tar.bz2 is
> > about the same size. It has a few differences, though (it's a new
> > version). Usually, one would:
> > - svn add tarball-1.1.tar.bz2
> > - svn del tarball-1.0.tar.bz2
> > - svn commit
> >
> > If I did a rename before to 1.1 and then updated the tarball, would it
> > be better? Like:
> > - svn rename tarball-1.0.tar.bz2 tarball-1.1.tar.bz2
> > - cp ~/tarball-1.1.tar.bz2 .
> > - svn commit
> >
> > Or even:
> > - svn rename tarball-1.0.tar.bz2 tarball-1.1.tar.bz2
> > - svn commit
> > - cp ~/tarball-1.1.tar.bz2 .
> > - svn commit
> >
> > I tried it here, but saw no noticeable difference in the repository
> > size. The end result was sizeof(1.0)+sizeof(1.1) in all cases. Is there
> > some other way or is that it?
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> users-unsubscribe@subversion.tigris.org
> > For additional commands, e-mail:
> users-help@subversion.tigris.org
> >
> >
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: best way to update large blobs in svn
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Tue, Jul 03, 2007 at 02:55:02PM +0200, Martin Schröder wrote:
> 2007/7/3, Andreas Hasenack <an...@mandriva.com.br>:
>> On Tue, Jul 03, 2007 at 10:49:53AM +1200, Talden wrote:
>> > Since the tarball is compressed (bz2) expecting Subversions diffing to
>> save
>> > you any space at all is probably a lost cause.
>>
>> Note that rsync can deal with this just fine, for example. If
>
> Bogus. rsync may be able to handle gz, but it doesn't handle bz2 well.
Sometimes it's a .tar.gz.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: best way to update large blobs in svn
Posted by Martin Schröder <ma...@oneiros.de>.
2007/7/3, Andreas Hasenack <an...@mandriva.com.br>:
> On Tue, Jul 03, 2007 at 10:49:53AM +1200, Talden wrote:
> > Since the tarball is compressed (bz2) expecting Subversions diffing to save
> > you any space at all is probably a lost cause.
>
> Note that rsync can deal with this just fine, for example. If
Bogus. rsync may be able to handle gz, but it doesn't handle bz2 well.
Best
Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: best way to update large blobs in svn
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Tue, Jul 03, 2007 at 10:49:53AM +1200, Talden wrote:
> Since the tarball is compressed (bz2) expecting Subversions diffing to save
> you any space at all is probably a lost cause.
Note that rsync can deal with this just fine, for example. If I change a
few bytes, it only transfers the differences. I guess the subversion
algorithm is different.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: best way to update large blobs in svn
Posted by Talden <ta...@gmail.com>.
Since the tarball is compressed (bz2) expecting Subversions diffing to save
you any space at all is probably a lost cause.
If you're going to expect many revisions of that tarball you're going to
need to consider at the least storing it without the compression - though
that will create it's own headaches in terms of working copy storage and
transmission (at least once) I expect.
--
Talden
On 7/3/07, Andreas Hasenack <an...@mandriva.com.br> wrote:
>
> I have a situation where I will probably have to store in svn a large
> blob, like 230Mb. It's a tarball that only contains source code, but I
> need it stored as the tarball.
>
> I'm wondering if there is any best procedure about updating this tarball
> considering the amount of space that will be used.
>
> For example, say tarball-1.0.tar.bz2 is 230Mb and tarball-1.1.tar.bz2 is
> about the same size. It has a few differences, though (it's a new
> version). Usually, one would:
> - svn add tarball-1.1.tar.bz2
> - svn del tarball-1.0.tar.bz2
> - svn commit
>
> If I did a rename before to 1.1 and then updated the tarball, would it
> be better? Like:
> - svn rename tarball-1.0.tar.bz2 tarball-1.1.tar.bz2
> - cp ~/tarball-1.1.tar.bz2 .
> - svn commit
>
> Or even:
> - svn rename tarball-1.0.tar.bz2 tarball-1.1.tar.bz2
> - svn commit
> - cp ~/tarball-1.1.tar.bz2 .
> - svn commit
>
> I tried it here, but saw no noticeable difference in the repository
> size. The end result was sizeof(1.0)+sizeof(1.1) in all cases. Is there
> some other way or is that it?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
>