You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by sebb <se...@gmail.com> on 2012/02/26 16:51:44 UTC
Options for moving large files between repos
The ASF is moving to using SVN for uploading releases [1]
The process involves uploading the release artifacts (archives, sigs, hashes) to
https://dist.apache.org/repos/dist/release/<project>
There is a parallel development area at
https://dist.apache.org/repos/dist/dev/<project>
which can be used for release candidates (RC).
The release manager (RM) can create RC files in the dev/ tree, and
publish them by moving them to the release/ tree once the vote
succeeds.
Now release archives can be quite large, so one wants to minimise
network transfers, and not have to checkout large workspaces either.
One way which seems to work OK is to use a separate directory for each
RC; this can then be renamed directly on the SVN server.
Thus the RM only has to import the RC files once into the dev/ tree;
all further copying is done on the server.
However, many projects currently use a different layout for the
release directory.
A common layout is to use
binaries/
source/
These directories may contain multiple concurrent release versions,
and the existing release will anyway need to be retained for a few
days during the release process to allow mirrors time to sync.
In this case, there is no parent directory which can easily be
renamed, so what are the options for minising network usage?
One could rename each file individually on the server, but that would
result in lots of separate commits, and would be tedious and error
prone.
Is there a way to rename multiple files as part of a single commit?
Or some other method?
[1] http://www.apache.org/dev/release.html#upload-ci
Re: Options for moving large files between repos
Posted by sebb <se...@gmail.com>.
On 8 March 2012 13:54, Mark Phippard <ma...@gmail.com> wrote:
> On Mon, Feb 27, 2012 at 10:03 AM, sebb <se...@gmail.com> wrote:
>
>> I've currently got the CollabNet build (1.7.2) for Windows, and it
>> does not include svnmucc as far as I can tell.
>
> As of the 1.7.4 release it will include it.
Thanks!
I've just reverted to the CollabNet build because I found that the
WANdisco builds have a problem with redirected output.
When the output of svn pl -v is redirected, the pathnames are printed
together before the properties, making the output all but useless.
I did not check if this bug affects other commands as well.
> --
> Thanks
>
> Mark Phippard
> http://markphip.blogspot.com/
Re: Options for moving large files between repos
Posted by Mark Phippard <ma...@gmail.com>.
On Mon, Feb 27, 2012 at 10:03 AM, sebb <se...@gmail.com> wrote:
> I've currently got the CollabNet build (1.7.2) for Windows, and it
> does not include svnmucc as far as I can tell.
As of the 1.7.4 release it will include it.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Options for moving large files between repos
Posted by sebb <se...@gmail.com>.
On 26 February 2012 21:52, Stefan Sperling <st...@elego.de> wrote:
> On Sun, Feb 26, 2012 at 07:52:41PM +0000, sebb wrote:
>> On 26 February 2012 16:05, Stefan Sperling <st...@elego.de> wrote:
>> > https://svn.apache.org/repos/asf/subversion/trunk/tools/client-side/svnmucc
>>
>> Is this available for all OSes?
>
> Yes. Things in tools/ are built by default with Subversion.
> Which package they end up in might depend on your OS, though.
I've currently got the CollabNet build (1.7.2) for Windows, and it
does not include svnmucc as far as I can tell.
(Later) just downloaded 1.7.3 from CollabNet and Wandisco (for comparison)
The CollabNet download is about twice the size, yet does not contain svnmucc.
So I've abandoned the CollabNet build.
Re: Options for moving large files between repos
Posted by Stefan Sperling <st...@elego.de>.
On Sun, Feb 26, 2012 at 07:52:41PM +0000, sebb wrote:
> On 26 February 2012 16:05, Stefan Sperling <st...@elego.de> wrote:
> > https://svn.apache.org/repos/asf/subversion/trunk/tools/client-side/svnmucc
>
> Is this available for all OSes?
Yes. Things in tools/ are built by default with Subversion.
Which package they end up in might depend on your OS, though.
> Is it documented anywhere?
$ svnmucc help
Multiple URL Command Client (for Subversion)
Usage: svnmucc [OPTION]... [ACTION]...
Actions:
cp REV URL1 URL2 copy URL1@REV to URL2
mkdir URL create new directory URL
mv URL1 URL2 move URL1 to URL2
rm URL delete URL
put SRC-FILE URL add or modify file URL with contents copied from
SRC-FILE (use "-" to read from standard input)
propset NAME VAL URL set property NAME on URL to value VAL
propsetf NAME VAL URL set property NAME on URL to value from file VAL
propdel NAME URL delete property NAME from URL
Options:
-h, --help display this text
-m, --message ARG use ARG as a log message
-F, --file ARG read log message from file ARG
-u, --username ARG commit the changes as username ARG
-p, --password ARG use ARG as the password
-U, --root-url ARG interpret all action URLs are relative to ARG
-r, --revision ARG use revision ARG as baseline for changes
--with-revprop A[=B] set revision property A in new revision to B
if specified, else to the empty string
-n, --non-interactive don't prompt the user about anything
-X, --extra-args ARG append arguments from file ARG (one per line;
use "-" to read from standard input)
--config-dir ARG use ARG to override the config directory
--config-option ARG use ARG so override a configuration option
--no-auth-cache do not cache authentication tokens
--version print version information
Re: Options for moving large files between repos
Posted by sebb <se...@gmail.com>.
On 26 February 2012 16:05, Stefan Sperling <st...@elego.de> wrote:
> On Sun, Feb 26, 2012 at 03:51:44PM +0000, sebb wrote:
>> Now release archives can be quite large, so one wants to minimise
>> network transfers, and not have to checkout large workspaces either.
>
> Why would anyone need to get a working copy of anything?
> All moves/copies can be performed server-side.
> The RM can import the file into a directory, and others can download it.
>
> Because separate revisions (RCs) of the same release usually use unique
> filenames, there is currently no benefit in updating an existing working
> copy instead of downloading the release artefact.
>
>> In this case, there is no parent directory which can easily be
>> renamed, so what are the options for minising network usage?
>
> What network usage are you trying to minimise?
> RM to SVN server?
Yes.
> Other developers to/from SVN server?
No, they would have to download just once from the dev area.
> SVN server to release mirror servers?
No.
>> One could rename each file individually on the server, but that would
>> result in lots of separate commits, and would be tedious and error
>> prone.
>
> In this scenario, I have no better idea than renaming files individually.
>
>> Is there a way to rename multiple files as part of a single commit?
>
> Yes, there is. See svnmucc:
> https://svn.apache.org/repos/asf/subversion/trunk/tools/client-side/svnmucc
Is this available for all OSes?
Is it documented anywhere?
Whatever process is chosen needs to be easy to use for multiple RMs.
Re: Options for moving large files between repos
Posted by Stefan Sperling <st...@elego.de>.
On Sun, Feb 26, 2012 at 03:51:44PM +0000, sebb wrote:
> Now release archives can be quite large, so one wants to minimise
> network transfers, and not have to checkout large workspaces either.
Why would anyone need to get a working copy of anything?
All moves/copies can be performed server-side.
The RM can import the file into a directory, and others can download it.
Because separate revisions (RCs) of the same release usually use unique
filenames, there is currently no benefit in updating an existing working
copy instead of downloading the release artefact.
> In this case, there is no parent directory which can easily be
> renamed, so what are the options for minising network usage?
What network usage are you trying to minimise?
RM to SVN server? Other developers to/from SVN server?
SVN server to release mirror servers?
> One could rename each file individually on the server, but that would
> result in lots of separate commits, and would be tedious and error
> prone.
In this scenario, I have no better idea than renaming files individually.
> Is there a way to rename multiple files as part of a single commit?
Yes, there is. See svnmucc:
https://svn.apache.org/repos/asf/subversion/trunk/tools/client-side/svnmucc