You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Oscar Lee <mi...@gmail.com> on 2019/01/09 18:10:39 UTC
Problems with using a symbolic link for .svn folder on TSVN
Hi,
I was told to post my issue here from a TSVN dev.
My company uses TortoiseSVN internally to keep our files updated. The .svn
folder for the project I have is massive (250GB) and as such I had to move
it off to an external HDD. I created a symbolic link to the new location so
that TortoiseSVN 'should' still continue to work.
I managed to run a clean-up, but when I tried to revert a file, it gave me
an error 'Failed to run the WC DB work queue associated with (file)" and
"Can't move (tmp file) to ... (original file): The system cannot move the
file to a different disk drive".
I found that this error is caused by Windows not letting a file be renamed
while it is being moved (
https://docs.microsoft.com/en-us/previous-versions/ms837428(v=msdn.10)).
Does anyone know a solution to this? Why is this an issue that only occurs
with a symbolic link setup?
Regards,
Oscar
Re: Problems with using a symbolic link for .svn folder on TSVN
Posted by Branko Čibej <br...@apache.org>.
On 09.01.2019 19:10, Oscar Lee wrote:
> Hi,
>
> I was told to post my issue here from a TSVN dev.
>
> My company uses TortoiseSVN internally to keep our files updated. The .svn
> folder for the project I have is massive (250GB) and as such I had to move
> it off to an external HDD. I created a symbolic link to the new location so
> that TortoiseSVN 'should' still continue to work.
Well, it "should" not, see below.
But if your .svn/ directory is much bigger than the rest of your working
copy, then 'svn cleanup --vacuum-pristines' will probably reduce its
size. In TSVN you'll have to select the checkbox "Vacuum pristine copies".
> I managed to run a clean-up, but when I tried to revert a file, it gave me
> an error 'Failed to run the WC DB work queue associated with (file)" and
> "Can't move (tmp file) to ... (original file): The system cannot move the
> file to a different disk drive".
>
> I found that this error is caused by Windows not letting a file be renamed
> while it is being moved (
> https://docs.microsoft.com/en-us/previous-versions/ms837428(v=msdn.10)).
> Does anyone know a solution to this? Why is this an issue that only occurs
> with a symbolic link setup?
This has nothing to do with symbolic links but with the fact that
Subversion, during normal operations, has to atomically rename (and
move) a file from somewhere in the .svn/ directory to its expected
location in the working copy. Windows can't do that if the source and
target of the rename are on different volumes, that's what the error
message is telling you. The explanation your link points to is
misleading, to put it mildly ... it's the move to a different volume
that fails, not the renaming of the file.
-- Brane
Re: Problems with using a symbolic link for .svn folder on TSVN
Posted by Branko Čibej <br...@apache.org>.
On 10.01.2019 13:40, minxinglee@gmail.com wrote:
> I see. Is it possible to change the implementation slightly as outlined by Boost SVN? (https://svn.boost.org/trac10/ticket/6809). I believe this is the same issue.
It is the same issue and no it is not possible to change the
implementation "slightly" because the integrity of the working copy
relies on atomic renames. That is why Subversion doesn't use the system-
(or user-) specific temporary directory (defined by %TMP% and %TEMP% on
Windows) for such files but creates them in .svn/, to make it as likely
as possible that they'll be on the same volume as the rest of the
working copy.
The only thing we could do would be to not create temporary files in
.svn/ but in the same directory as the target file, but then we'd have
problems with potential name collisions and with such temporary files
cluttering the working copy tree after aborted operations.
It's possible, but it's not a "slight" change like adding another flag
to MoveFileEx.
-- Brane
> -----Original Message-----
> From: Daniel Shahaf <d....@daniel.shahaf.name>
> Sent: Thursday, 10 January 2019 1:36 PM
> To: Oscar Lee <mi...@gmail.com>; users@subversion.apache.org
> Subject: Re: Problems with using a symbolic link for .svn folder on TSVN
>
> Oscar Lee wrote on Wed, 09 Jan 2019 19:10 +0100:
>> My company uses TortoiseSVN internally to keep our files updated. The
>> .svn folder for the project I have is massive (250GB) and as such I
>> had to move it off to an external HDD. I created a symbolic link to
>> the new location so that TortoiseSVN 'should' still continue to work.
> How large are the working copy files not under the .svn/ directory?
>
> If they're substantially smaller than 250GB, you might be running into this:
> https://subversion.apache.org/docs/release-notes/1.7#wc-pristines
>
>> I managed to run a clean-up, but when I tried to revert a file, it
>> gave me an error 'Failed to run the WC DB work queue associated with
>> (file)" and "Can't move (tmp file) to ... (original file): The system
>> cannot move the file to a different disk drive".
>>
>> I found that this error is caused by Windows not letting a file be
>> renamed while it is being moved (
>> https://docs.microsoft.com/en-us/previous-versions/ms837428(v=msdn.10)).
>> Does anyone know a solution to this?
>> Why is this an issue that only occurs with a symbolic link setup?
> Subversion assumes that it is possible to atomically move a file from the .svn directory to the working copy's checked out files. That would not possible when the .svn directory is on a different drive / filesystem.
>
RE: Problems with using a symbolic link for .svn folder on TSVN
Posted by mi...@gmail.com.
I see. Is it possible to change the implementation slightly as outlined by Boost SVN? (https://svn.boost.org/trac10/ticket/6809). I believe this is the same issue.
-----Original Message-----
From: Daniel Shahaf <d....@daniel.shahaf.name>
Sent: Thursday, 10 January 2019 1:36 PM
To: Oscar Lee <mi...@gmail.com>; users@subversion.apache.org
Subject: Re: Problems with using a symbolic link for .svn folder on TSVN
Oscar Lee wrote on Wed, 09 Jan 2019 19:10 +0100:
> My company uses TortoiseSVN internally to keep our files updated. The
> .svn folder for the project I have is massive (250GB) and as such I
> had to move it off to an external HDD. I created a symbolic link to
> the new location so that TortoiseSVN 'should' still continue to work.
How large are the working copy files not under the .svn/ directory?
If they're substantially smaller than 250GB, you might be running into this:
https://subversion.apache.org/docs/release-notes/1.7#wc-pristines
> I managed to run a clean-up, but when I tried to revert a file, it
> gave me an error 'Failed to run the WC DB work queue associated with
> (file)" and "Can't move (tmp file) to ... (original file): The system
> cannot move the file to a different disk drive".
>
> I found that this error is caused by Windows not letting a file be
> renamed while it is being moved (
> https://docs.microsoft.com/en-us/previous-versions/ms837428(v=msdn.10)).
> Does anyone know a solution to this?
> Why is this an issue that only occurs with a symbolic link setup?
Subversion assumes that it is possible to atomically move a file from the .svn directory to the working copy's checked out files. That would not possible when the .svn directory is on a different drive / filesystem.
Re: Problems with using a symbolic link for .svn folder on TSVN
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Oscar Lee wrote on Wed, 09 Jan 2019 19:10 +0100:
> My company uses TortoiseSVN internally to keep our files updated. The .svn
> folder for the project I have is massive (250GB) and as such I had to move
> it off to an external HDD. I created a symbolic link to the new location so
> that TortoiseSVN 'should' still continue to work.
How large are the working copy files not under the .svn/ directory?
If they're substantially smaller than 250GB, you might be running into this:
https://subversion.apache.org/docs/release-notes/1.7#wc-pristines
> I managed to run a clean-up, but when I tried to revert a file, it gave me
> an error 'Failed to run the WC DB work queue associated with (file)" and
> "Can't move (tmp file) to ... (original file): The system cannot move the
> file to a different disk drive".
>
> I found that this error is caused by Windows not letting a file be renamed
> while it is being moved (
> https://docs.microsoft.com/en-us/previous-versions/ms837428(v=msdn.10)).
> Does anyone know a solution to this?
> Why is this an issue that only occurs with a symbolic link setup?
Subversion assumes that it is possible to atomically move a file from
the .svn directory to the working copy's checked out files. That would
not possible when the .svn directory is on a different drive / filesystem.
Re: Problems with using a symbolic link for .svn folder on TSVN
Posted by Nathan Hartman <ha...@gmail.com>.
On Thu, Jan 10, 2019 at 10:48 AM Oscar Lee <mi...@gmail.com> wrote:
> The external drive is actually a company samba server that I connect over
> VPN. As such, it's super slow to read and even worse to write (maxes at
> 3MB/s). I don't care if the cleanup process or updating of the pristine
> files is slow, but I can't run the game off such a slow hard drive.
>
In that case, my suggestion is to invest <$200 in a USB3 5TB external hard
drive and work off of that.
> *The answer to Life, The Universe and Everything is 42*
>
How many roads must one take? :-p
Re: Problems with using a symbolic link for .svn folder on TSVN
Posted by Nathan Hartman <ha...@gmail.com>.
On Thu, Jan 10, 2019 at 7:39 AM <mi...@gmail.com> wrote:
> I cannot do a sparse checkout as I require every part of the information
> to get the software running (a game)
>
At risk of saying the obvious, may I make a suggestion: Work on the
(larger) external hard drive. In other words, have your entire working
copy, .svn directory and all, on the external drive.
RE: Problems with using a symbolic link for .svn folder on TSVN
Posted by mi...@gmail.com.
I cannot do a sparse checkout as I require every part of the information to get the software running (a game).
From: Paul Hammant <pa...@hammant.org>
Sent: Thursday, 10 January 2019 1:36 PM
To: Oscar Lee <mi...@gmail.com>
Cc: Subversion <us...@subversion.apache.org>
Subject: Re: Problems with using a symbolic link for .svn folder on TSVN
Alternative solution: do a Sparse Checkout - excising things in that tree that you don't really need. Google's monorepo is north of 100TB of history with over nine million source files at HEAD revision. Individual Googler's day to day checkout tens of megabytes only: https://trunkbaseddevelopment.com/expanding-contracting-monorepos/
Though they made tooling for that expanding/contracting stuff, most normal companies could do the same to share within the team with a single bat/sh script for checkout. After that, 'svn up', 'svn commit' work as they always have. Indeed, if that bat file using the command line svn exe did it stuff, TortoiseSvn afte that will work as you expect for update/commit.
The sym-link for something that Subversion needs to refer to a lot is a bad idea.
- Paul
Re: Problems with using a symbolic link for .svn folder on TSVN
Posted by Paul Hammant <pa...@hammant.org>.
Alternative solution: do a Sparse Checkout - excising things in that tree
that you don't really need. Google's monorepo is north of 100TB of history
with over nine million source files at HEAD revision. Individual Googler's
day to day checkout tens of megabytes only:
https://trunkbaseddevelopment.com/expanding-contracting-monorepos/
Though they made tooling for that expanding/contracting stuff, most normal
companies could do the same to share within the team with a single bat/sh
script for checkout. After that, 'svn up', 'svn commit' work as they always
have. Indeed, if that bat file using the command line svn exe did it stuff,
TortoiseSvn afte that will work as you expect for update/commit.
The sym-link for something that Subversion needs to refer to a lot is a bad
idea.
- Paul
RE: Problems with using a symbolic link for .svn folder on TSVN
Posted by mi...@gmail.com.
(Sorry forgot to reply all)
Yes, it’s on my computer when I do a checkout if that’s what you mean. If I do a fresh checkout, the .svn is the same size (over 250GB).
From: Paul Hammant <pa...@hammant.org>
Sent: Thursday, 10 January 2019 1:27 PM
To: Oscar Lee <mi...@gmail.com>
Cc: Subversion <us...@subversion.apache.org>
Subject: Re: Problems with using a symbolic link for .svn folder on TSVN
The .svn folder on the client side?
If you do a fresh checkout, how big is it then?
Re: Problems with using a symbolic link for .svn folder on TSVN
Posted by Paul Hammant <pa...@hammant.org>.
The .svn folder on the client side?
If you do a fresh checkout, how big is it then?