You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@subversion.apache.org by Stefan <lu...@gmx.de> on 2014/03/02 11:01:34 UTC

Re: Can't move temporary on update.

Hi,

sry for the belated answer. It took me a while before I had some time to 
investigate that issue deeper.

Following investigations were done with TSVN 1.8.5 (Subversion 1.8.8) 
and Server is now running Visual SVN 2.7.3 (based on SVN 1.8.5).

>> I traced the issue down to obvious problems with a certain directory in the
>> repository. Ignoring/Skipping that directory and the check-out as well as the
>> update operation work fine.
>>
>> Looking into the .svn/tmp-directory I do see a single existing file:
>> trz2A87.tmp but obviously there's no trace of the mentioned svn-E78ED003 file.
> Can you possibly come up with a reproduction recipe that doesn't require access
> to your repo (unless of course your repo is publicly accessible)?
I hope to be able to try that out later today. But no promises.

> The errors you're getting are coming from the run_file_install() step of the
> workqueue processor.  It's rather hard to understand what's going just from the
> error messages.  Based on what you're saying it seems like the temp file we're
> trying to move into place doesn't exist.  Since looking at the code we try to
> install the file, and then trap a ENOENT error from the rename call.  When the
> rename call fails with ENOENT we try to create the destination directory.
> Which fails and thus you see the errors.
>
> Right above the code generating the error it creates and writes the temp file.
>   So I would expect to be seeing an error message from that if the file isn't
> being created.
I debugged the code directly and didn't run into the breakpoint I set in 
run_file_install().
The code-path which returns the error seems to be somewhere else in my 
case (or did I get u wrong here).
 From what I can tell, it goes like this:
[...]
- svn_wc__db_pristine_install()
     - calls: svn_io_file_rename()
         - calls: svn_io_file_rename()
             - calls: apr_file_rename(from_path_apr, to_path_apr, pool);
                 - calls: if (MoveFileExW(wfrompath, wtopath, 
MOVEFILE_REPLACE_EXISTING || MOVEFILE_COPY_ALLOWED))
  That one returns a status code of 720,002. If I'm reading the 
APR-macros correctly, that corresponds to a window system error of 2 - 
which according to MSDN means: ERROR_FILE_NOT_FOUND.

Setting a breakpoint right before that Move-call in apr_file_rename 
suggests it's trying to move the following file:
G:\Egosoft\X4\Source\.svn\tmp\svn-C5D6631B
to
G:\Egosoft\X4\Source\.svn\pristine\00\0077903324a83da0419971daeb632ff51529d320.svn-base

Looking at the .svn/tmp-directory I do see that a file with the expected 
file contents was created, but is named differently:
trz5B5.tmp

 From that point, it's obvious that the move-call would fail, since the 
given filename seems to differ from that on the disk.

Taking ur statement into account I assume that SVN tries to actually put 
a differently named file into that folder and somehow that one seems to 
get mangled on my system to a different filename.
If so, I could try to debug the code a bit further. Would u have the 
function for me I'd investigate to trace down where the temp-file would 
be created?

> One question I do have is have you done anything unusual with how your file
> system is setup?  The .svn/tmp directory needs to be on the same file system as
> the rest of the working copy since we're doing atomic renames to put files into
> place.
There's nothing special from the file system set-up point of view. The 
drive is a simple local partition on an IDE drive. There are no symlinks 
or somesuch in place. File system is NTFS. The issue is reproducible on 
two different machines, both running Avast Antivirus. I don't know much 
about the set-up on the other machine, but I would expect it's somehow 
the same.

>>> we recently started getting this error when trying to update or check-out a
>>> repository.
>>> Weird thing is that the issue only occurs when we try to check-out/update the
>>> thing externally (meaning via VPN).
> So same machine, same working copy doesn't work over the VPN but does without a
> VPN?  Is the working copy perhaps on a network filesystem?
No, sry, wasn't clear enough. We didn't test whether it's at all related 
to a VPN-connection. We can't atm test it without a VPN connection to 
test whether it's at all related to that. I'll try to do a local set-up 
of a working copy to see whether it's reproducible in that environment 
too. Can't tell atm when I got the time to get it done, but might have a 
few minutes left today. Let's see.

Regards,
Stefan

Re: Can't move temporary on update.

Posted by olli hauer <oh...@gmx.de>.

On 2014-03-02 11:40, Stefan wrote:
> Hi,
> 
> I came up with an even easier repro case. It seems to suffice to trigger the problem by simply committing the problematic file to an empty local repository and having avast installed.
> To whom should I send the repro-case (it requires the zipped-exe-file).
> 


Hi Stefan,

if it works without installed avast or with a different virus scanner installed, then I suggest to open a ticket by avast.




>>
>> sry for the belated answer. It took me a while before I had some time to investigate that issue deeper.
>>
>> Following investigations were done with TSVN 1.8.5 (Subversion 1.8.8) and Server is now running Visual SVN 2.7.3 (based on SVN 1.8.5).
>>
>>>> I traced the issue down to obvious problems with a certain directory in the
>>>> repository. Ignoring/Skipping that directory and the check-out as well as the
>>>> update operation work fine.
>>>>
>>>> Looking into the .svn/tmp-directory I do see a single existing file:
>>>> trz2A87.tmp but obviously there's no trace of the mentioned svn-E78ED003 file.
>>> Can you possibly come up with a reproduction recipe that doesn't require access
>>> to your repo (unless of course your repo is publicly accessible)?
>> I hope to be able to try that out later today. But no promises.
>>
>>> The errors you're getting are coming from the run_file_install() step of the
>>> workqueue processor.  It's rather hard to understand what's going just from the
>>> error messages.  Based on what you're saying it seems like the temp file we're
>>> trying to move into place doesn't exist.  Since looking at the code we try to
>>> install the file, and then trap a ENOENT error from the rename call.  When the
>>> rename call fails with ENOENT we try to create the destination directory.
>>> Which fails and thus you see the errors.
>>>
>>> Right above the code generating the error it creates and writes the temp file.
>>>   So I would expect to be seeing an error message from that if the file isn't
>>> being created.
>> I debugged the code directly and didn't run into the breakpoint I set in run_file_install().
>> The code-path which returns the error seems to be somewhere else in my case (or did I get u wrong here).
>> From what I can tell, it goes like this:
>> [...]
>> - svn_wc__db_pristine_install()
>>     - calls: svn_io_file_rename()
>>         - calls: svn_io_file_rename()
>>             - calls: apr_file_rename(from_path_apr, to_path_apr, pool);
>>                 - calls: if (MoveFileExW(wfrompath, wtopath, MOVEFILE_REPLACE_EXISTING || MOVEFILE_COPY_ALLOWED))
>>  That one returns a status code of 720,002. If I'm reading the APR-macros correctly, that corresponds to a window system error of 2 - which according to MSDN means: ERROR_FILE_NOT_FOUND.
>>
>> Setting a breakpoint right before that Move-call in apr_file_rename suggests it's trying to move the following file:
>> G:\Egosoft\X4\Source\.svn\tmp\svn-C5D6631B
>> to
>> G:\Egosoft\X4\Source\.svn\pristine\00\0077903324a83da0419971daeb632ff51529d320.svn-base
>>
>> Looking at the .svn/tmp-directory I do see that a file with the expected file contents was created, but is named differently:
>> trz5B5.tmp
>>
>> From that point, it's obvious that the move-call would fail, since the given filename seems to differ from that on the disk.
>>
>> Taking ur statement into account I assume that SVN tries to actually put a differently named file into that folder and somehow that one seems to get mangled on my system to a different filename.
>> If so, I could try to debug the code a bit further. Would u have the function for me I'd investigate to trace down where the temp-file would be created?
>>
>>> One question I do have is have you done anything unusual with how your file
>>> system is setup?  The .svn/tmp directory needs to be on the same file system as
>>> the rest of the working copy since we're doing atomic renames to put files into
>>> place.
>> There's nothing special from the file system set-up point of view. The drive is a simple local partition on an IDE drive. There are no symlinks or somesuch in place. File system is NTFS. The issue is reproducible on two different machines, both running Avast Antivirus. I don't know much about the set-up on the other machine, but I would expect it's somehow the same.
>>
>>>>> we recently started getting this error when trying to update or check-out a
>>>>> repository.
>>>>> Weird thing is that the issue only occurs when we try to check-out/update the
>>>>> thing externally (meaning via VPN).
>>> So same machine, same working copy doesn't work over the VPN but does without a
>>> VPN?  Is the working copy perhaps on a network filesystem?
>> No, sry, wasn't clear enough. We didn't test whether it's at all related to a VPN-connection. We can't atm test it without a VPN connection to test whether it's at all related to that. I'll try to do a local set-up of a working copy to see whether it's reproducible in that environment too. Can't tell atm when I got the time to get it done, but might have a few minutes left today. Let's see.
>>
>> Regards,
>> Stefan
>>
> 
>

Re: Can't move temporary on update.

Posted by Branko Čibej <br...@wandisco.com>.

The share mode is the default from APR; presumably to make the default
behaviour similar to POSIX. That doesn't really matter: no other process
should be deleting Subversion's temp files. Unless it's malware ... or it's
quarantining an infected executable.

-- Brane
On 3 Mar 2014 07:43, "Stefan" <lu...@gmx.de> wrote:

> Hi Brane,
>
>>
>>>>  Is there some code path I'd trace down to confirm it's actually the
>>>>> virusscanner causing the rename? Where in the code path would the tmp-file
>>>>> be generated?
>>>>>
>>>> The call stack is probably:
>>>>     svn_io_open_unique_file3
>>>>     svn_stream_open_unique
>>>>     ....
>>>>     svn_wc__open_writable_base
>>>>     ...
>>>>     apply_textdelta
>>>>
>>>> The last function is private to subversion/libsvn_wc/update_editor.c.
>>>>
>>> Thanks that helped. I traced it down and it looks like when SVN is
>>> closing the stream to write the temp file, it gets removed from disk. I
>>> tried to trace the issue down a bit further using Process Monitor as
>>> suggested by Thorsten but am a bit buffled by the log. It seems to have no
>>> record of either a rename-operation or a delete operation on svn-BDF57639.
>>> A query on the file from the Avast succeeds while an almost directly
>>> following query on the same file from TSVN fails.
>>>
>>
>> Heh. I wonder if Avast is setting the delete-on-close flag on
>> Subversion's temp file. That would explain why it suddenly disappears.
>> Subversion definitely isn't doing that in this particular case; note the
>> svn_io_file_del_none parameter used in svn_wc__open_writable_base.
>>
> I can't see in the Process Monitor that Avast is actually setting that
> flag. Actually I don't see anyone setting that flag. But that idea let me
> to review the case, and I spotted an attribute which I overlooked earlier:
>
> Upon the creation of the temp file:
> G:\[delete]Test\checkout_TestRepo\.svn\tmp\svn-D0145269
>
> Desired Access:    Generic Read/Write
> Disposition:    Create
> Options:    Synchronous IO Non-Alert, Non-Directory File
> Attributes:    n/a
> ShareMode:    Read, Write, Delete
> AllocationSize:    0
> OpenResult:    Created
>
> ShareMode Delete... That made me a bit suspicious here. Why should SVN
> create that file with the share mode set to delete? Doesn't that suggest
> that any other process would be allowed to change the delete-state of that
> file while the handle is open? I don't know whether this is related to the
> problem or not, but I'm wondering the reasoning for that (or am I
> misreading that parameter --- i'm not that much into Windows file handling
> tbh).
>
> Regards,
> Stefan
>