Why do Virus Scanners hate Subversion

[Jesse Hogan <je...@gmail.com>]

Hello,
   When I do large checkouts and large merges on Windows NT using either
svn.exe or TSVN I often get error messages like this.

svn: In directory 'Databases/eMax/StoredProcs'
svn: Can't move source to dest
svn: Can't move
'Databases/eMax/StoredProcs/_svn/tmp/props/dbo.spConfigWriteStartingConfiguration.PRC.svn-work'
to
'Databases/eMax/StoredProcs/_svn/props/dbo.spConfigWriteStartingConfiguration.PRC.svn-work':
Permission denied

If I repeat the checkout or merge it will either work or fail on a different
file.

I've done a little research on these types of error messages and the only
explanation I've found is that a virus scanner is tampering with the newly
downloaded files. This seems to be what I am seeing because I can disable
the virus scanner to get the checkout to work. Am I right in blaming virus
scanners for these types of error message. Also, what exactly is happening.
Is the virus scanner placing a lock on these files which causes svn to
abend. Can anything be done to make svn more tolerant of virus scanners. I
hope so because this is one of the first things a user sees when (s)he
begins using svn which, I have witness, leads to bad impressions.

Re: Why do Virus Scanners hate Subversion

[Ma...@gfa-net.de]

D.J. Heap wrote:
> On 5/3/06, Ryan Schmidt <su...@ryandesign.com> wrote:
> > On May 2, 2006, at 19:31, Jesse Hogan wrote:
> >
> > > svn: In directory 'Databases/eMax/StoredProcs'
> > > svn: Can't move source to dest
> > > svn: Can't move 'Databases/eMax/StoredProcs/_svn/tmp/props/
> > > dbo.spConfigWriteStartingConfiguration.PRC.svn-work' to 'Databases/
> > > eMax/StoredProcs/_svn/props/
> > > dbo.spConfigWriteStartingConfiguration.PRC.svn-work': Permission
> > > denied
> >
> > > I've done a little research on these types of error messages and
> > > the only explanation I've found is that a virus scanner is
> > > tampering with the newly downloaded files. This seems to be what I
> > > am seeing because I can disable the virus scanner to get the
> > > checkout to work. Am I right in blaming virus scanners for these
> > > types of error message. Also, what exactly is happening. Is the
> > > virus scanner placing a lock on these files which causes svn to
> > > abend. Can anything be done to make svn more tolerant of virus
> > > scanners. I hope so because this is one of the first things a user
> > > sees when (s)he begins using svn which, I have witness, leads to
> > > bad impressions.
> >
> [snip]
> >
> > A workaround that Subversion could possibly employ would be to try
> > several times to move a file if it fails on the first attempt,
> > waiting a fraction of a second between attempts, and to only error
> > out if several consecutive attempts have failed.
> 
> 
> The native Win32 Subversion already does this and solves the
> virus-scanner/file indexers/tag-along-helper-apps issues in all known
> cases.  The cygwin version doesn't.  So don't use the cygwin version
> or convince the cygwin maintainers to include the workaround code.

I provided a patch to the dev group which activates the retries for
cygwin. With this patch applied the problem didn't occur any more
on my cygwin system (Yes, I encountered this problem from time to
time, too). So let's hope it will committed.

> 
> The root of the problem is Windows file-system semantics with regards
> to locking/deleting/moving already open files in combination with
> tag-along 'helper' applications that open every file that is created
> or changed in order to do something 'helpful' with it.  Most Unix
> file-systems don't have the problem because they support
> moving/deleting files that are open by other applications and they
> also have fewer (any?) tag-along type 'helper' applications.
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Why do Virus Scanners hate Subversion

["D.J. Heap" <dj...@gmail.com>]

On 5/3/06, Ryan Schmidt <su...@ryandesign.com> wrote:
> On May 2, 2006, at 19:31, Jesse Hogan wrote:
>
> > svn: In directory 'Databases/eMax/StoredProcs'
> > svn: Can't move source to dest
> > svn: Can't move 'Databases/eMax/StoredProcs/_svn/tmp/props/
> > dbo.spConfigWriteStartingConfiguration.PRC.svn-work' to 'Databases/
> > eMax/StoredProcs/_svn/props/
> > dbo.spConfigWriteStartingConfiguration.PRC.svn-work': Permission
> > denied
>
> > I've done a little research on these types of error messages and
> > the only explanation I've found is that a virus scanner is
> > tampering with the newly downloaded files. This seems to be what I
> > am seeing because I can disable the virus scanner to get the
> > checkout to work. Am I right in blaming virus scanners for these
> > types of error message. Also, what exactly is happening. Is the
> > virus scanner placing a lock on these files which causes svn to
> > abend. Can anything be done to make svn more tolerant of virus
> > scanners. I hope so because this is one of the first things a user
> > sees when (s)he begins using svn which, I have witness, leads to
> > bad impressions.
>
[snip]
>
> A workaround that Subversion could possibly employ would be to try
> several times to move a file if it fails on the first attempt,
> waiting a fraction of a second between attempts, and to only error
> out if several consecutive attempts have failed.


The native Win32 Subversion already does this and solves the
virus-scanner/file indexers/tag-along-helper-apps issues in all known
cases.  The cygwin version doesn't.  So don't use the cygwin version
or convince the cygwin maintainers to include the workaround code.

The root of the problem is Windows file-system semantics with regards
to locking/deleting/moving already open files in combination with
tag-along 'helper' applications that open every file that is created
or changed in order to do something 'helpful' with it.  Most Unix
file-systems don't have the problem because they support
moving/deleting files that are open by other applications and they
also have fewer (any?) tag-along type 'helper' applications.

DJ

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Why do Virus Scanners hate Subversion

[Ryan Schmidt <su...@ryandesign.com>]

On May 2, 2006, at 19:31, Jesse Hogan wrote:

> svn: In directory 'Databases/eMax/StoredProcs'
> svn: Can't move source to dest
> svn: Can't move 'Databases/eMax/StoredProcs/_svn/tmp/props/ 
> dbo.spConfigWriteStartingConfiguration.PRC.svn-work' to 'Databases/ 
> eMax/StoredProcs/_svn/props/ 
> dbo.spConfigWriteStartingConfiguration.PRC.svn-work': Permission  
> denied

> I've done a little research on these types of error messages and  
> the only explanation I've found is that a virus scanner is  
> tampering with the newly downloaded files. This seems to be what I  
> am seeing because I can disable the virus scanner to get the  
> checkout to work. Am I right in blaming virus scanners for these  
> types of error message. Also, what exactly is happening. Is the  
> virus scanner placing a lock on these files which causes svn to  
> abend. Can anything be done to make svn more tolerant of virus  
> scanners. I hope so because this is one of the first things a user  
> sees when (s)he begins using svn which, I have witness, leads to  
> bad impressions.

Possibly, the user should instead get a bad impression of an  
operating system that's plagued by viruses, and entertain the  
possibility of switching to one that is not.

http://www.apple.com/getamac/viruses.html

I believe the situation arises because the virus scanner is  
constantly on the lookout for new files being added to the system,  
and, when it finds them, scans them for possible threats. To scan the  
file, the scanner must open the file, read its contents and then  
close it. But Subversion's mode of operation is to rapidly create  
files in its tmp directory, then immediately move them elsewhere, as  
in the error message you copied above: Subversion created the file in  
the tmp/props directory and now wanted to move it to the props  
directory, but was denied. Presumably, right after Subversion created  
the file, the virus scanner noticed its existence and opened it and  
started reading it, but had not yet finished reading it by the time  
Subversion wanted to move it. As far as I know, on Windows, you  
cannot move a file that is open (and as far as I know, that's not a  
problem on Unix-like operating systems). Presumably, in the cases  
where it works properly, the virus scanner is either quick enough  
that it can read the entire file by the time Subversion tries to move  
it, or slow enough that it never notices the file in its original  
location in the tmp directory and only catches it later after it's  
been successfully moved.

A workaround that Subversion could possibly employ would be to try  
several times to move a file if it fails on the first attempt,  
waiting a fraction of a second between attempts, and to only error  
out if several consecutive attempts have failed.



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org