Re: Extremely slow merges

[St...@etas.de]

kfogel@collab.net writes:

> It would be helpful if you could first narrow down whether the problem
> originates in the client, or server, or both.  First downgrade one
> side back to whatever SVN you were using before; then revert and
> downgrade the other side; then downgrade both.  At each stage, test a
> merge (the exact same one each time), and see if its speed changes
> drastically.  This way we can narrow down the regression.

Accidentially, I performed a pretty huge merge operation 
(2000+ files / folder changes) today. In fact, it is still running ;) 

My config:

* Server: SVN 1.1.1 on W2k, 3GHz XEON, 2GB RAM, SCSI RAID, NTFS
* Client: SVN 1.1.1 on XPpro, 2.4GHz Opteron, 2GB RAM, IDE Disk, NTFS
* HTTP access via CL client over 100MBit Ethernet LAN

* BDB4.2 repository, 1.52 GB, 12000 revs
* trunk is 20k+ user files (mostly source code) in 1k+ user folders
* wc size is about 100MB

My observations:

* network load is low:
  50 .. 100 kBIT / sec
  ~10 packets in, 10 .. 15 packets out / sec

* server load is low
  0% CPU load with occational peeks up to 7%
  ~500MB total RAM usage (i.e. about 1.5G is used as file cache)
  disks are almost idle

* client load is low
  0 .. 5% typ. CPU usage, about half the time is at 0%
  SVN process uses ~5MB RAM
  disk is almost idle

Much more interesting:

* merge is slow (~2 secs per file) with updating, adding or
  deleting single files
* merge is fast (tens of files / sec) when adding or
  deleting whole sub-trees

My wild guess about what is happening:

* SVN client is slow when updating the item's parent entries file
  (i.e. when all other entries in that file are left unchanged)
* the is some sleep() operation involved here
  (since the seems to be no obvious bottleneck)

However, the title "extremely slow merges" sounds a bit overdone.
In most cases, where only few files get changed, the current merge
implementation is surely fast enough. Thus, a better title would be:
"Why is a merge much slower than an update?"

Hoping you find this information helpful,
Stefan.

Re: Extremely slow merges

[John Peacock <jp...@rowman.com>]

Stefan.Fuhrmann@etas.de wrote:
> My config:
> 
> * Server: SVN 1.1.1 on W2k, 3GHz XEON, 2GB RAM, SCSI RAID, NTFS
> * Client: SVN 1.1.1 on XPpro, 2.4GHz Opteron, 2GB RAM, IDE Disk, NTFS

What antivirus software are you running on the client?  Can you temporarily 
uninstall it (disabling may not be enough) and rerun the test?

The reason I ask is that the hooks that AV software use on Win32 platforms are 
very deep into the system.  These callbacks have an almost pathelogical effect 
on software which creates lots of temporary files then renames them (exactly 
what Subversion and for that matter CVSNT both do).  Disabling the AV software 
merely makes the callbacks return immediately, but still leaves the callback 
hooks in place, thus still slowing down the file access.

John

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org