SVN checkout speed issue

[Art Hatch <ar...@gmail.com>]

Hi Team!

We think there may be a bug, or a feature we don't understand:

When checking out a repository using TortoiseSVN on top of a Visual SVN
Server v3.6, running on Windows Server 2012R2, we notice that files with a
mime type set (usually application/octet-stream) download from the checkout
MUCH FASTER. For a repo with lots of mime types set (mostly
application/octet-stream, as mentioned), we see speeds around
1.5-3MByte/sec. For non-mime-typed files, it's capped at around 300kb/sec
max. If some files come through with mime types in the
slower-to-checkout-repo, the DL speed bursts up to 1.5-3MBytes/sec, as we
see in the faster repo.

This leads us to  believe that SVN's underlying checkout logic is running
some mime type evaluation logic on files without mimetype set, and that the
logic ran is extremely inefficient or slow by design.

*1. is this a bug, or a feature by default?*
*2. can we disable the logic doing the mime type check for files without
mime type data, to speed up checkouts?*
*3. can we force SVN to see files without mime types set at text or binary,
to stop the logic that tries to figure out a file's mime type
programattically?*
*4. Any other input or feedback to help us? :)*

Screenshot attached for showing the differences in checkout speed we're
seeing.

*NOTE: *In the screenshot, we kicked off the checkouts at roughly the same
time; as you can see, the "zebitpaymentsystem" checkout is MUCH quicker,
averaging around 3x the checkout speed of the other repo. The other repo
has far far less mime types on the files in SVN, hence why we think it's
related.

Re: SVN checkout speed issue

[Branko Čibej <br...@apache.org>]

On 31.07.2018 00:56, Art Hatch wrote:
> Hi Team!
>
> We think there may be a bug, or a feature we don't understand:
>
> When checking out a repository using TortoiseSVN on top of a Visual
> SVN Server v3.6, running on Windows Server 2012R2, we notice that
> files with a mime type set (usually application/octet-stream) download
> from the checkout MUCH FASTER. For a repo with lots of mime types set
> (mostly application/octet-stream, as mentioned), we see speeds around
> 1.5-3MByte/sec. For non-mime-typed files, it's capped at around
> 300kb/sec max. If some files come through with mime types in the
> slower-to-checkout-repo, the DL speed bursts up to 1.5-3MBytes/sec, as
> we see in the faster repo.
>
> This leads us to  believe that SVN's underlying checkout logic is
> running some mime type evaluation logic on files without mimetype set,
> and that the logic ran is extremely inefficient or slow by design.

Sorry, but you believe incorrectly. :)

> *1. is this a bug, or a feature by default?*

There's no clear answer to that. It may be related to some server
settings. It's definitely not by design.

> *2. can we disable the logic doing the mime type check for files
> without mime type data, to speed up checkouts?*

There's no such logic in the Subversion (but, see below). There may be
in the HTTP server, depending on how it's configured.

> *3. can we force SVN to see files without mime types set at text or
> binary, to stop the logic that tries to figure out a file's mime type
> programattically?*

As I said, there's no such code in Subversion.

Well, actually, there is; depending on how Subversion was compiled, it
may try to guess a file's MIME type at 'svn add' and set the
svn:mime-type property on the newly added file (in the working copy).
However, that code does not in any way affect checkouts.

> *4. Any other input or feedback to help us? :)*

If you're accessing the repository via http(s)://, I recommend you check
the Apache HTTPd server settings. It does have an option to try to guess
the MIME type of the file it's serving. However, I don't know if that
option is even available for files that come from a Subversion repository.

Also, overall transmission speed _may_ be related to file size; it's
possible that fewer large files will transfer more quickly than more
small files. Whether that's the case here is hard to see from your
measurements.

-- Brane