[users@httpd] mod_dav: resume transfers not working

[Stephen <hn...@yahoo.com>]

All,

Forgive me if I am not asking this question in the right way. If I should post it somewhere else, please let me know.

I have been racking my brain as to why some uploads via webdav are failing to resume.  I have some users that are uploading 10GB plus files and are trying to resume them when they have a connection timeout with no success. If there is no timeout on the connection, it works.  We see the following:

1) client starts the upload
2) client timeout (we pull a network cable to simulate)
3) partial file on the server
3) client tries to resume with an active network connection
4) client does a PROPFIND for the file on the server and gets a status 404
5) server deletes the file
6) client resume fails and starts over from scratch.


We pretty much have the standard config per documentation on the  server...http://httpd.apache.org/docs/2.2/mod/mod_dav.html

I have tried multiple clients that support resume (cyberduck and crossftp).  Both state "Some servers may not support resumable transfers and the file will be reloaded instead."  What is it specifically that has to be implemented on a server to support resumable transfers?

Cyberduck: 4.4.4
CrossFTP 1.91.2
Centos 5.7
Apache 2.2.15

Thanks,
stephen

Re: [users@httpd] mod_dav: resume transfers not working

[Ben Reser <be...@reser.org>]

On 12/17/13 7:29 AM, Stephen wrote:
> Forgive me if I am not asking this question in the right way. If I should post
> it somewhere else, please let me know.

You're in the right place.

> I have been racking my brain as to why some uploads via webdav are failing to
> resume.  I have some users that are uploading 10GB plus files and are trying to
> resume them when they have a connection timeout with no success. If there is no
> timeout on the connection, it works.  We see the following:
> 
> 1) client starts the upload
> 2) client timeout (we pull a network cable to simulate)
> 3) partial file on the server
> 3) client tries to resume with an active network connection
> 4) client does a PROPFIND for the file on the server and gets a status 404
> 5) server deletes the file
> 6) client resume fails and starts over from scratch.
> 
> 
> We pretty much have the standard config per documentation on the 
> server...http://httpd.apache.org/docs/2.2/mod/mod_dav.html
>
> I have tried multiple clients that support resume (cyberduck and crossftp). 
> Both state "Some servers may not support resumable transfers and the file will
> be reloaded instead."  What is it specifically that has to be implemented on a
> server to support resumable transfers?
> 
> Cyberduck: 4.4.4
> CrossFTP 1.91.2
> Centos 5.7
> Apache 2.2.15

First of all there really is no such thing as a resumable upload in DAV.  You
could implement one that work with mod_dav_fs but it wouldn't be based on the
webdav standard.  I'll explain how as we go.

There are two ways to send partial content when uploading.

1) Content-Range header.  You provide a Content-Range header along with your
request that specifies the begin and end positions in the file that your
content belongs.  mod_dav and mod_dav_fs support this, they just seek to the
start position in the file specified by the Content-Range header.  Technically
I don't think this is part of any RFC and violates RFC 2518/4918 which say the
the PUT should replace the entire contents of the file.  But this exists to
allow WebDAV to be used for mounted file systems (which makes it behave far
more like a normal file system).

2) HTTP PATCH method (see RFC 5789).  mod_dav does not support this.  The
problem here from a DAV standpoint is that the RFC leaves the actual body
format entirely up to the applications and servers and doesn't mandate any
particular format always be supported.  I'm sure PATCH is fine for people doing
RESTful APIs but unless DAV clients and servers agree on a fairly common format
it doesn't seem very useful.  Additionally PATCH isn't very useful if you want
to be able to resume an interuppted PATCH upload since it says if the PATCH
request is only partially received it should do nothing.

But in order to be able to resume an upload you need more than just the ability
to send partial content.  You need to server to preserve what you've already
uploaded and you need a way to know what you've uploaded.

First let's deal with getting the server to preserve what you've uploaded.
Right now for files being PUT'ed without a Content-Range header or files that
don't exist (regardless of the Content-Range header) when the connection fails
then the data you have already sent will be removed.  The intention here is to
allow the server to fail back to the existing state (the RFCs as far as I am
aware is silent on what you're supposed to do here).  Files that exist and that
are being modified with a PUT request that included a Content-Range header
relax this because the real file is being written to directly.  So whatever is
written by your client ends up on disk.

So in order to reliably get the server to preserve what has been sent even in
the case of an interrupted connection with mod_dav_fs a client would have to 1)
Ensure that the file already exists and if not create an empty file (PROPFIND
potentially followed by a PUT with no content).  2) Send the PUT with a
Content-Range for the entire file.

How you're supposed to know where to resume from seems like an open question.
You could make assumptions based on what you've seen TCP ACks for but that's
not reliable.  It's entirely possible for the servers TCP stack to ACK
something that the server application hasn't actually seen yet.  You could
allow for some fudge room but there would always been the possibility that the
upload would be missing something in the middle.

I suspect the only way to make it reliable is to split a large file up into
multiple PUTs with different Content-Ranges and if the piece doesn't succeed
then you resend the whole piece.

I couldn't find any explanation for CrossFTP as far as what they support.  For
Cyberduck I found this documentation:
https://trac.cyberduck.io/wiki/help/en/howto/transfer#Resume

Based on that documentation I think they're making a much narrower promise as
to what they mean by supporting resuming uploads.  They're saying they can
support appending to files (which can be done with Content-Range support we
have though it's not clear how they implement that) and they're saying they can
skip uploads that match the file file size (which can also be done, again with
no details on how they go about it).  But they never say anything really about
resuming an interrupted download like what you're asking about.

If I was writing a WebDAV client I probably wouldn't even try to implement
resuming interrupted PUTs because I don't think there is a uniform way you can
do it that would work across a variety of servers.  Too many details are left
to the server implementers to decide what to do.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org