WebDAV Delete - Timeout

["Marian Schedenig (qs)" <Ma...@qualysoft.com>]

Hi!

I'm not sure if this is really a Jackrabbit problem, but it seems to be.

Using WebDAV, I'm trying to delete a large folder (20k files and folders,
over 150MB). Obviously, this takes a while - longer than my HTTP timeout, so
the DELETE method, issued via Jakarta's HttpClient, fails with a timeout.

The HTTP and WebDAV specs seem to be very thin regarding DELETE behaviour,
but from what I can see, the server may return 202 (Accepted) if the delete
operation was accepted but not yet completed - presumably to prevent
timeouts, since I can't find another hint on how to handle long-running
deletes (either on the server or the client side).

So... is this a bug in my client? Or should Jackrabbit return a 202 in time
to prevent timeouts? It seems like currently, my only option would be to
guess a "really long" timeout.

Thanks,
Marian.

-- 
View this message in context: http://www.nabble.com/WebDAV-Delete---Timeout-tp25030725p25030725.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.

Re: WebDAV Delete - Timeout

["Marian Schedenig (qs)" <Ma...@qualysoft.com>]

Alexander Klimetschek wrote:
> 
> Why not a higher timeout? It only helps you when the connection is
> unexpectedly lost during an operation and gives you an earlier warning
> that the network connection has a problem.

I guess you're right. It's setting some high timeout (meaning that lost
connections are only discovered after a long time) that still might not be
high enough (if deletion takes too long) that doesn't quite agree with me.
But it seems Konqueror does just fine this way, so I'll just deal with it
this way.


A workaround is to delete "recursively" from deeper within the tree up
> to the parent you want to delete, if you know that a lot of data is
> involved.

Yes, I've been thinking about that. Also because it would give me some
progress information I can relay to the user. Problem is, I don't know how
much the directory a user deletes contains, so I have to either do a full
recursion (my first attempt, but recursively looking up the files takes many
times longer than actually deleting the root folder) or make an educated
guess for a recursion threshold.

I'll probably just deal with high timeouts, at least for now.

Thx,
Marian.

-- 
View this message in context: http://www.nabble.com/WebDAV-Delete---Timeout-tp25030725p25118041.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.

Re: WebDAV Delete - Timeout

[Alexander Klimetschek <ak...@day.com>]

On Thu, Aug 20, 2009 at 4:15 PM, Marian Schedenig
(qs)<Ma...@qualysoft.com> wrote:
> Right now, 10 seconds for my local tests. Obviously not very high, but
> considering the timeout will apply to all communication over that
> connection, setting it to something really high like 2 minutes wouldn't
> really play nice with real connection problems for normal operations.

Why not a higher timeout? It only helps you when the connection is
unexpectedly lost during an operation and gives you an earlier warning
that the network connection has a problem.

> Also, deletion *seems* to be carried out "in the background" (from a client
> perspective) after the timeout, but for my 150MB directory, it can take a
> *long* time. At times, server performances seems to degrade seriously as
> well, causing simple directory listing requests to time out as well.
> Restarting the server helps (though JR initialisation can take a while in
> this case), and sometimes the deletion operation seems to finish during
> startup - but sometimes, parts of the directory are still there and I have
> to repeat the process.

A workaround is to delete "recursively" from deeper within the tree up
to the parent you want to delete, if you know that a lot of data is
involved.

Regards,
Alex

-- 
Alexander Klimetschek
alexander.klimetschek@day.com

Re: WebDAV Delete - Timeout

["Marian Schedenig (qs)" <Ma...@qualysoft.com>]

Julian Reschke wrote:
> 
> What's the timeout value, btw?

Right now, 10 seconds for my local tests. Obviously not very high, but
considering the timeout will apply to all communication over that
connection, setting it to something really high like 2 minutes wouldn't
really play nice with real connection problems for normal operations.


That's possible, but I think most clients would be confused. I#ve never 
> seen that in the wild.

Neither have I (which isn't saying much since this is the first time I've
thought about the whole issue).


Yes, I think you need to increase the timeout.

Well, it gets worse. For starters, it's not just my client, but Windows
Explorer can't handle it either (no, I'm not a fan of Windows Explorer, and
even less so of its WebDAV functionality, but I have to assume people will
want to use it with our server).

Also, deletion *seems* to be carried out "in the background" (from a client
perspective) after the timeout, but for my 150MB directory, it can take a
*long* time. At times, server performances seems to degrade seriously as
well, causing simple directory listing requests to time out as well.
Restarting the server helps (though JR initialisation can take a while in
this case), and sometimes the deletion operation seems to finish during
startup - but sometimes, parts of the directory are still there and I have
to repeat the process.

Now the timeout issue for large collections may be a general HTTP/WebDAV
thing (I have no experience how other clients and servers deal with this),
but the impact on server performance worries me. I'll try to do more testing
on my Linux box at home and see if it behaves differently.

Cheers,
Marian.

-- 
View this message in context: http://www.nabble.com/WebDAV-Delete---Timeout-tp25030725p25062936.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.

Re: WebDAV Delete - Timeout

[Julian Reschke <ju...@gmx.de>]

Marian Schedenig (qs) wrote:
> Hi!
> 
> I'm not sure if this is really a Jackrabbit problem, but it seems to be.
> 
> Using WebDAV, I'm trying to delete a large folder (20k files and folders,
> over 150MB). Obviously, this takes a while - longer than my HTTP timeout, so
> the DELETE method, issued via Jakarta's HttpClient, fails with a timeout.

What's the timeout value, btw?

> The HTTP and WebDAV specs seem to be very thin regarding DELETE behaviour,
> but from what I can see, the server may return 202 (Accepted) if the delete
> operation was accepted but not yet completed - presumably to prevent
> timeouts, since I can't find another hint on how to handle long-running
> deletes (either on the server or the client side).

That's possible, but I think most clients would be confused. I#ve never 
seen that in the wild.

RFC 2518 suggested an intermediary status 102 Processing (see 
<http://greenbytes.de/tech/webdav/rfc2518.html#rfc.section.10.1>, but in 
RFC 4918 this was removed because nobody was implementing it). (And I 
don't think the servlet API allows producing it anyway).

> So... is this a bug in my client? Or should Jackrabbit return a 202 in time
> to prevent timeouts? It seems like currently, my only option would be to
> guess a "really long" timeout.

Yes, I think you need to increase the timeout.

BR, Julian