ra_dav mass-commit status

[Greg Stein <gs...@lyra.org>]

I just ran the mass-commit script (early version; not Karl's fancy one) and
things seem pretty reasonable now.

 948 successful commits
  41 svndiff failures(?)   (yes, Mike's patch is in there)
  11 failures to lock WC   (due to existing locks)
----
1000 attempted commits

The Apache log also shows 138 cases where a database close returned
DB_INCOMPLETE. Since it doesn't correlate to anything above, I suspect a
simple race condition between cleanup of a commit and the beginning of the
next commit. I will try adding "sleep 1" into my mass-commit to see if that
helps.


My next steps are:

* produce a short recipe for repro'ing the svndiff failure (and pass this to
  cmpilato for investigation)
* check in the mass-commit generation scripts
* investigate DB_INCOMPLETE, starting with the sleep

Ben: regarding the WC locking issue, I'm guessing you will figure out a way
to repro that. Let me know if you need assistance.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: ra_dav mass-commit status

[kf...@collab.net]

cmpilato@collab.net writes:
> Hey, Greg, check how svn_txdelta_parse_svndiff() is used in
> libsvn_fs/reps-strings.c:rep_read_range().  I'm trapping that specific
> error and ignoring it because I'm sending data in chunks, and I *know*
> there are times when I won't be sending the whole of the svndiff
> data.  I'm not sure that mod_dav_svn is doing anything special when
> the stream is closed -- would that be dav_svn_close_stream()?
> 
> Now, I don't know which is better: to catch this specific error (it is
> generated in exactly one place in the code), or to change the
> close_handler for svndiff stuffs so that it stops caring about the
> fact that the data ended early.  I do think it's good to have that
> final integrity check there, but if it is going to complicate the code
> in other ways, I'm not so sure.

Mike and I just discussed this some more, here's a summary of our
thought:

We're bound to lose if every caller of close_stream(svndiff_stream)
needs to check for and optionally ignore this error.  Someone will
always forget to do it.  I know that someone will be me at least
once. :-)

Instead, let's add a flag to open_stream(svndiff_stream).  The flag
will signify whether close_stream() should error on pending data or
just ignore it.  That way, callers will be forced by the function
prototype itself to make a choice when creating the stream.  This is
much less human-error prone than the current way.

-K

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: ra_dav mass-commit status

[cm...@collab.net]

Greg Stein <gs...@lyra.org> writes:

> Note that the first change doesn't trigger it. It is the second. And I just
> verified that using ra_local succeeds... hmm.

Hey, Greg, check how svn_txdelta_parse_svndiff() is used in
libsvn_fs/reps-strings.c:rep_read_range().  I'm trapping that specific
error and ignoring it because I'm sending data in chunks, and I *know*
there are times when I won't be sending the whole of the svndiff
data.  I'm not sure that mod_dav_svn is doing anything special when
the stream is closed -- would that be dav_svn_close_stream()?

Now, I don't know which is better: to catch this specific error (it is
generated in exactly one place in the code), or to change the
close_handler for svndiff stuffs so that it stops caring about the
fact that the data ended early.  I do think it's good to have that
final integrity check there, but if it is going to complicate the code
in other ways, I'm not so sure.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: ra_dav mass-commit status

[Greg Stein <gs...@lyra.org>]

On Fri, Aug 24, 2001 at 07:02:16PM -0500, cmpilato@collab.net wrote:
> Greg Stein <gs...@lyra.org> writes:
> >  948 successful commits
> >   41 svndiff failures(?)   (yes, Mike's patch is in there)
> >   11 failures to lock WC   (due to existing locks)
> > ----
> > 1000 attempted commits
> 
> Dangit.  Friggin' svndiff.
> 
> > * produce a short recipe for repro'ing the svndiff failure (and pass this to
> >   cmpilato for investigation)
> 
> Thank you, sir.

Got the script for ya :-)  [attached]

Note that the first change doesn't trigger it. It is the second. And I just
verified that using ra_local succeeds... hmm.

Probably means that it is in mod_dav_svn, or in the stream returned by
svn_txdelta_parse_svndiff(). mod_dav_svn simply creates that stream and
feeds it content from the network pipe. My earlier post with an actual
capture of those bytes seemed to indicate there was nothing wrong on the
client side (i.e. it got the right bytes out). Thus, it must be on the
server. Given that we correctly write text/plain into the FS, I would assume
that the mod_dav_svn logic for transferring bytes from the pipe is valid (we
are fed the bytes and then, in turn, feed them to the FS in one of two ways;
the branch with the svndiff parsing stream is actually the simpler of the
two (see mod_dav_svn/repos.c::dav_svn_write_stream)). That leaves me with an
initial suspicion / starting point of the svn_txdelta_parse_svndiff function
and its subcomponents.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/

Re: ra_dav mass-commit status

[cm...@collab.net]

Greg Stein <gs...@lyra.org> writes:

>  948 successful commits
>   41 svndiff failures(?)   (yes, Mike's patch is in there)
>   11 failures to lock WC   (due to existing locks)
> ----
> 1000 attempted commits

Dangit.  Friggin' svndiff.

> * produce a short recipe for repro'ing the svndiff failure (and pass this to
>   cmpilato for investigation)

Thank you, sir.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org