You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Karl Fogel <kf...@newton.ch.collab.net> on 2003/03/04 00:07:41 UTC
Re: svn commit: rev 5194 - trunk/subversion/libsvn_fs
brane@tigris.org writes:
> Log:
> We decided some time ago to kill off this txn_checkpoint.
>
> * subversion/libsvn_fs/fs.c (cleanup_fs): Don't call txn_checkpoint.
I had a local diff with this change, and have been testing it using
stress.pl. I didn't really see much improvement (though I'm not
totally convinced that my testing methodology was perfect). I did get
some deadlocks with the change, that I didn't seem to get without it.
I'm not objecting to the change -- I mean, it makes sense -- so much
as asking if you tested. Did it make a difference for you, and if so,
what sort?
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 5194 - trunk/subversion/libsvn_fs
Posted by mark benedetto king <mb...@boredom.org>.
On Tue, Mar 04, 2003 at 01:15:01PM -0600, Karl Fogel wrote:
> mark benedetto king <mb...@boredom.org> writes:
> > When I was using a 4.0.x BDB instead of 4.1.x, I was unable to run
> > either make svncheck or make davcheck to completion without this change.
>
This was a lie; the local change I had was the change of the parameters
from "every time" to "only every few minutes or few K" that Brandon Ehle
has been suggesting. A very simmilar change, though.
> ???
>
> Whoa -- you mean on a regular basis you couldn't run network tests
> without a local mod (or, you had to switch to 4.1.x)?
>
> That's pretty bad. Why wasn't anyone else seeing this, I wonder?
>
That's what I meant. David Summers and Brandon Ehle were both seeing
thing, IIRC.
--ben
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 5194 - trunk/subversion/libsvn_fs
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
mark benedetto king <mb...@boredom.org> writes:
> When I was using a 4.0.x BDB instead of 4.1.x, I was unable to run
> either make svncheck or make davcheck to completion without this change.
???
Whoa -- you mean on a regular basis you couldn't run network tests
without a local mod (or, you had to switch to 4.1.x)?
That's pretty bad. Why wasn't anyone else seeing this, I wonder?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 5194 - trunk/subversion/libsvn_fs
Posted by mark benedetto king <mb...@boredom.org>.
On Mon, Mar 03, 2003 at 06:07:41PM -0600, Karl Fogel wrote:
> brane@tigris.org writes:
> > Log:
> > We decided some time ago to kill off this txn_checkpoint.
> >
> > * subversion/libsvn_fs/fs.c (cleanup_fs): Don't call txn_checkpoint.
>
> I had a local diff with this change, and have been testing it using
> stress.pl. I didn't really see much improvement (though I'm not
> totally convinced that my testing methodology was perfect). I did get
> some deadlocks with the change, that I didn't seem to get without it.
>
> I'm not objecting to the change -- I mean, it makes sense -- so much
> as asking if you tested. Did it make a difference for you, and if so,
> what sort?
>
I know you were asking Brane, but:
When I was using a 4.0.x BDB instead of 4.1.x, I was unable to run
either make svncheck or make davcheck to completion without this change.
--ben
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 5194 - trunk/subversion/libsvn_fs
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Branko Čibej <br...@xbc.nu> writes:
> stress.pl isn't really the right kind of test for this. You need a huge
> repository to see the difference (or the timeouts).
Nah, stress.pl is always a "right kind of test" for a change that
affects the database. Sure, the purpose of the change may have
nothing to do with performance under stress.pl conditions... but
running stress.pl can still be useful in determining the real (as
opposed to projected) effects of the change.
> Deadlocks? I can't see how removing a txn_checkpoint could cause
> deadlocks. Are you sure it wasn't something else?
Well, honestly no, I'm not sure it wasn't something else, so I'm not
going to worry about it too much until we hear reports from the field.
I mean, I got the deadlocks with the change, and didn't get them
without it, but I only did one run each way. I don't feel it's enough
data to fret over yet, just something for us to make a mental note of.
> I can't recall the thread now, but I do remember people reporting that
> removing this checkpoint (or adding parameters to make "take" less
> often) would avoid client timeouts during long operations on large
> repositories.
>
> Now I wonder. Your report of deadlocks worries me a bit. Perhaps those
> people who reported problems could try again with the current code?
That would be good in any case, yeah.
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 5194 - trunk/subversion/libsvn_fs
Posted by Branko Čibej <br...@xbc.nu>.
Karl Fogel wrote:
>brane@tigris.org writes:
>
>
>>Log:
>>We decided some time ago to kill off this txn_checkpoint.
>>
>>* subversion/libsvn_fs/fs.c (cleanup_fs): Don't call txn_checkpoint.
>>
>>
>
>I had a local diff with this change, and have been testing it using
>stress.pl.
>
stress.pl isn't really the right kind of test for this. You need a huge
repository to see the difference (or the timeouts).
>I didn't really see much improvement (though I'm not
>totally convinced that my testing methodology was perfect). I did get
>some deadlocks with the change, that I didn't seem to get without it.
>
Deadlocks? I can't see how removing a txn_checkpoint could cause
deadlocks. Are you sure it wasn't something else?
>I'm not objecting to the change -- I mean, it makes sense -- so much
>as asking if you tested. Did it make a difference for you, and if so,
>what sort?
>
>
I can't recall the thread now, but I do remember people reporting that
removing this checkpoint (or adding parameters to make "take" less
often) would avoid client timeouts during long operations on large
repositories.
Now I wonder. Your report of deadlocks worries me a bit. Perhaps those
people who reported problems could try again with the current code?
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 5194 - trunk/subversion/libsvn_fs
Posted by Branko Čibej <br...@xbc.nu>.
(Short pause for something unprintable to pass down the socket...)
The only explanation I can think of is that we had a latent deadlock in
the code already, and the checkpoining (which is, for all intent and
purposes, a serialization point for the stress.pl threads) was only
hiding it. What we're /probably/ doing is not accessing the tables in
the same order in different code paths, which is an almost certain
recipe for deadlocks.
It's probably just another manifestation of our totally uncontrolled use
of (too large) transactions. If "just another" is the right word. Looks
like I'll have to dig into that mess sooner than I expected, ouch.
Karl Fogel wrote:
>Okay, I'm getting the error too, in three of my five stress.pl
>instances. Here's one example:
>
> Committing:
> Sending wcstress.4553/trunk/bar1/foo1
> Sending wcstress.4553/trunk/bar2/foo1
> Sending wcstress.4553/trunk/bar2/foo2
> Sending wcstress.4553/trunk/foo2
> Transmitting file data .subversion/libsvn_client/commit.c:670: \
> (apr_err=160020)
> svn: Item already exists in filesystem
> svn: Commit failed (details follow):
> subversion/libsvn_fs/bdb/nodes-table.c:146: (apr_err=160020)
> svn: successor id `0.0.u9' (for `0.0.tu') already exists \
> in filesystem '/home/kfogel/src/stress/repostress/db'
> subversion/libsvn_fs/bdb/bdb-err.c:61: (apr_err=160030)
> svn: Berkeley DB deadlock error
> svn: Berkeley DB error while reading node revision for \
> filesystem /home/kfogel/src/stress/repostress/db:
> DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock
> unexpected commit fail: exit status: 256
>
>They all look pretty similar to this. And I just looked at the output
>from my old stress tests, when I had (essentially) revision 5194 as a
>local mod, and it's the same error there too.
>
>I'll try again with revision 5194 subtracted; will post results.
>
>-Karl
>
>
>Philip Martin <ph...@codematters.co.uk> writes:
>
>
>>I just tried five instances of stress.pl (r5195) over ra_local and I
>>got a commit error that I have never seen before
>>
>>Updated to revision 193.
>>Committing:
>>Sending wcstress.5717/trunk/bar1/foo1
>>Sending wcstress.5717/trunk/bar2/foo2
>>Sending wcstress.5717/trunk/foo1
>>Sending wcstress.5717/trunk/foo2
>>Transmitting file data ...../svn/subversion/libsvn_client/commit.c:670: (apr_err=160020)
>>svn: Item already exists in filesystem
>>svn: Commit failed (details follow):
>>../svn/subversion/libsvn_fs/bdb/nodes-table.c:146: (apr_err=160020)
>>svn: successor id `2.0.166' (for `2.0.15r') already exists in filesystem '/home/pm/sw/subversion/obj/subversion/tests/repostress/db'
>>../svn/subversion/libsvn_fs/bdb/bdb-err.c:61: (apr_err=160030)
>>svn: Berkeley DB deadlock error
>>svn: Berkeley DB error while reading node revision for filesystem /home/pm/sw/subversion/obj/subversion/tests/repostress/db:
>>DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock
>>unexpected commit fail: exit status: 256
>>
>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
>For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
>
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 5194 - trunk/subversion/libsvn_fs
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Okay, I'm getting the error too, in three of my five stress.pl
instances. Here's one example:
Committing:
Sending wcstress.4553/trunk/bar1/foo1
Sending wcstress.4553/trunk/bar2/foo1
Sending wcstress.4553/trunk/bar2/foo2
Sending wcstress.4553/trunk/foo2
Transmitting file data .subversion/libsvn_client/commit.c:670: \
(apr_err=160020)
svn: Item already exists in filesystem
svn: Commit failed (details follow):
subversion/libsvn_fs/bdb/nodes-table.c:146: (apr_err=160020)
svn: successor id `0.0.u9' (for `0.0.tu') already exists \
in filesystem '/home/kfogel/src/stress/repostress/db'
subversion/libsvn_fs/bdb/bdb-err.c:61: (apr_err=160030)
svn: Berkeley DB deadlock error
svn: Berkeley DB error while reading node revision for \
filesystem /home/kfogel/src/stress/repostress/db:
DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock
unexpected commit fail: exit status: 256
They all look pretty similar to this. And I just looked at the output
from my old stress tests, when I had (essentially) revision 5194 as a
local mod, and it's the same error there too.
I'll try again with revision 5194 subtracted; will post results.
-Karl
Philip Martin <ph...@codematters.co.uk> writes:
> I just tried five instances of stress.pl (r5195) over ra_local and I
> got a commit error that I have never seen before
>
> Updated to revision 193.
> Committing:
> Sending wcstress.5717/trunk/bar1/foo1
> Sending wcstress.5717/trunk/bar2/foo2
> Sending wcstress.5717/trunk/foo1
> Sending wcstress.5717/trunk/foo2
> Transmitting file data ...../svn/subversion/libsvn_client/commit.c:670: (apr_err=160020)
> svn: Item already exists in filesystem
> svn: Commit failed (details follow):
> ../svn/subversion/libsvn_fs/bdb/nodes-table.c:146: (apr_err=160020)
> svn: successor id `2.0.166' (for `2.0.15r') already exists in filesystem '/home/pm/sw/subversion/obj/subversion/tests/repostress/db'
> ../svn/subversion/libsvn_fs/bdb/bdb-err.c:61: (apr_err=160030)
> svn: Berkeley DB deadlock error
> svn: Berkeley DB error while reading node revision for filesystem /home/pm/sw/subversion/obj/subversion/tests/repostress/db:
> DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock
> unexpected commit fail: exit status: 256
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 5194 - trunk/subversion/libsvn_fs
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
If you re-run, does the error (or a similar one) reproduce reliably?
If it's pretty easy to make it happen, then we can see if it happens
with this change subtracted...
I'll build right now and run five instances of stress.pl on head
(i.e., still with the change) to see if I can get something similar to
happen.
-Karl
Philip Martin <ph...@codematters.co.uk> writes:
> I just tried five instances of stress.pl (r5195) over ra_local and I
> got a commit error that I have never seen before
>
> Updated to revision 193.
> Committing:
> Sending wcstress.5717/trunk/bar1/foo1
> Sending wcstress.5717/trunk/bar2/foo2
> Sending wcstress.5717/trunk/foo1
> Sending wcstress.5717/trunk/foo2
> Transmitting file data ...../svn/subversion/libsvn_client/commit.c:670: (apr_err=160020)
> svn: Item already exists in filesystem
> svn: Commit failed (details follow):
> ../svn/subversion/libsvn_fs/bdb/nodes-table.c:146: (apr_err=160020)
> svn: successor id `2.0.166' (for `2.0.15r') already exists in filesystem '/home/pm/sw/subversion/obj/subversion/tests/repostress/db'
> ../svn/subversion/libsvn_fs/bdb/bdb-err.c:61: (apr_err=160030)
> svn: Berkeley DB deadlock error
> svn: Berkeley DB error while reading node revision for filesystem /home/pm/sw/subversion/obj/subversion/tests/repostress/db:
> DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock
> unexpected commit fail: exit status: 256
>
> Here's what the working copy that failed looks like after that failure
>
> svn st wcstress.5717/
> M wcstress.5717/trunk/bar1/foo1
> M wcstress.5717/trunk/bar2/foo2
> M wcstress.5717/trunk/foo1
> M wcstress.5717/trunk/foo2
>
> Although that particular commit failed, the client terminated
> gracefully, note the working copy is not locked. The repository was
> OK and the other four instances of stress.pl carried on beyond 800
> revisions.
>
> --
> Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 5194 - trunk/subversion/libsvn_fs
Posted by Philip Martin <ph...@codematters.co.uk>.
Karl Fogel <kf...@newton.ch.collab.net> writes:
> brane@tigris.org writes:
> > Log:
> > We decided some time ago to kill off this txn_checkpoint.
> >
> > * subversion/libsvn_fs/fs.c (cleanup_fs): Don't call txn_checkpoint.
>
> I had a local diff with this change, and have been testing it using
> stress.pl. I didn't really see much improvement (though I'm not
> totally convinced that my testing methodology was perfect). I did get
> some deadlocks with the change, that I didn't seem to get without it.
I just tried five instances of stress.pl (r5195) over ra_local and I
got a commit error that I have never seen before
Updated to revision 193.
Committing:
Sending wcstress.5717/trunk/bar1/foo1
Sending wcstress.5717/trunk/bar2/foo2
Sending wcstress.5717/trunk/foo1
Sending wcstress.5717/trunk/foo2
Transmitting file data ...../svn/subversion/libsvn_client/commit.c:670: (apr_err=160020)
svn: Item already exists in filesystem
svn: Commit failed (details follow):
../svn/subversion/libsvn_fs/bdb/nodes-table.c:146: (apr_err=160020)
svn: successor id `2.0.166' (for `2.0.15r') already exists in filesystem '/home/pm/sw/subversion/obj/subversion/tests/repostress/db'
../svn/subversion/libsvn_fs/bdb/bdb-err.c:61: (apr_err=160030)
svn: Berkeley DB deadlock error
svn: Berkeley DB error while reading node revision for filesystem /home/pm/sw/subversion/obj/subversion/tests/repostress/db:
DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock
unexpected commit fail: exit status: 256
Here's what the working copy that failed looks like after that failure
svn st wcstress.5717/
M wcstress.5717/trunk/bar1/foo1
M wcstress.5717/trunk/bar2/foo2
M wcstress.5717/trunk/foo1
M wcstress.5717/trunk/foo2
Although that particular commit failed, the client terminated
gracefully, note the working copy is not locked. The repository was
OK and the other four instances of stress.pl carried on beyond 800
revisions.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org