You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by kf...@collab.net on 2003/11/10 16:44:02 UTC
Philip, is issue #1556 easy for you?
Philip,
Sander, Mike, Ben, and I were just discussing issue #1556, the
"mysterious post-update/checkout delay on huge trees" bug.
Of course, it's not mysterious anymore -- we know it's due to the
post-update unlocking, which is (apparently?) triggered by a pool
cleanup function. The problem is, the unlocking happens after the
client has printed the last line, so in large trees the user just gets
an unexplained pause:
$ svn up
[...]
U subversion/include/svn_delta.h
U subversion/include/svn_wc.h
Updated to revision 7684.
[insert long delay here]
$
Speeding up the unlocking isn't in the cards, but that's okay. We're
just hoping to give the user some feedback, so they don't wonder
what's going on, as in:
$ svn up
[...]
U subversion/include/svn_delta.h
U subversion/include/svn_wc.h
Unlocking working copy (please wait)...
[insert long delay here]
Updated to revision 7684.
$
It's easy to add a notification code for this, but we're not show
where/how to call it, because we're not quite sure where the unlocking
gets triggered.
Granted that we could probably solve this with some research... but,
you probably already have the knowledge in your head and could get it
right the first time. Care to take it on? If not, that's fine, just
let me know so we can allocate time. (Or if you don't have time to do
it, but do have time to post some pointers, even that would help.)
Thanks for your consideration,
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Philip, is issue #1556 easy for you?
Posted by pm...@users.sourceforge.net.
> > That will mean a additional line in the output, and the way I use
> > Subversion it will mean that I get a "please wait" message immediately
> > before the command exits. Is this new output really necessary? It's
> > not really an improvement in every case.
>
> True. It would be a (very) mild annoyance to those who are not having
> a problem.
>
> But to those who *are* having a problem, it's a big difference -- to
> them, it looks like Subversion is stuck, and this would clue them in
> that it's not. After a few minutes (a realistic amount of time if you
> have a big enough tree combined with a slow enough disk) of mysterious
> inaction, anyone would be tempted to hit ^C.
>
> I'm certainly not wedded to the idea of an extra line of output, if we
> can think of a better solution to this problem.
How about testing if STDOUT (or STDERR) is a terminal, and just print a
rotating line ( one of -\|/, followed by \r ) there if it is?
So no output in logfiles, just a sign that things are being done.
Of course, there's nothing like this in svn now - so that would be a
precedent.
If there will be some kind of status-indicator for terminals we could as well
count the locks we made and later show a percentage (and a graph! Oh, I like
graphs :-) to show how we're doing ...
Regards,
PMarek
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Philip, is issue #1556 easy for you?
Posted by kf...@collab.net.
Philip Martin <ph...@codematters.co.uk> writes:
> That will mean a additional line in the output, and the way I use
> Subversion it will mean that I get a "please wait" message immediately
> before the command exits. Is this new output really necessary? It's
> not really an improvement in every case.
True. It would be a (very) mild annoyance to those who are not having
a problem.
But to those who *are* having a problem, it's a big difference -- to
them, it looks like Subversion is stuck, and this would clue them in
that it's not. After a few minutes (a realistic amount of time if you
have a big enough tree combined with a slow enough disk) of mysterious
inaction, anyone would be tempted to hit ^C.
I'm certainly not wedded to the idea of an extra line of output, if we
can think of a better solution to this problem.
The message doesn't have to actually say "please wait", of course,
that was just my example. It could simply say
Releasing locks...
Updated to revision 8888.
The pause, if there is one, would go between those two lines. At
least that would give people a language to express what's happening.
"Subversion seems to spend a long time releasing locks when I update."
"Oh, how big is your tree?" "Pretty big." "Okay, then it's normal."
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Philip, is issue #1556 easy for you?
Posted by Philip Martin <ph...@codematters.co.uk>.
kfogel@collab.net writes:
> Philip Martin <ph...@codematters.co.uk> writes:
>> time I measure (2500 directories in 5 seconds). Is this a Windows
>> difference? Or much slower disks? Or is it something other than the
>> unlocking code causing the delay?
>
> Probably much slower disk (Ben's notebook is famous for that).
I tried my old laptop and it is indeed much slower. As far as I can
tell there are interactions with the way the OS buffers and flushes
its disk cache. Things are much faster if the unlocking can complete
before the locking gets flushed, although I think there is still
directory data that needs to get written. It may well depend on which
filesystem is being used, my laptop uses ext3 and my workstation uses
reiserfs.
>> I'm not sure what you want the feedback to look like, just one single
>> "please wait" message? Is that just a single notify call added to
>> svn_client_update()? I'm guessing you don't want a notify call for
>> each directory unlocked.
>
> Oh -- just one single "please wait" message was what I was thinking
> (sorry if my previous mail's example didn't make that clear).
That will mean a additional line in the output, and the way I use
Subversion it will mean that I get a "please wait" message immediately
before the command exits. Is this new output really necessary? It's
not really an improvement in every case.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Philip, is issue #1556 easy for you?
Posted by kf...@collab.net.
Philip Martin <ph...@codematters.co.uk> writes:
> time I measure (2500 directories in 5 seconds). Is this a Windows
> difference? Or much slower disks? Or is it something other than the
> unlocking code causing the delay?
Probably much slower disk (Ben's notebook is famous for that).
> svn_client_update() calls svn_wc_adm_close() which calls do_close() on
> the working copy path. do_close() calls do_close() recursively on
> each child before removing its own lock.
Thanks!
> > Granted that we could probably solve this with some research... but,
> > you probably already have the knowledge in your head and could get
> > it right the first time. Care to take it on? If not, that's fine,
> > just let me know so we can allocate time. (Or if you don't have
> > time to do it, but do have time to post some pointers, even that
> > would help.)
>
> I'm not sure what you want the feedback to look like, just one single
> "please wait" message? Is that just a single notify call added to
> svn_client_update()? I'm guessing you don't want a notify call for
> each directory unlocked.
Oh -- just one single "please wait" message was what I was thinking
(sorry if my previous mail's example didn't make that clear).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Philip, is issue #1556 easy for you?
Posted by Philip Martin <ph...@codematters.co.uk>.
Philip Martin <ph...@codematters.co.uk> writes:
> Locking can get triggered by pool cleanup, but for update it's coded
> explicitly since svn_client_update calls svn_wc_adm_close. I see that
> svn_client__checkout_internal doesn't explictily call close, I think
> it should (although that won't make it any faster.)
Oops! svn_client__checkout_internal doesn't call svn_wc_adm_close
explictily because it uses svn_client_update and that calls
svn_wc_adm_close.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Philip, is issue #1556 easy for you?
Posted by Files <fi...@poetryunlimited.com>.
Not to appear too naive about these things, but shouldn't there be a point at
which XXXX number of locks translates to a single lock on the entire working
copy or the repository, or what not?
To make an analogy, on a RDMBS, if I ask for a large number of things, if my
settings on the RDBMS are correct, a table wide lock is issued and performance
does not degrade. But if the settings are wrong, and each row is locked
indiscriminately, performance degrades as a function of N, sometimes O(N), at
others O(2N) and I've even seen it as O(N^1.5) or as bad as O(N^2).
I don't know if it's specifically applicable to your problem. Just hoping it
might be helpful to think about.
--
Shamim Islam
BA BS
Philip Martin said:
> kfogel@collab.net writes:
>
>> Sander, Mike, Ben, and I were just discussing issue #1556, the
>> "mysterious post-update/checkout delay on huge trees" bug.
>
> I'm not sure what you want the feedback to look like, just one single
> "please wait" message? Is that just a single notify call added to
> svn_client_update()? I'm guessing you don't want a notify call for
> each directory unlocked.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Philip, is issue #1556 easy for you?
Posted by Philip Martin <ph...@codematters.co.uk>.
kfogel@collab.net writes:
> Sander, Mike, Ben, and I were just discussing issue #1556, the
> "mysterious post-update/checkout delay on huge trees" bug.
>
> Of course, it's not mysterious anymore -- we know it's due to the
> post-update unlocking, which is (apparently?) triggered by a pool
> cleanup function.
Locking can get triggered by pool cleanup, but for update it's coded
explicitly since svn_client_update calls svn_wc_adm_close. I see that
svn_client__checkout_internal doesn't explictily call close, I think
it should (although that won't make it any faster.)
> The problem is, the unlocking happens after the
> client has printed the last line, so in large trees the user just gets
> an unexplained pause:
>
> $ svn up
> [...]
> U subversion/include/svn_delta.h
> U subversion/include/svn_wc.h
> Updated to revision 7684.
> [insert long delay here]
> $
I don't get anything like the delay reported in the issue. I haven't
tried the mozilla tree, but I have tried trees with 2500 directories
and they unlock in about 5 seconds. That's using 7200rpm (i.e. old)
SCSI disks. There seems to be an order of magnitude difference in the
time reported in the issue (5000 directories in 2 minutes) and the
time I measure (2500 directories in 5 seconds). Is this a Windows
difference? Or much slower disks? Or is it something other than the
unlocking code causing the delay?
> Speeding up the unlocking isn't in the cards, but that's okay. We're
> just hoping to give the user some feedback, so they don't wonder
> what's going on, as in:
>
> $ svn up
> [...]
> U subversion/include/svn_delta.h
> U subversion/include/svn_wc.h
> Unlocking working copy (please wait)...
> [insert long delay here]
> Updated to revision 7684.
> $
>
> It's easy to add a notification code for this, but we're not show
> where/how to call it, because we're not quite sure where the unlocking
> gets triggered.
svn_client_update() calls svn_wc_adm_close() which calls do_close() on
the working copy path. do_close() calls do_close() recursively on
each child before removing its own lock.
> Granted that we could probably solve this with some research... but,
> you probably already have the knowledge in your head and could get
> it right the first time. Care to take it on? If not, that's fine,
> just let me know so we can allocate time. (Or if you don't have
> time to do it, but do have time to post some pointers, even that
> would help.)
I'm not sure what you want the feedback to look like, just one single
"please wait" message? Is that just a single notify call added to
svn_client_update()? I'm guessing you don't want a notify call for
each directory unlocked.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Philip, is issue #1556 easy for you?
Posted by kf...@collab.net.
kfogel@collab.net writes:
> It's easy to add a notification code for this, but we're not show
> where/how to call it
s/show/sure
-K, wondering what slip of the synapse prompted *that* one...
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org