You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Eric Noel <er...@rcs100.com> on 2004/11/11 23:20:21 UTC
repair timestamps of unchanged files
This was reported in 2002 as issue #791. Has anyone
revisited this since then? If you've got a bunch of
unchanged files that get an updated timestamp (through
touch, whatever), then svn becomes a huge dog... It was
taking 17 minutes for an update. I've fixed the problem in
my case, but it seems to me that a built-in way to correct
for this would be very appreciated.
Thanks,
--
Eric Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Simon Large <sl...@blazepoint.co.uk>.
Philip Martin wrote:
> "Simon Large" <sl...@blazepoint.co.uk> writes:
>> Can I just add a user's perspective here. How do we know that the
>> timestamps are mismatched?
> That was discussed in the last thread.
Sorry. It was a rhetorical question. Since the user _doesn't_ know, all
he sees is poor performance for no apparent reason, which is why an
automatic fix is preferable.
Simon
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Philip Martin <ph...@codematters.co.uk>.
"Simon Large" <sl...@blazepoint.co.uk> writes:
> Can I just add a user's perspective here. How do we know that the
> timestamps are mismatched?
That was discussed in the last thread.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Simon Large <sl...@blazepoint.co.uk>.
Julian Foad wrote:
> I was hoping there would be a better (more automatic and invisible)
> way than any of these four options, like making read-only operations
> (status, etc.) do the fix-up as and when they notice the discrepancy.
>
> But of these four options, I agree that "svn cleanup" is the best,
> because I really don't want to see any new commands or options for a
> "feature" that shouldn't need to exist (from a high-level-logic point
Can I just add a user's perspective here. How do we know that the
timestamps are mismatched? No status report shows that. All we see is
the symptom of increasing slowness in WC operations, so it is not
obvious that we would need to run "svn cleanup". Are you going to
document that somehow?
"If your working copy starts getting really slow, run svn cleanup to fix
the broken timestamps".
This sounds like more ammunition for the detractors who already complain
about speed problems. I agree with Julian here that if at all possible
it should be automatic, invisible and done as soon as the mismatch is
noticed.
Simon
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Branko Čibej <br...@xbc.nu>.
Philip Martin wrote:
>Branko ÄŒibej <br...@xbc.nu> writes:
>
>
>
>>I'm trying think of a good reason why we'd support any kind of
>>concurrent operation in the WC. If we just serialised everything, "svn
>>st" and friends could do this cleanup themselves.
>>
>>
>
>How about a GUI that runs status periodically (or in response to some
>sort of filesystem notification system), that could cause concurrent
>operation with some other process running, say, update. If status
>takes a write lock the update might fail with an "already locked"
>error, but when the user investigated there would be no lock.
>
>
If status takes a read lock and update /doesn't/ fail with "already
locked", then status might die due to reading half-baked data that
update hadn't finished yet. You can't have a reader and a writer in the
same WC at the same time, you can only have multiple readers.
Now I'll admit that if we serialize all access to the WC, then your
scenario with two statuses instead of a status and an update is valid.
However, I think that being able to automatically fix timestamps (and
other WC bogosities, such as leftover files in .svn/tmp) is more
important than running two statuses in parallel. Especially as you can
make the window for lock collision narrower by running "svn st -N" (and
I hope that locks only one directory, not the whole tree).
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Stefan <st...@gmx.ch>.
Ben Collins-Sussman wrote:
>
> On Nov 23, 2004, at 11:03 AM, Philip Martin wrote:
>
>> Branko ÄŒibej <br...@xbc.nu> writes:
>>
>>> I'm trying think of a good reason why we'd support any kind of
>>> concurrent operation in the WC. If we just serialised everything, "svn
>>> st" and friends could do this cleanup themselves.
>>
>>
>> How about a GUI that runs status periodically (or in response to some
>> sort of filesystem notification system), that could cause concurrent
>> operation with some other process running, say, update. If status
>> takes a write lock the update might fail with an "already locked"
>> error, but when the user investigated there would be no lock.
And that's exactly what TSVN is already doing. The user can refresh the
filesystem view anytime (s)he wants, even while another Subversion task
is running. Refreshing the view means 'svn status'.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Ben Collins-Sussman <su...@collab.net>.
On Nov 23, 2004, at 11:03 AM, Philip Martin wrote:
> Branko ÄŒibej <br...@xbc.nu> writes:
>
>> I'm trying think of a good reason why we'd support any kind of
>> concurrent operation in the WC. If we just serialised everything, "svn
>> st" and friends could do this cleanup themselves.
>
> How about a GUI that runs status periodically (or in response to some
> sort of filesystem notification system), that could cause concurrent
> operation with some other process running, say, update. If status
> takes a write lock the update might fail with an "already locked"
> error, but when the user investigated there would be no lock.
>
As a matter of fact, this is what the next version of sc-plugin
(TortoiseSVN for Mac OSX Finder) is probably going to do. I've been
working with the author, and he's planning to write an optional
background daemon that periodically runs 'svn status' on directories of
your choice... so that when you visit the working copy, the
icon-overlays have already been pre-computed.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Philip Martin <ph...@codematters.co.uk>.
Branko Äibej <br...@xbc.nu> writes:
> I'm trying think of a good reason why we'd support any kind of
> concurrent operation in the WC. If we just serialised everything, "svn
> st" and friends could do this cleanup themselves.
How about a GUI that runs status periodically (or in response to some
sort of filesystem notification system), that could cause concurrent
operation with some other process running, say, update. If status
takes a write lock the update might fail with an "already locked"
error, but when the user investigated there would be no lock.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Branko Čibej <br...@xbc.nu>.
Philip Martin wrote:
>I'm still uneasy about the idea. In addition to the problems
>mentioned in the old thread, "read locks" aren't really "locks" at
>all,
>
I'm trying think of a good reason why we'd support any kind of
concurrent operation in the WC. If we just serialised everything, "svn
st" and friends could do this cleanup themselves.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Greg Hudson <gh...@MIT.EDU>.
On Mon, 2004-11-22 at 19:51, Bryan Donlan wrote:
> Right, but if the last-modified timestamps of the text-base files were
> compared to the working copy files, instead of the entires file
> timestamps being compared to the wc files, it should be possible to
> update them without taking locks. This would come at the reduction of
> normal-case performance however, as it would have to stat() or
> equivalent twice as often.
I believe I came up with this idea myself a while ago, but it has some
other downsides. A "cp -r" of a working copy could cause the mod times
of the text base and working files to become identical even though the
working file has changed, or non-identical even if it hasn't.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Bryan Donlan <bd...@gmail.com>.
On Mon, 22 Nov 2004 23:37:50 +0000, Philip Martin
<ph...@codematters.co.uk> wrote:
> Bryan Donlan <bd...@gmail.com> writes:
>
> > Would it be possible to use the timestamps in the text-base files as a
> > baseline, instead of data in the entries file?
>
> No. The text-time in the entries file refers to the working file, not
> the text-base.
Right, but if the last-modified timestamps of the text-base files were
compared to the working copy files, instead of the entires file
timestamps being compared to the wc files, it should be possible to
update them without taking locks. This would come at the reduction of
normal-case performance however, as it would have to stat() or
equivalent twice as often.
--
bd
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Philip Martin <ph...@codematters.co.uk>.
Bryan Donlan <bd...@gmail.com> writes:
> Would it be possible to use the timestamps in the text-base files as a
> baseline, instead of data in the entries file?
No. The text-time in the entries file refers to the working file, not
the text-base.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Bryan Donlan <bd...@gmail.com>.
On Mon, 22 Nov 2004 23:01:45 +0000, Philip Martin
<ph...@codematters.co.uk> wrote:
> Julian Foad <ju...@btopenworld.com> writes:
>
> > I was hoping there would be a better (more automatic and invisible)
> > way than any of these four options, like making read-only operations
> > (status, etc.) do the fix-up as and when they notice the discrepancy.
> > In fact, I think it was discussed way back
>
> http://svn.haxx.se/dev/archive-2003-10/1528.shtml
>
> > that they could attempt to
> > convert their read lock to a write lock on discovering the problem,
> > and repair it if the write lock was obtained, else leave it for
> > another time.
>
> I'm still uneasy about the idea. In addition to the problems
> mentioned in the old thread, "read locks" aren't really "locks" at
> all, so any such conversion would need to reread the entries file and
> it could have changed--a tricky thing to handle. Further, write locks
> are responsible for working copy format upgrades, to allow the entries
> file to be rewritten, and having "svn status" do such an upgrade is
> really unfriendly in my opinon, although I suppose we could do the
> read-to-write conversion only if the formats already match.
Would it be possible to use the timestamps in the text-base files as a
baseline, instead of data in the entries file? This way, one would
just need to touch the text-base files, avoiding (most) concurrancy
issues.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Philip Martin <ph...@codematters.co.uk>.
Julian Foad <ju...@btopenworld.com> writes:
> I was hoping there would be a better (more automatic and invisible)
> way than any of these four options, like making read-only operations
> (status, etc.) do the fix-up as and when they notice the discrepancy.
> In fact, I think it was discussed way back
http://svn.haxx.se/dev/archive-2003-10/1528.shtml
> that they could attempt to
> convert their read lock to a write lock on discovering the problem,
> and repair it if the write lock was obtained, else leave it for
> another time.
I'm still uneasy about the idea. In addition to the problems
mentioned in the old thread, "read locks" aren't really "locks" at
all, so any such conversion would need to reread the entries file and
it could have changed--a tricky thing to handle. Further, write locks
are responsible for working copy format upgrades, to allow the entries
file to be rewritten, and having "svn status" do such an upgrade is
really unfriendly in my opinon, although I suppose we could do the
read-to-write conversion only if the formats already match.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Philip Martin <ph...@codematters.co.uk>.
Branko Äibej <br...@xbc.nu> writes:
> But I said before that if we allow non-modifying operations such as
> diff during a modifying operation such as update or commit, then the
> whole thing is broken anyway. We can't allow multiple readers /and/ a
> single writer to access the WC, only multiple readers /or/ a single
> writer. What I'm saying is that the multiple-reader idea isn't really
> useful anyway.
I'm not sure exactly what you are proposing. Do you want readers to
lock the WC? If so, won't that cause a performance problem? If not,
how else do you prevent multiple readers?
What about a WC to which I don't have write access?
Are you proposing that serialisation is the users' responsibility? If
so, how do I use my Subversion aware editor when my Subversion aware
file browser periodically scans the WC?
If readers don't lock the WC then I don't see what serialisation
gains. When a read-only operation determines that it needs write
access to repair a timestamp I'd be very unhappy if it simply assumed
that serialisation has been enforced. At present the algorithm to
upgrade a read-only lock to a write lock would be something like:
if working copy version != SVN_WC__VERSION:
cannot upgrade to write lock
else if cannot get physical lock:
cannot upgrade to write lock
else if entries read from file != entries in memory:
drop physical lock
cannot upgrade to write lock
else
successful upgrade to write lock
If we have "user enforced" serialisation where readers don't take a
physical lock then I think we would still need to use the same
algorithm.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Branko Čibej <br...@xbc.nu>.
Travis P wrote:
> Re: serializing
>
> When I'm committing and want to look at a diff, it's nice to be able
> to 'svn diff' in another terminal. I'd guess I'm not the only one
> that likes to do this. I assuming Serializing would not allow this.
> Also, some people might be doing this under-the-covers, for example,
> using a script like Julian's (
> http://svn.haxx.se/dev/archive-2004-10/0577.shtml ) and unlike him,
> using status and/or diff used to generate a default log message within
> the commit process as part of their editor invocation (rather than
> prior to commit is as Julian mentions is his habit).
But I said before that if we allow non-modifying operations such as diff
during a modifying operation such as update or commit, then the whole
thing is broken anyway. We can't allow multiple readers /and/ a single
writer to access the WC, only multiple readers /or/ a single writer.
What I'm saying is that the multiple-reader idea isn't really useful anyway.
> As an alternative: maybe an operation can reserve the right to repair
> timestamps and other processes would then proceed without bothering if
> that reservation were not available?
Ah, so you'd solve the problem by adding another lock to the WC.//
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Travis P <sv...@castle.fastmail.fm>.
On Nov 24, 2004, at 8:27 AM, Branko Čibej wrote:
> Philip Martin wrote:
>
>> Julian Foad <ju...@btopenworld.com> writes:
>>
>>
>>> Justin Erenkrantz wrote:
>>>
>>>>> - svn cleanup --repair, a new flag
>>>>> - svn cleanup, could be made to do it unconditionally
>>>>> - svn status --repair, a new flag
>>>>> - svn repair, a new function
>>>>>
>>>> I think 'svn cleanup' should do it unconditionally. -- justin
>>>>
>>> But of these four options, I agree that "svn cleanup" is the best,
>>>
>>
>> It's not as good as a fully automatic system, but it's the one I
>> dislike least. I've committed a first attempt at making it happen.
>>
> I still think this is a mistake, that we should serialise all WC
> accesses and repair the timestamps as soon as the difference is
> noticed.
Re: serializing
When I'm committing and want to look at a diff, it's nice to be able to
'svn diff' in another terminal. I'd guess I'm not the only one that
likes to do this. I assuming Serializing would not allow this. Also,
some people might be doing this under-the-covers, for example, using a
script like Julian's (
http://svn.haxx.se/dev/archive-2004-10/0577.shtml ) and unlike him,
using status and/or diff used to generate a default log message within
the commit process as part of their editor invocation (rather than
prior to commit is as Julian mentions is his habit).
As an alternative: maybe an operation can reserve the right to repair
timestamps and other processes would then proceed without bothering if
that reservation were not available?
-Travis
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Branko Čibej <br...@xbc.nu>.
Philip Martin wrote:
>Julian Foad <ju...@btopenworld.com> writes:
>
>
>
>>Justin Erenkrantz wrote:
>>
>>
>>>> - svn cleanup --repair, a new flag
>>>> - svn cleanup, could be made to do it unconditionally
>>>> - svn status --repair, a new flag
>>>> - svn repair, a new function
>>>>
>>>>
>>>I think 'svn cleanup' should do it unconditionally. -- justin
>>>
>>>
>>But of these four options, I agree that "svn cleanup" is the best,
>>
>>
>
>It's not as good as a fully automatic system, but it's the one I
>dislike least. I've committed a first attempt at making it happen.
>
>
I still think this is a mistake, that we should serialise all WC
accesses and repair the timestamps as soon as the difference is noticed.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Philip Martin <ph...@codematters.co.uk>.
Julian Foad <ju...@btopenworld.com> writes:
> Justin Erenkrantz wrote:
>>>
>>> - svn cleanup --repair, a new flag
>>> - svn cleanup, could be made to do it unconditionally
>>> - svn status --repair, a new flag
>>> - svn repair, a new function
>> I think 'svn cleanup' should do it unconditionally. -- justin
>
> But of these four options, I agree that "svn cleanup" is the best,
It's not as good as a fully automatic system, but it's the one I
dislike least. I've committed a first attempt at making it happen.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Julian Foad <ju...@btopenworld.com>.
Justin Erenkrantz wrote:
>>
>> - svn cleanup --repair, a new flag
>> - svn cleanup, could be made to do it unconditionally
>> - svn status --repair, a new flag
>> - svn repair, a new function
>
> I think 'svn cleanup' should do it unconditionally. -- justin
I was hoping there would be a better (more automatic and invisible) way than
any of these four options, like making read-only operations (status, etc.) do
the fix-up as and when they notice the discrepancy. In fact, I think it was
discussed way back that they could attempt to convert their read lock to a
write lock on discovering the problem, and repair it if the write lock was
obtained, else leave it for another time.
But of these four options, I agree that "svn cleanup" is the best, because I
really don't want to see any new commands or options for a "feature" that
shouldn't need to exist (from a high-level-logic point of view). It is just
the sort of clean-up that the "cleanup" command is intuitively meant to
perform, so a "--repair" flag or "repair" command would be superfluous and
ambiguous. "status --repair" is illogical because the problem is not
specifically related to "status", though it may be most noticeable in "status".
- Julian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
--On Wednesday, November 17, 2004 8:05 PM +0000 Philip Martin
<ph...@codematters.co.uk> wrote:
> Subversion operations will "repair" the time stamp when both a) the
> operation detects the problem, and b) the operation has a write lock,
> however not many operations meet both criteria. A better interface
> has been discussed in the past, but no conclusion was reached. I seem
> to remember suggesting
>
> - svn cleanup --repair, a new flag
> - svn cleanup, could be made to do it unconditionally
> - svn status --repair, a new flag
> - svn repair, a new function
I think 'svn cleanup' should do it unconditionally. -- justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: repair timestamps of unchanged files
Posted by Philip Martin <ph...@codematters.co.uk>.
Eric Noel <er...@rcs100.com> writes:
> This was reported in 2002 as issue #791. Has anyone revisited this
> since then? If you've got a bunch of unchanged files that get an
> updated timestamp (through touch, whatever), then svn becomes a huge
> dog... It was taking 17 minutes for an update. I've fixed the problem
> in my case, but it seems to me that a built-in way to correct for this
> would be very appreciated.
Subversion operations will "repair" the time stamp when both a) the
operation detects the problem, and b) the operation has a write lock,
however not many operations meet both criteria. A better interface
has been discussed in the past, but no conclusion was reached. I seem
to remember suggesting
- svn cleanup --repair, a new flag
- svn cleanup, could be made to do it unconditionally
- svn status --repair, a new flag
- svn repair, a new function
I think "svn revert --recursive" will make any repairs, but it will
also delete any local changes. I use
$ SVN_EDITOR=/bin/false svn commit
We really need a better interface.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org