You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Cedric Williams <Ce...@tceq.state.tx.us> on 2003/07/25 19:52:48 UTC
Re: How to backup a running script? and how to recover
out-of-sync file?
[cross-posted from users@]
>>> jrepenning@collab.net 07/25/03 01:32PM >>>
>Well, I think I'd start by checking out a fresh, new working copy,
>and see what sort of state it's in. Then you can make your changes
>as necessary. The problems here seem to be limited to the particular
>WC.
>
>But you've still got the more fundamental problem: it's not working
>to "self-commit" a script if it has keywords to expand. I don't
>think that's going to go away. Checking to see whether the file's
>writable might be done, but it would only allow SVN to refuse the
>checkin, which ain't necessarily what you had in mind!
Ok - I'm back in business. The trick was "fresh working copy." I checked out a new WC from the repository to another directory. The daily.sh in the WC was the HEAD version of the script, but the .svn/text-base version (of the new WC) matched the checksum that the 1st WC expected in the .svn/entries file. I copied text-base/daily.sh.svn-base from the new WC to the same location in the old WC, and tried the commit. Worked just fine.
=== Problem Recap
1 - I have a shell script that I use to back up a bunch of different working copies to my repository at the end of the day
2 - This script had the $Id$ keyword tag embedded in a comment
3 - After setting svn propset svn:keywords "Id" on the script I called it to commit itself to the repository.
4 - svn committed the script to the repository (the new WC has the expanded $Id$ tag embedded)
5 - After the commit succeeded, svn changed the .svn/entries file to have the checksum information for the checked-in version of the script (which has the $Id$ tag, though not expanded since the repository stores the tag info in the props)
6 - Then svn tried to replace the current WC version with the repository version + expanded tag - **this failed** (script was in use at the time and could not be overwritten - but this was not the real problem)
7 - svn failed, notified the bonehead user, and quit
8 - svn never copied the repository version to .svn/text-base, but left the prior version of the script (now revision HEAD - 2) THIS was the real problem - the .svn/entries file had the checksum for the HEAD - 1 copy, but .svn/text-base had the version of the file prior to that on the drive
=== A Fix that Worked but will Probably Give Jack Heartburn
9 - check out a new WC in another location
10 - copy the (repository version) file from .svn/text-base/ in the new WC to the .svn/text-base in the old WC
11 - check in the changed file in the old WC
12 - do the happy dance (all is now well)
=== Thoughts / Questions
- Another thing that might have worked (for svn) would be to "export" the version of the file that it thinks should be in text-base/ (HEAD -1) from the repository. That copy has the checksum that it is looking for from the entries file, but it doesn't tell the bonehead user why or how something went wrong
A - Should svn handle the move-previous-to-text-base before trying to do keyword expansion (since that might fail)?
B - Should svn cleanup be able to recognize / handle the situation when a checkin hasn't completed moving files around, and implement (or suggest) some strategy to get the .svn directory back in shape?
C - Attempting to obtain a write-lock on a file that will be updated (with keyword expansion, revert, or update) will change the "last-modified" date, even if no revisions are made, wont it?
C(1) - Even if the "last-modified" date changed when attempting a write-lock on a file, would this be a better indication that the keyword-expansion would succeed? (i.e., wouldn't svn have to hold that write lock through the whole commit process - not a great idea!)
D - Should this (ability to corrupt a WC this way) be added as an issue?
Ok - that's probably enough trouble stirred up for now... Thanks for all the great work on Subversion, everyone, and thanks for all the help Jack!
--Cedric
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: How to backup a running script? and how to recover out-of-sync
file?
Posted by Philip Martin <ph...@codematters.co.uk>.
Jack Repenning <jr...@collab.net> writes:
> At 2:43 AM +0100 7/26/03, Philip Martin wrote:
>>After the failed commit was the working copy locked, that's a
>>Subversion lock where 'svn st' shows state 'L'?
>
> This wasn't actually a failed commit, I think.
By "failed commit" I meant "'svn commit' producing an error".
> As the messages
> announced, the commit itself worked, it was only the fix-up of the WC
> (and text-base) that choked.
I realised that, I thought I explained the failure mechanism in my
mail :-/
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: How to backup a running script? and how to recover
out-of-sync file?
Posted by Jack Repenning <jr...@collab.net>.
At 2:43 AM +0100 7/26/03, Philip Martin wrote:
>After the failed commit was the working copy locked, that's a
>Subversion lock where 'svn st' shows state 'L'?
This wasn't actually a failed commit, I think. As the messages
announced, the commit itself worked, it was only the fix-up of the WC
(and text-base) that choked.
--
-==-
Jack Repenning
CollabNet, Inc.
8000 Marina Boulevard, Suite 600
Brisbane, California 94005
o: 650.228.2562
c: 408.835-8090
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: How to backup a running script? and how to recover
out-of-sync file?
Posted by Jack Repenning <jr...@collab.net>.
At 2:43 AM +0100 7/26/03, Philip Martin wrote:
>After the failed commit was the working copy locked, that's a
>Subversion lock where 'svn st' shows state 'L'?
This wasn't actually a failed commit, I think. As the messages
announced, the commit itself worked, it was only the fix-up of the WC
(and text-base) that choked.
--
-==-
Jack Repenning
CollabNet, Inc.
8000 Marina Boulevard, Suite 600
Brisbane, California 94005
o: 650.228.2562
c: 408.835-8090
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: How to backup a running script? and how to recover out-of-sync
file?
Posted by Philip Martin <ph...@codematters.co.uk>.
"Cedric Williams" <Ce...@tceq.state.tx.us> writes:
> 5 - After the commit succeeded, svn changed the .svn/entries file to
> have the checksum information for the checked-in version of the
> script (which has the $Id$ tag, though not expanded since the
> repository stores the tag info in the props)
> 6 - Then svn tried to replace the current WC version with the
> repository version + expanded tag - **this failed** (script was in
> use at the time and could not be overwritten - but this was not the
> real problem)
> 7 - svn failed, notified the bonehead user, and quit
> 8 - svn never copied the repository version to .svn/text-base, but
> left the prior version of the script (now revision HEAD - 2) THIS
> was the real problem - the .svn/entries file had the checksum for
> the HEAD - 1 copy, but .svn/text-base had the version of the file
> prior to that on the drive
[...]
> A - Should svn handle the move-previous-to-text-base before trying
> to do keyword expansion (since that might fail)?
> B - Should svn cleanup be able to recognize / handle the situation
> when a checkin hasn't completed moving files around, and implement
> (or suggest) some strategy to get the .svn directory back in shape?
After the failed commit was the working copy locked, that's a
Subversion lock where 'svn st' shows state 'L'?
In principle Subversion's log file mechanism should have meant that
the failed commit left the working copy locked. However I think I see
a bug that would have unlocked the working copy. Look at the
svn_wc_adm_close at the end of svn_client_commit, that will have the
effect of removing the lock even if an error occurred. There is a
comment there (one I think I added when I did the access baton
conversion) questioning the logic. I think the correct behaviour in
your case would be for the lock, and log file, to remain. Then there
is a chance that 'svn cleanup' would have fixed the working copy.
I don't really understand why there is special 'cleanup:' handling in
svn_client_commit, I guess it is to ensure that a failed commit always
calls abort_edit. Anyone know the history? I think it is an error
both to remove the locks, and to clear the temporary files, if a
commit error occurs.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: How to backup a running script? and how to recover
out-of-sync file?
Posted by Jack Repenning <jr...@collab.net>.
At 2:52 PM -0500 7/25/03, Cedric Williams wrote:
>=== A Fix that Worked but will Probably Give Jack Heartburn
You got that right! But I'm glad you're back in business.
>9 - check out a new WC in another location
>10 - copy the (repository version) file from .svn/text-base/ in the
>new WC to the .svn/text-base in the old WC
Why not copy the changed file into the new WC, and check in and happy
dance, without the unauthorized brain surgery on the text base?
>A - Should svn handle the move-previous-to-text-base before trying
>to do keyword expansion (since that might fail)?
Well, that's not a "move-previous", in the context of commit, that's
a "move-new": by the time the commit dust settles, the text base
should be identical to the working file. In particular for this
case, it should have the new version of the keyword expansions. So
if we suppose it were to be made before keyword expansion, then we'd
also have to suppose that the keyword expansion would be applied to
it, independently--seems suboptimal.
>B - Should svn cleanup be able to recognize / handle the situation
>when a checkin hasn't completed moving files around, and implement
>(or suggest) some strategy to get the .svn directory back in shape?
Perhaps. An RFE to that effect would certainly not be unreasonable.
>C - Attempting to obtain a write-lock on a file that will be updated
>(with keyword expansion, revert, or update) will change the
>"last-modified" date, even if no revisions are made, wont it?
Beats me. I don't think we've demonstrated yet that this even
happens. If you can provide a reproduction recipe that shows that it
does happen, that would be interesting. It would be tacky to diddle
the modtime just for commiting a file (without expanding keywords);
that would, e.g., force unnecessary rebuilds in the WC.
>C(1) - Even if the "last-modified" date changed when attempting a
>write-lock on a file, would this be a better indication that the
>keyword-expansion would succeed? (i.e., wouldn't svn have to hold
>that write lock through the whole commit process - not a great idea!)
You've lost me. Your problem was that Windows (on behalf of the
shell) had the write lock long before any Subversion code got a
chance to run, and held it until long after the last svn action had
flickered and died. Svn never got the lock; Windows blocked it. so
what's all this questioning about svn holding it?
--
-==-
Jack Repenning
CollabNet, Inc.
8000 Marina Boulevard, Suite 600
Brisbane, California 94005
o: 650.228.2562
c: 408.835-8090
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: How to backup a running script? and how to recover out-of-sync
file?
Posted by Philip Martin <ph...@codematters.co.uk>.
"Cedric Williams" <Ce...@tceq.state.tx.us> writes:
>>But you've still got the more fundamental problem: it's not working
>>to "self-commit" a script if it has keywords to expand. I don't
>>think that's going to go away. Checking to see whether the file's
>>writable might be done, but it would only allow SVN to refuse the
>>checkin, which ain't necessarily what you had in mind!
There is not much we can do if the OS prevents the client from
modifying a file in the working copy, that's always going to produce
an error. However if you can a) reproduce the problem, and b) build a
HEAD (later than revision 6584) client, then you should find that the
behaviour has improved.
Now when you get the error from "svn commit" the working copy will
remain locked. You will be unable to update or commit that working
copy until you successfully run "svn cleanup". For that to happen you
will need to remove the underlying problem (the inability to overwrite
the file), if you try to run cleanup before that it will fail and the
working copy will remain locked.
Once cleanup is successful the working copy will be useable again. It
may be out of date in places, that depends on where the original error
occurred during the post-commit processing, but an "svn update" will
fix that.
I don't have a Windows machine to test your exact scenario, but I can
reproduce something similar on Unix by using a debugger to simulate
errors.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: How to backup a running script? and how to recover
out-of-sync file?
Posted by Jack Repenning <jr...@collab.net>.
At 2:52 PM -0500 7/25/03, Cedric Williams wrote:
>=== A Fix that Worked but will Probably Give Jack Heartburn
You got that right! But I'm glad you're back in business.
>9 - check out a new WC in another location
>10 - copy the (repository version) file from .svn/text-base/ in the
>new WC to the .svn/text-base in the old WC
Why not copy the changed file into the new WC, and check in and happy
dance, without the unauthorized brain surgery on the text base?
>A - Should svn handle the move-previous-to-text-base before trying
>to do keyword expansion (since that might fail)?
Well, that's not a "move-previous", in the context of commit, that's
a "move-new": by the time the commit dust settles, the text base
should be identical to the working file. In particular for this
case, it should have the new version of the keyword expansions. So
if we suppose it were to be made before keyword expansion, then we'd
also have to suppose that the keyword expansion would be applied to
it, independently--seems suboptimal.
>B - Should svn cleanup be able to recognize / handle the situation
>when a checkin hasn't completed moving files around, and implement
>(or suggest) some strategy to get the .svn directory back in shape?
Perhaps. An RFE to that effect would certainly not be unreasonable.
>C - Attempting to obtain a write-lock on a file that will be updated
>(with keyword expansion, revert, or update) will change the
>"last-modified" date, even if no revisions are made, wont it?
Beats me. I don't think we've demonstrated yet that this even
happens. If you can provide a reproduction recipe that shows that it
does happen, that would be interesting. It would be tacky to diddle
the modtime just for commiting a file (without expanding keywords);
that would, e.g., force unnecessary rebuilds in the WC.
>C(1) - Even if the "last-modified" date changed when attempting a
>write-lock on a file, would this be a better indication that the
>keyword-expansion would succeed? (i.e., wouldn't svn have to hold
>that write lock through the whole commit process - not a great idea!)
You've lost me. Your problem was that Windows (on behalf of the
shell) had the write lock long before any Subversion code got a
chance to run, and held it until long after the last svn action had
flickered and died. Svn never got the lock; Windows blocked it. so
what's all this questioning about svn holding it?
--
-==-
Jack Repenning
CollabNet, Inc.
8000 Marina Boulevard, Suite 600
Brisbane, California 94005
o: 650.228.2562
c: 408.835-8090
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: How to backup a running script? and how to recover out-of-sync
file?
Posted by Philip Martin <ph...@codematters.co.uk>.
"Cedric Williams" <Ce...@tceq.state.tx.us> writes:
> 5 - After the commit succeeded, svn changed the .svn/entries file to
> have the checksum information for the checked-in version of the
> script (which has the $Id$ tag, though not expanded since the
> repository stores the tag info in the props)
> 6 - Then svn tried to replace the current WC version with the
> repository version + expanded tag - **this failed** (script was in
> use at the time and could not be overwritten - but this was not the
> real problem)
> 7 - svn failed, notified the bonehead user, and quit
> 8 - svn never copied the repository version to .svn/text-base, but
> left the prior version of the script (now revision HEAD - 2) THIS
> was the real problem - the .svn/entries file had the checksum for
> the HEAD - 1 copy, but .svn/text-base had the version of the file
> prior to that on the drive
[...]
> A - Should svn handle the move-previous-to-text-base before trying
> to do keyword expansion (since that might fail)?
> B - Should svn cleanup be able to recognize / handle the situation
> when a checkin hasn't completed moving files around, and implement
> (or suggest) some strategy to get the .svn directory back in shape?
After the failed commit was the working copy locked, that's a
Subversion lock where 'svn st' shows state 'L'?
In principle Subversion's log file mechanism should have meant that
the failed commit left the working copy locked. However I think I see
a bug that would have unlocked the working copy. Look at the
svn_wc_adm_close at the end of svn_client_commit, that will have the
effect of removing the lock even if an error occurred. There is a
comment there (one I think I added when I did the access baton
conversion) questioning the logic. I think the correct behaviour in
your case would be for the lock, and log file, to remain. Then there
is a chance that 'svn cleanup' would have fixed the working copy.
I don't really understand why there is special 'cleanup:' handling in
svn_client_commit, I guess it is to ensure that a failed commit always
calls abort_edit. Anyone know the history? I think it is an error
both to remove the locks, and to clear the temporary files, if a
commit error occurs.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org