You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by David Budworth <db...@eloan.com> on 2004/04/07 20:35:47 UTC
Possible bug in keyword replacement (revisited, with examples)
Sorry about reporting with less concrete info before, hope this is more
clear.
Attached is the exact script that generated this output. (works on
linux, not sure for cygwin).
I'm trying to re-produce my merge shows extra files issue, but it seems
to be something specific with my tree as I can't create the problem
from scratch.
Environment: MDK 10, svn 1.0.1. Also happens with webdab/apache and
other linux'
basic summary: If you turn on keyword substitution for 'Id', and add
the tag to your file as
$Id:$ (as in, the way CVS requires, therefore coded in many template
classes for us)
You will get diffs and conflicting merge updates on the $Id$ tag itself.
It appears that it gets stored in the repo as $Id:$, but when doing a
diff, the client strips it down to $Id$ and sees it as a conflicting
change.
Note: purely adding a space after the colon "$Id: $" makes this go
away, as well as leaving the colon out all together.
session is as follows (it's just a bash -v script, so it outputs
everything done):
[svn@svn test]$ ./keyword_issue.sh
#!/bin/bash -v
cd `dirname $0`
dirname $0
BASE=`pwd`
pwd
## setup vars to make rest of script easier to read
REPODIR=$BASE/keyword_repo
REPO=file://$REPODIR
WC=$BASE/keyword-wc
## (re)create repository
if [ -d $REPODIR ]; then rm -r $REPODIR; fi
if [ -d $WC ]; then rm -rf $WC; fi
svnadmin create $REPODIR
## get WC and add files to it
svn co $REPO $WC
Checked out revision 0.
cd $WC
echo 'File #1 $Id:$' > file1.txt
echo 'File #2 $Id: $' > file2.txt
svn add file?.txt
A file1.txt
A file2.txt
## enable keyword substitution
svn pset svn:keywords Id file?.txt
property 'svn:keywords' set on 'file1.txt'
property 'svn:keywords' set on 'file2.txt'
## commit changes
svn commit -m 'initial tree/files'
Adding file1.txt
Adding file2.txt
Transmitting file data ..
Committed revision 1.
## touch both files to force subversion to diff on status
touch file?.txt
## Perform a diff, file1.txt shows a diff on the keyword
svn diff
Index: file1.txt
===================================================================
--- file1.txt (revision 1)
+++ file1.txt (working copy)
@@ -1 +1 @@
-File #1 $Id:$
+File #1 $Id$
[svn@svn test]$
Re: Possible bug in keyword replacement (revisited, with examples)
Posted by David Budworth <db...@eloan.com>.
Just following up my post for those that may experience the same
problem and are looking for a solution in the mail archives.
Turns out, easiest thing was doing:
#get a copy of trunk
svn co http://repo/project/trunk
#touch all files that could possibly have bad tags (xml/java only in my
case)
find trunk -name "*.java" -exec touch {} \;
find trunk -name "*.xml" -exec touch {} \;
#svn stat will show "M"odified files for all the ones which have the
bad $Id:$ tags in the
#repo.
svn commit trunk -m 'fixing bad tags'
This solves the problem as the newly checked in files will get their
tags set properly on that revision. You are still kinda screwed if you
need to diff old versions vs head as they will still show as diffs.
But at least you can avoid the problem moving forward.
-David
On Apr 27, 2004, at 11:39 AM, David Budworth wrote:
> Resurrecting an old topic here...
>
> Given that my repository is littered with these tags, we'd like to go
> through and fix them all.
>
> Anyone has a good idea on how to do this en mass?
>
> At first i tried to do:
> svnadmin dump badrepo | perl -pe 's/\$Id:\$/\$Id\$/g' | svnadmin load
> goodrepo
>
> but of course, that doesn't work (didn't even think about md5
> hashes/content length on the items).
>
> Next best thing I can think of is to write a script that either
> a) modifies the dump file, properly setting the md5 and content length
> b) just scans for bad id's and logs the file names, then we go edit
> the source itself
>
> There are hundreds of offending files, and 400mb of repo. So I was
> hoping to do it by editing the stream, but I don't see how that can
> really be an option.
>
> Is there a more proper way to do this?
>
> We are willing to kill off all our branches to get this resolved. So
> maybe there's a way to force-recommit the entire tree? Since it seems
> that re-commiting the offending file seems to fix the repo version.
>
> On Apr 7, 2004, at 7:22 PM, John Peacock wrote:
>
>> Philip Martin wrote:
>>> John Peacock <jp...@rowman.com> writes:
>>>> Not really. If you dump the repository created using the
>>>> reproduction
>>>> recipe given in the issue:
>>>> ...
>>>>
>>>> You'll see that what is in the repository is not the expanded
>>>> keyword,
>>>> but a stub that looks like an keyword (for the first two). foo3
>>>> does
>>>> not contain a keyword at all, and will not be expanded in the WC.
>>> All I meant by "expanded" is that it had not been "detranslated".
>>> Change foo3 to be "$Id: xxx $" and now it's a keyword and it shows up
>>> in the diff.
>>
>> I just noticed that an additional commit will cleanse the taint (i.e.
>> unexpand the keywords in the repository), i.e. all files when dumped
>> will contain $Id$ and nothing more. This is a more subtle bug...
>>
>> John
>>
>> --
>> John Peacock
>> Director of Information Research and Technology
>> Rowman & Littlefield Publishing Group
>> 4720 Boston Way
>> Lanham, MD 20706
>> 301-459-3366 x.5010
>> fax 301-429-5747
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Possible bug in keyword replacement (revisited, with examples)
Posted by David Budworth <db...@eloan.com>.
Resurrecting an old topic here...
Given that my repository is littered with these tags, we'd like to go
through and fix them all.
Anyone has a good idea on how to do this en mass?
At first i tried to do:
svnadmin dump badrepo | perl -pe 's/\$Id:\$/\$Id\$/g' | svnadmin load
goodrepo
but of course, that doesn't work (didn't even think about md5
hashes/content length on the items).
Next best thing I can think of is to write a script that either
a) modifies the dump file, properly setting the md5 and content length
b) just scans for bad id's and logs the file names, then we go edit the
source itself
There are hundreds of offending files, and 400mb of repo. So I was
hoping to do it by editing the stream, but I don't see how that can
really be an option.
Is there a more proper way to do this?
We are willing to kill off all our branches to get this resolved. So
maybe there's a way to force-recommit the entire tree? Since it seems
that re-commiting the offending file seems to fix the repo version.
On Apr 7, 2004, at 7:22 PM, John Peacock wrote:
> Philip Martin wrote:
>> John Peacock <jp...@rowman.com> writes:
>>> Not really. If you dump the repository created using the
>>> reproduction
>>> recipe given in the issue:
>>> ...
>>>
>>> You'll see that what is in the repository is not the expanded
>>> keyword,
>>> but a stub that looks like an keyword (for the first two). foo3 does
>>> not contain a keyword at all, and will not be expanded in the WC.
>> All I meant by "expanded" is that it had not been "detranslated".
>> Change foo3 to be "$Id: xxx $" and now it's a keyword and it shows up
>> in the diff.
>
> I just noticed that an additional commit will cleanse the taint (i.e.
> unexpand the keywords in the repository), i.e. all files when dumped
> will contain $Id$ and nothing more. This is a more subtle bug...
>
> John
>
> --
> John Peacock
> Director of Information Research and Technology
> Rowman & Littlefield Publishing Group
> 4720 Boston Way
> Lanham, MD 20706
> 301-459-3366 x.5010
> fax 301-429-5747
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Possible bug in keyword replacement (revisited, with examples)
Posted by John Peacock <jp...@rowman.com>.
Philip Martin wrote:
> John Peacock <jp...@rowman.com> writes:
>
>
>>Not really. If you dump the repository created using the reproduction
>>recipe given in the issue:
>>...
>>
>>You'll see that what is in the repository is not the expanded keyword,
>>but a stub that looks like an keyword (for the first two). foo3 does
>>not contain a keyword at all, and will not be expanded in the WC.
>
>
> All I meant by "expanded" is that it had not been "detranslated".
> Change foo3 to be "$Id: xxx $" and now it's a keyword and it shows up
> in the diff.
>
I just noticed that an additional commit will cleanse the taint (i.e. unexpand
the keywords in the repository), i.e. all files when dumped will contain $Id$
and nothing more. This is a more subtle bug...
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Possible bug in keyword replacement (revisited, with examples)
Posted by Philip Martin <ph...@codematters.co.uk>.
John Peacock <jp...@rowman.com> writes:
> Not really. If you dump the repository created using the reproduction
> recipe given in the issue:
> ...
>
> You'll see that what is in the repository is not the expanded keyword,
> but a stub that looks like an keyword (for the first two). foo3 does
> not contain a keyword at all, and will not be expanded in the WC.
All I meant by "expanded" is that it had not been "detranslated".
Change foo3 to be "$Id: xxx $" and now it's a keyword and it shows up
in the diff.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Possible bug in keyword replacement (revisited, with examples)
Posted by John Peacock <jp...@rowman.com>.
Philip Martin wrote:
> John Peacock <jp...@rowman.com> writes:
>
>
>>It would appear that the diff code is treating it as an expanded
>>keyword and collapsing it in error.
>
>
> It's the commit code rather than the diff code. I've raised issue
> 1823.
>
Not really. If you dump the repository created using the reproduction recipe
given in the issue:
...
Node-path: foo1
Node-kind: file
Node-action: add
Prop-content-length: 10
Text-content-length: 7
Text-content-md5: f30442dd0b44c51adfd4656d363e671d
Content-length: 17
PROPS-END
$Id: $
Node-path: foo2
Node-kind: file
Node-action: add
Prop-content-length: 10
Text-content-length: 6
Text-content-md5: 0b3dea829a3557a69caed1bb464064a3
Content-length: 16
PROPS-END
$Id:$
Node-path: foo3
Node-kind: file
Node-action: add
Prop-content-length: 10
Text-content-length: 10
Text-content-md5: 3fe6718bbb5be7e768ae636905c222ff
Content-length: 20
PROPS-END
$Id: xxx$
...
You'll see that what is in the repository is not the expanded keyword, but a
stub that looks like an keyword (for the first two). foo3 does not contain a
keyword at all, and will not be expanded in the WC.
This is going to be a tricky one to fix, because the illegal keyword string can
be stored in the repository prior to the svn:keywords property being set (as you
did in the reproduction recipe). I think the appropriate response would be to
special case '$keyword:$' and '$keyword: $' in the same fashion, i.e. there will
be no keyword expansion in either case, since the keyword text is ill-formed in
the repository (per the book definitions).
However, for compatibility purposes, I believe both of those first two cases
should throw a warning that the keyword _cannot_ be expanded. The third one
should continue to be ignored as outside of our control.
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Possible bug in keyword replacement (revisited, with examples)
Posted by Philip Martin <ph...@codematters.co.uk>.
John Peacock <jp...@rowman.com> writes:
> It would appear that the diff code is treating it as an expanded
> keyword and collapsing it in error.
It's the commit code rather than the diff code. I've raised issue
1823.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Possible bug in keyword replacement (revisited, with examples)
Posted by John Peacock <jp...@rowman.com>.
David Budworth wrote:
> basic summary: If you turn on keyword substitution for 'Id', and add
> the tag to your file as
> $Id:$ (as in, the way CVS requires, therefore coded in many template
> classes for us)
> You will get diffs and conflicting merge updates on the $Id$ tag itself.
>
> It appears that it gets stored in the repo as $Id:$, but when doing a
> diff, the client strips it down to $Id$ and sees it as a conflicting
> change.
>
> Note: purely adding a space after the colon "$Id: $" makes this go away,
> as well as leaving the colon out all together.
>
This is mostly driver error (but the code could be improved to throw an error).
None of the special svn:keywords can include a colon between the $'s in the
repository (CVS behavior notwithstanding). The only time a colon (and trailing
space and additional text) will appear is when the keyword is expanded by the
client and then only in the WC. I think that there is some code which should
realize that this is not an expanded keyword (even though it looks like one),
but it may be failing.
It would appear that the diff code is treating it as an expanded keyword and
collapsing it in error. I would be curious whether the client expands it at all
when svn:keywords is set. It may be that only the diff code needs to be fixed.
I'll take a look tonight if I get a chance (since I am working on replacing
the existing svn:keywords code anyways).
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Possible bug in keyword replacement (revisited, with examples)
Posted by Jeremy Pereira <je...@jeremyp.net>.
On Apr 7, 2004, at 21:35, David Budworth wrote:
>
> basic summary: If you turn on keyword substitution for 'Id', and add
> the tag to your file as
> $Id:$ (as in, the way CVS requires, therefore coded in many template
> classes for us)
>
Minor point, but CVS does not require $Id:$. In fact the CVS book says
use $Id$.
>
--
Jeremy Pereira
http://www.jeremyp.net
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org