Call svn cat for all files modified or added in a given revision

[OBones <ob...@free.fr>]

Hello all,

Using svn cat I can see the state of a file at a given revision.
As it turns out, I have had in the past some files that were commit with 
inconsistent line endings and that are making analysis tools stop 
because of this.
What I want to do is find which (file, revision) pairs are impacted so 
that I can go back in the past and fix those invalid commits (using 
dump/reload).

However, I'm having a hard time finding a way to extract a list of 
modified or added files in a given revision so that I could give that 
list to the svn cat command that stops with an error when the above 
situation arises.
And then, once this works, I'd put all this in a loop going from the 
first revision to the last one.

I don't mind if this takes time, and I have no preference as to what 
operating system this should run under between Linux and Windows
I believe using Linux would be much simpler, but still, I can't figure 
out how to parse the result of svn log -v so that I can extract the 
filenames to give to svn cat.
If need be, I can work out of a working copy.

I would very much appreciate any help on this matter

Regards
Olivier

Re: Call svn cat for all files modified or added in a given revision

[OBones <ob...@free.fr>]

Andy Levy wrote:
> svn log --verbose -rX will give you a list of all paths touched in 
> revision X. You'll need to do some parsing to extract just the paths 
> vs. the other log data. If you're handy with XML, you can use svn log 
> --verbose --xml -rX and then use XPath to extract just the paths. 
I have taken this route and came up with the script below.
It might not be the best script ever, but it worked just fine for me, 
allowing to discover that there was only one revision in the entire 
history that was causing issue with svn cat.

Regards
Olivier

---------------------------------------------------

#!/bin/bash

# for this to work, you have to use xpathtool from the following page:
# http://www.semicomplete.com/projects/xpathtool/

# set those variables to match your installation
BASEURL=https://jvcl.svn.sourceforge.net/svnroot/jvcl
USER=
PASS=
MINREV=13281
XPATHTOOLDIR=~/xpathtool-20071102/xpathtool

echo Retrieving maximum revision
MAXREV=`svn log -r HEAD --xml --username=$USER --password=$PASS 
--no-auth-cache $BASEURL | $XPATHTOOLDIR/xpathtool.sh 
"/log/logentry/@revision"`

REV=$MINREV
while [ $REV -le $MAXREV ]; do
         echo Processing revision $REV out of $MAXREV

         FILES=`svn log --verbose -r $REV --xml --username=$USER 
--password=$PASS --no-auth-cache $BASEURL | $XPATHTOOLDIR/xpathtool.sh 
"/log/logentry/paths/path[@kind='file'][@action!='D']"`

         IFS=$'\n'
         for FILE in $FILES
         do
                 svn cat --username=$USER --password=$PASS 
--no-auth-cache $BASEURL$FILE@$REV > /dev/null
                 if [ $? -ne 0 ]; then
                         echo File $FILE at revision $REV gave an error
                 fi
         done
         let REV=REV+1
done

Re: Call svn cat for all files modified or added in a given revision

[OBones <ob...@free.fr>]

Andy Levy wrote:
> On Wed, Mar 21, 2012 at 08:02, OBones<ob...@free.fr>  wrote:
>> Andy Levy wrote:
>>> On Tue, Mar 20, 2012 at 10:39, OBones<ob...@free.fr>    wrote:
>>>> Hello all,
>>>>
>>>> Using svn cat I can see the state of a file at a given revision.
>>>> As it turns out, I have had in the past some files that were commit with
>>>> inconsistent line endings and that are making analysis tools stop because
>>>> of
>>>> this.
>>> Can you change analysis tools to one that can handle this condition?
>> Alas no, it's what ohloh.net is using...
> I'd suggest filing a bug report there then. Depending on the analysis
> being done, whitespace might be something that can be ignored or
> relaxed to stop this from breaking.
Actually, I went there directly before asking on this list and the 
answers I was given were that they could not change their process 
because their using "svn cat" and it's "svn cat" that is giving the error.

I'll try to insist on this, but I'm not really confident as to a 
positive issue...

Re: Call svn cat for all files modified or added in a given revision

[Andy Levy <an...@gmail.com>]

On Wed, Mar 21, 2012 at 08:02, OBones <ob...@free.fr> wrote:
> Andy Levy wrote:
>>
>> On Tue, Mar 20, 2012 at 10:39, OBones<ob...@free.fr>  wrote:
>>>
>>> Hello all,
>>>
>>> Using svn cat I can see the state of a file at a given revision.
>>> As it turns out, I have had in the past some files that were commit with
>>> inconsistent line endings and that are making analysis tools stop because
>>> of
>>> this.
>>
>> Can you change analysis tools to one that can handle this condition?
>
> Alas no, it's what ohloh.net is using...

I'd suggest filing a bug report there then. Depending on the analysis
being done, whitespace might be something that can be ignored or
relaxed to stop this from breaking.

Re: Call svn cat for all files modified or added in a given revision

[OBones <ob...@free.fr>]

Andy Levy wrote:
> On Tue, Mar 20, 2012 at 10:39, OBones<ob...@free.fr>  wrote:
>> Hello all,
>>
>> Using svn cat I can see the state of a file at a given revision.
>> As it turns out, I have had in the past some files that were commit with
>> inconsistent line endings and that are making analysis tools stop because of
>> this.
> Can you change analysis tools to one that can handle this condition?
Alas no, it's what ohloh.net is using...

>> What I want to do is find which (file, revision) pairs are impacted so that
>> I can go back in the past and fix those invalid commits (using dump/reload).
>>
>> However, I'm having a hard time finding a way to extract a list of modified
>> or added files in a given revision so that I could give that list to the svn
>> cat command that stops with an error when the above situation arises.
>> And then, once this works, I'd put all this in a loop going from the first
>> revision to the last one.
> svn log --verbose -rX will give you a list of all paths touched in
> revision X. You'll need to do some parsing to extract just the paths
> vs. the other log data. If you're handy with XML, you can use svn log
> --verbose --xml -rX and then use XPath to extract just the paths.
>
> To parse the output of svn log --verbose for this project, look for
> any line starting with one or more letters and/or spaces, followed by
> a space, followed by a slash. From that space to the end of the line
> is the path that was touched. The regex should be something close to
> this:
>
> \b[\w\s]+(\/*)\b
Thanks for the hints, I'll try to manage something out.

Re: Call svn cat for all files modified or added in a given revision

[Andy Levy <an...@gmail.com>]

On Tue, Mar 20, 2012 at 10:39, OBones <ob...@free.fr> wrote:
> Hello all,
>
> Using svn cat I can see the state of a file at a given revision.
> As it turns out, I have had in the past some files that were commit with
> inconsistent line endings and that are making analysis tools stop because of
> this.

Can you change analysis tools to one that can handle this condition?

> What I want to do is find which (file, revision) pairs are impacted so that
> I can go back in the past and fix those invalid commits (using dump/reload).
>
> However, I'm having a hard time finding a way to extract a list of modified
> or added files in a given revision so that I could give that list to the svn
> cat command that stops with an error when the above situation arises.
> And then, once this works, I'd put all this in a loop going from the first
> revision to the last one.

svn log --verbose -rX will give you a list of all paths touched in
revision X. You'll need to do some parsing to extract just the paths
vs. the other log data. If you're handy with XML, you can use svn log
--verbose --xml -rX and then use XPath to extract just the paths.

To parse the output of svn log --verbose for this project, look for
any line starting with one or more letters and/or spaces, followed by
a space, followed by a slash. From that space to the end of the line
is the path that was touched. The regex should be something close to
this:

\b[\w\s]+(\/*)\b