You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Jason Winnebeck <jp...@rit.edu> on 2007/07/09 19:41:06 UTC
svn mv "breaks" annotate?
I restructured a repository from a "normal" layout to a
"trunk/tags/branches" layout, but now annotate shows as if the entire
file was updated by myself on almost all files.
The repository was just converted from SourceSafe, using the excellent
vss2svn script. That process seemed to work as good as expected. So
then, I wanted to immediately restructure the projects to have a
"trunk/tags/branches" format. However, this seems to have broken
annotate somehow. What happens is that I am shown as the latest author
on all lines with the revision number being the number of the move, on
almost all files. But **some** files show the annotate as expected. Also
the log of the move showed a lot of A, D, and M statuses, and when
committing it took a long time for "sending file data." The size of the
transaction in the FSFS is 6MB, but I thought copies were "free" in SVN.
This is why I am thinking I might have done something wrong.
The two interesting points are that the last line of the files show a
different author, and that the only changes in the files were in the
$Revision$ keyword expansion.
I used the following shell script on the root directory of each project
in the repository:
cd "$1"
svn mkdir trunk
for x in *; do
if [ "$x" != "trunk" ]; then
svn mv "$x" trunk;
fi
done
svn mkdir branches
svn mkdir tags
My goal was to move all files at the root into a new directory trunk,
then create an empty branches and tags directory (I decided not to
import labels).
One of the reasons to switch from VSS was to have working annotate
support. Not having the history proper on the files is defeating the
point of the huge effort to convert the VSS history rather than just
importing the "tip."
Jason Winnebeck
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn mv "breaks" annotate?
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Jul 10, 2007, at 14:27, Jason Winnebeck wrote:
> Karl Fogel wrote:
>
>> Jason Winnebeck wrote:
>>
>>> Is it possible that the real problem here was that in the conversion
>>> process it assigned svn:eol-style native to the files? In the actual
>>> dump file and in the VSS repository the files were all stored as
>>> CRLF
>>> format. Also, after the conversion the files actually had the
>>> "$Revision: 123$" tag expanded. When I did the move is when SVN
>>> actually
>>> changed it to store "$Revision$" instead. I wonder if it changed the
>>> line endings too, but my attempts at "diffing" aren't picking
>>> anything
>>> up. When the svnadmin load runs, if the actual file data as CRLFs
>>> in the
>>> dumpfile but the props are set as EOL native, and I did the
>>> conversion
>>> and load on a UNIX system, is it actually showing that I edited all
>>> lines because I triggered SVN's normalization of all line endings?
>>
>> Mrm, I don't know. The cvs2svn converter sets the EOL style from the
>> start of the conversion, so it wouldn't be the explanation in that
>> case, but I don't know how vss2svn works.
>
> Actually this seems to be the problem. I redid the conversion and
> just made a trivial modification to a file and committed it without
> any moves. The entire file was one big diff where all lines were
> changed. Looking at the vss2svn source code it is not properly
> converting in all cases. I've patched it, but I'm not positive what
> the file format of the dump file should be. I don't know if the
> content should always be LF internally, or if it must match
> whatever eol-style is. I'm changing it so that it always outputs
> text files in LF and will see how that goes (based on the way that
> the .svn/text-base files are matching the dump file I am thinking I
> will be in the right for this change).
Yes: If svn:eol-style is set (to any value), then Subversion stores
the file in all-LF format internally (regardless of the value of
svn:eol-style).
> At least for sure, for now, I know that svn mv or the fact that svn
> doesn't support moves natively is not of any relationship to my
> problem.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn mv "breaks" annotate?
Posted by Karl Fogel <kf...@red-bean.com>.
Jason Winnebeck <jp...@rit.edu> writes:
> At least for sure, for now, I know that svn mv or the fact that svn
> doesn't support moves natively is not of any relationship to my
> problem.
And I'm going to keep the eol possibility in mind for next time this
comes up... (thanks for following up about vss2svn and eol styles).
-Karl
--
Subversion support & consulting <> http://producingoss.com/consulting.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn mv "breaks" annotate?
Posted by Jason Winnebeck <jp...@rit.edu>.
Karl Fogel wrote:
> "Jason Winnebeck" <jp...@rit.edu> writes:
>> Is it possible that the real problem here was that in the conversion
>> process it assigned svn:eol-style native to the files? In the actual
>> dump file and in the VSS repository the files were all stored as CRLF
>> format. Also, after the conversion the files actually had the
>> "$Revision: 123$" tag expanded. When I did the move is when SVN actually
>> changed it to store "$Revision$" instead. I wonder if it changed the
>> line endings too, but my attempts at "diffing" aren't picking anything
>> up. When the svnadmin load runs, if the actual file data as CRLFs in the
>> dumpfile but the props are set as EOL native, and I did the conversion
>> and load on a UNIX system, is it actually showing that I edited all
>> lines because I triggered SVN's normalization of all line endings?
>
> Mrm, I don't know. The cvs2svn converter sets the EOL style from the
> start of the conversion, so it wouldn't be the explanation in that
> case, but I don't know how vss2svn works.
Actually this seems to be the problem. I redid the conversion and just made a
trivial modification to a file and committed it without any moves. The entire
file was one big diff where all lines were changed. Looking at the vss2svn
source code it is not properly converting in all cases. I've patched it, but
I'm not positive what the file format of the dump file should be. I don't know
if the content should always be LF internally, or if it must match whatever
eol-style is. I'm changing it so that it always outputs text files in LF and
will see how that goes (based on the way that the .svn/text-base files are
matching the dump file I am thinking I will be in the right for this change).
At least for sure, for now, I know that svn mv or the fact that svn doesn't
support moves natively is not of any relationship to my problem.
Jason
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn mv "breaks" annotate?
Posted by Karl Fogel <kf...@red-bean.com>.
"Jason Winnebeck" <jp...@rit.edu> writes:
> Is it possible that the real problem here was that in the conversion
> process it assigned svn:eol-style native to the files? In the actual
> dump file and in the VSS repository the files were all stored as CRLF
> format. Also, after the conversion the files actually had the
> "$Revision: 123$" tag expanded. When I did the move is when SVN actually
> changed it to store "$Revision$" instead. I wonder if it changed the
> line endings too, but my attempts at "diffing" aren't picking anything
> up. When the svnadmin load runs, if the actual file data as CRLFs in the
> dumpfile but the props are set as EOL native, and I did the conversion
> and load on a UNIX system, is it actually showing that I edited all
> lines because I triggered SVN's normalization of all line endings?
Mrm, I don't know. The cvs2svn converter sets the EOL style from the
start of the conversion, so it wouldn't be the explanation in that
case, but I don't know how vss2svn works.
(I can't really dig deeply into your questions, unfortunately, due to
lack of time. Maybe someone else who has seen similar problems before
can.)
Good luck,
-Karl
--
Subversion support & consulting <> http://producingoss.com/consulting.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: svn mv "breaks" annotate?
Posted by Jason Winnebeck <jp...@rit.edu>.
-----Original Message-----
From: Karl Fogel [mailto:kfogel@red-bean.com]
> I guess my real reason for asking is to ask if I did something wrong
> and/or if there was a better way, but I suppose the final answer on
> this is "no."
No, you didn't do anything wrong. The "better way" would have simply
been the workaround: to do the dumpfile editing during the conversion.
-Karl
-------
Is it possible that the real problem here was that in the conversion
process it assigned svn:eol-style native to the files? In the actual
dump file and in the VSS repository the files were all stored as CRLF
format. Also, after the conversion the files actually had the
"$Revision: 123$" tag expanded. When I did the move is when SVN actually
changed it to store "$Revision$" instead. I wonder if it changed the
line endings too, but my attempts at "diffing" aren't picking anything
up. When the svnadmin load runs, if the actual file data as CRLFs in the
dumpfile but the props are set as EOL native, and I did the conversion
and load on a UNIX system, is it actually showing that I edited all
lines because I triggered SVN's normalization of all line endings?
Jason
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn mv "breaks" annotate?
Posted by Jason Winnebeck <jp...@rit.edu>.
Karl Fogel wrote:
> Jason Winnebeck <jp...@rit.edu> writes:
>> I have used svndumpfilter and svndumptool to transform dumps in the
>> past, but I didn't think about this one. The problem is that I
>> converted the repository a few days ago and opened it up so only
>> noticed this problem after the move as I tested annotate and history
>> right after conversion but not after moving everything. I think I'll
>> just have to "suck this one up" as even after I fix this problem we
>> will have a good number more renames left. In VSS a lot of operations
>> were horrible and this is one of many reasons why we wanted to switch
>> to SVN for so long is the promise of easy moves, refactors, and
>> branches, and it's disappointing to see this.
>
> Well, it would almost certainly be possible to get this all fixed,
> while not losing since the switchover with some fancy dumpfile
> maneuvering. However, it may not be worth it to you.
Well, I still do have the backups of everything I converted. I wonder if it is
possible to take the repository and get it to some state x (using a
different "move" method by modifying the dumpfile), then merge the changes
from x to HEAD from the "live" repository into the new one I am making, then
replace the live with the new (knowing there would be some revision number
adjustments). This sounds very tricky and "dangerous" though, and I think the
merge would have to be done by creating an incremental dump file (i.e.
svnadmin dump myrepos --revision 1001:2000 --incremental > dumpfile), then
loading the incremental dump into the "new" repository. If I understand
correctly, then as long as I get the tip of the repository into the same state
content-wise, that should work. But even before getting there I have to figure
out how to modify the dumpfile and I'm not sure that can be done easily; what
I need to do is transform the paths of "x/a/b/c" to "x/trunk/a/b/c" for most
paths x.
This is probably too much work, but now I'm still thinking about it from an
academic standpoint.
Jason
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn mv "breaks" annotate?
Posted by Karl Fogel <kf...@red-bean.com>.
Jason Winnebeck <jp...@rit.edu> writes:
> I have used svndumpfilter and svndumptool to transform dumps in the
> past, but I didn't think about this one. The problem is that I
> converted the repository a few days ago and opened it up so only
> noticed this problem after the move as I tested annotate and history
> right after conversion but not after moving everything. I think I'll
> just have to "suck this one up" as even after I fix this problem we
> will have a good number more renames left. In VSS a lot of operations
> were horrible and this is one of many reasons why we wanted to switch
> to SVN for so long is the promise of easy moves, refactors, and
> branches, and it's disappointing to see this.
Well, it would almost certainly be possible to get this all fixed,
while not losing since the switchover with some fancy dumpfile
maneuvering. However, it may not be worth it to you.
> I guess my real reason for asking is to ask if I did something wrong
> and/or if there was a better way, but I suppose the final answer on
> this is "no."
No, you didn't do anything wrong. The "better way" would have simply
been the workaround: to do the dumpfile editing during the conversion.
-Karl
--
Subversion support & consulting <> http://producingoss.com/consulting.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn mv "breaks" annotate?
Posted by Jason Winnebeck <jp...@rit.edu>.
Karl Fogel wrote:
> Eli Carter <el...@commprove.com> writes:
>> On Monday 09 July 2007, Jason Winnebeck wrote:
>>> I restructured a repository from a "normal" layout to a
>>> "trunk/tags/branches" layout, but now annotate shows as if the entire
>>> file was updated by myself on almost all files.
>>
>> If you're careful, you can modify a dump file to rename all the paths...
>
> Good point. And you don't have to be *so* careful, after all, you can
> work on copies of everything and test before making a repository live.
>
> Just dump the original repository, write a script to stick "/trunk" in
> front of every path (or perhaps some more complex formula will be
> necessary, but you get the idea), and load into a fresh repository.
>
> -Karl
>
I have used svndumpfilter and svndumptool to transform dumps in the past, but
I didn't think about this one. The problem is that I converted the repository
a few days ago and opened it up so only noticed this problem after the move as
I tested annotate and history right after conversion but not after moving
everything. I think I'll just have to "suck this one up" as even after I fix
this problem we will have a good number more renames left. In VSS a lot of
operations were horrible and this is one of many reasons why we wanted to
switch to SVN for so long is the promise of easy moves, refactors, and
branches, and it's disappointing to see this.
I guess my real reason for asking is to ask if I did something wrong and/or if
there was a better way, but I suppose the final answer on this is "no."
Jason Winnebeck
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn mv "breaks" annotate?
Posted by Karl Fogel <kf...@red-bean.com>.
Eli Carter <el...@commprove.com> writes:
> On Monday 09 July 2007, Jason Winnebeck wrote:
>> I restructured a repository from a "normal" layout to a
>> "trunk/tags/branches" layout, but now annotate shows as if the entire
>> file was updated by myself on almost all files.
>>
>> The repository was just converted from SourceSafe, using the excellent
>> vss2svn script. That process seemed to work as good as expected. So
>> then, I wanted to immediately restructure the projects to have a
>> "trunk/tags/branches" format.
>
> <advice type=dangerous>
> If you're careful, you can modify a dump file to rename all the paths...
> </advice>
Good point. And you don't have to be *so* careful, after all, you can
work on copies of everything and test before making a repository live.
Just dump the original repository, write a script to stick "/trunk" in
front of every path (or perhaps some more complex formula will be
necessary, but you get the idea), and load into a fresh repository.
-Karl
--
Subversion support & consulting <> http://producingoss.com/consulting.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn mv "breaks" annotate?
Posted by Eli Carter <el...@commprove.com>.
On Monday 09 July 2007, Jason Winnebeck wrote:
> I restructured a repository from a "normal" layout to a
> "trunk/tags/branches" layout, but now annotate shows as if the entire
> file was updated by myself on almost all files.
>
> The repository was just converted from SourceSafe, using the excellent
> vss2svn script. That process seemed to work as good as expected. So
> then, I wanted to immediately restructure the projects to have a
> "trunk/tags/branches" format.
<advice type=dangerous>
If you're careful, you can modify a dump file to rename all the paths...
</advice>
Eli
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn mv "breaks" annotate?
Posted by Karl Fogel <kf...@red-bean.com>.
"Jason Winnebeck" <jp...@rit.edu> writes:
> I restructured a repository from a "normal" layout to a
> "trunk/tags/branches" layout, but now annotate shows as if the entire
> file was updated by myself on almost all files.
>
> The repository was just converted from SourceSafe, using the excellent
> vss2svn script. That process seemed to work as good as expected. So
> then, I wanted to immediately restructure the projects to have a
> "trunk/tags/branches" format. However, this seems to have broken
> annotate somehow. What happens is that I am shown as the latest author
> on all lines with the revision number being the number of the move, on
> almost all files. But **some** files show the annotate as expected. Also
> the log of the move showed a lot of A, D, and M statuses, and when
> committing it took a long time for "sending file data." The size of the
> transaction in the FSFS is 6MB, but I thought copies were "free" in SVN.
> This is why I am thinking I might have done something wrong.
>
> The two interesting points are that the last line of the files show a
> different author, and that the only changes in the files were in the
> $Revision$ keyword expansion.
>
> I used the following shell script on the root directory of each project
> in the repository:
>
> cd "$1"
>
> svn mkdir trunk
>
> for x in *; do
> if [ "$x" != "trunk" ]; then
> svn mv "$x" trunk;
> fi
> done
>
> svn mkdir branches
> svn mkdir tags
>
> My goal was to move all files at the root into a new directory trunk,
> then create an empty branches and tags directory (I decided not to
> import labels).
>
> One of the reasons to switch from VSS was to have working annotate
> support. Not having the history proper on the files is defeating the
> point of the huge effort to convert the VSS history rather than just
> importing the "tip."
This is a bug in Subversion:
http://subversion.tigris.org/issues/show_bug.cgi?id=898
I'll bet that the annotate history is wrong on the files that were
directly renamed, and correct on files that are somewhere underneath
directories that got renamed.
It might be worth re-doing the import; I don't know how much effort
that would be for you.
Sorry not to have a better answer for you. We'd all like this fixed,
but it's a matter of resources.
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org