You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Russ Brown <pi...@gmail.com> on 2005/05/26 11:55:33 UTC
eol-style
'ello,
I've just helped a colleague out with a very bizarre issue, and wanted
to ask the group what (if anything) we're doing wrong.
Let me explain. He was editing a file and changed one line. He then
saved the file and performed svn diff on it. The output showed that
the entire file was changed.
Weird.
So, we eliminated the usual suspects. The editor was not changing the
line endings, stripping whitespace or converting tabs into spaces. We
tried appending a line to the file via command line tools and editing
the file using sed with the same result. We tried running the diff
through the system diff command with no change to the output.
Finally, we tried copying the (unmodified) file, changed one of the
copies on the disk and did the diff. The diff showed that just the one
line had changed. This was completely outside of Subversion, which
obviously pointed the finger at something to do with Subversion.
At this point I got the idea that the problem may be something to do
with the eol-style property. I checked it and it was indeed set; its
value being 'native'. So, I removed the property from the file and
tried the change again. This time, the diff worked correctly.
Now, a bit of background. All developers here use Linux, while the
designers use Windows. The file in question was a HTML template, and
so it likely to either have originated from a Windows machine or at
least been edited there at some point.
I thought that the point to the 'native' eol-style was to
transparently handle the situation where you potentially have users
from different operating systems editing files. If I'm wrong on this
I'd appreciate advice on how the eol-style property should be used
correctly. If I'm not wrong, I think I've found a bug. :-)
Oh, and I should say that my colleague is using Subversion client
1.1.4. The server is currently 1.1.3.
Thanks.
--
Russell Brown
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: eol-style
Posted by stephen meechan <mg...@dsl.pipex.com>.
>Now that I understand what the problem is, I'd like some advice on how
>to resolve it. It appears that the problem was caused by cvs2svn,
>which imported Windows-formatted files with the svn:eol-style property
>set but without normalising the endlines in the files themselves. This
>affects all windows-formatted files in the system (primarily
>templates).
>
...
>
>What I'm looking for is a way to correct the whole lot and commit the
>fix in one big changeset. That will result in loads of conflicts in
>the next few branch merges, but would fix the problem once and for all
>going forward. The question is, how to do that. svn doesn't want to
>commit the file unless the file gets changed, so I suspect that I'd
>have to change the files in some way (though that is hardly ideal).
>Anything else I can try?
>
>The other problem is how to detect the problem files. The only way I
>can think of is to look for all files that are Windows-formatted, have
>the svn:eol-style property set to native, and haven't been changed
>since the subversion import. Doesn't sound like a quick method (and
>I'm not aware of a command-line utility to check for windows format).
>Does anybody else have any suggestions?
I have the same problem, at the time the SVN client (1.0.6, I think) let me
fix the files on the trunk by removing the property, fixing the file and
setting the property again. But later clients seem to have become more
strict about how they treat the inconsistencies and stop when they hit the
problem.
The problem today is usually when looking back in the history of the file,
is not possible to retrieve the old revision or diff against it to see what
was changing.
My thought is that is should be possible to dump the repository and check
for files with the property set but with lines that aren't normalized, then
create a new dump file with the repaired files and load that as the new
repository.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: eol-style
Posted by Dale Worley <dw...@pingtel.com>.
> From: Russ Brown [mailto:pickscrape@gmail.com]
>
> What I'm looking for is a way to correct the whole lot and commit the
> fix in one big changeset. That will result in loads of conflicts in
> the next few branch merges, but would fix the problem once and for all
> going forward. The question is, how to do that. svn doesn't want to
> commit the file unless the file gets changed, so I suspect that I'd
> have to change the files in some way (though that is hardly ideal).
> Anything else I can try?
You could probably do something like this: Make a complete checkout. Use
"svn propget -R svn:eol-style" to list all the files that have svn:eol-style
set, and the values. Save that list. Do "svn propdel -R svn:eol-style" to
remove all the eol-styles. At this point your WC is OK. Now massage that
list you made into a set of "svn propset svn:eol-style <value> <file>"
commands and execute them. In the background, your client will edit the
EOLs on the files to make them consistent with the eol-styles you set, or at
least flag them as incorrect, which you can fix by hand or script. Now
check the mess in. That should make your repository consistent.
Dale
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: eol-style
Posted by Russ Brown <pi...@gmail.com>.
On 5/26/05, Dale Worley <dw...@pingtel.com> wrote:
> > From: Russ Brown [mailto:pickscrape@gmail.com]
> >
> > Could this have anything to do with the fact that we imported our
> > repository from our original CVS one using cvs2svn? I think this is
> > the first working-copy commit on that file since the repository was
> > originally imported.
>
> It seems quite likely. But you should inquire as to how svn:eol-style got
> set without normalizing the stored EOLs.
>
> > This may also explain a number of large conflicts we've been getting.
> > Would this mean that every single file in the repository that has that
> > property set will appear to have been completely changed the first
> > time it is commited since the import?
>
> Quite possibly.
>
> You might be able to clean up the situation by doing a complete checkout and
> seeing if the client discovers that the EOLs are wrong for the eol-style and
> cleans them up in the WC. In that case, you want to check in the WC as
> quickly as possible, so people start working off a base with correct EOLs.
>
>
All,
Now that I understand what the problem is, I'd like some advice on how
to resolve it. It appears that the problem was caused by cvs2svn,
which imported Windows-formatted files with the svn:eol-style property
set but without normalising the endlines in the files themselves. This
affects all windows-formatted files in the system (primarily
templates).
The problem manifsts itself in two ways (that we're aware of):
1. If a user makes a change to the file and commits, it appears that
the entire file has changed. This appears to cause conflicts in all
merges that involve this changeset.
2. The file foils svn revert and causes it to die with a message like:
svn: File 'xxxxx/.svn/text-base/yyyyyy.svn-base' has inconsistent newlines
What I'm looking for is a way to correct the whole lot and commit the
fix in one big changeset. That will result in loads of conflicts in
the next few branch merges, but would fix the problem once and for all
going forward. The question is, how to do that. svn doesn't want to
commit the file unless the file gets changed, so I suspect that I'd
have to change the files in some way (though that is hardly ideal).
Anything else I can try?
The other problem is how to detect the problem files. The only way I
can think of is to look for all files that are Windows-formatted, have
the svn:eol-style property set to native, and haven't been changed
since the subversion import. Doesn't sound like a quick method (and
I'm not aware of a command-line utility to check for windows format).
Does anybody else have any suggestions?
Thanks.
--
Russ
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: eol-style
Posted by Dale Worley <dw...@pingtel.com>.
> From: Russ Brown [mailto:pickscrape@gmail.com]
>
> Could this have anything to do with the fact that we imported our
> repository from our original CVS one using cvs2svn? I think this is
> the first working-copy commit on that file since the repository was
> originally imported.
It seems quite likely. But you should inquire as to how svn:eol-style got
set without normalizing the stored EOLs.
> This may also explain a number of large conflicts we've been getting.
> Would this mean that every single file in the repository that has that
> property set will appear to have been completely changed the first
> time it is commited since the import?
Quite possibly.
You might be able to clean up the situation by doing a complete checkout and
seeing if the client discovers that the EOLs are wrong for the eol-style and
cleans them up in the WC. In that case, you want to check in the WC as
quickly as possible, so people start working off a base with correct EOLs.
> If that's the case, I presume
> that if two developers change a file for the first time (on different
> branches) the merge would conflict because both changes will appear to
> have modified the entire file.
Well, the change to most of the lines would be non-conflicting, because it
would be the same in both WCs. But every place with a substantival change
would show a conflict, because Svn's merge will notice that the same line
has been changed in both WCs.
Dale
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: eol-style
Posted by Russ Brown <pi...@gmail.com>.
On 5/26/05, Dale Worley <dw...@pingtel.com> wrote:
> > From: Russ Brown [mailto:pickscrape@gmail.com]
> >
> > At this point I got the idea that the problem may be something to do
> > with the eol-style property. I checked it and it was indeed set; its
> > value being 'native'. So, I removed the property from the file and
> > tried the change again. This time, the diff worked correctly.
>
> It's not exactly clear to me how you got into this situation, but it appears
> that you managed to get the eol-style property set without checking in the
> file from a WC, so Subversion hadn't yet edited the file's contents to match
> the eol-style.
>
>
Could this have anything to do with the fact that we imported our
repository from our original CVS one using cvs2svn? I think this is
the first working-copy commit on that file since the repository was
originally imported.
This may also explain a number of large conflicts we've been getting.
Would this mean that every single file in the repository that has that
property set will appear to have been completely changed the first
time it is commited since the import? If that's the case, I presume
that if two developers change a file for the first time (on different
branches) the merge would conflict because both changes will appear to
have modified the entire file.
Or am I off the mark again?
--
Russ
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: eol-style
Posted by Ben Collins-Sussman <su...@collab.net>.
On May 26, 2005, at 12:11 PM, Dale Worley wrote:
>> From: Marc Sherman [mailto:msherman@projectile.ca]
>>
>> Could this happen if the file is initially added to the repo with no
>> eol-style property, and then the eol-style is subsequently
>> added using an URL on the svn propset command?
>>
>
> That seems possible. It might even be an error situation, if there
> is a
> rule that the contents of files in the repository with eol-style
> must have
> normalized EOLs.
>
Yes, that is exactly the case. If svn:eol-style=native, then
Subversion assumes that file is stored with all LF line-endings in
the repository. The client then changes line-endings in the working-
copy file if necessary.
I seem to recall an old bug in cvs2svn whereby it automatically sets
svn:eol-style=native on all files without the RCS '-kb' flags, but
then failed normalize the file to LF. Perhaps you were using an old
version of cvs2svn?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: eol-style
Posted by Dale Worley <dw...@pingtel.com>.
> From: Marc Sherman [mailto:msherman@projectile.ca]
>
> Could this happen if the file is initially added to the repo with no
> eol-style property, and then the eol-style is subsequently
> added using an URL on the svn propset command?
That seems possible. It might even be an error situation, if there is a
rule that the contents of files in the repository with eol-style must have
normalized EOLs.
Dale
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: eol-style
Posted by Marc Sherman <ms...@projectile.ca>.
Dale Worley wrote:
>
> It's not exactly clear to me how you got into this situation, but it
> appears that you managed to get the eol-style property set without
> checking in the file from a WC, so Subversion hadn't yet edited the
> file's contents to match the eol-style.
Could this happen if the file is initially added to the repo with no
eol-style property, and then the eol-style is subsequently added using
an URL on the svn propset command?
- Marc
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: eol-style
Posted by Dale Worley <dw...@pingtel.com>.
> From: Russ Brown [mailto:pickscrape@gmail.com]
>
> At this point I got the idea that the problem may be something to do
> with the eol-style property. I checked it and it was indeed set; its
> value being 'native'. So, I removed the property from the file and
> tried the change again. This time, the diff worked correctly.
It's not exactly clear to me how you got into this situation, but it appears
that you managed to get the eol-style property set without checking in the
file from a WC, so Subversion hadn't yet edited the file's contents to match
the eol-style.
See, when svn:eol-style is not set, end-of-line characters in the file are
just characters. But when you set svn:eol-style, the client and/or server
(I don't know which) always stores lines in the repository with one format
of EOL (LF only, I think, but it doesn't matter), and upon
check-in/check-out rewrites the EOL characters to/from the appropriate
characters based on the client's OS and the value of svn:eol-style.
The result is that when you initially set svn:eol-style, it may *imply* a
massive change to the *contents* of the file in te repository, changing all
the EOLs. It appears that the edit you performed was the point at which it
happened for this file.
Carefully extracting the various revisions of a file with svn cat -rxxx and
comparisons with svn diff will show this better...
Dale
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org