You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Ross Boylan <ro...@biostat.ucsf.edu> on 2007/01/17 23:17:01 UTC
File encoding
I have GNU/Linux system with LANG=en_US and another with
LANG=en_US.UTF-8.
A program produces some output, which is compared to an expected
output file (e.g., test1.Rout.save). The latter is kept in
subversion.
The output differs between the two systems because of the encoding
differences.
Is there anything I can do to make this work more smoothly? In
particular, it would be nice if the file (which is text) were created
with the correct encoding by the svn client.
The docs indicate that the server stores all filenames and logs as
UTF-8, but files appear to be kept as unmodified binaries. mime-type
can affect comparisons and eol-style can affect end of lines, but I
don't see anything dealing with the encoding per se (even if it is
part of the MIME type, and I'm not sure it is).
I know that historically version control systems attempt to be clever
about their files has often led to problems, but it seems to me
conversions between encoding schemes are fairly well-defined, and
would be useful. (Yes, I know some conversions are impossible!).
Thanks.
Ross Boylan
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: File encoding
Posted by Ross Boylan <ro...@biostat.ucsf.edu>.
On Wed, Jan 17, 2007 at 03:17:01PM -0800, Ross Boylan wrote:
> I have GNU/Linux system with LANG=en_US and another with
> LANG=en_US.UTF-8.
>
> A program produces some output, which is compared to an expected
> output file (e.g., test1.Rout.save). The latter is kept in
> subversion.
>
> The output differs between the two systems because of the encoding
> differences.
>
> Is there anything I can do to make this work more smoothly? In
> particular, it would be nice if the file (which is text) were created
> with the correct encoding by the svn client.
>
> The docs indicate that the server stores all filenames and logs as
> UTF-8, but files appear to be kept as unmodified binaries. mime-type
> can affect comparisons and eol-style can affect end of lines, but I
> don't see anything dealing with the encoding per se (even if it is
> part of the MIME type, and I'm not sure it is).
The MIME specification provides a charset parameter for text files
that seems to cover the encoding. Given subversion's current
approach, I assume that parameter is ignored, and it might even
confuse the code that decides whether a file is text or binary.
Ross
>
> I know that historically version control systems attempt to be clever
> about their files has often led to problems, but it seems to me
> conversions between encoding schemes are fairly well-defined, and
> would be useful. (Yes, I know some conversions are impossible!).
>
> Thanks.
> Ross Boylan
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: File encoding
Posted by Ulrich Eckhardt <ec...@satorlaser.com>.
On Thursday 18 January 2007 00:17, Ross Boylan wrote:
> I have GNU/Linux system with LANG=en_US and another with
> LANG=en_US.UTF-8.
>
> A program produces some output, which is compared to an expected
> output file (e.g., test1.Rout.save). The latter is kept in
> subversion.
>
> The output differs between the two systems because of the encoding
> differences.
Simple solution: always start tools with the "C" locale, this will give you
the best comparability between systems and is a locale that is always
installed.
Uli
**************************************************************************************
Visit our website at <http://www.satorlaser.de/>
**************************************************************************************
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich.
**************************************************************************************
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: File encoding
Posted by Ross Boylan <ro...@biostat.ucsf.edu>.
On Wed, Jan 17, 2007 at 06:28:43PM -0600, Ryan Schmidt wrote:
>
> On Jan 17, 2007, at 17:17, Ross Boylan wrote:
>
> >The docs indicate that the server stores all filenames and logs as
> >UTF-8, but files appear to be kept as unmodified binaries. mime-type
> >can affect comparisons and eol-style can affect end of lines, but I
> >don't see anything dealing with the encoding per se (even if it is
> >part of the MIME type, and I'm not sure it is).
> >
> >I know that historically version control systems attempt to be clever
> >about their files has often led to problems, but it seems to me
> >conversions between encoding schemes are fairly well-defined, and
> >would be useful. (Yes, I know some conversions are impossible!).
>
> Subversion does not have this feature.
I was afraid of that.
> Subversion stores file
> contents unchanged, except for line ending conversion if you ask for
> it. I believe attempting to provide encoding conversion is risky.
> There is no well-established way to indicate what encoding a text
> file is using. There are svn auto-props, but that wouldn't help in
> all cases. For example, I write php files, most of which are in
> iso-8859-1, but many of which are in utf-8 instead.
>
I don't see why a property like "encoding" wouldn't work with a smart
client. Some of the files would indicate iso-8859-1; others would be
tagged utf-8.
> As a solution outside of Subversion, you can use iconv to convert
> your files to the correct encoding for your situation before putting
> them in the repository.
That won't help, because the conversion needs to happen when the files
come out. I can store the file in iso8859 or utf-8, but I have to
pick one of them. When I check it out into the other environment, the
file won't be interpreted correctly.
Ross
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: File encoding
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Jan 17, 2007, at 17:17, Ross Boylan wrote:
> The docs indicate that the server stores all filenames and logs as
> UTF-8, but files appear to be kept as unmodified binaries. mime-type
> can affect comparisons and eol-style can affect end of lines, but I
> don't see anything dealing with the encoding per se (even if it is
> part of the MIME type, and I'm not sure it is).
>
> I know that historically version control systems attempt to be clever
> about their files has often led to problems, but it seems to me
> conversions between encoding schemes are fairly well-defined, and
> would be useful. (Yes, I know some conversions are impossible!).
Subversion does not have this feature. Subversion stores file
contents unchanged, except for line ending conversion if you ask for
it. I believe attempting to provide encoding conversion is risky.
There is no well-established way to indicate what encoding a text
file is using. There are svn auto-props, but that wouldn't help in
all cases. For example, I write php files, most of which are in
iso-8859-1, but many of which are in utf-8 instead.
As a solution outside of Subversion, you can use iconv to convert
your files to the correct encoding for your situation before putting
them in the repository.
--
To reply to the mailing list, please use your mailer's Reply To All
function
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org