You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Thomas Scheffler <th...@uni-jena.de> on 2008/02/01 13:56:40 UTC

Subversion and binary files

Hi,

for a larger project we migrated from CVS to Subversion with cvs2svn. All 
seemed to be gone well, but all xml files are handled as binary files. This 
must be due to their "svn:mime-type" value which is "application/xml".

As we have several of them in our repository it gets quiet anyoing, when you 
use some tools like viewvc which do not display "svn blame" outputs or 
svnnotify which refuses to print out the diffs (because svnlook is used).

What can I do to tell subversion that xml files are text files?

Greets

Thomas


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Subversion and binary files

Posted by John Peacock <jo...@havurah-software.org>.
Thomas Scheffler wrote:
> They should be. You cannot expect that arbitrary binary data can be put in an 
> XML file. They are mostly base64 encoded and wrapped in CDATA blocks. Diffing 
> those blocks can be long as well as diffing any large text documents.

No, I'm not talking about CDATA blocks.  I was pointing out that is 
legal under XML to have a series of attributes in the same container 
level be ordering indefinite.  There are other bits (but I can't 
remember off the top of my head), but effectively, certain portions of 
XML documents are effectively hashes, where the order is not 
significant, only the keys.

But the fact is that even if an XML document consists solely of text, it 
is not necessarily a text document.  This is why XML editors and XML 
diff programs are so complex.

John

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Subversion and binary files

Posted by Thomas Scheffler <th...@uni-jena.de>.
Am Freitag, 1. Februar 2008 schrieb John Peacock:
> Thomas Scheffler wrote:
> > As we have several of them in our repository it gets quiet anyoing, when
> > you use some tools like viewvc which do not display "svn blame" outputs
> > or svnnotify which refuses to print out the diffs (because svnlook is
> > used).
>
> Technically, speaking, XML files cannot always be treated as plain text
> files (since not all XML documents have strictly ordered contents).

They should be. You cannot expect that arbitrary binary data can be put in an 
XML file. They are mostly base64 encoded and wrapped in CDATA blocks. Diffing 
those blocks can be long as well as diffing any large text documents.

> > What can I do to tell subversion that xml files are text files?
>
> Remove the 'svn:mime-type' from the files; anything with a mime-type of
> "application/*" is considered binary.  Then use the --diff-cmd and
> --extensions command line arguments to call your favorite XML diff
> program (you can easily write some wrapper scripts that automate this to
> a large extent).

Removing the "svn:mime-type" is just not sufficient as this property is just 
evaluated once when the file is first added or imported to the repository. So 
removing or setting "text/*" or so will not help as the file stays "binary". 
I rely on tools that use "svnlook diff". So I need a solution where I can 
switch files into text mode after they are inserted. Where is that 
information stored? I am using the FSFS backend.

Greets

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Subversion and binary files

Posted by Paul Koning <Pa...@dell.com>.
>>>>> "Thomas" == Thomas Scheffler <th...@uni-jena.de> writes:

 Thomas> This is not stated in the documentation (svn book) I think,
 Thomas> but you can read it in the FAQ
 Thomas> http://subversion.tigris.org/faq.html#binary-files

 Thomas> "When you first add or import a file into Subversion, the
 Thomas> file is examined to determine if it is a binary file..."

 Thomas> They'll tell you there that "svn:eol-style" does not switch
 Thomas> binary files into text mode. So where is that switch?

Looks like setting the mime type to text/something (I tend to use
text/plan) should do the job.  Also, for any text file, setting
eol-style is a good idea.

cvs2svn can do this at the time of conversion (using the auto-props
definitions).

	  paul


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Subversion and binary files

Posted by Ryan Schmidt <su...@ryandesign.com>.
On Feb 1, 2008, at 09:05, Thomas Scheffler wrote:

> Removing the "svn:mime-type" is just not sufficient as this  
> property is just
> evaluated once when the file is first added or imported to the  
> repository. So
> removing or setting "text/*" or so will not help as the file stays  
> "binary".
> I rely on tools that use "svnlook diff". So I need a solution where  
> I can
> switch files into text mode after they are inserted. Where is that
> information stored? I am using the FSFS backend.


On Feb 1, 2008, at 10:03, Thomas Scheffler wrote:

> This is not stated in the documentation (svn book) I think, but you  
> can read
> it in the FAQ
> http://subversion.tigris.org/faq.html#binary-files
>
> "When you first add or import a file into Subversion, the file is  
> examined to
> determine if it is a binary file..."
>
> They'll tell you there that "svn:eol-style" does not switch binary  
> files into
> text mode. So where is that switch?

There is no switch, and there is no binary or text "mode". All files  
are stored binary in the repository. A client only handles a file as  
binary if has an svn:mime-type property and it does not begin with  
"text/". (There are a few other MIME types that are handled as text,  
too [1], but "application/xml" is not one of them.) If a file has no  
svn:mime-type property the client treats it as a text file.

If you want a file to be handled as a text file, give it a "text/ 
something" svn:mime-type, or delete the svn:mime-type.

If you want to correct this for all revisions in your repository, you  
can svnadmin dump the repo, change the revisions with svndumptool  
[2], svnadmin load it into a new repo, and check out all new working  
copies.


[1] http://subversion.tigris.org/faq.html#binary-files

[2] http://svn.borg.ch/svndumptool/



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Subversion and binary files

Posted by Thomas Scheffler <th...@uni-jena.de>.
Am Freitag, 1. Februar 2008 schrieb Paul Koning:
> >>>>> "John" == John Peacock <jo...@havurah-software.org> writes:
>  >>
>  >> What can I do to tell subversion that xml files are text files?
>
>  John> Remove the 'svn:mime-type' from the files; anything with a
>  John> mime-type of "application/*" is considered binary.
>
> That's a bit surprising.  I can't remember that the documentation is
> all that clear on the subject, but it's my impression that the easiest
> way to indicate a text file is to put a svn:eol-style property on it.
> I would have expected cvs2svn to do that (maybe you need to set up a
> config file to force that for .xml files).  In any case, if I ran into
> something like this I would start by setting that property.

This is not stated in the documentation (svn book) I think, but you can read 
it in the FAQ
http://subversion.tigris.org/faq.html#binary-files

"When you first add or import a file into Subversion, the file is examined to 
determine if it is a binary file..."

They'll tell you there that "svn:eol-style" does not switch binary files into 
text mode. So where is that switch?

Greets

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Subversion and binary files

Posted by Paul Koning <pk...@equallogic.com>.
>>>>> "John" == John Peacock <jo...@havurah-software.org> writes:

 >> What can I do to tell subversion that xml files are text files?

 John> Remove the 'svn:mime-type' from the files; anything with a
 John> mime-type of "application/*" is considered binary.  

That's a bit surprising.  I can't remember that the documentation is
all that clear on the subject, but it's my impression that the easiest
way to indicate a text file is to put a svn:eol-style property on it.
I would have expected cvs2svn to do that (maybe you need to set up a
config file to force that for .xml files).  In any case, if I ran into
something like this I would start by setting that property.

	  paul


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Subversion and binary files

Posted by John Peacock <jo...@havurah-software.org>.
Thomas Scheffler wrote:
> As we have several of them in our repository it gets quiet anyoing, when you 
> use some tools like viewvc which do not display "svn blame" outputs or 
> svnnotify which refuses to print out the diffs (because svnlook is used).

Technically, speaking, XML files cannot always be treated as plain text 
files (since not all XML documents have strictly ordered contents). 
There are a number of XML diff utilities out there that try and provide 
sensible diffs of XML documents, but Subversion doesn't [currently] have 
a pluggable diff architecture that would allow you to selectively switch 
to alternate diff programs for specific filetypes.  It is something that 
a fair number of people would be interested in, so I would expect it to 
come along at some point.  Patches welcome... ;-)

> What can I do to tell subversion that xml files are text files?

Remove the 'svn:mime-type' from the files; anything with a mime-type of 
"application/*" is considered binary.  Then use the --diff-cmd and 
--extensions command line arguments to call your favorite XML diff 
program (you can easily write some wrapper scripts that automate this to 
a large extent).

HTH

John

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org