[PATCH] Re: Issue 1954

[Philipp Marek <ph...@marek.priv.at>]

Here's a simple encoding/decoding patch; a property is defined, too.
It's only compile tested, though.


I'd like to ask you to vote for/against comitting *such a* patch; if the 
encoding format is defined it's much easier for all people to produce 
complying clients.


What is missing is the usage of these functions in subversion, the 
sending/receiving of the property, and some tests.
As some change in the wc code would be necessary, too  (to store 
such "extended" path names in the .svn directory index), I'd like to avoid 
getting into this.
(Also I'm sure that people who are familiar with these parts of the code are 
much faster doing these changes than me).
BTW, AFAIK the wc code is currently re-written or at least heavily changed.


Regards,

Phil



-- 
Versioning your /etc, /home or even your whole installation?
             Try fsvs (fsvs.tigris.org)!

[PATCH 2] Re: Issue 1954

[Philipp Marek <ph...@marek.priv.at>]

On Friday 21 July 2006 18:35 Philipp Marek wrote:
> Hmmm, I see what you mean.
> We need to allow one more character than before, to use that as escape
> character.
> All "old" clients can't make such filenames, any new ones do.
>
> Although I'm -1 on using a "special" (control) character for escaping, it
> would be the only "right" way to solve that problem.
> TBH, I don't believe that we'd have a problem with my proposed encoding
> scheme (practically) - is there anyone using "\\x09" in a filename? (Ok, a
> tab character is not that often used, too :-)
Here's another patch; this one escapes the escape character, too.
This should now be a correct way to escape characters; a path "\\hello" stays 
the same, a path "\\\ttab" gets encoded to "\\x5c\\x09tab", and the property 
defines without doubt whether a path needs to be decoded.

Other comments? Votes?


Regards,

Phil

-- 
Versioning your /etc, /home or even your whole installation?
             Try fsvs (fsvs.tigris.org)!

Re: [PATCH] Re: Issue 1954

[Philipp Marek <ph...@marek.priv.at>]

On Thursday 20 July 2006 20:56 Malcolm Rowe wrote:
> On Thu, Jul 20, 2006 at 08:29:07PM +0200, Philipp Marek wrote:
> > > 1. What happens if you have two files with the same encoded name? (one
> > > encoded, one not?).  For example, two files called 'foo\x09bar' and
> > > 'foo^Ibar' (where ^I represents a real TAB character).
> >
> > There has to be a property "svn:filename-encoded" set to "*", similar to
> > the way svn:executable works, to enable filename translation.
>
> Yes, but I assumed that this would be applied automatically where
> necessary?  
What do you mean by automatically?
All "old" programs who don't know about that encoding won't set 
svn:filename-encoded, so they always see "valid" filenames.

> So if you add a file called 'foo^Ibar', it would actually 
> create a file in the filesystem called 'foo\x09bar' with property
> filename-encoded set.  If not, assume that the user has done whatever
> magic is necessary to add this file.
>
> What happens when the user now adds a file that's really called
> 'foo\x09bar'? (note that you aren't allowed to require that '\' be
> escaped, since that wouldn't be compatible with existing repositories).
Hmmm, I see what you mean.
We need to allow one more character than before, to use that as escape 
character.
All "old" clients can't make such filenames, any new ones do.

Although I'm -1 on using a "special" (control) character for escaping, it 
would be the only "right" way to solve that problem.
TBH, I don't believe that we'd have a problem with my proposed encoding scheme 
(practically) - is there anyone using "\\x09" in a filename? (Ok, a tab 
character is not that often used, too :-)

> Okay, so anyone accessing the filesystem through the repos or fs layers
> sees the encoded filename.  That's potentially confusing ('ls' will
> return different filenames to 'svn ls', all the URL-using operations
> need to understand how to encode the filename), but I guess it does fix
> several of the problems I mentioned.
The svn binary, the fsvs binary, all repository viewers need to know about 
that convention, and need to parse filenames according to that.

> Regards,
> Malcolm
Thank you for your feedback.



Regards,

Phil


-- 
Versioning your /etc, /home or even your whole installation?
             Try fsvs (fsvs.tigris.org)!

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: [PATCH] Re: Issue 1954

[Malcolm Rowe <ma...@farside.org.uk>]

On Thu, Jul 20, 2006 at 08:29:07PM +0200, Philipp Marek wrote:
> > 1. What happens if you have two files with the same encoded name? (one
> > encoded, one not?).  For example, two files called 'foo\x09bar' and
> > 'foo^Ibar' (where ^I represents a real TAB character).
> There has to be a property "svn:filename-encoded" set to "*", similar to the 
> way svn:executable works, to enable filename translation.
> 

Yes, but I assumed that this would be applied automatically where
necessary?  So if you add a file called 'foo^Ibar', it would actually
create a file in the filesystem called 'foo\x09bar' with property
filename-encoded set.  If not, assume that the user has done whatever
magic is necessary to add this file.

What happens when the user now adds a file that's really called
'foo\x09bar'? (note that you aren't allowed to require that '\' be
escaped, since that wouldn't be compatible with existing repositories).

> > 3. The filesystem code now needs to know about properties in order to work
> > out to what a filename refers.  That seems wrong, since you can't carry
> > out operations on the pathname alone (for example, how do you interpret
> > what the [necessarily encoded] copyfrom path of a file refers to?
> > You'd need to find the copyfrom'd file before you could check what the
> > property value was, which you need to do before you know what the name
> > of the file is...).
> No. The working copy layer, which gets the properties on update, knows if the 
> property is set, and can do the translation or not.
> 

Okay, so anyone accessing the filesystem through the repos or fs layers
sees the encoded filename.  That's potentially confusing ('ls' will
return different filenames to 'svn ls', all the URL-using operations
need to understand how to encode the filename), but I guess it does fix
several of the problems I mentioned.

Regards,
Malcolm

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: [PATCH] Re: Issue 1954

[Philipp Marek <ph...@marek.priv.at>]

On Thursday 20 July 2006 18:04 Malcolm Rowe wrote:
> On Thu, Jul 20, 2006 at 04:39:10PM +0200, Philipp Marek wrote:
> > I'd like to ask you to vote for/against comitting *such a* patch; if the
> > encoding format is defined it's much easier for all people to produce
> > complying clients.
>
> I understand the need for such a feature
Thank you.

> , but don't see how it can be 
> introduced before a 2.0 release
Why? Please read the points below.

> (at which point, there's definitely a 
> cleaner way to do it).  A few immediate things come to mind with the
> current design, though there are probably more:
>
> 1. What happens if you have two files with the same encoded name? (one
> encoded, one not?).  For example, two files called 'foo\x09bar' and
> 'foo^Ibar' (where ^I represents a real TAB character).
There has to be a property "svn:filename-encoded" set to "*", similar to the 
way svn:executable works, to enable filename translation.

> 2. With the currently-proposed design, you can have two files with the
> same decoded name, for example, a file called 'foo' and another called
> '\x66oo', the latter with the encoded property set.  This is fixable to
> some extent by making such instances errors, in the same fashion as a
> non-minimal UTF-8 encoding, but the main problem in #1 still exists.

Re: [PATCH] Re: Issue 1954

[Malcolm Rowe <ma...@farside.org.uk>]

On Thu, Jul 20, 2006 at 04:39:10PM +0200, Philipp Marek wrote:
> I'd like to ask you to vote for/against comitting *such a* patch; if the 
> encoding format is defined it's much easier for all people to produce 
> complying clients.
> 

I understand the need for such a feature, but don't see how it can be
introduced before a 2.0 release (at which point, there's definitely a
cleaner way to do it).  A few immediate things come to mind with the
current design, though there are probably more:

1. What happens if you have two files with the same encoded name? (one
encoded, one not?).  For example, two files called 'foo\x09bar' and
'foo^Ibar' (where ^I represents a real TAB character).

2. With the currently-proposed design, you can have two files with the
same decoded name, for example, a file called 'foo' and another called
'\x66oo', the latter with the encoded property set.  This is fixable to
some extent by making such instances errors, in the same fashion as a
non-minimal UTF-8 encoding, but the main problem in #1 still exists.

3. The filesystem code now needs to know about properties in order to work
out to what a filename refers.  That seems wrong, since you can't carry
out operations on the pathname alone (for example, how do you interpret
what the [necessarily encoded] copyfrom path of a file refers to?
You'd need to find the copyfrom'd file before you could check what the
property value was, which you need to do before you know what the name
of the file is...).

4. If I save a dump out and load it into an earlier version of Subversion,
my load can fail as a result of at least #1 above (there may be other
problems too).  This might be solvable under the 'if you use new features
they might not be backward-compatible' mantra, but it would be a major
stretch to apply this to the dumpfile format, which is supposed to be
backward-compatible by default.

Regards,
Malcolm

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org