You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by "Ph. Marek" <ph...@bmlv.gv.at> on 2005/04/04 08:33:31 UTC
Escapes in configuration (svn:ignore in dir-auto-props)
Hello everybody,
here's a first patch to handle escapes in the configuration file.
I tried to keep it as simple as possible.
- May it cause problems, if a svn_string with len=8 has a \0 at index 4?
Shall I disallow NULs?
- This clears the way for dir-auto-props; IIRC that was the last concern.
Questions, remarks welcome.
Regards,
Phil
[[[
Allow escape sequences in configuration files.
* subversion/include/svn_string.h
(svn_stringbuf_do_backslash): New function.
* subversion/libsvn_subr/config_file.c
(parse_option): Call svn_stringbuf_do_backslash for option.
(parse_value): Call svn_stringbuf_do_backslash for value.
* subversion/libsvn_subr/svn_string.c
(svn_stringbuf_do_backslash): New function.
]]]
=== subversion/libsvn_subr/svn_string.c
==================================================================
--- subversion/libsvn_subr/svn_string.c (/mirror/trunk) (revision 537)
+++ subversion/libsvn_subr/svn_string.c (/local/trunk) (revision 537)
@@ -388,6 +388,99 @@
}
+/* returns 1 on success, 0 on error
+ * We allow \\, \000 (octal, 3 characters), \r, \n, and \x00 (2 characters) */
+int
+svn_stringbuf_do_backslash (svn_stringbuf_t *str)
+{
+ char *src, *dst;
+ int i;
+
+ src=dst=str->data;
+ while (*src)
+ {
+ if (src != dst)
+ *dst=*src;
+
+ if (*src == '\\')
+ {
+ src++;
+ str->len--;
+
+ switch (*src)
+ {
+ case '\\':
+ /* simply discard second \\ */
+ break;
+ case 'n':
+ *dst='\n';
+ break;
+ case 'r':
+ *dst='\r';
+ break;
+ case 'x':
+ src++;
+ /* I know isxdigit - but is there something that gives
+ * the value, too? sscanf seems too clumsy, and strtoi
+ * takes more characters, including an "0x" */
+ if (*src>='0' && *src<='9')
+ i=*src-'0';
+ else
+ if (*src>='a' && *src<='f')
+ i=*src-'a'+10;
+ else
+ if (*src>='A' && *src<='F')
+ i=*src-'A'+10;
+ else
+ return 0;
+ *dst=i << 4;
+
+ src++;
+ if (*src>='0' && *src<='9')
+ i=*src-'0';
+ else
+ if (*src>='a' && *src<='f')
+ i=*src-'a'+10;
+ else
+ if (*src>='A' && *src<='F')
+ i=*src-'A'+10;
+ else
+ return 0;
+ *dst |= i;
+ str->len -= 3; /* 4 chars got 1 */
+ break;
+ case '0' ... '3':
+ /* in octal we define the range 000 ... 377 */
+ *dst=(*src-'0') << 6;
+
+ src++;
+ if (*src>='0' && *src<='7')
+ *dst |= (*src-'0') << 3;
+ else
+ return 0;
+
+ src++;
+ if (*src>='0' && *src<='7')
+ *dst |= (*src-'0');
+ else
+ return 0;
+
+ str->len -= 3; /* 4 chars got 1 */
+ break;
+ default:
+ return 0;
+ }
+ }
+
+ src++;
+ dst++;
+ }
+
+ *dst='\0';
+ return 1;
+}
+
+
void
svn_stringbuf_appendstr (svn_stringbuf_t *targetstr,
const svn_stringbuf_t *appendstr)
=== subversion/libsvn_subr/config_file.c
==================================================================
--- subversion/libsvn_subr/config_file.c (/mirror/trunk) (revision 537)
+++ subversion/libsvn_subr/config_file.c (/local/trunk) (revision 537)
@@ -114,6 +114,18 @@
{
/* At end of file. The value is complete, there can't be
any continuation lines. */
+ if (!svn_stringbuf_do_backslash (ctx->value))
+ {
+ svn_error_t *err = SVN_NO_ERROR;
+
+ err = svn_error_createf (SVN_ERR_MALFORMED_FILE, NULL,
+ "%s:%d: invalid escape sequence (\\) in value",
+ svn_path_local_style (ctx->file,
+ ctx->cfg->pool),
+ ctx->line);
+ return err;
+ }
+
svn_config_set (ctx->cfg, ctx->section->data,
ctx->option->data, ctx->value->data);
}
@@ -190,6 +202,15 @@
svn_stringbuf_appendbytes (ctx->option, &char_from_int, 1);
}
+ if (!svn_stringbuf_do_backslash (ctx->option))
+ {
+ err = svn_error_createf (SVN_ERR_MALFORMED_FILE, NULL,
+ "%s:%d: invalid escape sequence (\\) in option name",
+ svn_path_local_style (ctx->file,
+ ctx->cfg->pool),
+ ctx->line);
+ }
+
if (ch != ':' && ch != '=')
{
ch = EOF;
=== subversion/include/svn_string.h
==================================================================
--- subversion/include/svn_string.h (/mirror/trunk) (revision 537)
+++ subversion/include/svn_string.h (/local/trunk) (revision 537)
@@ -337,6 +337,14 @@
const char *separator,
apr_pool_t *pool);
+/** @since New in 1.2.
+ *
+ * Uses the escapes in @a str, and inserts "special" characters.
+ * Currently understood are \n, \r, \x?? (hex), and \??? (octal)
+ * \\ gives a "normal" backslash. */
+int
+svn_stringbuf_do_backslash (svn_stringbuf_t *str);
+
/** @} */
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Escapes in configuration (svn:ignore in dir-auto-props)
Posted by Nicolás Lichtmaier <ni...@panoptico.reloco.com.ar>.
>+ err = svn_error_createf (SVN_ERR_MALFORMED_FILE, NULL,
>+ "%s:%d: invalid escape sequence (\\) in value",
>+ svn_path_local_style (ctx->file,
>+ ctx->cfg->pool),
>
>
>
>+ err = svn_error_createf (SVN_ERR_MALFORMED_FILE, NULL,
>+ "%s:%d: invalid escape sequence (\\) in option name",
>
>
You should mark these strings for translation with _().
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Escapes in configuration (svn:ignore in dir-auto-props)
Posted by Branko Čibej <br...@xbc.nu>.
Ph. Marek wrote:
>On Friday 08 April 2005 18:35, Branko Čibej wrote:
>
>
>>I think all your examples are about lists of values, rather than about
>>escaping in general. For example, when setting svn:ignore, you want a
>>list of values, not necessarly a newline-separated list. What if,
>>instead of an escaping scheme, we come up with an additional syntax for
>>value lists?
>>Instead of
>>
>> svn:ignore=
>> foo
>> bar
>>we could also accept
>> svn:ignore=@list("foo", "bar")
>>
>>
>That's an idea.
>
>
>
>>The list items enclosed in quotes could be subject to escaping (e.g.,
>>for the quites themselves). Something like this could be done in a
>>completely backward-compatible way (except for the pathologic case where
>>someone actually wants to ignore a file named @list(...); but the
>>workaround for that is @list("@list(...)")).
>>
>>
>Problem is, no matter what encoding we take, there'll be someone who uses
>exactly that values. So somebody bites the dust :-)
>
>
Oh, sure. In the end, it's a matter of probability.
>And IMO,
> * = svn:ignore=abc\n
> def\n
> ghi\n
>is not more or less readable than
> * = svn:ignore=@list("abc",
> "def",
> "ghi")
>(both are ugly, in some way)
>
>
Yes, but the first won't work as you expect, because the line
continuation always inserts one space between items on different lines
(and, incidentally, removes other whitespace -- you'd have to be very
careful about how you handle your escaped newlines).
>And how do you get " into filenames? The list isn't easily expandable - any
>script which wants to put new values into that has to account for the braces.
>
>
Um, I did say i'd allow escapint within the quotes. Those escapes, like
the @list itself, would be expanded when the value is interpreted, not
when it's parsed.
>I know that this will probably not matter much to most people - but if there's
>one thing I learned, it's "keep it simple and scriptable" :-).
>
>
>But to get as much backward-compatibility as possible, how about restricting
>this escape sequences to auto-props and dir-auto-props?
>Although it might be necessary to have special characters for the diff-cmd,
>and possibly tunnels - although that's exactly the points where pathnames
>(which have a backslash in windows) are used ...
>
>
Yup. That's where the difference between parsing and interpretation
becomes important.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Escapes in configuration (svn:ignore in dir-auto-props)
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
On Friday 08 April 2005 18:35, Branko Čibej wrote:
> Ph. Marek wrote:
> >1: It's very good if a configuration file is parsed in one way, and only
> > one. Handling some parts with escaping and others without leads to
> > confusion.
>
> I think you're confusing "parsing" and "interpreting the results". We
> already have different ways of interpreting the results; see, for
> example, svn_config_get_bool().
>
> >2: I don't know what can break if there are special characters in other
> >parameters, especially as svn_string uses a length field, but much
> > internal handling still happens on the ->data directly and is
> > \0-sensitive. That's why I asked if NUL characters should be disallowed
> > in unescaping.
>
> Do we want to autoprop binary properties? I'd guess not.
I'd believe it might be necessary. Do you define tab as binary? Form-feed? How
do you enter them into auto-props? (Without putting invisible characters into
the file, which get lost with copy+paste)
> >All in all, I believe that having 1 is of higher priority than 2.
> >In most cases it's much harder to adapt users to something than to adapt
> > some program to the users.
>
> I think all your examples are about lists of values, rather than about
> escaping in general. For example, when setting svn:ignore, you want a
> list of values, not necessarly a newline-separated list. What if,
> instead of an escaping scheme, we come up with an additional syntax for
> value lists?
> Instead of
>
> svn:ignore=
> foo
> bar
> we could also accept
> svn:ignore=@list("foo", "bar")
That's an idea.
> The list items enclosed in quotes could be subject to escaping (e.g.,
> for the quites themselves). Something like this could be done in a
> completely backward-compatible way (except for the pathologic case where
> someone actually wants to ignore a file named @list(...); but the
> workaround for that is @list("@list(...)")).
Problem is, no matter what encoding we take, there'll be someone who uses
exactly that values. So somebody bites the dust :-)
And IMO,
* = svn:ignore=abc\n
def\n
ghi\n
is not more or less readable than
* = svn:ignore=@list("abc",
"def",
"ghi")
(both are ugly, in some way)
And how do you get " into filenames? The list isn't easily expandable - any
script which wants to put new values into that has to account for the braces.
I know that this will probably not matter much to most people - but if there's
one thing I learned, it's "keep it simple and scriptable" :-).
But to get as much backward-compatibility as possible, how about restricting
this escape sequences to auto-props and dir-auto-props?
Although it might be necessary to have special characters for the diff-cmd,
and possibly tunnels - although that's exactly the points where pathnames
(which have a backslash in windows) are used ...
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Escapes in configuration (svn:ignore in dir-auto-props)
Posted by Branko Čibej <br...@xbc.nu>.
Ph. Marek wrote:
>On Friday 08 April 2005 10:32, Branko Čibej wrote:
>
>
>>Ph. Marek wrote:
>>
>>
>>>svn:ignore needs newlines.
>>>
>>>
>>Eh? I though we decided long ago that we won't support weirdness, e.g.
>>newlines, in file names.
>>
>>
>They're not part of the filenames, but seperate the patterns for filenames.
>eg svn:ignore=
> *.o
> *.a
> *.obj
> *~
> *.tmp
>
>
O.K., I see it now. Sorry for being dense.
>1: It's very good if a configuration file is parsed in one way, and only one.
>Handling some parts with escaping and others without leads to confusion.
>
>
I think you're confusing "parsing" and "interpreting the results". We
already have different ways of interpreting the results; see, for
example, svn_config_get_bool().
>2: I don't know what can break if there are special characters in other
>parameters, especially as svn_string uses a length field, but much internal
>handling still happens on the ->data directly and is \0-sensitive.
>That's why I asked if NUL characters should be disallowed in unescaping.
>
>
Do we want to autoprop binary properties? I'd guess not.
>All in all, I believe that having 1 is of higher priority than 2.
>In most cases it's much harder to adapt users to something than to adapt some
>program to the users.
>
>
I think all your examples are about lists of values, rather than about
escaping in general. For example, when setting svn:ignore, you want a
list of values, not necessarly a newline-separated list. What if,
instead of an escaping scheme, we come up with an additional syntax for
value lists?
Instead of
svn:ignore=
foo
bar
we could also accept
svn:ignore=@list("foo", "bar")
The list items enclosed in quotes could be subject to escaping (e.g.,
for the quites themselves). Something like this could be done in a
completely backward-compatible way (except for the pathologic case where
someone actually wants to ignore a file named @list(...); but the
workaround for that is @list("@list(...)")).
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Escapes in configuration (svn:ignore in dir-auto-props)
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
On Friday 08 April 2005 10:32, Branko Čibej wrote:
> Ph. Marek wrote:
> >svn:ignore needs newlines.
>
> Eh? I though we decided long ago that we won't support weirdness, e.g.
> newlines, in file names.
They're not part of the filenames, but seperate the patterns for filenames.
eg svn:ignore=
*.o
*.a
*.obj
*~
*.tmp
> > I expect that some user-defined file-property could
> >need special characters too;
>
> Subversion doesn't interpret user-defined properties. Some sort of user
> app does that. The "unescape string" function could be a public API in
> libsvn_subr, thus available to such a user app.
That's right.
But there should be a way to insert special characters into properties, and it
should be possible with auto-props as well.
> > I'd like to behave auto-properties the same for
> >files and directories, it's hard to explain why some things would be
> >un-escaped and others not.
>
> I still don't understand why we need escaping in auto props.
svn:ignore has patterns, which are seperated with newlines.
There is currently no way to insert newlines in auto-props, so there has to be
some extension.
> >I'd say, that the packagers (eg. debian) should test for this case and
> > provide some support for that.
> >But you're right, on upgrade it's very likely that something will break.
>
> Indeed. And we've never relied on packagers for backward compatibility,
> and I hope we don't start now.
And lots of people will just copy binaries. I know that it's not good, that's
why I told about the openssh-way of changing configuration files.
> >OpenSSH solved a similar problem by using a "authorized_keys2" file
> > instead of "authorized_keys", which was parsed some other way; and by now
> > they've deprecated the old format, and both files are parsed the same
> > way.
> >
> >So possibly we should a "config2" file which has this parsing - people who
> >need that, can rename the file and change the contents.
> >
> >
> >Do Python config files support any way of escaping?
>
> Not according to the Python manuals.
So there's no way to stay compatible with Python _and_ have special
characters. Ok.
> >I've thought about other encoding schemes, but it's always possible (and
> >sometimes likely) that something will break.
>
> Yes. That's why I really, really suggest we have a separate unescape
> function. I wouldn't even object to a svn_config_get_unescaped that
> would automatically call the unescape function before returinig the
> data, but I'm adamantly -1 on the idea that _all_ config values were
> unescaped during parsing.
I'm in two minds about that.
1: It's very good if a configuration file is parsed in one way, and only one.
Handling some parts with escaping and others without leads to confusion.
2: I don't know what can break if there are special characters in other
parameters, especially as svn_string uses a length field, but much internal
handling still happens on the ->data directly and is \0-sensitive.
That's why I asked if NUL characters should be disallowed in unescaping.
All in all, I believe that having 1 is of higher priority than 2.
In most cases it's much harder to adapt users to something than to adapt some
program to the users.
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Escapes in configuration (svn:ignore in dir-auto-props)
Posted by Branko Čibej <br...@xbc.nu>.
Ph. Marek wrote:
>On Friday 08 April 2005 01:54, Branko Čibej wrote:
>
>
>>Ph. Marek wrote:
>>
>>
>>>If I understand you correctly you want several places calling the
>>>un-escape function (for auto-props, dir-auto-props, and authentication)?
>>>
>>>
>>How do these features need escaping?
>>
>>
>svn:ignore needs newlines.
>
Eh? I though we decided long ago that we won't support weirdness, e.g.
newlines, in file names.
> I expect that some user-defined file-property could
>need special characters too;
>
Subversion doesn't interpret user-defined properties. Some sort of user
app does that. The "unescape string" function could be a public API in
libsvn_subr, thus available to such a user app.
> I'd like to behave auto-properties the same for
>files and directories, it's hard to explain why some things would be
>un-escaped and others not.
>
>
I still don't understand why we need escaping in auto props.
>The authentication was a wish of Max.
>
>
>
>>>Well, there's not much I can do.
>>>We need special characters for svn:ignore _and_ for the auth info
>>>(http://marc.theaimsgroup.com/?l=subversion-dev&m=111175528516724&w=2), so
>>>compatibility _has_ to give somewhere.
>>>
>>>
>>It's not just about compatibility with Python, it's also about
>>compatibility with previous versions of Subversion. It'd be very weird
>>if one upgraded svn and suddenly the config files stopped working
>>because, e.g., one was on Windows an put backslashed paths into the
>>config file.
>>
>>
>I'd say, that the packagers (eg. debian) should test for this case and provide
>some support for that.
>But you're right, on upgrade it's very likely that something will break.
>
>
Indeed. And we've never relied on packagers for backward compatibility,
and I hope we don't start now.
>OpenSSH solved a similar problem by using a "authorized_keys2" file instead of
>"authorized_keys", which was parsed some other way; and by now they've
>deprecated the old format, and both files are parsed the same way.
>
>So possibly we should a "config2" file which has this parsing - people who
>need that, can rename the file and change the contents.
>
>
>Do Python config files support any way of escaping?
>
>
Not according to the Python manuals.
>I've thought about other encoding schemes, but it's always possible (and
>sometimes likely) that something will break.
>
>
Yes. That's why I really, really suggest we have a separate unescape
function. I wouldn't even object to a svn_config_get_unescaped that
would automatically call the unescape function before returinig the
data, but I'm adamantly -1 on the idea that _all_ config values were
unescaped during parsing.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Escapes in configuration (svn:ignore in dir-auto-props)
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
On Friday 08 April 2005 01:54, Branko Čibej wrote:
> Ph. Marek wrote:
> >If I understand you correctly you want several places calling the
> > un-escape function (for auto-props, dir-auto-props, and authentication)?
>
> How do these features need escaping?
svn:ignore needs newlines. I expect that some user-defined file-property could
need special characters too; I'd like to behave auto-properties the same for
files and directories, it's hard to explain why some things would be
un-escaped and others not.
The authentication was a wish of Max.
> >Well, there's not much I can do.
> >We need special characters for svn:ignore _and_ for the auth info
> >(http://marc.theaimsgroup.com/?l=subversion-dev&m=111175528516724&w=2), so
> >compatibility _has_ to give somewhere.
>
> It's not just about compatibility with Python, it's also about
> compatibility with previous versions of Subversion. It'd be very weird
> if one upgraded svn and suddenly the config files stopped working
> because, e.g., one was on Windows an put backslashed paths into the
> config file.
I'd say, that the packagers (eg. debian) should test for this case and provide
some support for that.
But you're right, on upgrade it's very likely that something will break.
OpenSSH solved a similar problem by using a "authorized_keys2" file instead of
"authorized_keys", which was parsed some other way; and by now they've
deprecated the old format, and both files are parsed the same way.
So possibly we should a "config2" file which has this parsing - people who
need that, can rename the file and change the contents.
Do Python config files support any way of escaping?
I've thought about other encoding schemes, but it's always possible (and
sometimes likely) that something will break.
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Escapes in configuration (svn:ignore in dir-auto-props)
Posted by Branko Čibej <br...@xbc.nu>.
Ph. Marek wrote:
>On Monday 04 April 2005 22:27, Branko Čibej wrote:
>
>
>>Ph. Marek wrote:
>>
>>
>>>Questions, remarks welcome.
>>>
>>>
>>No, no, no, no and no.
>>
>>This kind of excaping has no place in the option file parser. If you
>>need special interpretation of the config values, do it with a set of
>>utility functions that modify strings (examples are utf en/decoding and
>>XML/URL (un)escaping).
>>
>>
>If I understand you correctly you want several places calling the un-escape
>function (for auto-props, dir-auto-props, and authentication)?
>
>
How do these features need escaping?
>Well, there's not much I can do.
>We need special characters for svn:ignore _and_ for the auth info
>(http://marc.theaimsgroup.com/?l=subversion-dev&m=111175528516724&w=2), so
>compatibility _has_ to give somewhere.
>
>
It's not just about compatibility with Python, it's also about
compatibility with previous versions of Subversion. It'd be very weird
if one upgraded svn and suddenly the config files stopped working
because, e.g., one was on Windows an put backslashed paths into the
config file.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Escapes in configuration (svn:ignore in dir-auto-props)
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
On Monday 04 April 2005 22:27, Branko Čibej wrote:
> Ph. Marek wrote:
> >Questions, remarks welcome.
>
> No, no, no, no and no.
>
> This kind of excaping has no place in the option file parser. If you
> need special interpretation of the config values, do it with a set of
> utility functions that modify strings (examples are utf en/decoding and
> XML/URL (un)escaping).
If I understand you correctly you want several places calling the un-escape
function (for auto-props, dir-auto-props, and authentication)?
That's possible, too, but I don't know which places all need that.
> -1. Our config file format was designed to be compatible with Ptyhon's
> ConfigParser format (except for the case-sensitivity, for which I'm
> still bainging my head against convenient walls). Your change breaks
> that compatibility.
Well, there's not much I can do.
We need special characters for svn:ignore _and_ for the auth info
(http://marc.theaimsgroup.com/?l=subversion-dev&m=111175528516724&w=2), so
compatibility _has_ to give somewhere.
Although you're right - if we change the behaviour only for some parts, most
of the config file stays the same. But I don't believe that buys us anything
- if I understand you correctly, a single escape in the config file leads to
parsing problems with Python?
And in my feeling it's much worse to parse some parts of a file this way and
others that way - a standard way would be much better.
BTW: Python has good SVN bindings. Why not add this parser to the bindings, so
that a python module can ask "how would you parse this config file?"
This would make for a clean way in the future, as we'd have a clean API.
> -- Brane
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org