You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Denis Vlasenko <vd...@googlemail.com> on 2006/07/03 15:28:20 UTC
BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Hello subversion people,
The below command failed for me:
svn co svn://busybox.net/trunk/busybox
in a rather strange way: it started to eat 100% CPU
doing nothing, ctrl-C doesn't work, kill doesn't kill it too,
only kill -9. Repeatable.
After much of fprintf() insertions I found out that
it enters infinite loop here:
/* Convert SRC_LENGTH bytes of SRC_DATA in CONVSET, store the result
in *DEST, which is allocated in POOL. */
static svn_error_t *
convert_to_stringbuf (apr_xlate_t *convset,
const char *src_data,
apr_size_t src_length,
svn_stringbuf_t **dest,
apr_pool_t *pool)
{
apr_size_t buflen = src_length;
apr_status_t apr_err;
apr_size_t srclen = src_length;
apr_size_t destlen = 0;
char *destbuf;
/* Initialize *DEST to an empty stringbuf. */
fprintf(stderr, "%s:%s():%d src_data='%s'\n", __FILE__, __FUNCTION__, __LINE__, src_data); //vda
*dest = svn_stringbuf_create ("", pool);
fprintf(stderr, "%s:%s():%d\n", __FILE__, __FUNCTION__, __LINE__); //vda
destbuf = (*dest)->data;
/* Not only does it not make sense to convert an empty string, but
apr-iconv is quite unreasonable about not allowing that. */
if (src_length == 0)
return SVN_NO_ERROR;
do
{
/* A 1:2 ratio of input characters to output characters should
be enough for most translations, and conveniently enough, if
it isn't, we'll grow the buffer size by 2 again. */
if (destlen == 0)
buflen *= 2;
/* Ensure that *DEST has sufficient storage for the translated
result. */
fprintf(stderr, "%s:%s():%d\n", __FILE__, __FUNCTION__, __LINE__); //vda
svn_stringbuf_ensure (*dest, buflen + 1);
fprintf(stderr, "%s:%s():%d\n", __FILE__, __FUNCTION__, __LINE__); //vda
/* Update the destination buffer pointer to the first character
after already-converted output. */
destbuf = (*dest)->data + (*dest)->len;
/* Set up state variables for xlate. */
destlen = buflen - (*dest)->len;
/* Attempt the conversion. */
fprintf(stderr, "%s:%s():%d src_length=%d srclen=%d destlen=%d\n", __FILE__, __FUNCTION__, __LINE__, src_length, srclen, destlen); //vda
apr_err = apr_xlate_conv_buffer (convset,
src_data + (src_length - srclen),
&srclen,
destbuf,
&destlen);
fprintf(stderr, "%s:%s():%d: apr_err=%d srclen=%d destlen=%d\n", __FILE__, __FUNCTION__, __LINE__, apr_err, srclen, destlen); //vda
/* Now, update the *DEST->len to track the amount of output data
churned out so far from this loop. */
(*dest)->len += ((buflen - (*dest)->len) - destlen);
} while (! apr_err && srclen);
/* If we exited the loop with an error, return the error. */
if (apr_err)
/* Can't use svn_error_wrap_apr here because it calls functions in
this file, leading to infinite recursion. */
{fprintf(stderr, "%s:%s():%d\n", __FILE__, __FUNCTION__, __LINE__); //vda
return svn_error_create (apr_err, NULL, "Can't recode string");
fprintf(stderr, "%s:%s():%d\n", __FILE__, __FUNCTION__, __LINE__); //vda
}
/* Else, exited due to success. Trim the result buffer down to the
right length. */
(*dest)->data[(*dest)->len] = '\0';
return SVN_NO_ERROR;
}
The output is:
../subversion/libsvn_subr/error.c:svn_error_wrap_apr():162
../subversion/libsvn_subr/error.c:svn_error_wrap_apr():164
../subversion/libsvn_subr/error.c:svn_error_wrap_apr():169
/.share/usr/src/subversion-1.1.3/apr/misc/unix/errorcodes.c:apr_strerror():390
../subversion/libsvn_subr/error.c:svn_error_wrap_apr():171 errbuf='.'
../subversion/libsvn_subr/utf.c:svn_utf_cstring_to_utf8():588 src='.'
../subversion/libsvn_subr/utf.c:convert_cstring():559 src='.'
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():319 src_data='.'
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():321
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():339
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():341
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():351 src_length=1 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():357: apr_err=0 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():339
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():341
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():351 src_length=1 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():357: apr_err=0 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():339
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():341
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():351 src_length=1 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():357: apr_err=0 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():339
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():341
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():351 src_length=1 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():357: apr_err=0 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():339
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():341
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():351 src_length=1 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():357: apr_err=0 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():339
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():341
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():351 src_length=1 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():357: apr_err=0 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():339
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():341
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():351 src_length=1 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():357: apr_err=0 srclen=1 destlen=2
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():339
../subversion/libsvn_subr/utf.c:convert_to_stringbuf():341
...<forever>...
I suspected that it has something to do with my locale which
is not C, and indeed, when I unset LC_xxx vars, it worked!
Environment in which it fails:
LC_ALL=ru_RU.koi8r
LC_MESSAGES=ru_RU.koi8r
LANG=ru_RU.koi8r
# locale -a
C
en_US.utf8
POSIX
ru_RU.koi8r
ru_RU.utf8
My system has glibc-2.4 installed if it is important.
In order to get some more data points, I removed all debugging
and then added just this single one in xlate.c:
translated = iconv(convset->ich, (ICONV_INBUF_TYPE)&inbufptr,
inbytes_left, &outbufptr, outbytes_left);
fprintf(stderr, "%s:%s():%d iconv() inbytes_left=%d outbytes_left=%d translated=%d errno=%d inbuf='%s'\n",
__FILE__, __FUNCTION__, __LINE__, *inbytes_left, *outbytes_left, translated, errno, inbuf); //vda
I am getting output like this:
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=5 translated=0 errno=7 inbuf='/root'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=84 inbuf='/root/.subversion'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=22 translated=0 errno=84 inbuf='/root/.subversion/auth'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=33 translated=0 errno=84 inbuf='/root/.subversion/auth/svn.simple'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=35 translated=0 errno=84 inbuf='/root/.subversion/auth/svn.username'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=37 translated=0 errno=84 inbuf='/root/.subversion/auth/svn.ssl.server'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=5 translated=0 errno=84 inbuf='/root'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=23 translated=0 errno=84 inbuf='/etc/subversion/servers'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=25 translated=0 errno=2 inbuf='/root/.subversion/servers'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=5 translated=0 errno=2 inbuf='/root'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=22 translated=0 errno=2 inbuf='/etc/subversion/config'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=24 translated=0 errno=2 inbuf='/root/.subversion/config'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=31 translated=0 errno=2 inbuf='svn://busybox.net/trunk/busybox'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=7 translated=0 errno=0 inbuf='busybox'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=7 translated=0 errno=2 inbuf='busybox'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=12 translated=0 errno=2 inbuf='busybox/.svn'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=12 translated=0 errno=2 inbuf='busybox/.svn'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=2 inbuf='busybox/.svn/lock'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=2 inbuf='busybox/.svn/lock'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=22 translated=0 errno=2 inbuf='busybox/.svn/text-base'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=2 inbuf='busybox/.svn/lock'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=22 translated=0 errno=2 inbuf='busybox/.svn/prop-base'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=2 inbuf='busybox/.svn/lock'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=18 translated=0 errno=2 inbuf='busybox/.svn/props'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=2 inbuf='busybox/.svn/lock'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=20 translated=0 errno=2 inbuf='busybox/.svn/wcprops'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=2 inbuf='busybox/.svn/lock'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=16 translated=0 errno=2 inbuf='busybox/.svn/tmp'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=2 inbuf='busybox/.svn/lock'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=26 translated=0 errno=2 inbuf='busybox/.svn/tmp/text-base'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=2 inbuf='busybox/.svn/lock'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=26 translated=0 errno=2 inbuf='busybox/.svn/tmp/prop-base'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=2 inbuf='busybox/.svn/lock'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=22 translated=0 errno=2 inbuf='busybox/.svn/tmp/props'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=2 inbuf='busybox/.svn/lock'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=24 translated=0 errno=2 inbuf='busybox/.svn/tmp/wcprops'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=24 translated=0 errno=2 inbuf='busybox/.svn/tmp/entries'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=24 translated=0 errno=2 inbuf='busybox/.svn/tmp/entries'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=20 translated=0 errno=2 inbuf='busybox/.svn/entries'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=20 translated=0 errno=2 inbuf='busybox/.svn/entries'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=2 inbuf='busybox/.svn/lock'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=23 translated=0 errno=2 inbuf='busybox/.svn/empty-file'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=27 translated=0 errno=2 inbuf='busybox/.svn/tmp/README.txt'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=27 translated=0 errno=2 inbuf='busybox/.svn/tmp/README.txt'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=23 translated=0 errno=2 inbuf='busybox/.svn/README.txt'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=23 translated=0 errno=2 inbuf='busybox/.svn/README.txt'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=23 translated=0 errno=2 inbuf='busybox/.svn/format.tmp'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=23 translated=0 errno=2 inbuf='busybox/.svn/format.tmp'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=19 translated=0 errno=2 inbuf='busybox/.svn/format'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=19 translated=0 errno=2 inbuf='busybox/.svn/format'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=2 inbuf='busybox/.svn/lock'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=11 translated=0 errno=2 inbuf='.svn/format'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=2 outbytes_left=2 translated=-1 errno=7 inbuf='5.Р╖Ь(Д╖╪JЦ╞ .С╖^M'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=2 outbytes_left=2 translated=-1 errno=7 inbuf='╖^M'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=2 outbytes_left=2 translated=-1 errno=7 inbuf='╖^M'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=2 outbytes_left=2 translated=-1 errno=7 inbuf='╖^M'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=2 outbytes_left=2 translated=-1 errno=7 inbuf='╖^M'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=2 outbytes_left=2 translated=-1 errno=7 inbuf='╖^M'
....forever...
^M in inbuf string is a Ctrl-M, not '^' + 'M'. errno==7 is E2BIG.
In the very same run with LC_xxx variables unset, I am getting:
...
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=17 translated=0 errno=2 inbuf='busybox/.svn/lock'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=11 translated=0 errno=2 inbuf='.svn/format'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=16 outbytes_left=33 translated=-1 errno=84 inbuf='5.З╖Ь8Л╖\>.╞ .Ш╖^M'
xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=19 translated=0 errno=84 inbuf='busybox/.svn/format'
...
and svn happily continues. NB: errno is different!
The code handles E2BIG as non-error:
fprintf(stderr, "%s:%s():%d iconv() inbytes_left=%d outbytes_left=%d translated=%d errno=%d inbuf='%s'\n",
__FILE__, __FUNCTION__, __LINE__, *inbytes_left, *outbytes_left, translated, errno, inbuf); //vda
if (translated == (apr_size_t)-1) {
int rv = errno;
switch (rv) {
case E2BIG: /* out of space on output */
status = 0; /* change table lookup code below if you
make this an error */
break;
Maybe this is a problem?
P.S. inbuf seems like garbage to me. Is that ok?
P.P.S. the full subroutine stack leading to this place is:
svn_io_file_open():
SVN_ERR(svn_path_cstring_from_utf8(&fname_apr, fname, pool));
apr_file_open(new_file, fname_apr, flag | APR_BINARY, perm, pool);
if (status) return svn_error_wrap_apr(status, _("Can't open file '%s'"), svn_path_local_style (fname, pool));
svn_error_wrap_apr():
make_error_internal(status, NULL);
apr_strerror(status, errbuf, sizeof (errbuf));
svn_utf_cstring_to_utf8(&msg_apr, errbuf, err->pool);
svn_utf_cstring_to_utf8():
SVN_ERR(get_ntou_xlate_handle_node(&node, pool));
err = convert_cstring (dest, src, node, pool);
convert_cstring():
SVN_ERR(convert_to_stringbuf(node, src, strlen (src), &destbuf, pool));
convert_to_stringbuf()
...
And it is executed on every file open error, even on those which are
*expected* to fail:
open("busybox/scripts/.svn/text-base/showasm.svn-base", O_RDONLY) = -1 ENOENT (No such file or directory)
There are a lot of such open() calls in normal svn operation
and svn never uses error string (does not print it).
Why do you bother with generating it in the first place?
Maybe you need a "int svn_io_file_open_silent()" which
does not do all this wasted work, but simply returns relevant
errno?
--
vda
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Denis Vlasenko <vd...@googlemail.com>.
On Monday 03 July 2006 22:40, Julian Foad wrote:
> Malcolm Rowe wrote:
> > This is a rather old version of Subversion - could you see if you still
> > get the same behaviour with 1.3.2?
>
> Malcolm, although Denis didn't say so, this was a re-post of a message that he
> posted on 2006-06-26, and on that day he sent this follow-up message:
I repost mainly because I somehow don't see my own messages on the ml.
--
vda
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Mon, Jul 03, 2006 at 09:40:31PM +0100, Julian Foad wrote:
> Malcolm Rowe wrote:
> >This is a rather old version of Subversion - could you see if you still
> >get the same behaviour with 1.3.2?
>
> Malcolm, although Denis didn't say so, this was a re-post of a message that
> he posted on 2006-06-26, and on that day he sent this follow-up message:
>
> Denis Vlasenko wrote:
> >Oh, forgot to add: subversion-1.3.2 did it to me,
> >so I backed out to 1.1.3, and I did all the bug hunting
> >on 1.1.3...
> >
> >I think that the failure is the same, though.
> >At least svn-1.3.2 hangs exactly like 1.1.3...
>
> But I don't know why he went back to the older version :-)
>
Heh, thanks, Julian. I thought I'd seen the message before, but I
obviously didn't remember the followup.
Anyway, I wasn't able to reproduce this myself, so I think we're dependent
upon Denis (or anyone else who can reproduce the problem) doing some
more debugging for us. The main thing I'd like to understand is why we
seemed to be trying to open a file with a garbage name (looks like we
might be looking for a wc root?).
Regards,
Malcolm
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in
locale!=C
Posted by Julian Foad <ju...@btopenworld.com>.
Malcolm Rowe wrote:
> This is a rather old version of Subversion - could you see if you still
> get the same behaviour with 1.3.2?
Malcolm, although Denis didn't say so, this was a re-post of a message that he
posted on 2006-06-26, and on that day he sent this follow-up message:
Denis Vlasenko wrote:
> Oh, forgot to add: subversion-1.3.2 did it to me,
> so I backed out to 1.1.3, and I did all the bug hunting
> on 1.1.3...
>
> I think that the failure is the same, though.
> At least svn-1.3.2 hangs exactly like 1.1.3...
But I don't know why he went back to the older version :-)
- Julian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Tue, Jul 04, 2006 at 03:53:17PM +0200, Denis Vlasenko wrote:
> * why does svn go to such great efforts to generate error string
> (utf-8 conversion, oh my) just to discard it later?
Because at the point that it generates the error, it doesn't know
that it's going to discard it (it usually _won't_ discard it, so this
particular case is an unusual situation).
> * why utf-8 conversion enters infinite loop on garbage input?
> Isn't it a bug?
Probably, but as the version of APR you're using doesn't appear to
support glibc 2.4, it's hard to say exactly where the fault is.
> * why I don't see my own mails from the ml? ;)
>
Pass :-). Ask Google.
Regards,
Malcolm
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Denis Vlasenko <vd...@googlemail.com>.
On Tuesday 04 July 2006 15:41, Malcolm Rowe wrote:
> > This is it. I am using stock svn-1.3.2 source, apr/CHANGES starts with:
> > Changes with APR 0.9.7
> > ...
> >
> > This means that APR version is 0.9.7. Am I toast? ;)
>
> Well, yes and no :-)
>
> In theory, it should be fairly easy to switch to an alternate APR version,
> but I'm not sure exactly what steps you need to follow having started
> with a release tarball. Possibly replace the 'apr' and 'apr-util'
> directories and re-run autogen.sh, though I can't promise that will work.
I'll wait for newer svn, then. Not a great problem.
The few non-closed questions remain:
* why does svn go to such great efforts to generate error string
(utf-8 conversion, oh my) just to discard it later?
* why utf-8 conversion enters infinite loop on garbage input?
Isn't it a bug?
* why I don't see my own mails from the ml? ;)
--
vda
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Tue, Jul 04, 2006 at 03:04:13PM +0200, Denis Vlasenko wrote:
> On Tuesday 04 July 2006 14:51, Malcolm Rowe wrote:
> > Sounds very much like this is a glibc problem (or possibly an
> > APR/glibc-2.4 problem).
> >
> > Which version of APR are you using? Ah, is it pre-APR 0.9.9?
> > From APR 0.9.9's CHANGES file:
> >
> > *) Fix apr_strerror() with glibc 2.4. [Joe Orton]
>
> This is it. I am using stock svn-1.3.2 source, apr/CHANGES starts with:
> Changes with APR 0.9.7
> ...
>
> This means that APR version is 0.9.7. Am I toast? ;)
Well, yes and no :-)
In theory, it should be fairly easy to switch to an alternate APR version,
but I'm not sure exactly what steps you need to follow having started
with a release tarball. Possibly replace the 'apr' and 'apr-util'
directories and re-run autogen.sh, though I can't promise that will work.
I don't know which distributions are shipping glibc 2.4 nowdays - perhaps
we should be distributing a later version of APR in the source tarballs?
Regards,
Malcolm
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Denis Vlasenko <vd...@googlemail.com>.
On Tuesday 04 July 2006 14:51, Malcolm Rowe wrote:
> On Tue, Jul 04, 2006 at 01:53:59PM +0200, Denis Vlasenko wrote:
> > It _is_ glibc 2.4. Looks like glibc people switched to POSIX version
> > of strerror_r. See here:
> >
> > http://sources.redhat.com/bugzilla/show_bug.cgi?id=364
>
> Ah, my mistake. I guess the APR comment is out of date.
>
> > But sometimes it returns >=0, yet buffer is not modified.
>
> Sounds very much like this is a glibc problem (or possibly an
> APR/glibc-2.4 problem).
>
> Which version of APR are you using? Ah, is it pre-APR 0.9.9?
> From APR 0.9.9's CHANGES file:
>
> *) Fix apr_strerror() with glibc 2.4. [Joe Orton]
This is it. I am using stock svn-1.3.2 source, apr/CHANGES starts with:
Changes with APR 0.9.7
...
This means that APR version is 0.9.7. Am I toast? ;)
--
vda
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Tue, Jul 04, 2006 at 03:28:38PM +0200, Denis Vlasenko wrote:
> This is it. We use POSIX prototype, yet somehow link with glibc's strerror_r
> which returns char*:
>
Yes, I imagine that's what's causing the problems here. The version
of APR we shipped with 1.3.2 doesn't appear to work - in some cases -
with glibc 2.4.
Regards,
Malcolm
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Denis Vlasenko <vd...@googlemail.com>.
On Tuesday 04 July 2006 14:51, Malcolm Rowe wrote:
> Sounds very much like this is a glibc problem (or possibly an
> APR/glibc-2.4 problem).
>
> Which version of APR are you using? Ah, is it pre-APR 0.9.9?
> From APR 0.9.9's CHANGES file:
>
> *) Fix apr_strerror() with glibc 2.4. [Joe Orton]
Something very fishy here...
static char *native_strerror(apr_status_t statcode, char *buf,
apr_size_t bufsize)
{
int r = strerror_r(statcode, buf, bufsize);
if (r < 0) {
//enum { r = 0 };
//if (strerror_r(statcode, buf, bufsize) < 0) {
fprintf(stderr, "%s: strerror_r()==%d, return stuffbuffer('APR does not understand this error code')\n", __FUNCTION__, r); //vda
return stuffbuffer(buf, bufsize,
"APR does not understand this error code");
}
else {
fprintf(stderr, "%s: strerror_r(%d,buf,%d)==%d, return '%s'\n", __FUNCTION__, statcode, bufsize, r, buf); //vda
return buf;
}
}
The above code works more-or-less okay:
...
apr_strerror: native_strerror
native_strerror: strerror_r()==-1481340491, return stuffbuffer('APR does not understand this error code')
...
but if I delete "int r = strerror_r(statcode, buf, bufsize); if (r < 0) {"
lines and uncomment next two ones, it starts to return someting >=0 from strerror_r()
but does not modify buf, and subsequently enters infinite lop.
This is it. We use POSIX prototype, yet somehow link with glibc's strerror_r
which returns char*:
# nm errorcodes.o | grep strerror
00000026 T apr_strerror
U gai_strerror
U hstrerror
U strerror_r
A small test program
#include <errno.h>
#include <string.h>
#include <stdio.h>
int main() {
char buf[1024] = "junk";
strerror_r(ERANGE, buf, 1024);
return 0;
}
has different symbol referenced, the POSIX one:
# nm t.o | grep strerror
U __xpg_strerror_r
--
vda
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Tue, Jul 04, 2006 at 01:53:59PM +0200, Denis Vlasenko wrote:
> It _is_ glibc 2.4. Looks like glibc people switched to POSIX version
> of strerror_r. See here:
>
> http://sources.redhat.com/bugzilla/show_bug.cgi?id=364
>
Ah, my mistake. I guess the APR comment is out of date.
> But sometimes it returns >=0, yet buffer is not modified.
Sounds very much like this is a glibc problem (or possibly an
APR/glibc-2.4 problem).
Which version of APR are you using? Ah, is it pre-APR 0.9.9?
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Denis Vlasenko <vd...@googlemail.com>.
On Tuesday 04 July 2006 13:43, Malcolm Rowe wrote:
> > My native_strerror currently looks like this:
> >
> > static char *native_strerror(apr_status_t statcode, char *buf,
> > apr_size_t bufsize)
> > {
> > if (strerror_r(statcode, buf, bufsize) < 0) {
> > fprintf(stderr, "%s: strerror_r()<0, return stuffbuffer('APR does not understand this error code')\n", __FUNCTION__); //vda
> > return stuffbuffer(buf, bufsize,
> > "APR does not understand this error code");
> > }
> > else {
> > fprintf(stderr, "%s: return '%s'\n", __FUNCTION__, buf); //vda
> > return buf;
> > }
> > }
> >
>
> If your native_strerror() looks like that, you must have STRERROR_R_RC_INT
> defined (in APR's include/arch/unix/apr_private.h), which implies you're
> using the platform-native C library for either AIX or Tru64 (from the
> comments in the above file).
>
> I thought you were using glibc? What OS are you using?
It _is_ glibc 2.4. Looks like glibc people switched to POSIX version
of strerror_r. See here:
http://sources.redhat.com/bugzilla/show_bug.cgi?id=364
But sometimes it returns >=0, yet buffer is not modified.
--
vda
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Tue, Jul 04, 2006 at 11:49:27AM +0200, Denis Vlasenko wrote:
> svn_io_file_open: fname='busybox/.svn/prop-base/AUTHORS.svn-base'
I assume that file doesn't exist?
> svn_error_wrap_apr: errbuf='Ф.╩╖ю╓ф╖Б.Б╖4ПА╞.ПА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.' (before), status=2
OS error 2 is usually 'file not found'.
> apr_strerror: native_strerror
> native_strerror: return 'Ф.╩╖ю╓ф╖Б.Б╖4ПА╞.ПА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.'
Implying that APR's native_strerror didn't change the buffer that was
passed to it, quite possibly the cause of these problems.
> My native_strerror currently looks like this:
>
> static char *native_strerror(apr_status_t statcode, char *buf,
> apr_size_t bufsize)
> {
> if (strerror_r(statcode, buf, bufsize) < 0) {
> fprintf(stderr, "%s: strerror_r()<0, return stuffbuffer('APR does not understand this error code')\n", __FUNCTION__); //vda
> return stuffbuffer(buf, bufsize,
> "APR does not understand this error code");
> }
> else {
> fprintf(stderr, "%s: return '%s'\n", __FUNCTION__, buf); //vda
> return buf;
> }
> }
>
If your native_strerror() looks like that, you must have STRERROR_R_RC_INT
defined (in APR's include/arch/unix/apr_private.h), which implies you're
using the platform-native C library for either AIX or Tru64 (from the
comments in the above file).
I thought you were using glibc? What OS are you using?
Regards,
Malcolm
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Denis Vlasenko <vd...@googlemail.com>.
On Tuesday 04 July 2006 10:59, Malcolm Rowe wrote:
> On Tue, Jul 04, 2006 at 10:16:41AM +0200, Denis Vlasenko wrote:
> > On Monday 03 July 2006 18:33, Malcolm Rowe wrote:
> > > > xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=11 translated=0 errno=2 inbuf='.svn/format'
> > > > xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=2 outbytes_left=2 translated=-1 errno=7 inbuf='5.Р╖Ь(Д╖╪JЦ╞ .С╖^M'
> > >
> > > ... could you get a backtrace from the svn_io_file_open() call that opens
> > > this file? Yes, if we're trying to open a file with a garbage name,
> > > that's probably a bug.
> >
> > The call chain is:
> >
> > svn_io_file_open():
>
> I was actually more interested in a backtrace - i.e. where the call was
> coming from. But it occurs to me that there's no way we can tell that
> the string above is actually supposed to be a filename, so that probably
> doesn't help.
>
> > The filename is passed to svn_error_wrap_apr() in third argument,
> > but svn_error_wrap_apr() is a variadic function and:
> >
> > svn_error_t *
> > svn_error_wrap_apr (apr_status_t status,
> > const char *fmt,
> > ...)
> > {
> > svn_error_t *err, *utf8_err;
> > va_list ap;
> > char errbuf[255];
> > const char *msg_apr, *msg;
> >
> > err = make_error_internal (status, NULL);
> >
> > if (fmt)
> > {
> > /* Grab the APR error message. */
> > apr_strerror (status, errbuf, sizeof (errbuf));
> > utf8_err = svn_utf_cstring_to_utf8 (&msg_apr, errbuf, err->pool); // <---- HERE
> > if (utf8_err)
> > msg_apr = NULL;
> > svn_error_clear (utf8_err);
> >
> > /* Append it to the formatted message. */
> > va_start (ap, fmt);
> > msg = apr_pvsprintf (err->pool, fmt, ap);
> > va_end (ap);
> > err->message = apr_psprintf (err->pool, "%s%s%s", msg,
> > (msg_apr) ? ": " : "",
> > (msg_apr) ? msg_apr : "");
> > }
> >
> > return err;
> > }
> >
> > See? we call svn_utf_cstring_to_utf8() but it has no chance in hell
> > to get a filename, because va_start() is not yet called,
> > and therefore it has no way to obtain 3rd argument.
> >
>
> No, that's fine. The generated error message is in the form
> "<fmt,variadic arguments>: <APR error message>". In our example, it
> might be something like "Can't open file 'foo': File not found".
>
> The first section (including the line you highlighted) only looks up
> the APR error message, which doesn't contain any format strings, and
> doesn't reference the filename.
>
> > I inserted a fprintf(stderr, "%s: errbuf='%s'\n", __FUNCTION__, errbuf)
> > directly after apr_strerror (status, errbuf, sizeof (errbuf))
> > call and I see this:
> >
> > svn_io_file_open: fname='busybox/.svn/dir-props'
> > svn_error_wrap_apr: errbuf='ФЁ╧╖юдд╖.╗Ю╖..╟╞h.╟╞A.Ю╖юдд╖.╗Ю╖Л╖Ю╖a.'
>
> That is interesting though. I'd be interested to know what 'status' is at
> this point, and whether errbuf is just what was in the buffer originally,
> or whether it's been filled by apr_strerror() [and in that case, is it
> possible that it's the native error message in the native codepage?].
Added a bit more debugging. In ru locale:
svn_io_file_open: fname='busybox/.svn/prop-base/AUTHORS.svn-base'
svn_error_wrap_apr: errbuf='Ф.╩╖ю╓ф╖Б.Б╖4ПА╞.ПА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.' (before), status=2
apr_strerror: native_strerror
native_strerror: return 'Ф.╩╖ю╓ф╖Б.Б╖4ПА╞.ПА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.'
svn_error_wrap_apr: errbuf='Ф.╩╖ю╓ф╖Б.Б╖4ПА╞.ПА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=6 outbytes_left=0 translated=-1 errno=7 inbuf='Ф.╩╖ю╓ф╖Б.Б╖4ПА╞.ПА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=0 outbytes_left=64 translated=0 errno=7 inbuf='л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=0 outbytes_left=39 translated=0 errno=7 inbuf='busybox/.svn/prop-base/AUTHORS.svn-base'
svn_io_file_open: fname='busybox/.svn/prop-base/AUTHORS.svn-base'
svn_error_wrap_apr: errbuf='Ф.╩╖ю╓ф╖Б.Б╖.ОА╞эОА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.' (before), status=2
apr_strerror: native_strerror
native_strerror: return 'Ф.╩╖ю╓ф╖Б.Б╖.ОА╞эОА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.'
svn_error_wrap_apr: errbuf='Ф.╩╖ю╓ф╖Б.Б╖.ОА╞эОА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=7 outbytes_left=2 translated=-1 errno=7 inbuf='Ф.╩╖ю╓ф╖Б.Б╖.ОА╞эОА╞A~Б╖ю╓ф╖Б.Б╖л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=7 outbytes_left=2 translated=-1 errno=7 inbuf='╖л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=7 outbytes_left=2 translated=-1 errno=7 inbuf='╖л.Б╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=7 outbytes_left=2 translated=-1 errno=7 inbuf='╖л.Б╖a.'
...infinite loop...
NB: in C locale it _also_ has some garbage in errbuf, but iconv()
returns different errno and therefore we do not loop.
svn_io_file_open: fname='busybox/.svn/prop-base/AUTHORS.svn-base'
svn_error_wrap_apr: errbuf='Фsф╖ю.я╖БgМ╖.п╪╞`п╪╞A^М╖ю.я╖БgМ╖лgМ╖a.' (before), status=2
apr_strerror: native_strerror
native_strerror: return 'Фsф╖ю.я╖БgМ╖.п╪╞`п╪╞A^М╖ю.я╖БgМ╖лgМ╖a.'
svn_error_wrap_apr: errbuf='Фsф╖ю.я╖БgМ╖.п╪╞`п╪╞A^М╖ю.я╖БgМ╖лgМ╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=38 outbytes_left=76 translated=-1 errno=84 inbuf='Фsф╖ю.я╖БgМ╖.п╪╞`п╪╞A^М╖ю.я╖БgМ╖лgМ╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=0 outbytes_left=39 translated=0 errno=84 inbuf='busybox/.svn/prop-base/AUTHORS.svn-base'
svn_io_file_open: fname='busybox/.svn/prop-base/AUTHORS.svn-base'
svn_error_wrap_apr: errbuf='Фsф╖ю.я╖БgМ╖pо╪╞╪о╪╞A^М╖ю.я╖БgМ╖лgМ╖a.' (before), status=2
apr_strerror: native_strerror
native_strerror: return 'Фsф╖ю.я╖БgМ╖pо╪╞╪о╪╞A^М╖ю.я╖БgМ╖лgМ╖a.'
svn_error_wrap_apr: errbuf='Фsф╖ю.я╖БgМ╖pо╪╞╪о╪╞A^М╖ю.я╖БgМ╖лgМ╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=38 outbytes_left=76 translated=-1 errno=84 inbuf='Фsф╖ю.я╖БgМ╖pо╪╞╪о╪╞A^М╖ю.я╖БgМ╖лgМ╖a.'
apr_xlate_conv_buffer:353 iconv() inbytes_left=0 outbytes_left=35 translated=0 errno=84 inbuf='busybox/.svn/props/AUTHORS.svn-work'
and it continues!
My native_strerror currently looks like this:
static char *native_strerror(apr_status_t statcode, char *buf,
apr_size_t bufsize)
{
if (strerror_r(statcode, buf, bufsize) < 0) {
fprintf(stderr, "%s: strerror_r()<0, return stuffbuffer('APR does not understand this error code')\n", __FUNCTION__); //vda
return stuffbuffer(buf, bufsize,
"APR does not understand this error code");
}
else {
fprintf(stderr, "%s: return '%s'\n", __FUNCTION__, buf); //vda
return buf;
}
}
Hmm.... "man strerr_r" says:
#include <string.h>
char *strerror(int errnum);
char *strerror_r(int errnum, char *buf, size_t buflen);
/* GNU-specific strerror_r() */
#define _XOPEN_SOURCE 600
#include <string.h>
int strerror_r(int errnum, char *buf, size_t buflen);
/* XSI-compliant strerror_r() */
...
The XSI-compliant strerror_r() is preferred for portable applications.
It returns the error string in the user-supplied buffer buf of length
buflen.
The GNU-specific strerror_r() returns a pointer to a string containing
the error message. This may be either a pointer to a string that the
function stores in buf, or a pointer to some (immutable) static string
(in which case buf is unused). If the function stores a string in buf,
then at most buflen bytes are stored (the string may be truncated if
buflen is too small) and the string always includes a terminating null
byte.
Can this be the reason?
--
vda
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Tue, Jul 04, 2006 at 10:16:41AM +0200, Denis Vlasenko wrote:
> On Monday 03 July 2006 18:33, Malcolm Rowe wrote:
> > > xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=11 translated=0 errno=2 inbuf='.svn/format'
> > > xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=2 outbytes_left=2 translated=-1 errno=7 inbuf='5.Р╖Ь(Д╖╪JЦ╞ .С╖^M'
> >
> > ... could you get a backtrace from the svn_io_file_open() call that opens
> > this file? Yes, if we're trying to open a file with a garbage name,
> > that's probably a bug.
>
> The call chain is:
>
> svn_io_file_open():
I was actually more interested in a backtrace - i.e. where the call was
coming from. But it occurs to me that there's no way we can tell that
the string above is actually supposed to be a filename, so that probably
doesn't help.
> The filename is passed to svn_error_wrap_apr() in third argument,
> but svn_error_wrap_apr() is a variadic function and:
>
> svn_error_t *
> svn_error_wrap_apr (apr_status_t status,
> const char *fmt,
> ...)
> {
> svn_error_t *err, *utf8_err;
> va_list ap;
> char errbuf[255];
> const char *msg_apr, *msg;
>
> err = make_error_internal (status, NULL);
>
> if (fmt)
> {
> /* Grab the APR error message. */
> apr_strerror (status, errbuf, sizeof (errbuf));
> utf8_err = svn_utf_cstring_to_utf8 (&msg_apr, errbuf, err->pool); // <---- HERE
> if (utf8_err)
> msg_apr = NULL;
> svn_error_clear (utf8_err);
>
> /* Append it to the formatted message. */
> va_start (ap, fmt);
> msg = apr_pvsprintf (err->pool, fmt, ap);
> va_end (ap);
> err->message = apr_psprintf (err->pool, "%s%s%s", msg,
> (msg_apr) ? ": " : "",
> (msg_apr) ? msg_apr : "");
> }
>
> return err;
> }
>
> See? we call svn_utf_cstring_to_utf8() but it has no chance in hell
> to get a filename, because va_start() is not yet called,
> and therefore it has no way to obtain 3rd argument.
>
No, that's fine. The generated error message is in the form
"<fmt,variadic arguments>: <APR error message>". In our example, it
might be something like "Can't open file 'foo': File not found".
The first section (including the line you highlighted) only looks up
the APR error message, which doesn't contain any format strings, and
doesn't reference the filename.
> I inserted a fprintf(stderr, "%s: errbuf='%s'\n", __FUNCTION__, errbuf)
> directly after apr_strerror (status, errbuf, sizeof (errbuf))
> call and I see this:
>
> svn_io_file_open: fname='busybox/.svn/dir-props'
> svn_error_wrap_apr: errbuf='ФЁ╧╖юдд╖.╗Ю╖..╟╞h.╟╞A.Ю╖юдд╖.╗Ю╖Л╖Ю╖a.'
That is interesting though. I'd be interested to know what 'status' is at
this point, and whether errbuf is just what was in the buffer originally,
or whether it's been filled by apr_strerror() [and in that case, is it
possible that it's the native error message in the native codepage?].
Regards,
Malcolm
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Denis Vlasenko <vd...@googlemail.com>.
On Monday 03 July 2006 18:33, Malcolm Rowe wrote:
> On Mon, Jul 03, 2006 at 05:28:20PM +0200, Denis Vlasenko wrote:
> > /.share/usr/src/subversion-1.1.3/apr/misc/unix/errorcodes.c:apr_strerror():390
>
> This is a rather old version of Subversion - could you see if you still
> get the same behaviour with 1.3.2?
Yes, I forgot to mention it. I tried the same on 1.3.2,
still fails.
> And if you do...
>
> > xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=11 translated=0 errno=2 inbuf='.svn/format'
> > xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=2 outbytes_left=2 translated=-1 errno=7 inbuf='5.Р╖Ь(Д╖╪JЦ╞ .С╖^M'
>
> ... could you get a backtrace from the svn_io_file_open() call that opens
> this file? Yes, if we're trying to open a file with a garbage name,
> that's probably a bug.
The call chain is:
svn_io_file_open():
SVN_ERR(svn_path_cstring_from_utf8(&fname_apr, fname, pool));
apr_file_open(new_file, fname_apr, flag | APR_BINARY, perm, pool);
if (status) return svn_error_wrap_apr(status, _("Can't open file '%s'"), svn_path_local_style (fname, pool));
svn_error_wrap_apr():
make_error_internal(status, NULL);
apr_strerror(status, errbuf, sizeof (errbuf));
svn_utf_cstring_to_utf8(&msg_apr, errbuf, err->pool);
svn_utf_cstring_to_utf8():
SVN_ERR(get_ntou_xlate_handle_node(&node, pool));
err = convert_cstring (dest, src, node, pool);
convert_cstring():
SVN_ERR(convert_to_stringbuf(node, src, strlen (src), &destbuf, pool));
convert_to_stringbuf()
...
I think I see the problem:
svn_error_t *
svn_io_file_open (apr_file_t **new_file, const char *fname,
apr_int32_t flag, apr_fileperms_t perm,
apr_pool_t *pool)
{
const char *fname_apr;
apr_status_t status;
SVN_ERR (svn_path_cstring_from_utf8 (&fname_apr, fname, pool));
status = apr_file_open (new_file, fname_apr, flag | APR_BINARY, perm, pool);
if (status)
return svn_error_wrap_apr (status, _("Can't open file '%s'"),
svn_path_local_style (fname, pool));
else
return SVN_NO_ERROR;
}
Above:
The filename is passed to svn_error_wrap_apr() in third argument,
but svn_error_wrap_apr() is a variadic function and:
svn_error_t *
svn_error_wrap_apr (apr_status_t status,
const char *fmt,
...)
{
svn_error_t *err, *utf8_err;
va_list ap;
char errbuf[255];
const char *msg_apr, *msg;
err = make_error_internal (status, NULL);
if (fmt)
{
/* Grab the APR error message. */
apr_strerror (status, errbuf, sizeof (errbuf));
utf8_err = svn_utf_cstring_to_utf8 (&msg_apr, errbuf, err->pool); // <---- HERE
if (utf8_err)
msg_apr = NULL;
svn_error_clear (utf8_err);
/* Append it to the formatted message. */
va_start (ap, fmt);
msg = apr_pvsprintf (err->pool, fmt, ap);
va_end (ap);
err->message = apr_psprintf (err->pool, "%s%s%s", msg,
(msg_apr) ? ": " : "",
(msg_apr) ? msg_apr : "");
}
return err;
}
See? we call svn_utf_cstring_to_utf8() but it has no chance in hell
to get a filename, because va_start() is not yet called,
and therefore it has no way to obtain 3rd argument.
I inserted a fprintf(stderr, "%s: errbuf='%s'\n", __FUNCTION__, errbuf)
directly after apr_strerror (status, errbuf, sizeof (errbuf))
call and I see this:
svn_io_file_open: fname='busybox/.svn/dir-props'
svn_error_wrap_apr: errbuf='ФЁ╧╖юдд╖.╗Ю╖..╟╞h.╟╞A.Ю╖юдд╖.╗Ю╖Л╖Ю╖a.'
apr_xlate_conv_buffer():353 iconv() inbytes_left=4 outbytes_left=1 translated=-1 errno=7 inbuf='ФЁ╧╖юдд╖.╗Ю╖..╟╞h.╟╞A.Ю╖юдд╖.╗Ю╖Л╖Ю╖a.'
apr_xlate_conv_buffer():353 iconv() inbytes_left=4 outbytes_left=1 translated=-1 errno=7 inbuf='Ю╖a.'
apr_xlate_conv_buffer():353 iconv() inbytes_left=4 outbytes_left=1 translated=-1 errno=7 inbuf='Ю╖a.'
apr_xlate_conv_buffer():353 iconv() inbytes_left=4 outbytes_left=1 translated=-1 errno=7 inbuf='Ю╖a.'
That probably explains why the bug shows variable behaviour
(it happens in semi-random place). Seems like it is using
some uninitialized data (stack?).
However, no matter what garbage we feed to utf8 converter,
it still have to convert it, should not loop forever, right?
--
vda
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BUG: svn enters unkillable state, tracked down to UTF conv in locale!=C
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Mon, Jul 03, 2006 at 05:28:20PM +0200, Denis Vlasenko wrote:
> /.share/usr/src/subversion-1.1.3/apr/misc/unix/errorcodes.c:apr_strerror():390
This is a rather old version of Subversion - could you see if you still
get the same behaviour with 1.3.2?
And if you do...
> xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=0 outbytes_left=11 translated=0 errno=2 inbuf='.svn/format'
> xlate.c:apr_xlate_conv_buffer():353 iconv() inbytes_left=2 outbytes_left=2 translated=-1 errno=7 inbuf='5.Р╖Ь(Д╖╪JЦ╞ .С╖^M'
... could you get a backtrace from the svn_io_file_open() call that opens
this file? Yes, if we're trying to open a file with a garbage name,
that's probably a bug.
Regards,
Malcolm
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org