You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by David Wilson <dw...@botanicus.net> on 2005/01/14 23:41:33 UTC
[BUG] Resource consumption problem in clients/cmdline/prompt.c::prompt().
Good evening folks,
At around midnight on the 13th of January, my nightly svn cronjob
spiralled out of control and brought my host's FreeBSD machine to a near
halt. On further exploration, it turns out that my SSL certificate
expired on that day (https://china.botanicus.net/).
Cron starts processes with stdin connected to /dev/null, or some
equivalent that causes 0-byte reads.
Poking through the Subversion sources, I found a while loop in
clients/cmdline/prompt.c::prompt() that repeats until the platform
specific EOL is detected.
That loop calls prompt.c::wait_for_input(), which calls apr_poll(),
which uses `pool' that was passed to prompt(). The problem appears to
appear in the tests that follow line 114:
114: status = apr_file_getc (&c, fp);
I don't yet understand enough about APR/svn to determine what is going
wrong, but I assume apr_file_getc() is returning success and setting
'&c' to a NUL character, or returning a status value that is not
accounted for.
To reproduce:
1) Start a terminal with /usr/bin/top running (or something that can
see svn processes' resource usage).
2) In another terminal, execute:
$ svn co https://china.botanicus.net/svn/cdr_mysql < /dev/null
I suppose anything that calls prompt() will trigger the bug, but this
particular command line will do it. Well, at least until I renew my SSL
certificate. :)
Thanks,
David.
--
Implementations are always preferred over rhetoric.
-- Robert Brewer on comp.lang.python
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH] Detect EOF during propmt
Posted by Julian Foad <ju...@btopenworld.com>.
Julian Foad wrote:
> Sorry, Philip - not enough words for someone like me to understand.
> (Apologies if it is obvious from some previous message that I haven't
> read.) [...]
I've now found a previous thread that gives the background: "[BUG] Resource
consumption problem in clients/cmdline/prompt.c::prompt()." started by David
Wilson yesterday.
- Julian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH] Detect EOF during propmt
Posted by Philip Martin <ph...@codematters.co.uk>.
"C. Michael Pilato" <cm...@collab.net> writes:
> Julian Foad <ju...@btopenworld.com> writes:
>
>> Philip Martin wrote:
>> > David Wilson <dw...@botanicus.net> writes:
>> >
>> >> $ svn co https://china.botanicus.net/svn/cdr_mysql < /dev/null
>> > If you are running svn from a crontab then using --non-interactive
>> > will cause it to fail rather than spin.
>>
> The --non-interactive options tells Subversion to never prompt for
> auth information. In this case, if Subversion doesn't prompt, it
> doesn't hang.
The original bug does not involve hanging, rather, the client is
running flat-out using CPU to continually ignore eof and allocate
memory. If the client process isn't killed it will probably bring the
box running it to a grinding halt.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH] Detect EOF during propmt
Posted by "C. Michael Pilato" <cm...@collab.net>.
Julian Foad <ju...@btopenworld.com> writes:
> Philip Martin wrote:
> > David Wilson <dw...@botanicus.net> writes:
> >
> >> $ svn co https://china.botanicus.net/svn/cdr_mysql < /dev/null
> > If you are running svn from a crontab then using --non-interactive
> > will cause it to fail rather than spin.
>
> Sorry, Philip - not enough words for someone like me to understand.
> (Apologies if it is obvious from some previous message that I haven't
> read.) Probably someone who is familiar with the problem will
> understand and reply. If not, what's the behaviour seen by David
> Wilson, what is the desired behaviour, what sort of failure, what
> spins for what?
The problem seen is that Subversion needs auth info to perform the
checkout, so by default, it prompts for that information. The problem
is that, while running from a cron job, there's no way to actually
respond to those prompts, so Subversion just hangs waiting for input
it will never get.
The --non-interactive options tells Subversion to never prompt for
auth information. In this case, if Subversion doesn't prompt, it
doesn't hang. But then, it doesn't get the auth info it needs either,
so it will simply return an error. But at least it doesn't hang!
Of course, if the repository requires only simple username/password
auth input, then the cron job can use --non-interactive, --username,
and --password to achieve success. More complex scenarios might
involve tweaking runtime config options (for the likes of cert path
and passphrase info, etc).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH] Detect EOF during propmt
Posted by Julian Foad <ju...@btopenworld.com>.
Philip Martin wrote:
> David Wilson <dw...@botanicus.net> writes:
>
>> $ svn co https://china.botanicus.net/svn/cdr_mysql < /dev/null
>
> If you are running svn from a crontab then using --non-interactive
> will cause it to fail rather than spin.
Sorry, Philip - not enough words for someone like me to understand. (Apologies
if it is obvious from some previous message that I haven't read.) Probably
someone who is familiar with the problem will understand and reply. If not,
what's the behaviour seen by David Wilson, what is the desired behaviour, what
sort of failure, what spins for what?
- Julian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
[PATCH] Detect EOF during propmt
Posted by Philip Martin <ph...@codematters.co.uk>.
David Wilson <dw...@botanicus.net> writes:
> $ svn co https://china.botanicus.net/svn/cdr_mysql < /dev/null
If you are running svn from a crontab then using --non-interactive
will cause it to fail rather than spin.
Does anybody understand why EOF is ignored? Even if it's some magic
Window's stuff, I don't think it's sensible to use c after EOF. How
about this:
Index: subversion/clients/cmdline/prompt.c
===================================================================
--- subversion/clients/cmdline/prompt.c (revision 12736)
+++ subversion/clients/cmdline/prompt.c (working copy)
@@ -112,7 +112,7 @@
return svn_error_wrap_apr (status, _("Can't read stdin"));
status = apr_file_getc (&c, fp);
- if (status && ! APR_STATUS_IS_EOF(status))
+ if (status)
return svn_error_wrap_apr (status, _("Can't read stdin"));
if (saw_first_half_of_eol)
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [BUG] Resource consumption problem in clients/cmdline/prompt.c::prompt().
Posted by Michael W Thelen <mi...@pietdepsi.com>.
David Wilson wrote:
> At around midnight on the 13th of January, my nightly svn cronjob
> spiralled out of control and brought my host's FreeBSD machine to a near
> halt. On further exploration, it turns out that my SSL certificate
> expired on that day (https://china.botanicus.net/).
>
> Cron starts processes with stdin connected to /dev/null, or some
> equivalent that causes 0-byte reads.
I had an extremely similar problem back on July 19. If I remember
correctly, it happened when I ran svn checkout in a cron job, but with
an incorrect username/password. I vaguely remember discussing the
problem in IRC with Ben Reser (I think). Looking through the email
archives, I can't find any messages about it, though. We decided not to
file an issue for the problem at that time, but I don't remember why.
Maybe it's an APR problem, not a Subversion problem?
I have a GDB backtrace from back then, which would have been 1.0.5. It
may still be relevant, maybe not. I'm attaching it to this message so
it can be archived. Please forgive me if this is not useful.
--
Michael W Thelen
It is a mistake to think you can solve any major problems just with
potatoes. -- Douglas Adams