You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Kylo Ginsberg <ky...@gmail.com> on 2007/06/26 18:10:32 UTC
stranded svnserve processes?
I'm running svnserve from xinetd and I see a slow accumulation of
stranded svnserve processes (a half dozen a day or so). This becomes a
problem because I have xinetd's cps (connections per second) set to
drop connections if there are 100 in a second, so once enough of these
build up (e.g. 60 this morning), the normal fluctuations of svn usage
end up causing peaks that hit the cps threshold and then users are
getting "svn: connection closed unexpectedly".
I've tried reproducing this but it's not as simple as just Ctrl-C'ing
a client during a lengthy operation. In any event, any suggestion on
how to handle this? I could start a cron job to either restart xinetd
nightly, or just periodically purge older svnserve processes, but I'd
welcome a better solution.
This is subversion 1.4.3, and xinetd 2.3.13 on FC4, kernel 2.6.14.
Thanks,
Kylo
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: stranded svnserve processes?
Posted by Kylo Ginsberg <ky...@gmail.com>.
Good call. Sample backtrace below; of the 6 I checked all had identical
backtrace, although the repository paths in frame 2 varied somewhat (that
is, they weren't identical, but they were all in the same area; this could
reflect something about the repo or it could correlate with one particular
user (or set of users) tools).
I didn't see this in the Issue Tracker on tigris. Should this thread move
to the dev list?
Kylo
(gdb) bt
#0 0x0000003ba230b1a0 in __read_nocancel () from /lib64/libpthread.so.0
#1 0x00002aaaab28dfde in apr_file_read (thefile=0x5163d8, buf=0x524468,
nbytes=0x7fffffa35c28) at readwrite.c:124
#2 0x00002aaaab03ee17 in readbuf_input (conn=0x524448,
data=0x524468 "( success ( ) ) ( ) ) false ) ) t/trunk/omneon/shared )
) areader ) ) ) omneon/panther ) ", len=0x7fffffa35c28)
at subversion/libsvn_ra_svn/marshal.c:255
#3 0x00002aaaab03eedd in readbuf_fill (conn=0x524448, pool=Variable "pool"
is not available.
)
at subversion/libsvn_ra_svn/marshal.c:274
#4 0x00002aaaab03ef8b in readbuf_getchar (conn=0x524448, pool=Variable
"pool" is not available.
)
at subversion/libsvn_ra_svn/marshal.c:284
#5 0x00002aaaab03efbe in readbuf_getchar_skip_whitespace (conn=0x524448,
pool=0x5364b8, result=0x7fffffa35c9f "")
at subversion/libsvn_ra_svn/marshal.c:294
#6 0x00002aaaab03fa3d in svn_ra_svn_read_item (conn=0x524448,
pool=0x5364b8,
item=0x7fffffa35ce8) at subversion/libsvn_ra_svn/marshal.c:621
#7 0x00002aaaab040088 in svn_ra_svn_read_tuple (conn=Variable "conn" is not
available.
)
at subversion/libsvn_ra_svn/marshal.c:741
#8 0x00002aaaab040558 in svn_ra_svn_handle_commands (conn=0x524448,
pool=Variable "pool" is not available.
)
at subversion/libsvn_ra_svn/marshal.c:831
#9 0x0000000000409987 in serve (conn=0x524448, params=0x7fffffa35f70,
pool=0x5161c8) at subversion/svnserve/serve.c:2291
#10 0x00000000004048af in main (argc=4, argv=0x7fffffa36118)
at subversion/svnserve/main.c:491
On 6/26/07, Ulrich Eckhardt <ec...@satorlaser.com> wrote:
>
> On Tuesday 26 June 2007 20:10, Kylo Ginsberg wrote:
> > I'm running svnserve from xinetd and I see a slow accumulation of
> > stranded svnserve processes (a half dozen a day or so). This becomes a
> > problem because I have xinetd's cps (connections per second) set to
> > drop connections if there are 100 in a second, so once enough of these
> > build up (e.g. 60 this morning), the normal fluctuations of svn usage
> > end up causing peaks that hit the cps threshold and then users are
> > getting "svn: connection closed unexpectedly".
> >
> > I've tried reproducing this but it's not as simple as just Ctrl-C'ing
> > a client during a lengthy operation. In any event, any suggestion on
> > how to handle this?
>
> Sounds like a bug in svnserve. Once it detects that its input is at EOF,
> it
> should terminate. I'd suggest that you 'gdb svnserve' and then at the gdb
> prompt 'attach' to one of the hanging svnserves. Then you can at least get
> a
> backtrace so developers can tell where it is hanging.
> Also, I believe that xinetd can log the PIDs of started servers, thus you
> might be able to make out which operation was performed and from where.
>
> Uli
>
> --
> Sator Laser GmbH
> Geschäftsführer: Ronald Boers, Amtsgericht Hamburg HR B62 932
>
>
> **************************************************************************************
> Visit our website at <http://www.satorlaser.de/>
>
> **************************************************************************************
> Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten
> bestimmt und kann vertrauliche Informationen enthalten. Bitte
> benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte
> Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf
> weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt
> werden.
> E-Mails können durch Dritte gelesen werden und Viren sowie
> nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese
> Folgen nicht verantwortlich.
>
>
> **************************************************************************************
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
>
Re: stranded svnserve processes?
Posted by Ulrich Eckhardt <ec...@satorlaser.com>.
On Tuesday 26 June 2007 20:10, Kylo Ginsberg wrote:
> I'm running svnserve from xinetd and I see a slow accumulation of
> stranded svnserve processes (a half dozen a day or so). This becomes a
> problem because I have xinetd's cps (connections per second) set to
> drop connections if there are 100 in a second, so once enough of these
> build up (e.g. 60 this morning), the normal fluctuations of svn usage
> end up causing peaks that hit the cps threshold and then users are
> getting "svn: connection closed unexpectedly".
>
> I've tried reproducing this but it's not as simple as just Ctrl-C'ing
> a client during a lengthy operation. In any event, any suggestion on
> how to handle this?
Sounds like a bug in svnserve. Once it detects that its input is at EOF, it
should terminate. I'd suggest that you 'gdb svnserve' and then at the gdb
prompt 'attach' to one of the hanging svnserves. Then you can at least get a
backtrace so developers can tell where it is hanging.
Also, I believe that xinetd can log the PIDs of started servers, thus you
might be able to make out which operation was performed and from where.
Uli
--
Sator Laser GmbH
Geschäftsführer: Ronald Boers, Amtsgericht Hamburg HR B62 932
**************************************************************************************
Visit our website at <http://www.satorlaser.de/>
**************************************************************************************
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich.
**************************************************************************************
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org