You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Julian Foad <ju...@btopenworld.com> on 2006/03/23 01:19:50 UTC
svn hung at end of commit operation
Committing to the Subversion repository just now using Subversion trunk r18992,
I got:
> ~/src/subversion> svn ci -F p && rm p
> Sending subversion/include/svn_xml.h
> Sending subversion/libsvn_client/prop_commands.c
> Sending subversion/libsvn_subr/xml.c
> Transmitting file data ...
and no further output. "svn log" from a separate terminal shows that the
commit has succeeded in the repository, creating r18995. Attaching GDB gives:
> (gdb) bt
> #0 0xffffe410 in ?? ()
> #1 0xbfbf34c8 in ?? ()
> #2 0x00000000 in ?? ()
> #3 0xbfbf3430 in ?? ()
> #4 0x4051df2d in ___newselect_nocancel () from /lib/tls/libc.so.6
> #5 0x402e55ce in readable_raw ()
> from /home/julianfoad/build/subversion/neon/src/.libs/libneon.so.24
> #6 0x402e56e5 in read_raw ()
> from /home/julianfoad/build/subversion/neon/src/.libs/libneon.so.24
> #7 0x402e5a72 in ne_sock_readline ()
> from /home/julianfoad/build/subversion/neon/src/.libs/libneon.so.24
> #8 0x402df910 in send_request ()
> from /home/julianfoad/build/subversion/neon/src/.libs/libneon.so.24
> #9 0x402dff65 in ne_begin_request ()
> from /home/julianfoad/build/subversion/neon/src/.libs/libneon.so.24
> #10 0x402e0321 in ne_request_dispatch ()
> from /home/julianfoad/build/subversion/neon/src/.libs/libneon.so.24
> #11 0x40105bf7 in parsed_request (sess=0x827f730, method=0x40107d44 "MERGE",
> url=0x8085270 "/repos/svn/trunk/subversion",
> body=0x8274948 "<?xml version=\"1.0\" encoding=\"utf-8\"?><D:merge xmlns:D=\"DAV:\"><D:source><D:href>/repos/svn/!svn/act/f214a7fe-9e0f-0410-bd38-c950c51accf2</D:href></D:source><D:no-auto-merge/><D:no-checkout/><D:prop><D"...,
> body_file=0x0, set_parser=0, elements=0x4010a460, use_neon_shim=1,
> validate_compat_cb=0x400fe27f <validate_element>,
> startelm_compat_cb=0x400fe4da <start_element>,
> endelm_compat_cb=0x400fe5fe <end_element>, startelm_cb=0, cdata_cb=0,
> endelm_cb=0, baton=0xbfbf3820, extra_headers=0x82748c8, status_code=0x0,
> spool_response=0, pool=0x8072230)
> at /home/julianfoad/src/subversion/subversion/libsvn_ra_dav/util.c:715
> #12 0x4010602a in svn_ra_dav__parsed_request_compat (sess=0x827f730,
> method=0x40107d44 "MERGE", url=0x8085270 "/repos/svn/trunk/subversion",
> body=0x8274948 "<?xml version=\"1.0\" encoding=\"utf-8\"?><D:merge xmlns:D=\"DAV:\"><D:source><D:href>/repos/svn/!svn/act/f214a7fe-9e0f-0410-bd38-c950c51accf2</D:href></D:source><D:no-auto-merge/><D:no-checkout/><D:prop><D"...,
> body_file=0x0, set_parser=0, elements=0x4010a460,
> validate_cb=0x400fe27f <validate_element>,
> startelm_cb=0x400fe4da <start_element>,
> endelm_cb=0x400fe5fe <end_element>, baton=0xbfbf3820,
> extra_headers=0x82748c8, status_code=0x0, spool_response=0,
> pool=0x8072230)
> at /home/julianfoad/src/subversion/subversion/libsvn_ra_dav/util.c:876
> #13 0x400fedb4 in svn_ra_dav__merge_activity (new_rev=0x8274800,
> committed_date=0x8274804, committed_author=0x8274808,
> post_commit_err=0x827480c, ras=0x8093a98,
> repos_url=0x8085270 "/repos/svn/trunk/subversion",
> activity_url=0x8093e78 "/repos/svn/!svn/act/f214a7fe-9e0f-0410-bd38-c950c51accf2", valid_targets=0x8093ba0, lock_tokens=0x8093958, keep_locks=0,
> disable_merge_response=0, pool=0x8072230)
> at /home/julianfoad/src/subversion/subversion/libsvn_ra_dav/merge.c:737
> #14 0x400f7321 in commit_close_edit (edit_baton=0x8093b68, pool=0x8072230)
> at /home/julianfoad/src/subversion/subversion/libsvn_ra_dav/commit.c:1462
> #15 0x40027659 in svn_client__do_commit (
> base_url=0x80938d0 "http://svn.collab.net/repos/svn/trunk/subversion",
> commit_items=0x824ddb8, adm_access=0x80a84e8, editor=0x809d5b0,
> edit_baton=0x8093b68,
> notify_path_prefix=0x809d718 "/home/julianfoad/src/subversion",
> tempfiles=0xbfbf3a84, digests=0xbfbf3a80, ctx=0x80871d8, pool=0x8072230)
> at /home/julianfoad/src/subversion/subversion/libsvn_client/commit_util.c:1344
> #16 0x40024b8b in svn_client_commit3 (commit_info_p=0xbfbf3b7c,
> targets=0x808a940, recurse=1, keep_locks=0, ctx=0x80871d8,
> pool=0x8072230)
> at /home/julianfoad/src/subversion/subversion/libsvn_client/commit.c:1499
> #17 0x0804ceec in svn_cl__commit (os=0x8072348, baton=0xbfbf3c90,
> pool=0x8072230)
> at /home/julianfoad/src/subversion/subversion/svn/commit-cmd.c:94
> #18 0x0805292e in main (argc=4, argv=0xbfbf3e34)
> at /home/julianfoad/src/subversion/subversion/svn/main.c:1440
Any ideas?
I will note that I see a similar effect occasionally with the package
installation tool "apt": sometimes it downloads 100% of a file and then freezes
waiting for some sort of end indication, which makes me suspect some subtle
problem with my network connection - software, options, firewall, router, etc.
However, I have never seen such a problem with Subversion.
- Julian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn hung at end of commit operation
Posted by "Peter N. Lundblad" <pe...@famlundblad.se>.
Molle Bestefich writes:
> Peter Samuelson wrote:
> > Molle Bestefich wrote:
> > > Try something like:
> > > (/usr/local/svn/hook-scripts/mailer.py commit "${REPOS}" "${REV}" &) &
> >
> > I thought that was only useful if your shell uses job control.
> > Noninteractive shells shouldn't.
>
> The purpose of the above is not to start the process in a background
> job but to detach it from the shell that runs it:
The problem in 1.3.0 is that the server reads the hooks stderr and
*then* waits for its child. It was changed from doing it the other way
around because that would deadlock when the hook wrote a lot to stderr
(causing the pipe to get full).
In 1.3.0, an extra write end to the stderr pipe was left open in the
child, meaning that if you redirect stderr when trunning the
background task, it would still inherit the write end of the
pipe,causing the server to block for input.
> ~ # (sleep 5 &) &
> [1] 416
> [1]+ Done ( sleep 5 & )
> ~ # jobs
> ~ # ps
> PID PPID STIME COMMAND
> 4084 4083 10:16 bash
> 2600 1 10:16 sleep
> 4092 4084 10:17 ps
> ~ #
Does this really make the problem go away?
Anyway, this is fixed in 1.3.1, but you still need to redirect (or
close, but that seems like a bad idea) stderr when running the
background job.
Thanks,
//Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn hung at end of commit operation
Posted by Peter Samuelson <pe...@p12n.org>.
[Molle Bestefich]
> Peter Samuelson wrote:
> > I thought that was only useful if your shell uses job control.
> > Noninteractive shells shouldn't.
>
> The purpose of the above is not to start the process in a background
> job but to detach it from the shell that runs it:
Exactly - the PPID does not really matter, the shell only keeps track
of its child processes if they are considered background jobs. And
noninteractive shells don't use job control.
By the way, "(sleep 5 &) &" and "(sleep 5 &)" give you exactly the same
effect. So even in an interactive shell, when I do have to worry about
job control, I don't use the second &. Though actually, if I'm in an
interactive shell, it is usually bash, so I'm more in the habit of
typing "sleep 5 & disown" instead. (disown is a bashism.)
Re: svn hung at end of commit operation
Posted by Molle Bestefich <mo...@gmail.com>.
Peter Samuelson wrote:
> Molle Bestefich wrote:
> > Try something like:
> > (/usr/local/svn/hook-scripts/mailer.py commit "${REPOS}" "${REV}" &) &
>
> I thought that was only useful if your shell uses job control.
> Noninteractive shells shouldn't.
The purpose of the above is not to start the process in a background
job but to detach it from the shell that runs it:
~ # (sleep 5 &) &
[1] 416
[1]+ Done ( sleep 5 & )
~ # jobs
~ # ps
PID PPID STIME COMMAND
4084 4083 10:16 bash
2600 1 10:16 sleep
4092 4084 10:17 ps
~ #
Note the PPID for 'sleep'.
> I agree that redirecting stdout and stderr should be sufficient.
> Come to think of it, perhaps just at the top of the hook script:
> exec < /dev/null > /dev/null 2>&1
> which then affects the whole shell and all subprocesses.
Perhaps.
What happens when the hook script process dies early (to allow svn to continue)?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn hung at end of commit operation
Posted by Peter Samuelson <pe...@p12n.org>.
[Molle Bestefich]
> Try something like:
> (/usr/local/svn/hook-scripts/mailer.py commit "${REPOS}" "${REV}" &) &
I thought that was only useful if your shell uses job control.
Noninteractive shells shouldn't. I agree that redirecting stdout and
stderr should be sufficient. Come to think of it, perhaps just at the
top of the hook script:
exec < /dev/null > /dev/null 2>&1
which then affects the whole shell and all subprocesses.
Re: svn hung at end of commit operation
Posted by Molle Bestefich <mo...@gmail.com>.
Garrett Rooney wrote:
> C. Michael Pilato wrote:
> > I added " > /dev/null 2>&1" to each of those lines. Would you expect that
> > to be sufficient?
>
> It looks like that should work, assuming we're using a version of
> Subversion that has fixed the issues we had with that... And it
> doesn't look like we have. Need to upgrade to 1.3.1 to get that fix.
Try something like:
(/usr/local/svn/hook-scripts/mailer.py commit "${REPOS}" "${REV}" &) &
that is, "(cmd &) &" for each process..
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn hung at end of commit operation
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 3/22/06, C. Michael Pilato <cm...@collab.net> wrote:
> Garrett Rooney wrote:
> > That probably isn't sufficient to actually let the commit continue
> > while those tasks are still running. It's not redirecting stdout and
> > stderr of the backgrounded jobs, so they'll still be held open, at
> > least with subversion 1.3.x anyway.
>
> I added " > /dev/null 2>&1" to each of those lines. Would you expect that
> to be sufficient?
It looks like that should work, assuming we're using a version of
Subversion that has fixed the issues we had with that... And it
doesn't look like we have. Need to upgrade to 1.3.1 to get that fix.
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn hung at end of commit operation
Posted by "C. Michael Pilato" <cm...@collab.net>.
Garrett Rooney wrote:
> That probably isn't sufficient to actually let the commit continue
> while those tasks are still running. It's not redirecting stdout and
> stderr of the backgrounded jobs, so they'll still be held open, at
> least with subversion 1.3.x anyway.
I added " > /dev/null 2>&1" to each of those lines. Would you expect that
to be sufficient?
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Re: svn hung at end of commit operation
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 3/22/06, C. Michael Pilato <cm...@collab.net> wrote:
> Garrett Rooney wrote:
> > On 3/22/06, Julian Foad <ju...@btopenworld.com> wrote:
> >
> >>Committing to the Subversion repository just now using Subversion trunk r18992,
> >>I got:
> >>
> >>
> >>>~/src/subversion> svn ci -F p && rm p
> >>>Sending subversion/include/svn_xml.h
> >>>Sending subversion/libsvn_client/prop_commands.c
> >>>Sending subversion/libsvn_subr/xml.c
> >>>Transmitting file data ...
> >
> >
> > I've seen svn.collab.net seem a bit sluggish at the end of commits
> > lately, like the post-commit hooks were taking longer than you'd
> > expect them to. Never had it totally hang though, that's pretty
> > weird. Was this a one time thing, or was it reproducable?
>
> Here's the post-commit hook in use on svn.collab.net (for all repositories):
>
> ---------------------------------------------------------------------
> #!/bin/sh
>
> LANG=en_US; export LANG
>
> REPOS=${1}
> REV=${2}
>
> # Send a commit mail.
> if [ -f ${REPOS}/POST-COMMIT-EMAIL ]; then
> /usr/local/svn/hook-scripts/mailer.py commit "${REPOS}" "${REV}" &
> fi
>
> # See http://cia.navi.cx/doc/how-cia-works for what this is about:
> if [ -f ${REPOS}/POST-COMMIT-CIABOT ]; then
> /usr/local/svn/hook-scripts/ciabot_svn.py ${REPOS} ${REV} "svn" &
> fi
>
> # Run the ViewVC commits database indexer.
> if [ -f ${REPOS}/POST-COMMIT-VIEWVC-DB ]; then
> /usr/local/viewvc-1.0-dev/bin/svndbadmin rebuild ${REPOS} ${REV} &
> fi
> ---------------------------------------------------------------------
>
> As you can see, all things are backgrounded. But that does mean there are
> three more processes all asking the repository about commit details for the
> newly committed revision at the same time. svndbadmin is a new addition
> post-migration, but still -- a difference of many *minutes* in post-commit
> because of a backgrounded task? That seems really unlikely.
That probably isn't sufficient to actually let the commit continue
while those tasks are still running. It's not redirecting stdout and
stderr of the backgrounded jobs, so they'll still be held open, at
least with subversion 1.3.x anyway.
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn hung at end of commit operation
Posted by "C. Michael Pilato" <cm...@collab.net>.
Garrett Rooney wrote:
> On 3/22/06, Julian Foad <ju...@btopenworld.com> wrote:
>
>>Committing to the Subversion repository just now using Subversion trunk r18992,
>>I got:
>>
>>
>>>~/src/subversion> svn ci -F p && rm p
>>>Sending subversion/include/svn_xml.h
>>>Sending subversion/libsvn_client/prop_commands.c
>>>Sending subversion/libsvn_subr/xml.c
>>>Transmitting file data ...
>
>
> I've seen svn.collab.net seem a bit sluggish at the end of commits
> lately, like the post-commit hooks were taking longer than you'd
> expect them to. Never had it totally hang though, that's pretty
> weird. Was this a one time thing, or was it reproducable?
Here's the post-commit hook in use on svn.collab.net (for all repositories):
---------------------------------------------------------------------
#!/bin/sh
LANG=en_US; export LANG
REPOS=${1}
REV=${2}
# Send a commit mail.
if [ -f ${REPOS}/POST-COMMIT-EMAIL ]; then
/usr/local/svn/hook-scripts/mailer.py commit "${REPOS}" "${REV}" &
fi
# See http://cia.navi.cx/doc/how-cia-works for what this is about:
if [ -f ${REPOS}/POST-COMMIT-CIABOT ]; then
/usr/local/svn/hook-scripts/ciabot_svn.py ${REPOS} ${REV} "svn" &
fi
# Run the ViewVC commits database indexer.
if [ -f ${REPOS}/POST-COMMIT-VIEWVC-DB ]; then
/usr/local/viewvc-1.0-dev/bin/svndbadmin rebuild ${REPOS} ${REV} &
fi
---------------------------------------------------------------------
As you can see, all things are backgrounded. But that does mean there are
three more processes all asking the repository about commit details for the
newly committed revision at the same time. svndbadmin is a new addition
post-migration, but still -- a difference of many *minutes* in post-commit
because of a backgrounded task? That seems really unlikely.
Any ideas?
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Re: svn hung at end of commit operation
Posted by Daniel Rall <dl...@collab.net>.
On Thu, 23 Mar 2006, Julian Foad wrote:
> Garrett Rooney wrote:
> >On 3/22/06, Julian Foad <ju...@btopenworld.com> wrote:
> >
> >>>Sending subversion/libsvn_subr/xml.c
> >>>Transmitting file data ...
> >
> >I've seen svn.collab.net seem a bit sluggish at the end of commits
> >lately, like the post-commit hooks were taking longer than you'd
> >expect them to. Never had it totally hang though, that's pretty
>
> I waited several minutes. Probably five or ten minutes by the time I'd
> finished looking at it in GDB and detached from it and finally killed the
> process.
>
> >weird. Was this a one time thing, or was it reproducable?
>
> I don't want to make any more commits to the main repository at the moment.
> I've only made about three commits in the last fortnight and they all went
> smoothly IIRC.
>
> At the moment I'm treating this as a freak occurrence; if you've no ideas
> don't worry about it yet. If it happens again then I or we will need to
> investigate.
Incidently, I experienced the same symptoms Julian describes here
within the last hour or so.
--
Daniel Rall
Re: svn hung at end of commit operation
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
On 3/22/06, Julian Foad <ju...@btopenworld.com> wrote:
> At the moment I'm treating this as a freak occurrence; if you've no ideas don't
> worry about it yet. If it happens again then I or we will need to investigate.
I was commenting about the slowdowns on IRC as well. So, yah,
something's horked with svn.collab.net. It generally takes 2-3
minutes for a commit to go through. I've been using ra_serf - so if
it's happening with ra_dav, it's not the client library.
Maybe this is C-Mike's way of telling us to stop committing so much.
=) -- justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn hung at end of commit operation
Posted by Julian Foad <ju...@btopenworld.com>.
Garrett Rooney wrote:
> On 3/22/06, Julian Foad <ju...@btopenworld.com> wrote:
>
>>>Sending subversion/libsvn_subr/xml.c
>>>Transmitting file data ...
>
> I've seen svn.collab.net seem a bit sluggish at the end of commits
> lately, like the post-commit hooks were taking longer than you'd
> expect them to. Never had it totally hang though, that's pretty
I waited several minutes. Probably five or ten minutes by the time I'd
finished looking at it in GDB and detached from it and finally killed the process.
> weird. Was this a one time thing, or was it reproducable?
I don't want to make any more commits to the main repository at the moment.
I've only made about three commits in the last fortnight and they all went
smoothly IIRC.
At the moment I'm treating this as a freak occurrence; if you've no ideas don't
worry about it yet. If it happens again then I or we will need to investigate.
- Julian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn hung at end of commit operation
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 3/22/06, Julian Foad <ju...@btopenworld.com> wrote:
> Committing to the Subversion repository just now using Subversion trunk r18992,
> I got:
>
> > ~/src/subversion> svn ci -F p && rm p
> > Sending subversion/include/svn_xml.h
> > Sending subversion/libsvn_client/prop_commands.c
> > Sending subversion/libsvn_subr/xml.c
> > Transmitting file data ...
I've seen svn.collab.net seem a bit sluggish at the end of commits
lately, like the post-commit hooks were taking longer than you'd
expect them to. Never had it totally hang though, that's pretty
weird. Was this a one time thing, or was it reproducable?
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org