You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Jon Nicoll <jn...@hanoverdisplays.com> on 2011/10/21 11:34:52 UTC
why does my SVN client process die an hour after completion of commit?
Hi all
(This question is a combination of an SVN and a linux question)
I am running andLinux (a linux distribution which co-exists with Windows 7) on a PC and have a project with a large number (several tens of thousands) of files which I wanted to put under SVN.
I have an SVN server running over http, created a repo, and performed an initial commit of the files via an 'in-place commit' process:
cd ~/myproj
svn co http://revisioncontrol/svn/myproj/trunk .
svn add mylargesubdir
time svn ci
This process took a long time (close to seven hours). At the end I got this prompt on my client console:
. . . . . Killed
(ie. many dots, over a period of hours, as the data was transmitted, followed by a message that my process had been killed.)
The server log seems to indicate that the commit completed successfully at time ~9:48pm; there are no error messages .
If I look in /var/log/messages on my client machine, I see this:
{{{
Oct 20 21:00:48 andLinux -- MARK --
Oct 20 21:20:48 andLinux -- MARK --
Oct 20 21:40:48 andLinux -- MARK --
Oct 20 22:00:49 andLinux -- MARK --
Oct 20 22:21:09 andLinux -- MARK --
Oct 20 22:41:37 andLinux -- MARK --
Oct 20 22:43:23 andLinux kernel: kded4 invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0
Oct 20 22:43:23 andLinux kernel: dd invoked oom-killer: gfp_mask=0x200d2, order=0, oomkilladj=0
Oct 20 22:43:23 andLinux kernel: [<c0103b7a>] show_trace_log_lvl+0x1a/0x30
Oct 20 22:43:23 andLinux kernel: [<c0103cb2>] show_trace+0x12/0x20
Oct 20 22:43:23 andLinux kernel: [<c0104ae5>] dump_stack+0x15/0x20
Oct 20 22:43:23 andLinux kernel: [<c014197d>] out_of_memory+0x19d/0x200
Oct 20 22:43:23 andLinux kernel: [<c014319a>] __alloc_pages+0x2da/0x330
Oct 20 22:43:23 andLinux kernel: [<c0152b11>] read_swap_cache_async+0xa1/0xe0
Oct 20 22:43:23 andLinux kernel: [<c01492f5>] swapin_readahead+0x55/0x70
Oct 20 22:43:23 andLinux kernel: [<c014b98e>] __handle_mm_fault+0x82e/0xa00
Oct 20 22:43:23 andLinux kernel: [<c010b541>] do_page_fault+0x351/0x690
Oct 20 22:43:23 andLinux kernel: [<c02e81fa>] error_code+0x6a/0x70
Oct 20 22:43:23 andLinux kernel: [<c019d315>] kmsg_read+0x25/0x50
Oct 20 22:43:23 andLinux kernel: [<c015e105>] vfs_read+0xb5/0x140
Oct 20 22:43:23 andLinux kernel: [<c015e4ed>] sys_read+0x3d/0x70
Oct 20 22:43:23 andLinux kernel: [<c01028f2>] syscall_call+0x7/0xb
Oct 20 22:43:23 andLinux kernel: =======================
Oct 20 22:43:23 andLinux kernel: Mem-info:
Oct 20 22:43:23 andLinux kernel: Normal per-cpu:
Oct 20 22:43:23 andLinux kernel: CPU 0: Hot: hi: 90, btch: 15 usd: 5 Cold: hi: 30, btch: 7 usd: 6
Oct 20 22:43:23 andLinux kernel: Active:30295 inactive:30529 dirty:0 writeback:0 unstable:0
Oct 20 22:43:23 andLinux kernel: free:509 slab:1850 mapped:42 pagetables:315 bounce:0
Oct 20 22:43:23 andLinux kernel: Normal free:2036kB min:2036kB low:2544kB high:3052kB active:121180kB inactive:122116kB present:260096kB pages_scanned:407510 all_unreclaimable? yes
Oct 20 22:43:23 andLinux kernel: lowmem_reserve[]: 0
Oct 20 22:43:23 andLinux kernel: Normal: 11*4kB 3*8kB 1*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2036kB
Oct 20 22:43:23 andLinux kernel: Swap cache: add 1043236, delete 1043236, find 25790/146554, race 0+0
Oct 20 22:43:23 andLinux kernel: Free swap = 0kB
Oct 20 22:43:23 andLinux kernel: Total swap = 262120kB
Oct 20 22:43:23 andLinux kernel: Free swap: 0kB
Oct 20 22:43:23 andLinux kernel: 65536 pages of RAM
Oct 20 22:43:23 andLinux kernel: 0 pages of HIGHMEM
Oct 20 22:43:23 andLinux kernel: 1542 reserved pages
Oct 20 22:43:23 andLinux kernel: 117 pages shared
Oct 20 22:43:23 andLinux kernel: 0 pages swap cached
Oct 20 22:43:23 andLinux kernel: 0 pages dirty
Oct 20 22:43:23 andLinux kernel: 0 pages writeback
Oct 20 22:43:23 andLinux kernel: 42 pages mapped
Oct 20 22:43:23 andLinux kernel: 1850 pages slab
Oct 20 22:43:23 andLinux kernel: 315 pages pagetables
# etc.
}}}
'oom-killer' is the Linux Out Of Memory killer. So, it looks like the kernel on my client machine killed the SVN process at around 10:43pm.
Clearly there may be an issue with the memory available on my client machine, and I need to look into this. But my question is related to the timings indicated here.
- The server says the process completed successfully at ~9:48pm
- my client svn process was killed at ~10:43pm.
- There were no other tasks running on the client that might have caused problems AFAIK (I left it on running just the checkin overnight)
So, my question is: is there something in the 'svn commit' protocol which causes the client process to do a lot of work, potentially causing the client machine to run out of memory, up to one hour after the server has determined that a checkin is complete?
If nothing else I'd like to learn a bit more about the svn/http protocol, and what is going on here. Thanks for any pointers.
jon N
Please consider the environment before printing this email
--------------------------------------------------------------------------------
The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited.
If you received this in error, please contact the sender or postmaster (postmaster@hanoverdisplays.com) and delete the material from any computer.
Although we routinely screen for viruses, addressees should check this e-mail and any attachment for viruses. We make no warranty as to absence of viruses in this e-mail or any attachments.
Our Company's email policy is to permit incidental personal use. If this email is of a personal nature, it must not be relied upon as expressing the views or opinions of the company.
RE: why does my SVN client process die an hour after completion of commit?
Posted by Dominik Psenner <dp...@gmail.com>.
>If I look in /var/log/messages on my client machine, I see this:
>{{{
>Oct 20 21:00:48 andLinux -- MARK --
>Oct 20 21:20:48 andLinux -- MARK --
>Oct 20 21:40:48 andLinux -- MARK --
>Oct 20 22:00:49 andLinux -- MARK --
>Oct 20 22:21:09 andLinux -- MARK --
>Oct 20 22:41:37 andLinux -- MARK --
>Oct 20 22:43:23 andLinux kernel: kded4 invoked oom-killer:
>gfp_mask=0x201d2, order=0, oomkilladj=0
>Oct 20 22:43:23 andLinux kernel: dd invoked oom-killer: gfp_mask=0x200d2,
>order=0, oomkilladj=0
>Oct 20 22:43:23 andLinux kernel: [<c0103b7a>] show_trace_log_lvl+0x1a/0x30
>Oct 20 22:43:23 andLinux kernel: [<c0103cb2>] show_trace+0x12/0x20
>Oct 20 22:43:23 andLinux kernel: [<c0104ae5>] dump_stack+0x15/0x20
>Oct 20 22:43:23 andLinux kernel: [<c014197d>] out_of_memory+0x19d/0x200
>Oct 20 22:43:23 andLinux kernel: [<c014319a>] __alloc_pages+0x2da/0x330
>Oct 20 22:43:23 andLinux kernel: [<c0152b11>]
>read_swap_cache_async+0xa1/0xe0
>Oct 20 22:43:23 andLinux kernel: [<c01492f5>] swapin_readahead+0x55/0x70
>Oct 20 22:43:23 andLinux kernel: [<c014b98e>]
>__handle_mm_fault+0x82e/0xa00
>Oct 20 22:43:23 andLinux kernel: [<c010b541>] do_page_fault+0x351/0x690
>Oct 20 22:43:23 andLinux kernel: [<c02e81fa>] error_code+0x6a/0x70
>Oct 20 22:43:23 andLinux kernel: [<c019d315>] kmsg_read+0x25/0x50
>Oct 20 22:43:23 andLinux kernel: [<c015e105>] vfs_read+0xb5/0x140
>Oct 20 22:43:23 andLinux kernel: [<c015e4ed>] sys_read+0x3d/0x70
>Oct 20 22:43:23 andLinux kernel: [<c01028f2>] syscall_call+0x7/0xb
>Oct 20 22:43:23 andLinux kernel: =======================
>Oct 20 22:43:23 andLinux kernel: Mem-info:
>Oct 20 22:43:23 andLinux kernel: Normal per-cpu:
>Oct 20 22:43:23 andLinux kernel: CPU 0: Hot: hi: 90, btch: 15 usd:
>5 Cold: hi: 30, btch: 7 usd: 6
>Oct 20 22:43:23 andLinux kernel: Active:30295 inactive:30529 dirty:0
>writeback:0 unstable:0
>Oct 20 22:43:23 andLinux kernel: free:509 slab:1850 mapped:42
>pagetables:315 bounce:0
>Oct 20 22:43:23 andLinux kernel: Normal free:2036kB min:2036kB low:2544kB
>high:3052kB active:121180kB inactive:122116kB present:260096kB
Is this a virtual machine? 260096kB of RAM was probably not enough.
Therefore it used a lot of swap memory, which explains why the process took
so long.
>pages_scanned:407510 all_unreclaimable? yes
>Oct 20 22:43:23 andLinux kernel: lowmem_reserve[]: 0
>Oct 20 22:43:23 andLinux kernel: Normal: 11*4kB 3*8kB 1*16kB 1*32kB 0*64kB
>1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2036kB
>Oct 20 22:43:23 andLinux kernel: Swap cache: add 1043236, delete 1043236,
>find 25790/146554, race 0+0
>Oct 20 22:43:23 andLinux kernel: Free swap = 0kB
*ouch* Looks like you've had no swap left, either. Probably you ran out of
memory during the commit and the process was killed by the kernel when the
process tried to allocate even more.
Cheers
Re: why does my SVN client process die an hour after completion of commit?
Posted by Philip Martin <ph...@wandisco.com>.
Jon Nicoll <jn...@hanoverdisplays.com> writes:
> So for now I would be best off to regard my 'initial working
> directory' as suspect, and if I checkout from the server into a new
> directory, that directory should be correct, yes?
You will probably need to run cleanup on the old working copy to remove
locks but it should be OK. Items that have been not been updated to
commited revision will get pulled from the server on the next update.
--
Philip
RE: why does my SVN client process die an hour after completion of commit?
Posted by Jon Nicoll <jn...@hanoverdisplays.com>.
Hi Stefan, Philip
Thanks for your postings:
> On Fri, Oct 21, 2011 at 10:34:52AM +0100, Jon Nicoll wrote:
>> So, my question is: is there something in the 'svn commit' protocol
>> which causes the client process to do a lot of work, potentially
>> causing the client machine to run out of memory, up to one hour after
>> the server has determined that a checkin is complete?
>
> This is a known issue in Subversion 1.6.
> Adding many files in one commit uses too much memory, see:
> http://subversion.tigris.org/issues/show_bug.cgi?id=1964#desc12
The reason it happens after a commit is complete on the server is that the client has to modify the metadata for all the items in the working copy that were committed. When the commit starts the items being committed all have local modifications of some kind, when the commit is complete they must all be change to unmodified. This can involve a lot of work, writing pristine text and properties, updating the revision numbers. This updating cannot happen until the client knows that the commit was successful.
====
Ah, that makes sense, thanks.
So for now I would be best off to regard my 'initial working directory' as suspect, and if I checkout from the server into a new directory, that directory should be correct, yes?
(this commit of a large number of files is not something I will have do very often, if at all, after this initial commit).
I'm using SVN client v.1.5.4 BTW - a bit older that I expected. I will also look into getting this updated to 1.7.
Thanks & Regards
Jon N
Please consider the environment before printing this email
--------------------------------------------------------------------------------
The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited.
If you received this in error, please contact the sender or postmaster (postmaster@hanoverdisplays.com) and delete the material from any computer.
Although we routinely screen for viruses, addressees should check this e-mail and any attachment for viruses. We make no warranty as to absence of viruses in this e-mail or any attachments.
Our Company's email policy is to permit incidental personal use. If this email is of a personal nature, it must not be relied upon as expressing the views or opinions of the company.
Re: why does my SVN client process die an hour after completion of commit?
Posted by Philip Martin <ph...@wandisco.com>.
Stefan Sperling <st...@elego.de> writes:
> On Fri, Oct 21, 2011 at 10:34:52AM +0100, Jon Nicoll wrote:
>> So, my question is: is there something in the 'svn commit' protocol
>> which causes the client process to do a lot of work, potentially
>> causing the client machine to run out of memory, up to one hour after
>> the server has determined that a checkin is complete?
>
> This is a known issue in Subversion 1.6.
> Adding many files in one commit uses too much memory, see:
> http://subversion.tigris.org/issues/show_bug.cgi?id=1964#desc12
The reason it happens after a commit is complete on the server is that
the client has to modify the metadata for all the items in the working
copy that were committed. When the commit starts the items being
committed all have local modifications of some kind, when the commit is
complete they must all be change to unmodified. This can involve a lot
of work, writing pristine text and properties, updating the revision
numbers. This updating cannot happen until the client knows that the
commit was successful.
--
Philip
Re: why does my SVN client process die an hour after completion of
commit?
Posted by Stefan Sperling <st...@elego.de>.
On Fri, Oct 21, 2011 at 10:34:52AM +0100, Jon Nicoll wrote:
> So, my question is: is there something in the 'svn commit' protocol
> which causes the client process to do a lot of work, potentially
> causing the client machine to run out of memory, up to one hour after
> the server has determined that a checkin is complete?
This is a known issue in Subversion 1.6.
Adding many files in one commit uses too much memory, see:
http://subversion.tigris.org/issues/show_bug.cgi?id=1964#desc12
As a workaround, use 'svn import', or upgrade to Subversion 1.7.