You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Mark Phippard <ma...@gmail.com> on 2012/05/04 22:37:48 UTC
Trunk bug, serf, sqlite, or just me?
I wanted to play around with the commit cmpilato made earlier today:
http://svn.apache.org/viewvc?view=revision&revision=1333936
Using Serf, this should cause it to skip fetching files from the
server if they are already in the pristine store.
I am on OSX 10.7, and using MacPorts for my dependencies.
$ port installed sqlite3
The following ports are currently installed:
sqlite3 @3.7.11_0 (active)
$ port installed serf1
The following ports are currently installed:
serf1 @1.0.3_0 (active)
$ port installed apr
The following ports are currently installed:
apr @1.4.6_1 (active)
I have built trunk@1334150
I then run the following set of commands to setup a sparse working
copy with a shared pristines store.
$ svn co --depth=immediates http://svn.apache.org/repos/asf/subversion
A subversion/README
A subversion/tags
A subversion/developer-resources
A subversion/mk.xiv
A subversion/svn-logos
A subversion/branches
A subversion/site
A subversion/trunk
U subversion
Checked out revision 1334157.
$ svn up --set-depth=infinity trunk/
Updating 'trunk':
...
A trunk/subversion/bindings/swig/proxy
Updated to revision 1334157.
$ svn up --set-depth=immediates branches/
Updating 'branches':
A branches/svnmucc
...
A branches/compressed-pristines
Updated to revision 1334158.
So far so good. Now, I am ready to update a branch that should share
many pristines with trunk
$ cd branches
$ svn up --set-depth=infinity ev2-export/
Updating 'ev2-export':
subversion/svn/update-cmd.c:163: (apr_err=200030)
subversion/libsvn_client/update.c:611: (apr_err=200030)
subversion/libsvn_client/update.c:552: (apr_err=200030)
subversion/libsvn_client/update.c:413: (apr_err=200030)
subversion/libsvn_wc/adm_crawler.c:858: (apr_err=200030)
subversion/libsvn_ra_serf/update.c:2639: (apr_err=200030)
subversion/libsvn_ra_serf/util.c:1837: (apr_err=200030)
subversion/libsvn_ra_serf/util.c:1818: (apr_err=200030)
subversion/libsvn_ra_serf/update.c:1249: (apr_err=200030)
subversion/libsvn_ra_serf/update.c:594: (apr_err=200030)
subversion/libsvn_delta/cancel.c:145: (apr_err=200030)
subversion/libsvn_wc/update_editor.c:2521: (apr_err=200030)
subversion/libsvn_wc/wc_db.c:12180: (apr_err=200030)
subversion/libsvn_wc/wc_db_util.c:202: (apr_err=200030)
subversion/libsvn_subr/sqlite.c:1003: (apr_err=200030)
subversion/libsvn_wc/wc_db_util.c:185: (apr_err=200030)
subversion/libsvn_wc/wc_db.c:12151: (apr_err=200030)
subversion/libsvn_subr/sqlite.c:229: (apr_err=200030)
subversion/libsvn_subr/sqlite.c:213: (apr_err=200030)
subversion/libsvn_subr/sqlite.c:249: (apr_err=200030)
svn: E200030: sqlite: unable to open database file
subversion/libsvn_subr/sqlite.c:585: (apr_err=200030)
svn: E200030: sqlite: unable to open database file
subversion/libsvn_wc/lock.c:1608: (apr_err=200030)
subversion/libsvn_wc/wc_db.c:11986: (apr_err=200030)
subversion/libsvn_subr/sqlite.c:229: (apr_err=200030)
subversion/libsvn_subr/sqlite.c:213: (apr_err=200030)
subversion/libsvn_subr/sqlite.c:249: (apr_err=200030)
svn: E200030: sqlite: unable to open database file
subversion/libsvn_subr/sqlite.c:585: (apr_err=200030)
svn: E200030: sqlite: unable to open database file
This could be due to my SQLite version? I was getting similar errors
with my own repository but after it installed a few of the files.
Based on that earlier testing, I then tried this:
$ svn cleanup
$ cd ev2-export/
$ svn up --set-depth=infinity .
Updating '.':
A NOTICE
... (just a few files snipped)
A notes/obliterate/presentations/why.odp
subversion/svn/update-cmd.c:163: (apr_err=24)
subversion/libsvn_client/update.c:611: (apr_err=24)
subversion/libsvn_client/update.c:552: (apr_err=24)
subversion/libsvn_client/update.c:413: (apr_err=24)
subversion/libsvn_wc/adm_crawler.c:858: (apr_err=24)
subversion/libsvn_ra_serf/update.c:2639: (apr_err=24)
subversion/libsvn_ra_serf/util.c:1837: (apr_err=24)
subversion/libsvn_ra_serf/util.c:1818: (apr_err=24)
subversion/libsvn_ra_serf/update.c:1281: (apr_err=24)
subversion/libsvn_wc/update_editor.c:3688: (apr_err=24)
subversion/libsvn_wc/adm_files.c:342: (apr_err=24)
subversion/libsvn_subr/stream.c:888: (apr_err=24)
subversion/libsvn_subr/io.c:4320: (apr_err=24)
subversion/libsvn_subr/io.c:4135: (apr_err=24)
svn: E000024: Can't create temporary file from template
'/Users/markphip/work/trunk-testing/subversion/.svn/tmp/svn-XXXXXX':
Too many open files
So this gets further before getting a different error.
Does anyone have ideas on things to check? Mike tried some of these
commands earlier on his Linux box and does not get any errors, just
speed goodness.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Trunk bug, serf, sqlite, or just me?
Posted by Greg Stein <gs...@gmail.com>.
On Fri, May 4, 2012 at 5:16 PM, C. Michael Pilato <cm...@collab.net> wrote:
> On 05/04/2012 04:37 PM, Mark Phippard wrote:
>> $ svn up --set-depth=infinity ev2-export/
>> Updating 'ev2-export':
>
> [...]
>
>> subversion/libsvn_subr/sqlite.c:585: (apr_err=200030)
>> svn: E200030: sqlite: unable to open database file
>>
>> This could be due to my SQLite version? I was getting similar errors
>> with my own repository but after it installed a few of the files.
>
> I can't explain this one.
Likely there were too many open files, so the database couldn't be
opened. Simple timing on what file was trying to be opened.
>
>> $ svn up --set-depth=infinity .
>> Updating '.':
>> A NOTICE
>> ... (just a few files snipped)
>> A notes/obliterate/presentations/why.odp
>> subversion/svn/update-cmd.c:163: (apr_err=24)
>> subversion/libsvn_client/update.c:611: (apr_err=24)
>> subversion/libsvn_client/update.c:552: (apr_err=24)
>> subversion/libsvn_client/update.c:413: (apr_err=24)
>> subversion/libsvn_wc/adm_crawler.c:858: (apr_err=24)
>> subversion/libsvn_ra_serf/update.c:2639: (apr_err=24)
>> subversion/libsvn_ra_serf/util.c:1837: (apr_err=24)
>> subversion/libsvn_ra_serf/util.c:1818: (apr_err=24)
>> subversion/libsvn_ra_serf/update.c:1281: (apr_err=24)
>> subversion/libsvn_wc/update_editor.c:3688: (apr_err=24)
>> subversion/libsvn_wc/adm_files.c:342: (apr_err=24)
>> subversion/libsvn_subr/stream.c:888: (apr_err=24)
>> subversion/libsvn_subr/io.c:4320: (apr_err=24)
>> subversion/libsvn_subr/io.c:4135: (apr_err=24)
>> svn: E000024: Can't create temporary file from template
>> '/Users/markphip/work/trunk-testing/subversion/.svn/tmp/svn-XXXXXX':
>> Too many open files
>>
>> So this gets further before getting a different error.
>
> I *might* be able to explain at least a portion of this one.
>
> My code calls the (new) get_wc_contents RA callback to get a read stream for
> the pristine text matching the specified SHA1. This will open the pristine
> file. The file remains open until its contents are consumed (whenever
> Serf/ra_serf get around to doing so per the pipelined approach taken to
> GET/HEAD/PROPFINDs during updates). I suppose if the code which is cramming
> pipelined requests (the result of parsing the initial REPORT) into the queue
> outpaces the code that's handling those requests, you could wind up with too
> many open files.
That's exactly what is happening. See my review your of r1333936
commit, from about an hour ago. I also suggest the fix, and copied
Mark on that email.
ra_serf will only allow 1000 outstanding requests during update
processing, but on Mac OS, the default file handle limit is 256.
That's why it probably worked on CMike's machine, but not on a Mac.
Cheers,
-g
Re: Trunk bug, serf, sqlite, or just me?
Posted by "C. Michael Pilato" <cm...@collab.net>.
On 05/04/2012 04:37 PM, Mark Phippard wrote:
> $ svn up --set-depth=infinity ev2-export/
> Updating 'ev2-export':
[...]
> subversion/libsvn_subr/sqlite.c:585: (apr_err=200030)
> svn: E200030: sqlite: unable to open database file
>
> This could be due to my SQLite version? I was getting similar errors
> with my own repository but after it installed a few of the files.
I can't explain this one.
> $ svn up --set-depth=infinity .
> Updating '.':
> A NOTICE
> ... (just a few files snipped)
> A notes/obliterate/presentations/why.odp
> subversion/svn/update-cmd.c:163: (apr_err=24)
> subversion/libsvn_client/update.c:611: (apr_err=24)
> subversion/libsvn_client/update.c:552: (apr_err=24)
> subversion/libsvn_client/update.c:413: (apr_err=24)
> subversion/libsvn_wc/adm_crawler.c:858: (apr_err=24)
> subversion/libsvn_ra_serf/update.c:2639: (apr_err=24)
> subversion/libsvn_ra_serf/util.c:1837: (apr_err=24)
> subversion/libsvn_ra_serf/util.c:1818: (apr_err=24)
> subversion/libsvn_ra_serf/update.c:1281: (apr_err=24)
> subversion/libsvn_wc/update_editor.c:3688: (apr_err=24)
> subversion/libsvn_wc/adm_files.c:342: (apr_err=24)
> subversion/libsvn_subr/stream.c:888: (apr_err=24)
> subversion/libsvn_subr/io.c:4320: (apr_err=24)
> subversion/libsvn_subr/io.c:4135: (apr_err=24)
> svn: E000024: Can't create temporary file from template
> '/Users/markphip/work/trunk-testing/subversion/.svn/tmp/svn-XXXXXX':
> Too many open files
>
> So this gets further before getting a different error.
I *might* be able to explain at least a portion of this one.
My code calls the (new) get_wc_contents RA callback to get a read stream for
the pristine text matching the specified SHA1. This will open the pristine
file. The file remains open until its contents are consumed (whenever
Serf/ra_serf get around to doing so per the pipelined approach taken to
GET/HEAD/PROPFINDs during updates). I suppose if the code which is cramming
pipelined requests (the result of parsing the initial REPORT) into the queue
outpaces the code that's handling those requests, you could wind up with too
many open files.
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Enterprise Cloud Development