You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by John Merrells <jo...@merrells.com> on 2010/05/15 23:13:36 UTC
write to compacting database
Hello,
If I write a document to a database whilst it's being compacted what http result code will be sent back?
John
--
John Merrells
http://johnmerrells.com
+1.415.244.5808
Re: write to compacting database
Posted by John Merrells <jo...@merrells.com>.
On May 19, 2010, at 6:55 PM, Randall Leeds wrote:
> Might be hitting the maximum number of file descriptors.
> You could bump this by using the ulimit command. You'll have to modify
> your init script if you launch couch as a daemon that way. Just run
> `ulimit -n XXXX` before erlang starts.
Good point. I'll bump it up and let it run for a day or so...
John
--
John Merrells
http://johnmerrells.com
+1.415.244.5808
Re: write to compacting database
Posted by Randall Leeds <ra...@gmail.com>.
Might be hitting the maximum number of file descriptors.
You could bump this by using the ulimit command. You'll have to modify
your init script if you launch couch as a daemon that way. Just run
`ulimit -n XXXX` before erlang starts.
Re: write to compacting database
Posted by John Merrells <jo...@merrells.com>.
On May 16, 2010, at 4:37 PM, Randall Leeds wrote:
>>
>> max_dbs_open = 100
>>
>> and I have ~130 databases. Could that explain it?
>
> Easy way to test :)
I have bumped this up, but I'm still getting period patches of 500's from couch.
I still see the gen_server timeouts occurring, but I now see these too....
[Thu, 20 May 2010 00:16:36 GMT] [debug] [<0.25663.146>] httpd 500 error response:^M
{"error":"unknown_error","reason":"system_limit"}^M
So... possibly I'm just overloading this machine and this is couch's failure mode?
John
--
John Merrells
http://johnmerrells.com
+1.415.244.5808
Re: write to compacting database
Posted by John Merrells <jo...@merrells.com>.
On May 16, 2010, at 4:37 PM, Randall Leeds wrote:
> On Sun, May 16, 2010 at 14:44, John Merrells <jo...@merrells.com> wrote:
>> In the couch log I see patches where it's returning 500 for PUTs and GETs... e.g.
>
> ...snip...
>
>> But there's nothing there to suggest why it's happening. The same documents will work fine later when the client retries.
>
> You could try debug logging.
With debugging on...
[Tue, 18 May 2010 06:08:53 GMT] [debug] [<0.12523.115>] httpd 500 error response:
{"error":"timeout","reason":"{gen_server,call,[<0.13110.110>,{open_ref_count,<0.12523.115>}]}"}
[Tue, 18 May 2010 06:08:53 GMT] [debug] [<0.12575.115>] httpd 500 error response:
{"error":"timeout","reason":"{gen_server,call,\n [couch_server,\n {open,<<\"articles-2\">>,\n [{user_ctx,\n {user_ctx,<<\"admin\">>,\n [<<\"_admin\">>],\n <<\"{couch_httpd_auth, default_authentication_handler}\">>}}]}]}"}
...so it's timing out.
>
>> My only other theory is that I have more open databases than the configuration allows for.
>
> Seems possible.
>
>>
>> max_dbs_open = 100
>>
>> and I have ~130 databases. Could that explain it?
>
> Easy way to test :)
Yeah, I'll bump it up and see if the badness goes away.
John
--
John Merrells
http://johnmerrells.com
+1.415.244.5808
Re: write to compacting database
Posted by Randall Leeds <ra...@gmail.com>.
On Sun, May 16, 2010 at 14:44, John Merrells <jo...@merrells.com> wrote:
> In the couch log I see patches where it's returning 500 for PUTs and GETs... e.g.
...snip...
> But there's nothing there to suggest why it's happening. The same documents will work fine later when the client retries.
You could try debug logging.
> My only other theory is that I have more open databases than the configuration allows for.
Seems possible.
>
> max_dbs_open = 100
>
> and I have ~130 databases. Could that explain it?
Easy way to test :)
Re: write to compacting database
Posted by John Merrells <jo...@merrells.com>.
On May 15, 2010, at 5:30 PM, Randall Leeds wrote:
> On Sat, May 15, 2010 at 16:27, John Merrells <jo...@merrells.com> wrote:
>> Hmm.... I get 500's back quite often... only ever from put's... was wondering if that was it. Must be something else. Are there any common cases where a 500 comes back?
>
> Not off the top of my head.
> If you can find any relevant log lines or documents that reproduce the
> problem you should start a thread about it or maybe ask around in
> #couchdb on freenode and people might have ideas.
In the couch log I see patches where it's returning 500 for PUTs and GETs... e.g.
[Sun, 25 Apr 2010 19:20:59 GMT] [info] [<0.29954.5>] 192.168.130.122 - - 'PUT' /images/http%3a%2f%2fexternal.ak.fbcdn.net%2fsa
fe_image.php%3fd%3de9dfe1296dd93b9677bb6ce281aaf500%26w%3d130%26h%3d130%26url%3dhttp%253A%252F%252Fcdn.picapp.com%252Fftp%252FImages%252F9%252F0%252F3%252F0%252FSOCCER_2010_0af0.jpg%253FadImageId%253D12422580%2526imageId%253D7985100 409
[Sun, 25 Apr 2010 19:21:00 GMT] [info] [<0.29961.5>] 192.168.130.122 - - 'GET' /pages-27/http%3a%2f%2fnats320.blogspot.com%2ffeeds%2fposts%2fdefault 500
[Sun, 25 Apr 2010 19:21:00 GMT] [info] [<0.29973.5>] 192.168.130.122 - - 'GET' /pages-45/http%3a%2f%2ffeeds2.feedburner.com%2fmadridgalacticos 500
[Sun, 25 Apr 2010 19:21:00 GMT] [info] [<0.29935.5>] 192.168.130.122 - - 'PUT' /pages-52/http%3a%2f%2fmlb-vids.com%2ffeed 500
[Sun, 25 Apr 2010 19:21:00 GMT] [info] [<0.29938.5>] 192.168.130.122 - - 'GET' /pages-46/http%3a%2f%2fpoeticfootball.blogspot.com%2ffeeds%2fposts%2fdefault 500
[Sun, 25 Apr 2010 19:21:00 GMT] [info] [<0.29943.5>] 192.168.130.122 - - 'PUT' /pages-8/http%3a%2f%2fwww.soccerjones.com%2f%3ffeed%3drss2 500
[Sun, 25 Apr 2010 19:21:00 GMT] [info] [<0.29932.5>] 192.168.130.122 - - 'GET' /pages-17/http%3a%2f%2fnineguysshort.blogspot.c
om%2ffeeds%2fposts%2fdefault 500
[Sun, 25 Apr 2010 19:21:00 GMT] [info] [<0.29986.5>] 192.168.130.122 - - 'PUT' /pages-36/http%3a%2f%2foutonpenalties.blogspot.com%2ffeeds%2fposts%2fdefault/page?rev=963-b9053ca84ca6af969f531bd4a90a0608 500
[Sun, 25 Apr 2010 19:21:00 GMT] [info] [<0.29985.5>] 192.168.130.122 - - 'PUT' /pages-61/http%3a%2f%2flyon.theoffside.com%2ffeed 500
[Sun, 25 Apr 2010 19:21:00 GMT] [info] [<0.29958.5>] 192.168.130.122 - - 'PUT' /pages-31/http%3a%2f%2fsoccerfansintheusa.blogspot.com%2ffeeds%2fposts%2fdefault/page?rev=861-974ee7587021a145626d68e538f6cbd7 500
[Sun, 25 Apr 2010 19:21:06 GMT] [info] [<0.30213.5>] 192.168.130.122 - - 'PUT' /pages-31/http%3a%2f%2frevolution.theoffside.com%2ffeed/page?rev=950-f511675e78e2d806c49f9773439f56f7 500
[Sun, 25 Apr 2010 19:21:06 GMT] [info] [<0.30141.5>] 192.168.130.122 - - 'PUT' /pages-34/http%3a%2f%2fwww.pinstripealley.com%2frss%2ffanposts 500
[Sun, 25 Apr 2010 19:21:06 GMT] [info] [<0.30141.5>] 192.168.130.122 - - 'PUT' /pages-34/http%3a%2f%2fwww.pinstripealley.com%2frss%2ffanposts 500
[Sun, 25 Apr 2010 19:21:06 GMT] [info] [<0.29948.5>] 192.168.130.122 - - 'PUT' /pages-44/http%3a%2f%2ffeeds.feedburner.com%2fSoccerblogger 500
[Sun, 25 Apr 2010 19:21:06 GMT] [info] [<0.30148.5>] 192.168.130.122 - - 'PUT' /images/http%3a%2f%2fexternal.ak.fbcdn.net%2fsafe_image.php%3fd%3d53c0fea305b8e711032701da46c02591%26w%3d130%26h%3d130%26url%3dhttp%253A%252F%252Fcdn.picapp.com%252Fftp%252FImages%252Fe%252F1%252Fe%252Fa%252FBarcelona_vs_Arsenal_eaf7.jpg%253FadImageId%253D12407869%2526imageId%253D8445390 500
[Sun, 25 Apr 2010 19:21:06 GMT] [info] [<0.29961.5>] 192.168.130.122 - - 'PUT' /pages-45/http%3a%2f%2fdodgers.gearupforsports.com%2fblog%2ffeed%2f/page?rev=654-d7d4432f117d3fbad2dea35859e999dc 500
[Sun, 25 Apr 2010 19:21:06 GMT] [info] [<0.29930.5>] 192.168.130.122 - - 'PUT' /pages-3/http%3a%2f%2fwww.mlsenespanol.net%2ffeeds%2fposts%2fdefault 500
[Sun, 25 Apr 2010 19:21:06 GMT] [info] [<0.29944.5>] 192.168.130.122 - - 'PUT' /pages-62/http%3a%2f%2fhastaelgolsiempre.com%2ffeed%2f 500
[Sun, 25 Apr 2010 19:21:07 GMT] [info] [<0.29932.5>] 192.168.130.122 - - 'PUT' /pages-17/http%3a%2f%2ffeeds.feedburner.com%2fworldcupcorner 500
[Sun, 25 Apr 2010 19:34:39 GMT] [info] [<0.30222.5>] 192.168.130.122 - - 'GET' /images/http%3a%2f%2fhits.guardian.co.uk%2fb%2fss%2fguardiangufeeds%2f1%2fH.20.3%2f66657%3fAQB%3d1%26pccr%3dtrue%26vidn%3d25E327C185010EA9-40000101A00007D9%26g%3dnone%26ns%3dguardian%26pageName%3dSouth%2bAfrica%2bracing%2bto%2bsell%2b500%252C000%2bWorld%2bCup%2btickets%253AArticle%253A1385557%26ch%3dFootball%26c3%3dGuardian%26c4%3dWorld%2b 200
But there's nothing there to suggest why it's happening. The same documents will work fine later when the client retries.
My only other theory is that I have more open databases than the configuration allows for.
max_dbs_open = 100
and I have ~130 databases. Could that explain it?
John
--
John Merrells
http://johnmerrells.com
+1.415.244.5808
Re: write to compacting database
Posted by Randall Leeds <ra...@gmail.com>.
On Sat, May 15, 2010 at 16:27, John Merrells <jo...@merrells.com> wrote:
> Hmm.... I get 500's back quite often... only ever from put's... was wondering if that was it. Must be something else. Are there any common cases where a 500 comes back?
Not off the top of my head.
If you can find any relevant log lines or documents that reproduce the
problem you should start a thread about it or maybe ask around in
#couchdb on freenode and people might have ideas.
Re: write to compacting database
Posted by John Merrells <jo...@merrells.com>.
On May 15, 2010, at 2:36 PM, Randall Leeds wrote:
> Compaction has no effect on DB access.
Hmm.... I get 500's back quite often... only ever from put's... was wondering if that was it. Must be something else. Are there any common cases where a 500 comes back?
John
--
John Merrells
http://johnmerrells.com
+1.415.244.5808
Re: write to compacting database
Posted by Randall Leeds <ra...@gmail.com>.
Compaction has no effect on DB access.
On May 15, 2010 2:14 PM, "John Merrells" <jo...@merrells.com> wrote:
Hello,
If I write a document to a database whilst it's being compacted what http
result code will be sent back?
John
--
John Merrells
http://johnmerrells.com
+1.415.244.5808