You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by James Marca <jm...@translab.its.uci.edu> on 2009/11/09 20:07:21 UTC
compaction quirkiness
Hi All,
I'm running a recent svn checkout of couchdb (actually a git pull from
halorgium / couchdb). (1)
I have twelve large databases that I am populating with data (one db
per month of raw data...manual "sharding").
As each data-loading perl job finishes, it sends off a call to compact
the database as writing is done.
Unfortunately, multiple compaction jobs seem to crash couchdb, which
cancels the compaction (but doesn't seem to hurt the data loading
processes). I'm not sure why the server crashes, as I can't find
anything significant in my log files. All I see is:
...
[Fri, 06 Nov 2009 18:28:32 GMT] [info] [<0.7156.2>] 127.0.0.1 - -
'PUT' /d12_2007_04morehash/1209013_2007-04-29T03:00:00 201
[Fri, 06 Nov 2009 18:28:38 GMT] [info] [<0.31.0>] Apache CouchDB has
started on http://127.0.0.1:5984/
[Fri, 06 Nov 2009 18:28:39 GMT] [info] [<0.100.0>] 127.0.0.1 - - 'PUT'
/d12_2007_04morehash/1214006_2007-04-29T06:00:00 201
...
I have logging on info, not debug, but putting logging on debug didn't
appear to give more helpful information. I'd expect a crash condition
to spit out some sort of "error" level log message anyway.
If I run just one compaction at a time, all is well. Also, restarting
the crashed compactions restarts them at the right place (it picks up
where it left off). So the problem isn't devastating by any means,
but it seems like a bug that multiple compactions crash the server.
I'm willing to upgrade to a newer git pull, or try different things to
try to isolate the problem, short of mucking with the source code (I
do not know erlang).
Details of install:
CouchDB: {"couchdb":"Welcome","version":"0.11.0b1e2a54d1-git"}
(last commit dated Oct 29 in the logs)
Erlang: Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:8:8] [rq:8]
[async-threads:0] [kernel-poll:false]
OS: Gentoo Linux amd64 10.0/no-multilib (reasonably up to date, gcc
version 4.3.4 in case that matters to erlang/couchdb)
Regards,
James Marca
(1) Yes, I'm aware I can expect things to break running a git version
of CouchDB. I'm not complaining, rather just hoping my problem can
lead to finding and fixing a bug.
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
Re: compaction quirkiness
Posted by James Marca <jm...@translab.its.uci.edu>.
JIRA bug report filed COUCHDB-575.
On Mon, Nov 09, 2009 at 02:14:20PM -0500, Adam Kocoloski wrote:
> On Nov 9, 2009, at 2:07 PM, James Marca wrote:
>
> >Hi All,
> >
> >I'm running a recent svn checkout of couchdb (actually a git pull from
> >halorgium / couchdb). (1)
> >
> >I have twelve large databases that I am populating with data (one db
> >per month of raw data...manual "sharding").
> >
> >As each data-loading perl job finishes, it sends off a call to compact
> >the database as writing is done.
> >
> >Unfortunately, multiple compaction jobs seem to crash couchdb, which
> >cancels the compaction (but doesn't seem to hurt the data loading
> >processes). I'm not sure why the server crashes, as I can't find
> >anything significant in my log files. All I see is:
> >
> >...
> >
> >[Fri, 06 Nov 2009 18:28:32 GMT] [info] [<0.7156.2>] 127.0.0.1 - -
> >'PUT' /d12_2007_04morehash/1209013_2007-04-29T03:00:00 201
> >
> >[Fri, 06 Nov 2009 18:28:38 GMT] [info] [<0.31.0>] Apache CouchDB has
> >started on http://127.0.0.1:5984/
> >
> >[Fri, 06 Nov 2009 18:28:39 GMT] [info] [<0.100.0>] 127.0.0.1 - - 'PUT'
> >/d12_2007_04morehash/1214006_2007-04-29T06:00:00 201
> >
> >...
> >
> >I have logging on info, not debug, but putting logging on debug didn't
> >appear to give more helpful information. I'd expect a crash condition
> >to spit out some sort of "error" level log message anyway.
> >
> >If I run just one compaction at a time, all is well. Also, restarting
> >the crashed compactions restarts them at the right place (it picks up
> >where it left off). So the problem isn't devastating by any means,
> >but it seems like a bug that multiple compactions crash the server.
> >
> >I'm willing to upgrade to a newer git pull, or try different things to
> >try to isolate the problem, short of mucking with the source code (I
> >do not know erlang).
> >
> >Details of install:
> >
> >CouchDB: {"couchdb":"Welcome","version":"0.11.0b1e2a54d1-git"}
> >(last commit dated Oct 29 in the logs)
> >
> >Erlang: Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:8:8] [rq:8]
> >[async-threads:0] [kernel-poll:false]
> >
> >OS: Gentoo Linux amd64 10.0/no-multilib (reasonably up to date, gcc
> >version 4.3.4 in case that matters to erlang/couchdb)
> >
> >
> >Regards,
> >James Marca
> >
> >(1) Yes, I'm aware I can expect things to break running a git version
> >of CouchDB. I'm not complaining, rather just hoping my problem can
> >lead to finding and fixing a bug.
>
> Hi James, that does sound odd. Thanks for the detailed report, do you
> mind filing a JIRA ticket so we don't forget about this? Thanks,
>
> Adam
>
> https://issues.apache.org/jira/browse/COUCHDB
--
James E. Marca, PhD
Researcher
Institute of Transportation Studies
AIRB Suite 4000
University of California
Irvine, CA 92697-3600
jmarca@translab.its.uci.edu
(949) 824-6287
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
Re: compaction quirkiness
Posted by James Marca <jm...@translab.its.uci.edu>.
On Mon, Nov 09, 2009 at 02:14:20PM -0500, Adam Kocoloski wrote:
> On Nov 9, 2009, at 2:07 PM, James Marca wrote:
...
>
> Hi James, that does sound odd. Thanks for the detailed report, do you
> mind filing a JIRA ticket so we don't forget about this? Thanks,
>
> Adam
>
> https://issues.apache.org/jira/browse/COUCHDB
I will.
James
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
Re: compaction quirkiness
Posted by Adam Kocoloski <ko...@apache.org>.
On Nov 9, 2009, at 2:07 PM, James Marca wrote:
> Hi All,
>
> I'm running a recent svn checkout of couchdb (actually a git pull from
> halorgium / couchdb). (1)
>
> I have twelve large databases that I am populating with data (one db
> per month of raw data...manual "sharding").
>
> As each data-loading perl job finishes, it sends off a call to compact
> the database as writing is done.
>
> Unfortunately, multiple compaction jobs seem to crash couchdb, which
> cancels the compaction (but doesn't seem to hurt the data loading
> processes). I'm not sure why the server crashes, as I can't find
> anything significant in my log files. All I see is:
>
> ...
>
> [Fri, 06 Nov 2009 18:28:32 GMT] [info] [<0.7156.2>] 127.0.0.1 - -
> 'PUT' /d12_2007_04morehash/1209013_2007-04-29T03:00:00 201
>
> [Fri, 06 Nov 2009 18:28:38 GMT] [info] [<0.31.0>] Apache CouchDB has
> started on http://127.0.0.1:5984/
>
> [Fri, 06 Nov 2009 18:28:39 GMT] [info] [<0.100.0>] 127.0.0.1 - - 'PUT'
> /d12_2007_04morehash/1214006_2007-04-29T06:00:00 201
>
> ...
>
> I have logging on info, not debug, but putting logging on debug didn't
> appear to give more helpful information. I'd expect a crash condition
> to spit out some sort of "error" level log message anyway.
>
> If I run just one compaction at a time, all is well. Also, restarting
> the crashed compactions restarts them at the right place (it picks up
> where it left off). So the problem isn't devastating by any means,
> but it seems like a bug that multiple compactions crash the server.
>
> I'm willing to upgrade to a newer git pull, or try different things to
> try to isolate the problem, short of mucking with the source code (I
> do not know erlang).
>
> Details of install:
>
> CouchDB: {"couchdb":"Welcome","version":"0.11.0b1e2a54d1-git"}
> (last commit dated Oct 29 in the logs)
>
> Erlang: Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:8:8] [rq:8]
> [async-threads:0] [kernel-poll:false]
>
> OS: Gentoo Linux amd64 10.0/no-multilib (reasonably up to date, gcc
> version 4.3.4 in case that matters to erlang/couchdb)
>
>
> Regards,
> James Marca
>
> (1) Yes, I'm aware I can expect things to break running a git version
> of CouchDB. I'm not complaining, rather just hoping my problem can
> lead to finding and fixing a bug.
Hi James, that does sound odd. Thanks for the detailed report, do you
mind filing a JIRA ticket so we don't forget about this? Thanks,
Adam
https://issues.apache.org/jira/browse/COUCHDB