You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Norman Barker <no...@gmail.com> on 2009/10/06 23:18:16 UTC

couchdb slowed down

I have upgraded couchdb from 0.9 to the trunk and the document write
speed has really slowed down.

My sequence is to call an erlang function get_docs_revs (my code) to
find if the document exists as follows

get_doc_revs(Db, DocId) ->
  case DocId of
    undefined ->
      Id = couch_util:new_uuid(),
      Revs = {0, []};
    _ ->
      case couch_db:get_doc_info(Db, DocId) of
        {ok, Info} ->
           Id = Info#doc_info.id,
           [{rev_info, {Start, Rev}, _seq, _deleted, _body_sp}|_] =
Info#doc_info.revs,
           Revs = {Start, [Rev]};
        _ ->
          Id = DocId,
          Revs = {0, []}
      end
  end,
  {Id, Revs}.

and then to write this doc to couch by calling write_doc_to_couch(Db,
#doc{id=Id, revs=Revs, body={JsonObj}}, []) where this is defined as;

write_doc_to_couch(Db, Doc, Options) ->
   case couch_db:update_doc(Db,  Doc, Options) of
      {ok, NewRev} ->
         {Doc#doc.id, NewRev};
      Error ->
         throw({error, Error})
   end.

With this sequence I was writing about 500 docs in 5 seconds with 0.9,
now it has slowed down to 30 seconds with trunk, my JSON generating
code hasn't changed at all, the only thing that has had to change
since moving from 0.9 to trunk is the get_docs_revs above - has this
become a bottle neck in couchdb?

My temporary work around is to spawn the writes, but this still
doesn't really solve why it is has slowed down so much. Is there a
'cheaper' way of getting the current revision?

thanks,

Norman

Re: couchdb slowed down

Posted by Norman Barker <no...@gmail.com>.
On Tue, Oct 6, 2009 at 7:19 PM, Adam Kocoloski <ko...@apache.org> wrote:
> On Oct 6, 2009, at 5:18 PM, Norman Barker wrote:
>
>> I have upgraded couchdb from 0.9 to the trunk and the document write
>> speed has really slowed down.
>>
>> My sequence is to call an erlang function get_docs_revs (my code) to
>> find if the document exists as follows
>>
>> get_doc_revs(Db, DocId) ->
>>  case DocId of
>>   undefined ->
>>     Id = couch_util:new_uuid(),
>>     Revs = {0, []};
>>   _ ->
>>     case couch_db:get_doc_info(Db, DocId) of
>>       {ok, Info} ->
>>          Id = Info#doc_info.id,
>>          [{rev_info, {Start, Rev}, _seq, _deleted, _body_sp}|_] =
>> Info#doc_info.revs,
>>          Revs = {Start, [Rev]};
>>       _ ->
>>         Id = DocId,
>>         Revs = {0, []}
>>     end
>>  end,
>>  {Id, Revs}.
>>
>> and then to write this doc to couch by calling write_doc_to_couch(Db,
>> #doc{id=Id, revs=Revs, body={JsonObj}}, []) where this is defined as;
>>
>> write_doc_to_couch(Db, Doc, Options) ->
>>  case couch_db:update_doc(Db,  Doc, Options) of
>>     {ok, NewRev} ->
>>        {Doc#doc.id, NewRev};
>>     Error ->
>>        throw({error, Error})
>>  end.
>>
>> With this sequence I was writing about 500 docs in 5 seconds with 0.9,
>> now it has slowed down to 30 seconds with trunk, my JSON generating
>> code hasn't changed at all, the only thing that has had to change
>> since moving from 0.9 to trunk is the get_docs_revs above - has this
>> become a bottle neck in couchdb?
>>
>> My temporary work around is to spawn the writes, but this still
>> doesn't really solve why it is has slowed down so much. Is there a
>> 'cheaper' way of getting the current revision?
>>
>> thanks,
>>
>> Norman
>
> Hi Norman, you might check the value for delayed_commits in the [couchdb]
> section of the .ini files. If set to false your serial write performance
> will be slower because CouchDB will fsync before returning from
> update_doc/3.  If set to true CouchDB will fsync at most once per second.
>  It's a tradeoff between durability and throughput.  For a while
> delayed_commits was set to false on trunk.  Best,
>
> Adam
>

Hi Adam,

great, that makes a big difference.

thanks,

Norman

Re: couchdb slowed down

Posted by Adam Kocoloski <ko...@apache.org>.
On Oct 6, 2009, at 5:18 PM, Norman Barker wrote:

> I have upgraded couchdb from 0.9 to the trunk and the document write
> speed has really slowed down.
>
> My sequence is to call an erlang function get_docs_revs (my code) to
> find if the document exists as follows
>
> get_doc_revs(Db, DocId) ->
>  case DocId of
>    undefined ->
>      Id = couch_util:new_uuid(),
>      Revs = {0, []};
>    _ ->
>      case couch_db:get_doc_info(Db, DocId) of
>        {ok, Info} ->
>           Id = Info#doc_info.id,
>           [{rev_info, {Start, Rev}, _seq, _deleted, _body_sp}|_] =
> Info#doc_info.revs,
>           Revs = {Start, [Rev]};
>        _ ->
>          Id = DocId,
>          Revs = {0, []}
>      end
>  end,
>  {Id, Revs}.
>
> and then to write this doc to couch by calling write_doc_to_couch(Db,
> #doc{id=Id, revs=Revs, body={JsonObj}}, []) where this is defined as;
>
> write_doc_to_couch(Db, Doc, Options) ->
>   case couch_db:update_doc(Db,  Doc, Options) of
>      {ok, NewRev} ->
>         {Doc#doc.id, NewRev};
>      Error ->
>         throw({error, Error})
>   end.
>
> With this sequence I was writing about 500 docs in 5 seconds with 0.9,
> now it has slowed down to 30 seconds with trunk, my JSON generating
> code hasn't changed at all, the only thing that has had to change
> since moving from 0.9 to trunk is the get_docs_revs above - has this
> become a bottle neck in couchdb?
>
> My temporary work around is to spawn the writes, but this still
> doesn't really solve why it is has slowed down so much. Is there a
> 'cheaper' way of getting the current revision?
>
> thanks,
>
> Norman

Hi Norman, you might check the value for delayed_commits in the  
[couchdb] section of the .ini files. If set to false your serial write  
performance will be slower because CouchDB will fsync before returning  
from update_doc/3.  If set to true CouchDB will fsync at most once per  
second.  It's a tradeoff between durability and throughput.  For a  
while delayed_commits was set to false on trunk.  Best,

Adam