You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by GitBox <gi...@apache.org> on 2022/09/21 19:31:38 UTC
[GitHub] [couchdb] nickva opened a new issue, #4183: Optimize _bulk_get
nickva opened a new issue, #4183:
URL: https://github.com/apache/couchdb/issues/4183
As part of optimizing the replicator and implementing [`_bulk_get`](https://github.com/apache/couchdb/pull/4144) support for it, noticed that `_bulk_get` is implement in the most inefficient way possible: it makes sequential [`fabric:open_revs/4`](https://github.com/apache/couchdb/blob/cb176a69ab6f9a6631826db7df3a133ab1fee3b4/src/chttpd/src/chttpd_db.erl#L2439) calls for each `{doc_id, rev}`, when instead it could be grouping documents by shard range an issuing a bulk `open_revs` request per shard range, not unlike what `_bulk_docs` does.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] nickva commented on issue #4183: Optimize _bulk_get
Posted by GitBox <gi...@apache.org>.
nickva commented on issue #4183:
URL: https://github.com/apache/couchdb/issues/4183#issuecomment-1254141992
For instance, here is a snippet of logs from running a basic benchmark with couchdyno:
```
POST /cdyno-0000001/_bulk_get?latest=true&revs=true&attachments=false 200 ok 420
POST /cdyno-0000002/_bulk_docs 201 ok 122
POST /cdyno-0000001/_bulk_get?latest=true&revs=true&attachments=false 200 ok 429
POST /cdyno-0000002/_bulk_docs 201 ok 126
POST /cdyno-0000001/_bulk_get?latest=true&revs=true&attachments=false 200 ok 419
POST /cdyno-0000002/_bulk_docs 201 ok 124
POST /cdyno-0000002/_bulk_docs 201 ok 130
POST /cdyno-0000001/_bulk_get?latest=true&revs=true&attachments=false 200 ok 387
POST /cdyno-0000001/_bulk_get?latest=true&revs=true&attachments=false 200 ok 375
POST /cdyno-0000002/_bulk_docs 201 ok 137
```
The numbers at the end are the milliseconds it took to process the request. It looks like the `_bulk_get` (read) requests are almost 2x-3x as expensive as a `_bulk_docs` (write) requests. So it seems there is room for some optimization there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] nickva closed issue #4183: Optimize _bulk_get
Posted by GitBox <gi...@apache.org>.
nickva closed issue #4183: Optimize _bulk_get
URL: https://github.com/apache/couchdb/issues/4183
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org