You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by GitBox <gi...@apache.org> on 2020/05/05 10:41:41 UTC
[GitHub] [couchdb] garrensmith opened a new pull request #2869: fetch local docs in fold_docs function
garrensmith opened a new pull request #2869:
URL: https://github.com/apache/couchdb/pull/2869
<!-- Thank you for your contribution!
Please file this form by replacing the Markdown comments
with your text. If a section needs no action - remove it.
Also remember, that CouchDB uses the Review-Then-Commit (RTC) model
of code collaboration. Positive feedback is represented +1 from committers
and negative is a -1. The -1 also means veto, and needs to be addressed
to proceed. Once there are no objections, the PR can be merged by a
CouchDB committer.
See: http://couchdb.apache.org/bylaws.html#decisions for more info. -->
## Overview
<!-- Please give a short brief for the pull request,
what problem it solves or how it makes things better. -->
## Testing recommendations
<!-- Describe how we can test your changes.
Does it provides any behaviour that the end users
could notice? -->
## Related Issues or Pull Requests
<!-- If your changes affects multiple components in different
repositories please put links to those issues or pull requests here. -->
## Checklist
- [x] Code is written and works correctly
- [x] Changes are covered by tests
- [ ] Any new configurable parameters are documented in `rel/overlay/etc/default.ini`
- [ ] A PR for documentation changes has been made in https://github.com/apache/couchdb-documentation
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] nickva commented on a change in pull request #2869: fetch local docs in fold_docs function
Posted by GitBox <gi...@apache.org>.
nickva commented on a change in pull request #2869:
URL: https://github.com/apache/couchdb/pull/2869#discussion_r421560988
##########
File path: test/elixir/test/all_docs_test.exs
##########
@@ -275,22 +280,88 @@ defmodule AllDocsTest do
assert doc["string"] == "1"
end
+ @tag :with_db
+ test "_local_docs POST with keys and limit", context do
+ expected = [
+ %{
+ "doc" => %{"_id" => "_local/one", "_rev" => "0-1", "value" => "one"},
+ "id" => "_local/one",
+ "key" => "_local/one",
+ "value" => %{"rev" => "0-1"}
+ },
+ %{
+ "doc" => %{"_id" => "_local/two", "_rev" => "0-1", "value" => "two"},
+ "id" => "_local/two",
+ "key" => "_local/two",
+ "value" => %{"rev" => "0-1"}
+ },
+ %{
+ "doc" => %{
+ "_id" => "three",
+ "_rev" => "1-878d3724976748bc881841046a276ceb",
+ "value" => "three"
+ },
+ "id" => "three",
+ "key" => "three",
+ "value" => %{"rev" => "1-878d3724976748bc881841046a276ceb"}
+ },
+ %{"error" => "not_found", "key" => "missing"},
+ %{"error" => "not_found", "key" => "_local/missing"}
+ ]
+
+ db_name = context[:db_name]
+
+ docs = [
+ %{
+ _id: "_local/one",
+ value: "one"
+ },
+ %{
+ _id: "_local/two",
+ value: "two"
+ },
+ %{
+ _id: "three",
+ value: "three"
+ }
+ ]
+
+ resp = Couch.post("/#{db_name}/_bulk_docs", body: %{docs: docs})
+ assert resp.status_code in [201, 202]
+
+ resp =
+ Couch.post(
+ "/#{db_name}/_all_docs",
+ body: %{
+ :keys => ["_local/one", "_local/two", "three", "missing", "_local/missing"],
+ :include_docs => true
+ }
+ )
+
+ assert resp.status_code == 200
+ rows = resp.body["rows"]
+ IO.inspect rows
Review comment:
A debug line? I notice elixir's credo test didn't like that:
```
Checking 74 source files (this might take a while) ...
Warnings - please take a look
┃
┃ [W] ↗ There should be no calls to IO.inspect/1.
┃ test/elixir/test/all_docs_test.exs:343:5 #(AllDocsTest)
Please report incorrect results: https://github.com/rrrene/credo/issues
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] nickva commented on a change in pull request #2869: fetch local docs in fold_docs function
Posted by GitBox <gi...@apache.org>.
nickva commented on a change in pull request #2869:
URL: https://github.com/apache/couchdb/pull/2869#discussion_r420959016
##########
File path: src/fabric/src/fabric2_db.erl
##########
@@ -996,12 +986,12 @@ fold_docs(Db, DocIds, UserFun, UserAcc0, Options) ->
user_fun => UserFun
},
- FinalAcc1 = lists:foldl(fun(DocId, Acc) ->
+ FinalAcc1 = lists:foldl(fun (DocId, Acc) ->
Review comment:
Minor nit: whitespace change
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] nickva commented on a change in pull request #2869: fetch local docs in fold_docs function
Posted by GitBox <gi...@apache.org>.
nickva commented on a change in pull request #2869:
URL: https://github.com/apache/couchdb/pull/2869#discussion_r421561824
##########
File path: src/fabric/src/fabric2_db.erl
##########
@@ -1262,6 +1252,47 @@ drain_all_deleted_info_futures(FutureQ, UserFun, Acc) ->
end.
+fold_docs_get_revs(Db, <<?LOCAL_DOC_PREFIX, _/binary>> = DocId, _) ->
+ fabric2_fdb:get_local_doc_rev_future(Db, DocId);
+
+fold_docs_get_revs(Db, DocId, true) ->
+ fabric2_fdb:get_all_revs_future(Db, DocId);
+
+fold_docs_get_revs(Db, DocId, false) ->
+ fabric2_fdb:get_winning_revs_future(Db, DocId, 1).
+
+
+fold_docs_get_revs_wait(_Db, <<?LOCAL_DOC_PREFIX, _/binary>>, RevsFuture) ->
+ Rev = fabric2_fdb:get_local_doc_rev_wait(RevsFuture),
+ [Rev];
+
+fold_docs_get_revs_wait(Db, _DocId, RevsFuture) ->
+ fabric2_fdb:get_revs_wait(Db, RevsFuture).
+
+
+fold_docs_get_doc_body_future(Db, <<?LOCAL_DOC_PREFIX, _/binary>> = DocId,
+ [Rev]) ->
+ fabric2_fdb:get_local_doc_body_future(Db, DocId, Rev);
+
+fold_docs_get_doc_body_future(Db, DocId, Revs) ->
+ Winner = get_rev_winner(Revs),
+ fabric2_fdb:get_doc_body_future(Db, DocId, Winner).
+
+
+fold_docs_get_doc_body_wait(Db, <<?LOCAL_DOC_PREFIX, _/binary>> = DocId, [Rev],
+ _DocOpts, BodyFuture) ->
+ case fabric2_fdb:get_local_doc_body_wait(Db, DocId, Rev, BodyFuture) of
+ {not_found,missing} -> {not_found, missing};
Review comment:
Minor nit: space between `not_found` and `missing`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] garrensmith commented on pull request #2869: fetch local docs in fold_docs function
Posted by GitBox <gi...@apache.org>.
garrensmith commented on pull request #2869:
URL: https://github.com/apache/couchdb/pull/2869#issuecomment-625254683
@nickva thanks I've updated with a fix for the deleted doc. Great spot on the deleted doc. I've updated the elixir test to test for that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] garrensmith commented on pull request #2869: fetch local docs in fold_docs function
Posted by GitBox <gi...@apache.org>.
garrensmith commented on pull request #2869:
URL: https://github.com/apache/couchdb/pull/2869#issuecomment-624639422
@nickva and @davisp thanks for the review. I've updated the PR to make the local docs async and also added some eunit tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] davisp commented on a change in pull request #2869: fetch local docs in fold_docs function
Posted by GitBox <gi...@apache.org>.
davisp commented on a change in pull request #2869:
URL: https://github.com/apache/couchdb/pull/2869#discussion_r420246021
##########
File path: src/fabric/src/fabric2_db.erl
##########
@@ -996,17 +996,32 @@ fold_docs(Db, DocIds, UserFun, UserAcc0, Options) ->
user_fun => UserFun
},
- FinalAcc1 = lists:foldl(fun(DocId, Acc) ->
- #{
- revs_q := RevsQ,
- revs_count := RevsCount
- } = Acc,
- Future = FetchRevs(DocId),
- NewAcc = Acc#{
- revs_q := queue:in({DocId, Future}, RevsQ),
- revs_count := RevsCount + 1
- },
- drain_fold_docs_revs_futures(TxDb, NewAcc)
+ FinalAcc1 = lists:foldl(fun
+ (<<?LOCAL_DOC_PREFIX, _/binary>> = DocId, Acc) ->
+ DocResp = case fabric2_fdb:get_local_doc(TxDb, DocId) of
+ #doc{} = Doc -> {ok, Doc};
+ _Else -> {not_found, missing}
+ end,
+ #{
+ user_acc := UserAcc,
+ user_fun := UserFun
+ } = Acc,
+
+ NewUserAcc = maybe_stop(UserFun(DocId, DocResp, UserAcc)),
Review comment:
This is broken. You can't mix synchronous and asynchronous fetches like this. `POST /dbname/_all_docs` has a (perhaps implicit?) contract for returning docs in the order specified in the body. Mixing these operations means that we'd be returning any `_local` docs in some random position within the previous 100 or so docs returned.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org