[GitHub] AlexanderKaraberov opened a new issue #1142: fabric:query_view/4 fails if number of docs in the DB is more than 60

[GitBox <gi...@apache.org>]

AlexanderKaraberov opened a new issue #1142: fabric:query_view/4 fails if number of docs in the DB is more than 60
URL: https://github.com/apache/couchdb/issues/1142
 
 
   ## Expected Behaviour
   Currently I'm implementing new view filter functionality using CouchDB 2.2 (master branch) codebase. Namely I want to filter `_changes` feed using additional query params like this:
   `_changes?include_docs=true&filter=_view&view=index/type&key="d"'`, `start_key_docid` and so on. 
   I managed to implement a working solution. I parse query string to `#mrargs` record (I reuse `fitler_args` field) and then pass this to this function (I wrote an additional case for a 5-tuple).
   ```erlang
   filter(Db, DocInfo, {FilterType, Style, DDoc, VName, FilterArgs})
           when FilterType == view; FilterType == fast_view
   ```
   And then I use `query_view/4` from `fabric.erl` proxy layer in order to execute a view and receive this kind of response `{row,[{id,<<"doc2">>},{key,<<"d">>}]}]`  and then I implemented something similar to this: `{ok, Passes} = couch_query_servers:filter_view(DDoc, VName, Docs),` functionality.
   I would expect this to work on any number of docs (in my case an I have in average ~50000 docs and up to 200000 docs).
   
   ## Current Behaviour
   My implementation works flawlessly with very small test databases (1-55 docs). I can apply additional filters (`type`, `keys` and so on) as a query param and all my tests succeed but when I have more than 50-60 docs this function fails. I have `results [ ` printed in logs and then just an empty space. I receive no timeout as well. Just in case I also checked in `fabric_view_map:go` that timeout value for erlang:send_after is fairly large:
   ```erlang
   case rexi_utils:recv(Workers, #shard.ref, fun handle_message/3,
           State, infinity, 1000 * 60 * 60) of
   ``` 
   Currently for the sake of testing I'm using only map views but i see the same `rexi_util_recv` call in `fabric_view_reduce:go2` as well.
   
   ## Possible Solution
   It looks like Erlang `gen_server`'s `handle_call` doesn't have enough time to receive a response chunk and just infinitely calls `changes_callback` with no result but I might be wrong. Probably that's because of clustered configuration (I have a default 3-node setup) and now `query_view` takes additional time to query all nodes but timeout for `erlang:send_after` wasn't increased.
   
   ## Steps to Reproduce (for bugs)
   You can very easily reproduce this. 
   1. Just insert this call to query_view in `couch_changes:filter` with filter type == `view`:
   ```erlang
   [_, DName] = binary:split(element(2, DDoc), <<"/">>),
    QArgs = #mrargs{},
    {ok, VResp} = fabric:query_view(<<db1>>, DName, VName, QArgs),
   ```
   2. Test with small db (<50 docs): create a simple ddoc `index`, make query:
   `curl -u admin:rm http://127.0.0.1:5983/db1/_design/index/_view/type` - everything works
   2 Add more docs (e.g. 90), make the same query and observe that `_changes` feed stuck.
   
   ## Context
   As I said in the beginning I need this functionality to work because I want to implement my additional logic for view filtering.
   P.S Yes I know that I may use mango selectors for this (in fact i already do this):
   http://docs.couchdb.org/en/2.1.1/api/database/changes.html#selector
   But I want to know query_view and hence my implementation doesn't work on a fairly small DB.
   
   ## Your Environment
   CouchDB 2.2 (fork from master). But I already tested this with 2.1 release version and behaviour is completely the same unfortunately.
   
   Any help from CouchDB core devs will be much appreciated!

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services