[GitHub] mmarinchenko commented on issue #1209: Erlang native query server consumes too much RAM when filters a document with lots of revisions

[GitBox <gi...@apache.org>]

mmarinchenko commented on issue #1209: Erlang native query server consumes too much RAM when filters a document with lots of revisions
URL: https://github.com/apache/couchdb/issues/1209#issuecomment-372201494
 
 
   Thank you for the response, Janl!
   
   But why are you asking this? I've pointed out to the source code :)
   
   I'm talking about any revision, regular, conflicted, or deleted. It doesn't matter in this particular context. Of course, with many conflicts the situation will be even worse. If the `style=all_docs` parameter is specified in request along with native filter then CouchDB calls `to_binary()` for each document with all its revisions. All at once!
   
   I'm not claiming that a document with such amount of revisions (especially conflicts) is a good state. No. This happened to us by accident, I guess. I'm not responsible for the application side of things, so I don't know why this happened.
   
   I'm just trying to say that loading a document with all of its associated revisions into memory at once and then convert all this data into plain list using `to_binary()` without checking the size of loaded data, any buffering, etc. is a very bad idea for a database management system.
   
   If we look at changes enumerator we'll see that situation even worse. Imagine that we need to filter several sequential changes with such a big bad document through native query server. CouchDB will load all the revisions and call `to_binary()` for all the revisions on each change. For what matter?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services