[GitHub] [couchdb] davisp commented on issue #2232: Elusive refc binary memory leaks in index_updater and replicator_worker processes

[GitBox <gi...@apache.org>]

davisp commented on issue #2232: Elusive refc binary memory leaks in index_updater and replicator_worker processes
URL: https://github.com/apache/couchdb/issues/2232#issuecomment-538550840
 
 
   The couch_replicator_worker is probably straightforward to fix. We could just add a hibernate after a batch is flushed [1] to trigger a thorough GC. There was a similar GC optimization done quite a long time ago for this same issue.
   
   For the couch_index_updater, is that the gen_server process, or the anonymous worker process actually performing the updates?
   
   If it were the gen_server, then likely adding a similar hibernate option after every update would likely be the best approach. I'm somewhat uncertain what'd be accumulating binaries there. Perhaps a largish reduce value? Though it does have all the usual hallmarks of a process that passes binaries around but doesn't do many reductions of its own.
   
   If its not the gen_server then we'll have to do a bit more investigating because that process already invokes garbage collection after every document processed. So having it getting into a state where there's lots of accumulated binaries would be fairly unexpected. Not to manage that its already doing quite a few reductions which should also trigger GC fairly quickly. Certainly I'd expect quickly enough that a GC-the-world would release anything meaningful on a regular basis.
   
   [1] https://github.com/apache/couchdb/blob/2.3.1/src/couch_replicator/src/couch_replicator_worker.erl#L174

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services