Assistance in tuning CouchDB heap and/or replicator config

[Mat Degerholm <Ma...@kiwirail.co.nz>]

Hi guys,

Just looking for a little assistance in tuning the CouchDB heap and/or replicator config.

When performing an initial/full replication to an empty 2nd node using the _replicator database, the 2nd node crashes with the following error:

hend=0x00007f280d67f628
stop=0x00007f280d67f418
htop=0x00007f280d6a27b0
heap=0x00007f280d592028
beam/erl_gc.c, line 427: <0.194.0>: Overrun stack and heap
heart: Tue Oct 21 13:29:55 2014: Erlang has closed.
heart: Tue Oct 21 13:29:56 2014: Executed "/opt/couchdb/bin/couchdb -k" -> 0. Terminating.

As my replication source node does have a sizeable data set (16,000 docs, 180,000 updates, total size around 1GB), I found I could prevent the crash by reducing the replication worker batch size.  But I found I had to drop down 2 orders of magnitude (from the default of 500 down to 5) before the problem disappeared!

[replicator]
; With lower batch sizes checkpoints are done more frequently. Lower batch sizes
; also reduce the total amount of used RAM memory.
worker_batch_size = 5

I'm nervous having to tweak the replicator setup so far from the defaults, so is there any way I can tune the stack/heap?  I have included my ulimit output in case you could recommend any changes (note that my stack size and max memory are unlimited).

Thanks in advance,
Mat


Technical Details

CouchDB version:            1.6.0
Erlang:                                  OTP 17.0
OS:                                         RHEL 6.5
ERL_MAX_PORTS:           4096
ERL_FLAGS:                        "+A 4"


ulimit output:
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 30502
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 4096
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited