You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@couchdb.apache.org by va...@apache.org on 2024/10/01 16:01:55 UTC

(couchdb) 02/02: Use off-heap messages for the couch_event_server

This is an automated email from the ASF dual-hosted git repository.

vatamane pushed a commit to branch improve-couch-event-server
in repository https://gitbox.apache.org/repos/asf/couchdb.git

commit 6c9e9c815bde6d292cfe78558c1708017d83394d
Author: Nick Vatamaniuc <va...@gmail.com>
AuthorDate: Tue Oct 1 11:58:39 2024 -0400

    Use off-heap messages for the couch_event_server
    
    We noticed couch_event_server getting overloaded with a lot of GC work on the
    cluster. Off-heap messages should help speed up GC for the process since the
    message queue doesn't partificate in GC.
    
    While at it, document the rest of the existing off-heap modules in
    `default.ini`. Do not put these in the docs for now. Just keep them in the
    `default.ini`, they are a bit too low level and require deeper Erlang VM
    knowledge to tweak.
---
 rel/overlay/etc/default.ini                | 16 ++++++++++++++++
 src/couch_event/src/couch_event_server.erl |  1 +
 2 files changed, 17 insertions(+)

diff --git a/rel/overlay/etc/default.ini b/rel/overlay/etc/default.ini
index 48f16da1a..fef064473 100644
--- a/rel/overlay/etc/default.ini
+++ b/rel/overlay/etc/default.ini
@@ -1083,3 +1083,19 @@ url = {{nouveau_url}}
 ; from the _first_ authentication failure.
 ; note: changing this setting requires a couchdb restart.
 ;max_lifetime = 300000
+
+[off_heap_mqd]
+; Set Erlang process_flag(message_queue_data, off_heap) for the respective
+; module if the setting is true. Off-heap messages do not participate in the
+; process garbage collection. If processes expect to get a large message queue,
+; it usually make sense to set to true, as otherwise GC may take too long.
+; However in some cases that could affect latency as it takes an extra copy
+; step to copy them into the process heap.
+;rexi_server = true
+;couch_log_server = true
+;couch_server = true
+;couch_event_server = true
+;ddoc_cache_lru = true
+;mem3_shards = true
+;nouveau_index_manager = true
+;dreyfus_index_manager = true
diff --git a/src/couch_event/src/couch_event_server.erl b/src/couch_event/src/couch_event_server.erl
index a3d1d8464..dfdf0ac02 100644
--- a/src/couch_event/src/couch_event_server.erl
+++ b/src/couch_event/src/couch_event_server.erl
@@ -33,6 +33,7 @@ start_link() ->
     gen_server:start_link({local, ?MODULE}, ?MODULE, nil, []).
 
 init(_) ->
+    couch_util:set_mqd_off_heap(?MODULE),
     {ok, #st{
         by_pid = #{},
         by_dbname = #{}