You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by "nickva (via GitHub)" <gi...@apache.org> on 2025/08/18 23:13:28 UTC

[PR] BTree engine term cache [couchdb]

nickva opened a new pull request, #5625:
URL: https://github.com/apache/couchdb/pull/5625

   Cache the top b-tree nodes and header terms. Items are inserted into the cache on first use and then they have to be updated when accessed. If they are not updated frequently enough they are evicted.
   
   In order to get or update any kv leaf nodes in the b-trees, we always have to read the top few kp nodes closer to the root. Even with parallel preads and data being in the page cache the cost of system calls, marshaling the term, going through the erlang IO system can add up.
   
   The cache is limited in size. The size defaults to 64Mb and is configurable by the user. If the cache is full, no more entries will be added until more space frees up.
   
   Since multiple processes are accessing the data concurrently, ets tables are sharded by the number of schedulers, not unlike how we shard our couch_server processes. Each ets table has an associated cleaner process which evicts unused entries. Cleaners traverse table entries and "decay" the usage counters exponentially, by repeatedly shifting the value to the right with the `bsr` op. Entries with a usage counter equal 0 are then removed.
   
   In order to only insert the top nodes in the cache the couch_btree module's lookup and streaming functions were updated with a `depth` parameter. If that way we avoid thrashing node which are less likely to be reused through the cache: if the depth > 2 or the node is a kv_node it skips the cache altogether.
   
   To get an idea on tuning the cache size for various workload there are two new metrics to count hits and misses:
   
   ```
   % http $DB/_node/_local/_stats/couchdb/bt_engine_cache
   
   {
       "hits": {
           "desc": "number of bt_engine cache hits",
           "type": "counter",
           "value": 233310
       },
       "misses": {
           "desc": "number of bt_engine cache misses",
           "type": "counter",
           "value": 7386
       }
   }
   ```
   
   Those are the metrics after  running `fabric_bench:go(#{q=>8, doc_size=>small, docs=>100000})`.
   
   A quick and dirty comparison with main:
   
   ```
   > fabric_bench:go(#{q=>8, doc_size=>small, docs=>100000}).
    *** Parameters
    * batch_size       : 1000
    * doc_size         : small
    * docs             : 100000
    * individual_docs  : 1000
    * n                : default
    * q                : 8
   
    *** Environment
    * Nodes        : 1
    * Bench ver.   : 1
    * N            : 1
    * Q            : 8
    * OS           : unix/darwin
    * Couch ver.   : 3.5.0-d16fc1f
    * Couch git sha: d16fc1f
    * VM details   : Erlang/OTP 26 [erts-14.2.5.11] [source] [64-bit] [smp:12:12] [ds:12:12:16] [async-threads:1]
   
    *** Inserting 100000 docs
    * Add 100000 docs, ok:100/accepted:0     (Hz):   22000
    * Get random doc 100000X                 (Hz):    3800
    * All docs                               (Hz):  130000
    * All docs w/ include_docs               (Hz):   48000
    * Changes                                (Hz):   16000
    * Single doc updates 1000X               (Hz):     530
    * Time to run all benchmarks            (sec):      40
   ```
   
   With the btree cache:
   
   ```
   fabric_bench:go(#{q=>8, doc_size=>small, docs=>100000}).
    *** Parameters
    * batch_size       : 1000
    * doc_size         : small
    * docs             : 100000
    * individual_docs  : 1000
    * n                : default
    * q                : 8
   
    *** Environment
    * Nodes        : 1
    * Bench ver.   : 1
    * N            : 1
    * Q            : 8
    * OS           : unix/darwin
    * Couch ver.   : 3.5.0-d16fc1f
    * Couch git sha: d16fc1f
    * VM details   : Erlang/OTP 26 [erts-14.2.5.11] [source] [64-bit] [smp:12:12] [ds:12:12:16] [async-threads:1]
   
    *** Inserting 100000 docs
    * Add 100000 docs, ok:100/accepted:0     (Hz):   24000
    * Get random doc 100000X                 (Hz):    4400
    * All docs                               (Hz):  140000
    * All docs w/ include_docs               (Hz):   49000
    * Changes                                (Hz):   29000
    * Single doc updates 1000X               (Hz):     680
    * Time to run all benchmarks            (sec):      33
   ```
   
   The idea to use a depth parameter and an ets table came from Paul J. Davis (@davisp).
   
   <!-- Thank you for your contribution!
   
        Please file this form by replacing the Markdown comments
        with your text. If a section needs no action - remove it.
   
        Also remember, that CouchDB uses the Review-Then-Commit (RTC) model
        of code collaboration. Positive feedback is represented +1 from committers
        and negative is a -1. The -1 also means veto, and needs to be addressed
        to proceed. Once there are no objections, the PR can be merged by a
        CouchDB committer.
   
        See: http://couchdb.apache.org/bylaws.html#decisions for more info. -->
   
   ## Overview
   
   <!-- Please give a short brief for the pull request,
        what problem it solves or how it makes things better. -->
   
   ## Testing recommendations
   
   <!-- Describe how we can test your changes.
        Does it provide any behaviour that the end users
        could notice? -->
   
   ## Related Issues or Pull Requests
   
   <!-- If your changes affect multiple components in different
        repositories please put links to those issues or pull requests here.  -->
   
   ## Checklist
   
   - [ ] Code is written and works correctly
   - [ ] Changes are covered by tests
   - [ ] Any new configurable parameters are documented in `rel/overlay/etc/default.ini`
   - [ ] Documentation changes were made in the `src/docs` folder
   - [ ] Documentation changes were backported (separated PR) to affected branches
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298380859


##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,118 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+    {
+        foreach,
+        fun setup/0,
+        fun teardown/1,
+        [
+            ?TDEF_FE(t_created),
+            ?TDEF_FE(t_insert_and_lookup),
+            ?TDEF_FE(t_decay_and_removal_works, 10),
+            ?TDEF_FE(t_pid_cleanup_works, 10)
+        ]
+    }.
+
+setup() ->
+    Ctx = test_util:start_applications([config]),
+    couch_bt_engine_cache:create_tables(),
+    N = tuple_size(persistent_term:get(?PTERM_KEY)),
+    Pids = lists:map(
+        fun(I) ->
+            {ok, Pid} = couch_bt_engine_cache:start_link(I),
+            Pid
+        end,
+        lists:seq(1, N)
+    ),
+    {Ctx, Pids}.
+
+teardown({Ctx, [_ | _] = Pids}) ->
+    lists:foreach(
+        fun(Pid) ->
+            catch unlink(Pid),
+            exit(Pid, kill)
+        end,
+        Pids
+    ),
+    clear_tables(),
+    config:delete("bt_engine_cache", "max_size", false),
+    config:delete("bt_engine_cache", "leave_percent", false),
+    test_util:stop_applications(Ctx).
+
+clear_tables() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Fun = fun({cache, Tid, _}) -> ets:delete(Tid) end,
+            lists:foreach(Fun, tuple_to_list(Caches));
+        undefined ->
+            ok
+    end.
+
+t_created(_) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    N = erlang:system_info(schedulers),
+    ?assert(tuple_size(Caches) >= 1),
+    ?assertEqual(N, tuple_size(Caches)),
+    Stats = couch_bt_engine_cache:status(),
+    ?assertMatch(#{size := 0, memory := Mem} when is_integer(Mem), Stats).
+
+t_insert_and_lookup(_) ->
+    % Can only insert pids and positions
+    ?assertError(function_clause, couch_bt_engine_cache:insert(not_a_pid, 1, foo)),
+    ?assertError(function_clause, couch_bt_engine_cache:insert(self(), xyz, foo)),
+    ?assertMatch(#{size := 0}, couch_bt_engine_cache:status()),
+
+    ?assert(couch_bt_engine_cache:insert({pid, 42}, term)),
+    ?assertMatch(#{size := 1}, couch_bt_engine_cache:status()),
+
+    % Already inserted
+    ?assertNot(couch_bt_engine_cache:insert({pid, 42}, term)),
+
+    % Should be there when we look it up
+    ?assertEqual(term, couch_bt_engine_cache:lookup({pid, 42})),
+
+    % Other stuff we didn't insert should return undefined
+    ?assertEqual(undefined, couch_bt_engine_cache:lookup({pid, 43})).
+
+t_decay_and_removal_works(_) ->
+    config:set("bt_engine_cache", "leave_percent", "0", false),
+    Term = [foo, bar, baz, lists:seq(1, 100)],
+    [couch_bt_engine_cache:insert({pid, I}, Term) || I <- lists:seq(1, 10000)],
+    WaitFun = fun() ->
+        #{size := Size} = couch_bt_engine_cache:status(),
+        case Size > 0 of
+            true -> wait;
+            false -> ok
+        end
+    end,
+    test_util:wait(WaitFun, 7500),
+    ?assertMatch(#{size := 0}, couch_bt_engine_cache:status()).
+
+t_pid_cleanup_works(_) ->
+    Pid = spawn(fun() -> timer:sleep(2000) end),
+    [couch_bt_engine_cache:insert({Pid, I}, baz) || I <- lists:seq(1, 1000)],
+    WaitFun = fun() ->
+        #{size := Size} = couch_bt_engine_cache:status(),
+        case Size > 0 of
+            true -> wait;
+            false -> ok
+        end
+    end,
+    test_util:wait(WaitFun, 7500),

Review Comment:
   This one has a longer timeout set as well in `?TDEF_FE(t_pid_cleanup_works, 10)`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307697680


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+    insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+    Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+    case get_cache(Key) of
+        #cache{tid = Tid, max_size = Max} ->
+            case ets:info(Tid, memory) < Max of
+                true ->
+                    case ets:insert_new(Tid, {Key, Priority1, Term}) of
+                        true ->
+                            true;
+                        false ->
+                            bump_usage(Tid, Key),
+                            false
+                    end;
+                false ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+                    false
+            end;
+        undefined ->
+            false
+    end.
+
+reset(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} -> reset_usage(Tid, Key);
+        undefined -> true
+    end.
+
+lookup(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} ->
+            case ets:lookup_element(Tid, Key, 3, undefined) of
+                undefined ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+                    undefined;
+                Term ->
+                    bump_usage(Tid, Key),
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+                    Term
+            end;
+        undefined ->
+            undefined
+    end.
+
+status() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+            {Sizes, Memory} = lists:unzip(SizeMem),
+            #{
+                size => lists:sum(Sizes),
+                memory => lists:sum(Memory)
+            };
+        undefined ->
+            #{}
+    end.
+
+% Supervisor helper functions
+
+create_tables() ->
+    BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+    persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+    [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+    Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+    #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+    {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    Cache = #cache{tid = Tid} = element(N, Caches),
+    ets:delete_all_objects(Tid),
+    loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+    decay(Tid),
+    Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+    clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+    remove_dead(Cache),
+    WaitLeft = max(10, Next - now_msec()),
+    timer:sleep(WaitLeft),
+    loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+    case now_msec() < Until of
+        true ->
+            case should_clean(Cache) of
+                true ->
+                    ets:match_delete(Tid, {'_', 0, '_'});
+                false ->
+                    timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+                    clean(Cache, Until)
+            end;
+        false ->
+            false
+    end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+    ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+    All = pids(Tid),
+    Alive = sets:filter(fun is_process_alive/1, All),
+    Dead = sets:subtract(All, Alive),
+    % In OTP 27+ use sets:map/2
+    Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+    sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+    Acc = couch_util:new_set(),
+    try
+        ets:foldl(fun pids_fold/2, Acc, Tid)
+    catch
+        error:badarg -> Acc
+    end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+    sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+    Acc.
+
+% Private helper functions
+
+new() ->
+    Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+    Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+    % Some per-table overhead for the table metadata
+    Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+    #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Index = erlang:phash2(Term, tuple_size(Caches)),
+            #cache{} = element(1 + Index, Caches);
+        undefined ->
+            undefined
+    end.
+
+bump_usage(Tid, Key) ->
+    % We're updating the second field incrementing it by 1 and clamping it
+    % at ?MAX_PRIORITY. We don't set the default for the update_counter
+    % specifically to avoid creating bogus entries just from updating the
+    % counter, so expect the error:badarg here sometimes.
+    UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+reset_usage(Tid, Key) ->
+    % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+    % subtract that and clamp it at 0. Do not provide a default since if an
+    % entry is missing we don't want to create a bogus one from this operation.
+    UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+info(#cache{tid = Tid}) ->
+    Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+    Size = ets:info(Tid, size),
+    {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+    Head = {'$1', '$2', '$3'},
+    Cond = [{'>', '$2', 0}],
+    Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+    ets:select_replace(Tid, [{Head, Cond, Result}]).

Review Comment:
   That would work, not bad idea, but we haven't been typically been using the fun2ms parse transforms before (only in a few tests before). Also, would this interfere with hot code loading if we ever have to do it?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3199749004

   Nice work.
   
   I'd like to see a bigger benchmark before deciding but the performance gain is quite modest (except for changes) to warrant the extra complexity and failure modes (on the flipside it does validate our deliberate reliance on the OS page cache for this so far). I'd be curious to see the same benchmarks but with the code modified to not expire the cached items so aggressively (to find the upper bound to how much this could improve things). It would also be useful to know how deep the btrees are in question (or some other indication of the cache hit/miss ratio to give us context when assessing the throughput figures). As for why changes gets such a boost, that needs some thinking. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298432070


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning

Review Comment:
   Thank you. You're right that is confusing.
   
   I think something like this might be more clear:
   ```
   % How often cleaners check if the ets size increases. This is used in cases
   % when initially, at the start the cleaning interval, ets table size is below
   % "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
   % new set of requests come in soon after, but before the next clean-up interval
   % they'll fail since the cache would be full. To be able to react quicker and
   % make room, cleaners poll table sizes a bit more often.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3202798994

   I pretty-fied a quick btree stats reporter I had since we were wondering what the tree looked like from above (this is q=8, 100k docs) just one shard copy:
   
   ```json
      "sizes": {
           "active": 5812423,
           "external": 4040930,
           "file": 7188696,
           "id_tree": {
               "1": {
                   "kp_node": {
                       "cnt": 1,
                       "max": 5,
                       "min": 5
                   }
               },
               "2": {
                   "kp_node": {
                       "cnt": 5,
                       "max": 21,
                       "min": 15
                   }
               },
               "3": {
                   "kp_node": {
                       "cnt": 91,
                       "max": 23,
                       "min": 11
                   }
               },
               "4": {
                   "kv_node": {
                       "cnt": 1450,
                       "max": 15,
                       "min": 1
                   }
               }
           },
           "seq_tree": {
               "1": {
                   "kp_node": {
                       "cnt": 1,
                       "max": 2,
                       "min": 2
                   }
               },
               "2": {
                   "kp_node": {
                       "cnt": 2,
                       "max": 26,
                       "min": 2
                   }
               },
               "3": {
                   "kp_node": {
                       "cnt": 28,
                       "max": 47,
                       "min": 6
                   }
               },
               "4": {
                   "kv_node": {
                       "cnt": 975,
                       "max": 15,
                       "min": 2
                   }
               }
           }
       }
   ```
   
   The key is depth, then node type, then `cnt` is the number of nodes at that level, `min` is the smallest node size (number of kvs/kps), `max` is the largest size.
   
   It's not as shallow as we'd expect due to how complete_root works, and chunk_size is probably not the best any longer (doesn't count for compression). I was going to look into maybe having a different chunk size, or different per node type (kps get more), but that's for a different PR.
   
   So caching the top 2 nodes makes sense, there are not that many and they bring the biggest benefit. Top 3 could also be an option but maybe start smaller at first.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298114554


##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,118 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+    {
+        foreach,
+        fun setup/0,
+        fun teardown/1,
+        [
+            ?TDEF_FE(t_created),
+            ?TDEF_FE(t_insert_and_lookup),
+            ?TDEF_FE(t_decay_and_removal_works, 10),
+            ?TDEF_FE(t_pid_cleanup_works, 10)
+        ]
+    }.
+
+setup() ->
+    Ctx = test_util:start_applications([config]),
+    couch_bt_engine_cache:create_tables(),
+    N = tuple_size(persistent_term:get(?PTERM_KEY)),
+    Pids = lists:map(
+        fun(I) ->
+            {ok, Pid} = couch_bt_engine_cache:start_link(I),
+            Pid
+        end,
+        lists:seq(1, N)
+    ),
+    {Ctx, Pids}.
+
+teardown({Ctx, [_ | _] = Pids}) ->
+    lists:foreach(
+        fun(Pid) ->
+            catch unlink(Pid),
+            exit(Pid, kill)
+        end,
+        Pids
+    ),
+    clear_tables(),
+    config:delete("bt_engine_cache", "max_size", false),
+    config:delete("bt_engine_cache", "leave_percent", false),
+    test_util:stop_applications(Ctx).
+
+clear_tables() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Fun = fun({cache, Tid, _}) -> ets:delete(Tid) end,
+            lists:foreach(Fun, tuple_to_list(Caches));
+        undefined ->
+            ok
+    end.
+
+t_created(_) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    N = erlang:system_info(schedulers),
+    ?assert(tuple_size(Caches) >= 1),
+    ?assertEqual(N, tuple_size(Caches)),
+    Stats = couch_bt_engine_cache:status(),
+    ?assertMatch(#{size := 0, memory := Mem} when is_integer(Mem), Stats).
+
+t_insert_and_lookup(_) ->
+    % Can only insert pids and positions

Review Comment:
   The comment and following two assertions are confusing. The `couch_bt_engine_cache:insert/{2, 3}` expects the key to be in the form `{pid(), pos_integer()}`, while comment imply that it can be `pid()`. Please update comment.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298135326


##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,118 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+    {
+        foreach,
+        fun setup/0,
+        fun teardown/1,
+        [
+            ?TDEF_FE(t_created),
+            ?TDEF_FE(t_insert_and_lookup),
+            ?TDEF_FE(t_decay_and_removal_works, 10),
+            ?TDEF_FE(t_pid_cleanup_works, 10)
+        ]
+    }.
+
+setup() ->
+    Ctx = test_util:start_applications([config]),
+    couch_bt_engine_cache:create_tables(),
+    N = tuple_size(persistent_term:get(?PTERM_KEY)),
+    Pids = lists:map(
+        fun(I) ->
+            {ok, Pid} = couch_bt_engine_cache:start_link(I),
+            Pid
+        end,
+        lists:seq(1, N)
+    ),
+    {Ctx, Pids}.
+
+teardown({Ctx, [_ | _] = Pids}) ->
+    lists:foreach(
+        fun(Pid) ->
+            catch unlink(Pid),
+            exit(Pid, kill)
+        end,
+        Pids
+    ),
+    clear_tables(),
+    config:delete("bt_engine_cache", "max_size", false),
+    config:delete("bt_engine_cache", "leave_percent", false),
+    test_util:stop_applications(Ctx).
+
+clear_tables() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Fun = fun({cache, Tid, _}) -> ets:delete(Tid) end,
+            lists:foreach(Fun, tuple_to_list(Caches));
+        undefined ->
+            ok
+    end.
+
+t_created(_) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    N = erlang:system_info(schedulers),
+    ?assert(tuple_size(Caches) >= 1),
+    ?assertEqual(N, tuple_size(Caches)),
+    Stats = couch_bt_engine_cache:status(),
+    ?assertMatch(#{size := 0, memory := Mem} when is_integer(Mem), Stats).
+
+t_insert_and_lookup(_) ->
+    % Can only insert pids and positions
+    ?assertError(function_clause, couch_bt_engine_cache:insert(not_a_pid, 1, foo)),
+    ?assertError(function_clause, couch_bt_engine_cache:insert(self(), xyz, foo)),
+    ?assertMatch(#{size := 0}, couch_bt_engine_cache:status()),
+
+    ?assert(couch_bt_engine_cache:insert({pid, 42}, term)),
+    ?assertMatch(#{size := 1}, couch_bt_engine_cache:status()),
+
+    % Already inserted
+    ?assertNot(couch_bt_engine_cache:insert({pid, 42}, term)),
+
+    % Should be there when we look it up
+    ?assertEqual(term, couch_bt_engine_cache:lookup({pid, 42})),
+
+    % Other stuff we didn't insert should return undefined
+    ?assertEqual(undefined, couch_bt_engine_cache:lookup({pid, 43})).
+
+t_decay_and_removal_works(_) ->
+    config:set("bt_engine_cache", "leave_percent", "0", false),
+    Term = [foo, bar, baz, lists:seq(1, 100)],
+    [couch_bt_engine_cache:insert({pid, I}, Term) || I <- lists:seq(1, 10000)],
+    WaitFun = fun() ->
+        #{size := Size} = couch_bt_engine_cache:status(),
+        case Size > 0 of
+            true -> wait;
+            false -> ok
+        end
+    end,
+    test_util:wait(WaitFun, 7500),
+    ?assertMatch(#{size := 0}, couch_bt_engine_cache:status()).
+
+t_pid_cleanup_works(_) ->
+    Pid = spawn(fun() -> timer:sleep(2000) end),
+    [couch_bt_engine_cache:insert({Pid, I}, baz) || I <- lists:seq(1, 1000)],
+    WaitFun = fun() ->
+        #{size := Size} = couch_bt_engine_cache:status(),
+        case Size > 0 of
+            true -> wait;
+            false -> ok
+        end
+    end,
+    test_util:wait(WaitFun, 7500),

Review Comment:
   Timeout value is greater than 5 sec



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298132326


##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,118 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+    {
+        foreach,
+        fun setup/0,
+        fun teardown/1,
+        [
+            ?TDEF_FE(t_created),
+            ?TDEF_FE(t_insert_and_lookup),
+            ?TDEF_FE(t_decay_and_removal_works, 10),
+            ?TDEF_FE(t_pid_cleanup_works, 10)
+        ]
+    }.
+
+setup() ->
+    Ctx = test_util:start_applications([config]),
+    couch_bt_engine_cache:create_tables(),
+    N = tuple_size(persistent_term:get(?PTERM_KEY)),
+    Pids = lists:map(
+        fun(I) ->
+            {ok, Pid} = couch_bt_engine_cache:start_link(I),
+            Pid
+        end,
+        lists:seq(1, N)
+    ),
+    {Ctx, Pids}.
+
+teardown({Ctx, [_ | _] = Pids}) ->
+    lists:foreach(
+        fun(Pid) ->
+            catch unlink(Pid),
+            exit(Pid, kill)
+        end,
+        Pids
+    ),
+    clear_tables(),
+    config:delete("bt_engine_cache", "max_size", false),
+    config:delete("bt_engine_cache", "leave_percent", false),
+    test_util:stop_applications(Ctx).
+
+clear_tables() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Fun = fun({cache, Tid, _}) -> ets:delete(Tid) end,
+            lists:foreach(Fun, tuple_to_list(Caches));
+        undefined ->
+            ok
+    end.
+
+t_created(_) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    N = erlang:system_info(schedulers),
+    ?assert(tuple_size(Caches) >= 1),
+    ?assertEqual(N, tuple_size(Caches)),
+    Stats = couch_bt_engine_cache:status(),
+    ?assertMatch(#{size := 0, memory := Mem} when is_integer(Mem), Stats).
+
+t_insert_and_lookup(_) ->
+    % Can only insert pids and positions
+    ?assertError(function_clause, couch_bt_engine_cache:insert(not_a_pid, 1, foo)),
+    ?assertError(function_clause, couch_bt_engine_cache:insert(self(), xyz, foo)),
+    ?assertMatch(#{size := 0}, couch_bt_engine_cache:status()),
+
+    ?assert(couch_bt_engine_cache:insert({pid, 42}, term)),
+    ?assertMatch(#{size := 1}, couch_bt_engine_cache:status()),
+
+    % Already inserted
+    ?assertNot(couch_bt_engine_cache:insert({pid, 42}, term)),
+
+    % Should be there when we look it up
+    ?assertEqual(term, couch_bt_engine_cache:lookup({pid, 42})),
+
+    % Other stuff we didn't insert should return undefined
+    ?assertEqual(undefined, couch_bt_engine_cache:lookup({pid, 43})).
+
+t_decay_and_removal_works(_) ->
+    config:set("bt_engine_cache", "leave_percent", "0", false),
+    Term = [foo, bar, baz, lists:seq(1, 100)],
+    [couch_bt_engine_cache:insert({pid, I}, Term) || I <- lists:seq(1, 10000)],
+    WaitFun = fun() ->
+        #{size := Size} = couch_bt_engine_cache:status(),
+        case Size > 0 of
+            true -> wait;
+            false -> ok
+        end
+    end,
+    test_util:wait(WaitFun, 7500),

Review Comment:
   The default timeout for a test in eunit is 5 seconds, the value you use for wait function is greater.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298189107


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning

Review Comment:
   The sentence is unclear.
   
   If no entries were cleaned, how often should we poll the ETS size to determine when to start cleaning.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3200929782

   > I find the 33% improvement in single doc reads is significant.
   
   I think it's pretty good too and shows how much overhead even getting data from page cache can be. 
   
   >  like you @rnewson I'm somewhat surprised by the `_changes` seeing an 80% increase in reads per sec (I also wonder why `_all_docs` is so much higher than `_changes` in the first place).
   
   _changes for some reason is pretty variable in this benchmark for some reason, we'll have to investigate a bit why that is


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307427835


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+    insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+    Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+    case get_cache(Key) of
+        #cache{tid = Tid, max_size = Max} ->
+            case ets:info(Tid, memory) < Max of
+                true ->
+                    case ets:insert_new(Tid, {Key, Priority1, Term}) of
+                        true ->
+                            true;
+                        false ->
+                            bump_usage(Tid, Key),
+                            false
+                    end;
+                false ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+                    false
+            end;
+        undefined ->
+            false
+    end.
+
+reset(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} -> reset_usage(Tid, Key);
+        undefined -> true
+    end.
+
+lookup(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} ->
+            case ets:lookup_element(Tid, Key, 3, undefined) of
+                undefined ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+                    undefined;
+                Term ->
+                    bump_usage(Tid, Key),
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+                    Term
+            end;
+        undefined ->
+            undefined
+    end.
+
+status() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+            {Sizes, Memory} = lists:unzip(SizeMem),
+            #{
+                size => lists:sum(Sizes),
+                memory => lists:sum(Memory)
+            };
+        undefined ->
+            #{}
+    end.
+
+% Supervisor helper functions
+
+create_tables() ->
+    BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+    persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+    [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+    Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+    #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+    {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    Cache = #cache{tid = Tid} = element(N, Caches),
+    ets:delete_all_objects(Tid),
+    loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+    decay(Tid),
+    Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+    clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+    remove_dead(Cache),
+    WaitLeft = max(10, Next - now_msec()),
+    timer:sleep(WaitLeft),
+    loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+    case now_msec() < Until of
+        true ->
+            case should_clean(Cache) of
+                true ->
+                    ets:match_delete(Tid, {'_', 0, '_'});
+                false ->
+                    timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+                    clean(Cache, Until)
+            end;
+        false ->
+            false
+    end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+    ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+    All = pids(Tid),
+    Alive = sets:filter(fun is_process_alive/1, All),
+    Dead = sets:subtract(All, Alive),
+    % In OTP 27+ use sets:map/2
+    Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+    sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+    Acc = couch_util:new_set(),
+    try
+        ets:foldl(fun pids_fold/2, Acc, Tid)
+    catch
+        error:badarg -> Acc
+    end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+    sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+    Acc.
+
+% Private helper functions
+
+new() ->
+    Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+    Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+    % Some per-table overhead for the table metadata
+    Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+    #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Index = erlang:phash2(Term, tuple_size(Caches)),
+            #cache{} = element(1 + Index, Caches);
+        undefined ->
+            undefined
+    end.
+
+bump_usage(Tid, Key) ->
+    % We're updating the second field incrementing it by 1 and clamping it
+    % at ?MAX_PRIORITY. We don't set the default for the update_counter
+    % specifically to avoid creating bogus entries just from updating the
+    % counter, so expect the error:badarg here sometimes.
+    UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+reset_usage(Tid, Key) ->
+    % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+    % subtract that and clamp it at 0. Do not provide a default since if an
+    % entry is missing we don't want to create a bogus one from this operation.
+    UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+info(#cache{tid = Tid}) ->
+    Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+    Size = ets:info(Tid, size),
+    {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+    Head = {'$1', '$2', '$3'},
+    Cond = [{'>', '$2', 0}],
+    Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+    ets:select_replace(Tid, [{Head, Cond, Result}]).

Review Comment:
   I think matchspec would read better here.
   
   ```erlang
   MS = ets:fun2ms(
       fun({Key, Priority, Term}) when Priority > 0 ->
          % bsr bit-shifts right so we divide by 2, then 4, 8, etc.
          {Key, Priority bsr 1, Term}
       end),
   ets:select_replace(Tid, MS).
   ```
   
   The matchspec above produces the same query at compile time (if you would do `
   -include_lib("stdlib/include/ms_transform.hrl").`).
   
   ```
   8> ets:fun2ms(fun({Key, Priority, Term}) when Priority > 0 -> {Key, Priority bsr 1, Term} end).
   [{{'$1','$2','$3'},
     [{'>','$2',0}],
     [{{'$1',{'bsr','$2',1},'$3'}}]}]
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298376754


##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,118 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+    {
+        foreach,
+        fun setup/0,
+        fun teardown/1,
+        [
+            ?TDEF_FE(t_created),
+            ?TDEF_FE(t_insert_and_lookup),
+            ?TDEF_FE(t_decay_and_removal_works, 10),
+            ?TDEF_FE(t_pid_cleanup_works, 10)
+        ]
+    }.
+
+setup() ->
+    Ctx = test_util:start_applications([config]),
+    couch_bt_engine_cache:create_tables(),
+    N = tuple_size(persistent_term:get(?PTERM_KEY)),
+    Pids = lists:map(
+        fun(I) ->
+            {ok, Pid} = couch_bt_engine_cache:start_link(I),
+            Pid
+        end,
+        lists:seq(1, N)
+    ),
+    {Ctx, Pids}.
+
+teardown({Ctx, [_ | _] = Pids}) ->
+    lists:foreach(
+        fun(Pid) ->
+            catch unlink(Pid),
+            exit(Pid, kill)
+        end,
+        Pids
+    ),
+    clear_tables(),
+    config:delete("bt_engine_cache", "max_size", false),
+    config:delete("bt_engine_cache", "leave_percent", false),
+    test_util:stop_applications(Ctx).
+
+clear_tables() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Fun = fun({cache, Tid, _}) -> ets:delete(Tid) end,
+            lists:foreach(Fun, tuple_to_list(Caches));
+        undefined ->
+            ok
+    end.
+
+t_created(_) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    N = erlang:system_info(schedulers),
+    ?assert(tuple_size(Caches) >= 1),
+    ?assertEqual(N, tuple_size(Caches)),
+    Stats = couch_bt_engine_cache:status(),
+    ?assertMatch(#{size := 0, memory := Mem} when is_integer(Mem), Stats).
+
+t_insert_and_lookup(_) ->
+    % Can only insert pids and positions

Review Comment:
   Good spot. Thank you, I switched between two implementation previously so this is a remnant of that.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3216348604

   One missing part of the testing was to see how the PR fares with multiple dbs when it hits the full cache limit, so I ran the benchmark again (with the latest version) with 1 database (Q=64, 1M docs) and then 100 dbs also with Q=64, 10k docs per db. Since the working set is larger and we hit the full cache a few times, it's less effective, but still shows an improvement both latency and throughput.
   
   #### MAIN
   
   ##### 1 DB
   
   ```
      HTTP
       http_req_duration..............: avg=1.1s     min=3.8ms    med=1.12s    max=3.97s    p(90)=2.32s    p(95)=2.52s
         { expected_response:true }...: avg=1.1s     min=3.8ms    med=1.12s    max=3.97s    p(90)=2.31s    p(95)=2.52s
         { name:bulk_docs }...........: avg=194.82ms min=122.29ms med=194.57ms max=506.31ms p(90)=210.71ms p(95)=216.96ms
         { name:get_doc }.............: avg=39.48ms  min=3.8ms    med=38.29ms  max=3.25s    p(90)=50.8ms   p(95)=54.52ms
         { name:post_doc }............: avg=1.37s    min=39.94ms  med=1.28s    max=3.88s    p(90)=2.15s    p(95)=2.37s
         { name:put_doc }.............: avg=1.91s    min=117.86ms med=1.99s    max=3.97s    p(90)=2.57s    p(95)=2.73s
       http_req_failed................: 0.13%   5492 out of 4172744
       http_reqs......................: 4172744 4918.641943/s
   
       EXECUTION
       iteration_duration.............: avg=4.32s    min=1.42s    med=4.35s    max=8.74s    p(90)=4.97s    p(95)=5.17s
       iterations.....................: 1390580 1639.152824/s
       vus............................: 3175    min=0               max=10000
       vus_max........................: 10000   min=10000           max=10000
   
       NETWORK
       data_received..................: 2.7 GB  3.1 MB/s
       data_sent......................: 738 MB  870 kB/s
   
   running (14m08.4s), 00000/10000 VUs, 1390580 complete and 0 interrupted iterations
   ```
   
   ##### 100 DBs
   
   ```
       HTTP
       http_req_duration..............: avg=739.75ms min=3.41ms   med=722.39ms max=4.29s    p(90)=1.53s   p(95)=1.74s
         { expected_response:true }...: avg=739.01ms min=5.91ms   med=722.02ms max=4.28s    p(90)=1.53s   p(95)=1.74s
         { name:bulk_docs }...........: avg=176.97ms min=104.51ms med=180.09ms max=222.71ms p(90)=198ms   p(95)=204.05ms
         { name:get_doc }.............: avg=72.49ms  min=6.42ms   med=72.87ms  max=2.5s     p(90)=90.76ms p(95)=97.98ms
         { name:post_doc }............: avg=1.01s    min=5.91ms   med=895.98ms max=4.28s    p(90)=1.6s    p(95)=1.8s
         { name:put_doc }.............: avg=1.13s    min=57.11ms  med=1.02s    max=4.29s    p(90)=1.71s   p(95)=1.91s
       http_req_failed................: 0.11%   6299 out of 5597273
       http_reqs......................: 5597273 6640.279304/s
   
       EXECUTION
       iteration_duration.............: avg=3.22s    min=1.7s     med=3.15s    max=7.57s    p(90)=4.04s   p(95)=4.33s
       iterations.....................: 1865324 2212.911958/s
       vus............................: 0       min=0               max=10000
       vus_max........................: 10000   min=10000           max=10000
   
       NETWORK
       data_received..................: 3.5 GB  4.2 MB/s
       data_sent......................: 980 MB  1.2 MB/s
   
   running (14m02.9s), 00000/10000 VUs, 1865324 complete and 0 interrupted iterations
   ```
   
   #### BTREE CACHE
   
   ##### 1 DB
   
   ```
       HTTP
       http_req_duration..............: avg=586.26ms min=3.41ms  med=683.49ms max=3.62s    p(90)=1.14s    p(95)=1.27s
         { expected_response:true }...: avg=585.63ms min=3.41ms  med=682.96ms max=3.04s    p(90)=1.14s    p(95)=1.27s
         { name:bulk_docs }...........: avg=118.95ms min=88.45ms med=115.99ms max=430.22ms p(90)=123.29ms p(95)=125.34ms
         { name:get_doc }.............: avg=12.71ms  min=3.41ms  med=10.05ms  max=2.11s    p(90)=19.59ms  p(95)=22.72ms
         { name:post_doc }............: avg=832.66ms min=7.93ms  med=808.87ms max=3.04s    p(90)=1.19s    p(95)=1.32s
         { name:put_doc }.............: avg=913.62ms min=45.74ms med=896.9ms  max=3.62s    p(90)=1.25s    p(95)=1.36s
       http_req_failed................: 0.09%   6333 out of 6530402
       http_reqs......................: 6530402 8459.167081/s
   
       EXECUTION
       iteration_duration.............: avg=2.76s    min=1.14s   med=2.73s    max=6.74s    p(90)=3.24s    p(95)=3.42s
       iterations.....................: 2176466 2819.288849/s
       vus............................: 262     min=0               max=10000
       vus_max........................: 10000   min=10000           max=10000
   
       NETWORK
       data_received..................: 4.1 GB  5.3 MB/s
       data_sent......................: 1.1 GB  1.4 MB/s
   
   running (12m52.0s), 00000/10000 VUs, 2176466 complete and 0 interrupted iterations
   ```
   ```
   {
     "full": {
       "value": 0,
       "type": "counter",
       "desc": "number of times bt_engine cache was full"
     },
     "hits": {
       "value": 37521910,
       "type": "counter",
       "desc": "number of bt_engine cache hits"
     },
     "misses": {
       "value": 1364431,
       "type": "counter",
       "desc": "number of bt_engine cache misses"
     }
   }
   ```
   
   ##### 100 DBs
   
   ```
       HTTP
       http_req_duration..............: avg=578.9ms  min=2.82ms  med=552.61ms max=3.93s    p(90)=1.25s    p(95)=1.44s
         { expected_response:true }...: avg=578.27ms min=3.34ms  med=552.29ms max=3.82s    p(90)=1.25s    p(95)=1.44s
         { name:bulk_docs }...........: avg=118.11ms min=88.52ms med=117.76ms max=144.29ms p(90)=126.05ms p(95)=129.4ms
         { name:get_doc }.............: avg=39.67ms  min=3.34ms  med=38.55ms  max=2.34s    p(90)=53.12ms  p(95)=58.21ms
         { name:post_doc }............: avg=798.7ms  min=6.24ms  med=668.4ms  max=3.65s    p(90)=1.31s    p(95)=1.49s
         { name:put_doc }.............: avg=898.61ms min=6.29ms  med=779.32ms max=3.93s    p(90)=1.41s    p(95)=1.59s
       http_req_failed................: 0.10%   6988 out of 6583385
       http_reqs......................: 6583385 8474.816169/s
   
       EXECUTION
       iteration_duration.............: avg=2.73s    min=1.47s   med=2.67s    max=7.45s    p(90)=3.45s    p(95)=3.71s
       iterations.....................: 2194028 2824.380462/s
       vus............................: 0       min=0               max=10000
       vus_max........................: 10000   min=10000           max=10000
   
       NETWORK
       data_received..................: 4.1 GB  5.3 MB/s
       data_sent......................: 1.1 GB  1.5 MB/s
   ```
   
   ```
   {
     "full": {
       "value": 726465,
       "type": "counter",
       "desc": "number of times bt_engine cache was full"
     },
     "hits": {
       "value": 27097580,
       "type": "counter",
       "desc": "number of bt_engine cache hits"
     },
     "misses": {
       "value": 14718983,
       "type": "counter",
       "desc": "number of bt_engine cache misses"
     }
   }
   ```
   
   <img width="930" height="816" alt="multi_db_btree" src="https://github.com/user-attachments/assets/ff55f72c-edf9-4d39-abd0-d81292cd8121" />
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298234945


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning
+% them. This is to quickly react and make room for potentially new entries
+% being added without having to for a whole interval
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+    insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+    Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+    case get_cache(Key) of
+        #cache{tid = Tid, max_size = Max} ->
+            case ets:info(Tid, memory) < Max of
+                true ->
+                    case ets:insert_new(Tid, {Key, Priority1, Term}) of
+                        true ->
+                            true;
+                        false ->
+                            bump_usage(Tid, Key),
+                            false
+                    end;
+                false ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+                    false
+            end;
+        undefined ->
+            false
+    end.
+
+reset(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} -> reset_usage(Tid, Key);
+        undefined -> true
+    end.
+
+lookup(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} ->
+            case ets:lookup_element(Tid, Key, 3, undefined) of
+                undefined ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+                    undefined;
+                Term ->
+                    bump_usage(Tid, Key),
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+                    Term
+            end;
+        undefined ->
+            undefined
+    end.
+
+status() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+            {Sizes, Memory} = lists:unzip(SizeMem),
+            #{
+                size => lists:sum(Sizes),
+                memory => lists:sum(Memory)
+            };
+        undefined ->
+            #{}
+    end.
+
+% Supervisor helper functions
+
+create_tables() ->
+    BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+    persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+    [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+    Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+    #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+    {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    Cache = #cache{tid = Tid} = element(N, Caches),
+    ets:delete_all_objects(Tid),
+    loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+    decay(Tid),
+    Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+    clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+    remove_dead(Cache),
+    WaitLeft = max(10, Next - now_msec()),
+    timer:sleep(WaitLeft),
+    loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+    case now_msec() < Until of
+        true ->
+            case should_clean(Cache) of
+                true ->
+                    ets:match_delete(Tid, {'_', 0, '_'});
+                false ->
+                    timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+                    clean(Cache, Until)
+            end;
+        false ->
+            false
+    end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+    ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+    All = pids(Tid),
+    Alive = sets:filter(fun is_process_alive/1, All),
+    Dead = sets:subtract(All, Alive),
+    % In OTP 27+ use sets:map/2
+    Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+    sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+    Acc = couch_util:new_set(),
+    try
+        ets:foldl(fun pids_fold/2, Acc, Tid)
+    catch
+        error:badarg -> Acc
+    end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+    sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+    Acc.
+
+% Private helper functions
+
+new() ->
+    Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+    Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+    % Some per-table overhead for the table metadata
+    Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+    #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Index = erlang:phash2(Term, tuple_size(Caches)),
+            #cache{} = element(1 + Index, Caches);
+        undefined ->
+            undefined
+    end.
+
+bump_usage(Tid, Key) ->
+    % We're updating the second field incrementing it by 1 and clamping it
+    % at ?MAX_PRIORITY. We don't set the default for the update_counter
+    % specifically to avoid creating bogus entries just from updating the
+    % counter, so expect the error:badarg here sometimes.
+    UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+reset_usage(Tid, Key) ->
+    % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,

Review Comment:
   Nice approach 💯 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "mikerhodes (via GitHub)" <gi...@apache.org>.
mikerhodes commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3200592575

   > ...the performance gain is quite modest (except for changes) to warrant the extra complexity and failure modes (on the flipside it does validate our deliberate reliance on the OS page cache for this so far).
   
   I find the 33% improvement in single doc reads is significant. It's definitely in line with, if not slightly greater than, what I'd expect for this kind of change. It's also where I'd expect to see the greatest benefit of this change --- like you @rnewson I'm somewhat surprised by the `_changes` seeing an 80% increase in reads per sec (I also wonder why `_all_docs` is so much higher).
   
   One other interesting thing in general with these numbers is how many more docs we read per second with the `_all_docs` call --- 49k/s vs 4k/s. Presumably the random reads are going through an `r=2` read, whereas `_all_docs` include_docs reads are going to the local shard that was chosen for `_all_docs` using an implicit `r=1`? Feels like there's something important in that disparity.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3202493282

   I wonder also if supplying the fadvise hints as we read data might give a boost without adding an in-vm cache. (e.g, `WILLNEED` for non-leaf btree nodes and `DONTNEED` for all leaf data (doc bodies, attachment chunks).
   
   for _all_docs and _changes traversals (which under the covers are walking the by_id and by_seq trees in their respective orders) I wonder why caching does help. We "should" read the btree optimally there, we're not skipping around the btree. there's no seek penalty these days, etc. It might be a code wart we could solve. Or, perhaps, we could cache in-process more safely/easily. repeated calls to pread_term with the same offset by the same process that has an open #db record will yield the same result each time (until a reopen).
   
   I'd focus on benchmarking and more statistics, so we know what we're comparing. We'd want to know how many times we read the same term from disk in the current codebase versus how often we avoided it with the cache.
   
   Finally, ets_lru exists and has both size and time expiration. why invent a new way?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298179724


##########
rel/overlay/etc/default.ini:
##########
@@ -130,6 +130,26 @@ view_index_dir = {{view_index_dir}}
 ; downgrade or even open the databases in question.
 ;prohibit_downgrade = true
 
+[bt_engine_cache]
+; Memory used for btree engine cache. This is a cache for top levels of
+; database btrees (id tree, seq tree) and a few terms from the db header. Value
+; is in bytes.
+;max_size = 67108864
+;
+; Items not accessed in a while get are eventually evicted. However, if the
+; memory used is below this percentage, then even the unused items are left in
+; the cache. The trade-off here is when a new workload starts, it may find the
+; cache with some stale items during the first few seconds and not be able to
+; insert its entries in.
+;leave_percent = 30
+;
+; Cache database btree nodes up to this depth only. Depth starts at 1 at root,
+; then 2 the next level down, and so on. Only intermediate (pointer) nodes will

Review Comment:
   maybe add an `at`
   
   Depth starts at 1 at root, then 2 *at* the next level down, and so on.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307672524


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+    insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+    Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+    case get_cache(Key) of
+        #cache{tid = Tid, max_size = Max} ->
+            case ets:info(Tid, memory) < Max of
+                true ->
+                    case ets:insert_new(Tid, {Key, Priority1, Term}) of
+                        true ->
+                            true;
+                        false ->
+                            bump_usage(Tid, Key),
+                            false
+                    end;
+                false ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+                    false
+            end;
+        undefined ->
+            false
+    end.
+
+reset(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} -> reset_usage(Tid, Key);
+        undefined -> true
+    end.
+
+lookup(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} ->
+            case ets:lookup_element(Tid, Key, 3, undefined) of
+                undefined ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+                    undefined;
+                Term ->
+                    bump_usage(Tid, Key),
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+                    Term
+            end;
+        undefined ->
+            undefined
+    end.
+
+status() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+            {Sizes, Memory} = lists:unzip(SizeMem),
+            #{
+                size => lists:sum(Sizes),
+                memory => lists:sum(Memory)
+            };
+        undefined ->
+            #{}
+    end.
+
+% Supervisor helper functions
+
+create_tables() ->
+    BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+    persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+    [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+    Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+    #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+    {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    Cache = #cache{tid = Tid} = element(N, Caches),
+    ets:delete_all_objects(Tid),
+    loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+    decay(Tid),
+    Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+    clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+    remove_dead(Cache),
+    WaitLeft = max(10, Next - now_msec()),
+    timer:sleep(WaitLeft),
+    loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+    case now_msec() < Until of
+        true ->
+            case should_clean(Cache) of
+                true ->
+                    ets:match_delete(Tid, {'_', 0, '_'});
+                false ->
+                    timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+                    clean(Cache, Until)
+            end;
+        false ->
+            false
+    end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+    ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+    All = pids(Tid),
+    Alive = sets:filter(fun is_process_alive/1, All),
+    Dead = sets:subtract(All, Alive),
+    % In OTP 27+ use sets:map/2
+    Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+    sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+    Acc = couch_util:new_set(),
+    try
+        ets:foldl(fun pids_fold/2, Acc, Tid)
+    catch
+        error:badarg -> Acc
+    end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+    sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+    Acc.
+
+% Private helper functions
+
+new() ->
+    Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+    Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+    % Some per-table overhead for the table metadata
+    Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+    #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Index = erlang:phash2(Term, tuple_size(Caches)),
+            #cache{} = element(1 + Index, Caches);
+        undefined ->
+            undefined
+    end.
+
+bump_usage(Tid, Key) ->
+    % We're updating the second field incrementing it by 1 and clamping it
+    % at ?MAX_PRIORITY. We don't set the default for the update_counter
+    % specifically to avoid creating bogus entries just from updating the
+    % counter, so expect the error:badarg here sometimes.
+    UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+reset_usage(Tid, Key) ->
+    % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+    % subtract that and clamp it at 0. Do not provide a default since if an
+    % entry is missing we don't want to create a bogus one from this operation.
+    UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+info(#cache{tid = Tid}) ->
+    Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+    Size = ets:info(Tid, size),
+    {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+    Head = {'$1', '$2', '$3'},
+    Cond = [{'>', '$2', 0}],
+    Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+    ets:select_replace(Tid, [{Head, Cond, Result}]).
+
+shard_count() ->
+    erlang:system_info(schedulers).
+
+wait_interval(Interval) ->
+    Jitter = rand:uniform(max(1, round(Interval * 0.5))),

Review Comment:
   Good idea



##########
rel/overlay/etc/default.ini:
##########
@@ -130,6 +130,26 @@ view_index_dir = {{view_index_dir}}
 ; downgrade or even open the databases in question.
 ;prohibit_downgrade = true
 
+[bt_engine_cache]
+; Memory used for btree engine cache. This is a cache for top levels of
+; database btrees (id tree, seq tree) and a few terms from the db header. Value
+; is in bytes.
+;max_size = 67108864
+;
+; Items not accessed in a while are eventually evicted. However, if the memory
+; used is below this percentage, then even the unused items are left in the
+; cache. The trade-off here is when a new workload starts, it may find the
+; cache with some stale items during the first few seconds and not be able to
+; insert its entries in.
+;leave_percent = 30
+;
+; Cache database btree nodes up to this depth only. Depth starts at 1 at root,
+; then at 2 the next level down, and so on. Only intermediate (pointer) nodes
+; will be cached, those are the nodes which point to other nodes, as opposed to
+; leaf key-value nodes, which hold revision trees. To disabe db btree node

Review Comment:
   thank you!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva merged PR #5625:
URL: https://github.com/apache/couchdb/pull/5625


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3202941355

   > Finally, ets_lru exists and has both size and time expiration. why invent a new way?
   
   I looked at ets_lru but opted to go with something which has O(1) reads and inserts. And it's kind of a mix of a frequency based lru with a delayed time component, not a plain LRU. For an LRU the update would need to bump a sorted ets or list of some sort, while with a frequency counter we can rely on an ets:update_counter/3 call then let the scanners for each sharded ets do the work. I mostly thought of how we do the rate limiting tables for couch_replicator and the couch_server sharding.
   
   > I'd focus on benchmarking and more statistics, so we know what we're comparing. We'd want to know how many times we read the same term from disk in the current codebase versus how often we avoided it with the cache.
   
   Yeah, definitely will try to run more benchmarks. These are ones were run on my laptop, I'll see if I can get my hands on a real cluster. I think some of the stats there already tell a good story for instance for the benchmark above we get:
   
   ```
   > % http $DB/_node/_local/_stats/couchdb/bt_engine_cache
   
   {
       "full": {
           "desc": "number of times bt_engine cache was full",
           "type": "counter",
           "value": 0
       },
       "hits": {
           "desc": "number of bt_engine cache hits",
           "type": "counter",
           "value": 233296
       },
       "misses": {
           "desc": "number of bt_engine cache misses",
           "type": "counter",
           "value": 7343
       }
   }
   ```
   
   That is caching level 2 btree tops we already get 233296 kp_node hits, that's how many of them we can fetch from the cache not from the disk.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3212432680

   I made two more improvements inspired by some discussions with Mike and Bob:
   
     * Added a priority field to the insert call. So the depth 1 nodes get a higher initial priority. Priority  higher than 1 is basically the initial usage count, so the entry would survive a few more cleanup intervals.
     * When b-trees are updated there is an effect that a root node that was in use would now become abandoned suddenly. To help evict it faster, since we know how the b-tree get updated, we can reset its usage count to 0 (not remove it because some clients may still be reading from the old snapshot, otherwise the entry stays at least until the next cleanup interval but no more than that)
     * If the cache is not very full previously we just leave the entries in there case  they may be reused if there is no pressure on the cache. However with a longer interval of 1-2 seconds a new set of updates would be prevented from using the cache for a bit. So the change is that now if we didn't do any cleanup (we're half empty) then instead of sleeping for 1-2 seconds we checking the cache size every few hundred milliseconds, so we can react faster to new entries wanting to use the cache
     * To avoid the cache users all calling is_process_alive(Fd), which is still a process signal send/receive I made the cleaner processes do it periodically. They gather the processes in a set then delete the dead ones, that gets the send/recv signal path out of main data path.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298179724


##########
rel/overlay/etc/default.ini:
##########
@@ -130,6 +130,26 @@ view_index_dir = {{view_index_dir}}
 ; downgrade or even open the databases in question.
 ;prohibit_downgrade = true
 
+[bt_engine_cache]
+; Memory used for btree engine cache. This is a cache for top levels of
+; database btrees (id tree, seq tree) and a few terms from the db header. Value
+; is in bytes.
+;max_size = 67108864
+;
+; Items not accessed in a while get are eventually evicted. However, if the
+; memory used is below this percentage, then even the unused items are left in
+; the cache. The trade-off here is when a new workload starts, it may find the
+; cache with some stale items during the first few seconds and not be able to
+; insert its entries in.
+;leave_percent = 30
+;
+; Cache database btree nodes up to this depth only. Depth starts at 1 at root,
+; then 2 the next level down, and so on. Only intermediate (pointer) nodes will

Review Comment:
   maybe add an `at`
   
   Depth starts at 1 at root, then 2 `at` the next level down, and so on.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298168947


##########
rel/overlay/etc/default.ini:
##########
@@ -130,6 +130,26 @@ view_index_dir = {{view_index_dir}}
 ; downgrade or even open the databases in question.
 ;prohibit_downgrade = true
 
+[bt_engine_cache]
+; Memory used for btree engine cache. This is a cache for top levels of
+; database btrees (id tree, seq tree) and a few terms from the db header. Value
+; is in bytes.
+;max_size = 67108864
+;
+; Items not accessed in a while get are eventually evicted. However, if the

Review Comment:
   I think the verb `get` is not needed there.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2308008760


##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,117 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+    {
+        foreach,
+        fun setup/0,
+        fun teardown/1,
+        [
+            ?TDEF_FE(t_created),
+            ?TDEF_FE(t_insert_and_lookup),
+            ?TDEF_FE(t_decay_and_removal_works, 10),
+            ?TDEF_FE(t_pid_cleanup_works, 10)
+        ]
+    }.
+
+setup() ->
+    Ctx = test_util:start_applications([config]),
+    couch_bt_engine_cache:create_tables(),
+    N = tuple_size(persistent_term:get(?PTERM_KEY)),

Review Comment:
   Worked out pretty well. I also renamed it from `status/0` to `info/0` like you suggested, that's a more consistent name.
   
   ```
   > couch_bt_engine_cache:info().
   #{memory => 23103896,size => 11618,full => 0,hits => 4734895,
     misses => 784110,max_memory => 71204864,shard_count => 16}
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3234459217

   > Great work. I have couple of nitpicks, otherwise looks really good. I apologize about my mistake to use another my account initially (@iilyak-ibm is also me).
   
   No worries at all. Thank you for taking a look and for the nice suggestions. I updated the PR accordingly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3208291841

   I found a 12 node cluster (80 cpus, 128GB ram per node) and ran a high concurrency benchmark with 10k virtual users reading, updating and creating documents.
   
   To generate the load used k6 with this script: https://gist.github.com/nickva/d5312248635534315e76d25b31a86faa
     - starts with a 1M docs
     - then 10k users connect to the cluster and each do:
         * read a random document from the initial 1M
         * update the document
         * insert a new document (with a random uuid)
     - run for 10 minutes after initial setup and report stats
   
   Results for main:
   
   ```
       HTTP
       http_req_duration..............: avg=1.1s     min=5.01ms   med=1.09s    max=3.81s    p(90)=2.32s    p(95)=2.52s
         { expected_response:true }...: avg=1.1s     min=5.01ms   med=1.09s    max=3.81s    p(90)=2.32s    p(95)=2.52s
         { name:bulk_docs }...........: avg=188.58ms min=124.78ms med=188.34ms max=517.28ms p(90)=204.56ms p(95)=208.42ms
         { name:get_doc }.............: avg=40.97ms  min=5.01ms   med=39.47ms  max=3.49s    p(90)=52.12ms  p(95)=55.7ms
         { name:post_doc }............: avg=1.34s    min=29.05ms  med=1.25s    max=3.81s    p(90)=2.13s    p(95)=2.35s
         { name:put_doc }.............: avg=1.92s    min=75.03ms  med=2.01s    max=3.81s    p(90)=2.58s    p(95)=2.73s
       http_req_failed................: 0.13%   5617 out of 4183555
       http_reqs......................: 4183555 4969.270526/s
   
       EXECUTION
       iteration_duration.............: avg=4.31s    min=1.31s    med=4.34s    max=8.35s    p(90)=4.94s    p(95)=5.12s
       iterations.....................: 1394184 1656.026384/s
       vus............................: 390     min=0               max=10000
       vus_max........................: 10000   min=10000           max=10000
   
       NETWORK
       data_received..................: 2.6 GB  3.1 MB/s
       data_sent......................: 721 MB  856 kB/s
   
   running (14m01.9s), 00000/10000 VUs, 1394184 complete and 0 interrupted iterations
   ```
   
   Results with the bt cache
   
   ```
      HTTP
       http_req_duration..............: avg=743.25ms min=3.62ms   med=896.05ms max=3.4s     p(90)=1.41s    p(95)=1.56s
         { expected_response:true }...: avg=742.56ms min=3.62ms   med=895.55ms max=3.03s    p(90)=1.41s    p(95)=1.56s
         { name:bulk_docs }...........: avg=112.65ms min=93.67ms  med=109.18ms max=424.51ms p(90)=117.06ms p(95)=119.81ms
         { name:get_doc }.............: avg=13.6ms   min=3.62ms   med=11.68ms  max=2.35s    p(90)=15.95ms  p(95)=17.6ms
         { name:post_doc }............: avg=1.09s    min=16.85ms  med=1.06s    max=2.98s    p(90)=1.49s    p(95)=1.64s
         { name:put_doc }.............: avg=1.11s    min=117.82ms med=1.08s    max=3.4s     p(90)=1.51s    p(95)=1.65s
       http_req_failed................: 0.09%   5148 out of 5580982
       http_reqs......................: 5580982 7284.775804/s
   
       EXECUTION
       iteration_duration.............: avg=3.23s    min=1.63s    med=3.19s    max=6.92s    p(90)=3.8s     p(95)=3.99s
       iterations.....................: 1859993 2427.822201/s
       vus............................: 959     min=0               max=10000
       vus_max........................: 10000   min=10000           max=10000
   
       NETWORK
       data_received..................: 3.5 GB  4.5 MB/s
       data_sent......................: 927 MB  1.2 MB/s
   
   running (12m46.1s), 00000/10000 VUs, 1859993 complete and 0 interrupted iterations
   ```
   
   <img width="792" height="1279" alt="btree_bench" src="https://github.com/user-attachments/assets/89bebf02-114d-4b67-84b1-e1f65a142bc8" />
   
   Based on the metrics:
   
    * We can do 9.6k vs  http 7.3k requests / sec in favor of the btree cache.
   
    * In the read/write metrics the first grey bump is the initial bulk load. We can do 18k vs 12k inserts /sec in favor of the cache. During the main part of the benchmark 10k db reads vs 8k db reads also in favor of the btree-cache.
   
    * In the latency graph all latencies, but especially the higher ones p90, p99 are improved (1.4 sec vs 2.2 sec).
   
     * Memory usage is a bit better with the cache probably because we pile up more client processes.
   
   After the benchmark ran I queried the stats and got:
   
   ```
   curl http://$node/_stats/couchdb/bt_engine_cache
   {
     "full": {
       "value": 0,
       "type": "counter",
       "desc": "number of times bt_engine cache was full"
     },
     "hits": {
       "value": 24371805,
       "type": "counter",
       "desc": "number of bt_engine cache hits"
     },
     "misses": {
       "value": 374470,
       "type": "counter",
       "desc": "number of bt_engine cache misses"
     }
   }
   ```
   
   We saved 24M btree block reads and didn't hit the full limit/


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "big-r81 (via GitHub)" <gi...@apache.org>.
big-r81 commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3209048118

   Is there a break even point where things get worser? Can you run those tests on smaller clusters too?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3209109828

   > Nice work! Is there a break even point where things get worser? Can you run those tests on smaller clusters too?
   
   I'll try to find a smaller cluster but sometime setting it up my not work as well as our dedicated perf cluster. I could run some more benchmarks on my ubuntu laptop, that could be an example of a small machine.
   
   >  Is there a break even point where things get worser? 
   
   I'd imagine if we shuffled every node through the cache, things would get worse. As we'd fill it up with nodes used only a times. We see the distribution of counts in one of the above comments, at level 2 or 3 we may have only 40-50 kp nodes and one level lower there are thousands. The `full` metric can sort of help determine if the cache size may need to  be bumped or if the depth may need to be reduced to make it more effective.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3206587494

   fair comment on ets_lru. this seems a good thing to pursue given the hit rate.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2308265489


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+    insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+    Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+    case get_cache(Key) of
+        #cache{tid = Tid, max_size = Max} ->
+            case ets:info(Tid, memory) < Max of
+                true ->
+                    case ets:insert_new(Tid, {Key, Priority1, Term}) of
+                        true ->
+                            true;
+                        false ->
+                            bump_usage(Tid, Key),
+                            false
+                    end;
+                false ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+                    false
+            end;
+        undefined ->
+            false
+    end.
+
+reset(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} -> reset_usage(Tid, Key);
+        undefined -> true
+    end.
+
+lookup(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} ->
+            case ets:lookup_element(Tid, Key, 3, undefined) of
+                undefined ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+                    undefined;
+                Term ->
+                    bump_usage(Tid, Key),
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+                    Term
+            end;
+        undefined ->
+            undefined
+    end.
+
+status() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+            {Sizes, Memory} = lists:unzip(SizeMem),
+            #{
+                size => lists:sum(Sizes),
+                memory => lists:sum(Memory)
+            };
+        undefined ->
+            #{}
+    end.
+
+% Supervisor helper functions
+
+create_tables() ->
+    BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+    persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+    [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+    Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+    #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+    {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    Cache = #cache{tid = Tid} = element(N, Caches),
+    ets:delete_all_objects(Tid),
+    loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+    decay(Tid),
+    Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+    clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+    remove_dead(Cache),
+    WaitLeft = max(10, Next - now_msec()),
+    timer:sleep(WaitLeft),
+    loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+    case now_msec() < Until of
+        true ->
+            case should_clean(Cache) of
+                true ->
+                    ets:match_delete(Tid, {'_', 0, '_'});
+                false ->
+                    timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+                    clean(Cache, Until)
+            end;
+        false ->
+            false
+    end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+    ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+    All = pids(Tid),
+    Alive = sets:filter(fun is_process_alive/1, All),
+    Dead = sets:subtract(All, Alive),
+    % In OTP 27+ use sets:map/2
+    Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+    sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+    Acc = couch_util:new_set(),
+    try
+        ets:foldl(fun pids_fold/2, Acc, Tid)
+    catch
+        error:badarg -> Acc
+    end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+    sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+    Acc.
+
+% Private helper functions
+
+new() ->
+    Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+    Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+    % Some per-table overhead for the table metadata
+    Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+    #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Index = erlang:phash2(Term, tuple_size(Caches)),
+            #cache{} = element(1 + Index, Caches);
+        undefined ->
+            undefined
+    end.
+
+bump_usage(Tid, Key) ->
+    % We're updating the second field incrementing it by 1 and clamping it
+    % at ?MAX_PRIORITY. We don't set the default for the update_counter
+    % specifically to avoid creating bogus entries just from updating the
+    % counter, so expect the error:badarg here sometimes.
+    UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+reset_usage(Tid, Key) ->
+    % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+    % subtract that and clamp it at 0. Do not provide a default since if an
+    % entry is missing we don't want to create a bogus one from this operation.
+    UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+info(#cache{tid = Tid}) ->
+    Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+    Size = ets:info(Tid, size),
+    {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+    Head = {'$1', '$2', '$3'},
+    Cond = [{'>', '$2', 0}],
+    Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+    ets:select_replace(Tid, [{Head, Cond, Result}]).

Review Comment:
   > Hot code loading shouldn't be an issue.
   It will not be an issue since the beam file with fun2ms and without is identical.



##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+    insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+    Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+    case get_cache(Key) of
+        #cache{tid = Tid, max_size = Max} ->
+            case ets:info(Tid, memory) < Max of
+                true ->
+                    case ets:insert_new(Tid, {Key, Priority1, Term}) of
+                        true ->
+                            true;
+                        false ->
+                            bump_usage(Tid, Key),
+                            false
+                    end;
+                false ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+                    false
+            end;
+        undefined ->
+            false
+    end.
+
+reset(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} -> reset_usage(Tid, Key);
+        undefined -> true
+    end.
+
+lookup(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} ->
+            case ets:lookup_element(Tid, Key, 3, undefined) of
+                undefined ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+                    undefined;
+                Term ->
+                    bump_usage(Tid, Key),
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+                    Term
+            end;
+        undefined ->
+            undefined
+    end.
+
+status() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+            {Sizes, Memory} = lists:unzip(SizeMem),
+            #{
+                size => lists:sum(Sizes),
+                memory => lists:sum(Memory)
+            };
+        undefined ->
+            #{}
+    end.
+
+% Supervisor helper functions
+
+create_tables() ->
+    BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+    persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+    [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+    Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+    #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+    {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    Cache = #cache{tid = Tid} = element(N, Caches),
+    ets:delete_all_objects(Tid),
+    loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+    decay(Tid),
+    Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+    clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+    remove_dead(Cache),
+    WaitLeft = max(10, Next - now_msec()),
+    timer:sleep(WaitLeft),
+    loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+    case now_msec() < Until of
+        true ->
+            case should_clean(Cache) of
+                true ->
+                    ets:match_delete(Tid, {'_', 0, '_'});
+                false ->
+                    timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+                    clean(Cache, Until)
+            end;
+        false ->
+            false
+    end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+    ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+    All = pids(Tid),
+    Alive = sets:filter(fun is_process_alive/1, All),
+    Dead = sets:subtract(All, Alive),
+    % In OTP 27+ use sets:map/2
+    Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+    sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+    Acc = couch_util:new_set(),
+    try
+        ets:foldl(fun pids_fold/2, Acc, Tid)
+    catch
+        error:badarg -> Acc
+    end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+    sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+    Acc.
+
+% Private helper functions
+
+new() ->
+    Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+    Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+    % Some per-table overhead for the table metadata
+    Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+    #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Index = erlang:phash2(Term, tuple_size(Caches)),
+            #cache{} = element(1 + Index, Caches);
+        undefined ->
+            undefined
+    end.
+
+bump_usage(Tid, Key) ->
+    % We're updating the second field incrementing it by 1 and clamping it
+    % at ?MAX_PRIORITY. We don't set the default for the update_counter
+    % specifically to avoid creating bogus entries just from updating the
+    % counter, so expect the error:badarg here sometimes.
+    UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+reset_usage(Tid, Key) ->
+    % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+    % subtract that and clamp it at 0. Do not provide a default since if an
+    % entry is missing we don't want to create a bogus one from this operation.
+    UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+info(#cache{tid = Tid}) ->
+    Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+    Size = ets:info(Tid, size),
+    {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+    Head = {'$1', '$2', '$3'},
+    Cond = [{'>', '$2', 0}],
+    Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+    ets:select_replace(Tid, [{Head, Cond, Result}]).

Review Comment:
   > Hot code loading shouldn't be an issue.
   
   It will not be an issue since the beam file with fun2ms and without is identical.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307671142


##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,117 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+    {
+        foreach,
+        fun setup/0,
+        fun teardown/1,
+        [
+            ?TDEF_FE(t_created),
+            ?TDEF_FE(t_insert_and_lookup),
+            ?TDEF_FE(t_decay_and_removal_works, 10),
+            ?TDEF_FE(t_pid_cleanup_works, 10)
+        ]
+    }.
+
+setup() ->
+    Ctx = test_util:start_applications([config]),
+    couch_bt_engine_cache:create_tables(),
+    N = tuple_size(persistent_term:get(?PTERM_KEY)),

Review Comment:
   That could work, good idea!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3200915315

   > I'd like to see a bigger benchmark before deciding but the performance gain is quite modest (except for changes) to warrant the extra complexity and failure modes (on the flipside it does validate our deliberate reliance on the OS page cache for this so far). 
   
   Definitely would need a better benchmark, this was a quick and dirty one mainly to mainly show how many reads we saved with a mix of operations on 100k docs. Since I didn't do any page cache evicting all the block were in the page cache, so  it validates our use of the page but it also shows that there is quite an overhead in going through all the IO layers. And since we're aware of the b-tree/header structure at this layer we can make better decisions on what to cache.
   
   >  I'd be curious to see the same benchmarks but with the code modified to not expire the cached items so aggressively (to find the upper bound to how much this could improve things).
   
   Maybe we can make the "don't clean up if fill ratio is below 1/4" setting configurable, and with a large enough cache and a value like 3/4 we could look at the effects.
   
   >  It would also be useful to know how deep the btrees are in question (or some other indication of the cache hit/miss ratio to give us context when assessing the throughput figures). As for why changes gets such a boost, that needs some thinking.
   
   Good idea, maybe we'd want a diagnostic get depth traversal API to get an idea. 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3209915418

   these results are really good, so I'm ok with the added complexity.
   
   As for what depth to cache to; we are forced to choose a level because of the eviction mechanism (which was chosen for speed of execution I think). If, instead, we evicted more intelligently, we could just use a memory bound (which ops could move up/down if necessary). that is, try to evict non-leaf nodes we haven't hit recently. The Java caching library Caffeine operates this way and has some research on how optimal it is (https://github.com/ben-manes/caffeine/wiki/Efficiency : Window TinyLfu). An eviction scheme like this takes the guesswork out of things, and would basically tune itself.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298163717


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning
+% them. This is to quickly react and make room for potentially new entries
+% being added without having to for a whole interval
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+    insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+    Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+    case get_cache(Key) of
+        #cache{tid = Tid, max_size = Max} ->
+            case ets:info(Tid, memory) < Max of
+                true ->
+                    case ets:insert_new(Tid, {Key, Priority1, Term}) of
+                        true ->
+                            true;
+                        false ->
+                            bump_usage(Tid, Key),
+                            false
+                    end;
+                false ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+                    false
+            end;
+        undefined ->
+            false
+    end.
+
+reset(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} -> reset_usage(Tid, Key);
+        undefined -> true
+    end.
+
+lookup(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} ->
+            case ets:lookup_element(Tid, Key, 3, undefined) of
+                undefined ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+                    undefined;
+                Term ->
+                    bump_usage(Tid, Key),
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+                    Term
+            end;
+        undefined ->
+            undefined
+    end.
+
+status() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+            {Sizes, Memory} = lists:unzip(SizeMem),
+            #{
+                size => lists:sum(Sizes),
+                memory => lists:sum(Memory)
+            };
+        undefined ->
+            #{}
+    end.
+
+% Supervisor helper functions
+
+create_tables() ->
+    BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+    persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+    [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+    Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+    #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+    {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    Cache = #cache{tid = Tid} = element(N, Caches),
+    ets:delete_all_objects(Tid),
+    loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+    decay(Tid),
+    Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+    clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+    remove_dead(Cache),
+    WaitLeft = max(10, Next - now_msec()),
+    timer:sleep(WaitLeft),
+    loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+    case now_msec() < Until of
+        true ->
+            case should_clean(Cache) of
+                true ->
+                    ets:match_delete(Tid, {'_', 0, '_'});
+                false ->
+                    timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+                    clean(Cache, Until)
+            end;
+        false ->
+            false
+    end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+    ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+    All = pids(Tid),
+    Alive = sets:filter(fun is_process_alive/1, All),
+    Dead = sets:subtract(All, Alive),
+    % In OTP 27+ use sets:map/2
+    Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+    sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+    Acc = couch_util:new_set(),
+    try
+        ets:foldl(fun pids_fold/2, Acc, Tid)
+    catch
+        error:badarg -> Acc
+    end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+    sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+    Acc.
+
+% Private helper functions
+
+new() ->
+    Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+    Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+    % Some per-table overhead for the table metadata
+    Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+    #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Index = erlang:phash2(Term, tuple_size(Caches)),
+            #cache{} = element(1 + Index, Caches);
+        undefined ->
+            undefined
+    end.
+
+bump_usage(Tid, Key) ->
+    % We're updating the second field incrementing it by 1 and clamping it
+    % at ?MAX_PRIORITY. We don't set the default for the update_counter
+    % specifically to avoid creating bogus entries just from updating the
+    % counter, so expect the error:badarg here sometimes.
+    UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+reset_usage(Tid, Key) ->
+    % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+    % subtract that and clamp it at 0. Do not provide a default since if an
+    % entry is missing we don't want to creat a bogus from this operation.

Review Comment:
   A typo `creat` -> `create`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298383419


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning
+% them. This is to quickly react and make room for potentially new entries
+% being added without having to for a whole interval
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+    insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+    Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+    case get_cache(Key) of
+        #cache{tid = Tid, max_size = Max} ->
+            case ets:info(Tid, memory) < Max of
+                true ->
+                    case ets:insert_new(Tid, {Key, Priority1, Term}) of
+                        true ->
+                            true;
+                        false ->
+                            bump_usage(Tid, Key),
+                            false
+                    end;
+                false ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+                    false
+            end;
+        undefined ->
+            false
+    end.
+
+reset(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} -> reset_usage(Tid, Key);
+        undefined -> true
+    end.
+
+lookup(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} ->
+            case ets:lookup_element(Tid, Key, 3, undefined) of
+                undefined ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+                    undefined;
+                Term ->
+                    bump_usage(Tid, Key),
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+                    Term
+            end;
+        undefined ->
+            undefined
+    end.
+
+status() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+            {Sizes, Memory} = lists:unzip(SizeMem),
+            #{
+                size => lists:sum(Sizes),
+                memory => lists:sum(Memory)
+            };
+        undefined ->
+            #{}
+    end.
+
+% Supervisor helper functions
+
+create_tables() ->
+    BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+    persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+    [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+    Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+    #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+    {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    Cache = #cache{tid = Tid} = element(N, Caches),
+    ets:delete_all_objects(Tid),
+    loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+    decay(Tid),
+    Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+    clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+    remove_dead(Cache),
+    WaitLeft = max(10, Next - now_msec()),
+    timer:sleep(WaitLeft),
+    loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+    case now_msec() < Until of
+        true ->
+            case should_clean(Cache) of
+                true ->
+                    ets:match_delete(Tid, {'_', 0, '_'});
+                false ->
+                    timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+                    clean(Cache, Until)
+            end;
+        false ->
+            false
+    end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+    ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+    All = pids(Tid),
+    Alive = sets:filter(fun is_process_alive/1, All),
+    Dead = sets:subtract(All, Alive),
+    % In OTP 27+ use sets:map/2
+    Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+    sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+    Acc = couch_util:new_set(),
+    try
+        ets:foldl(fun pids_fold/2, Acc, Tid)
+    catch
+        error:badarg -> Acc
+    end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+    sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+    Acc.
+
+% Private helper functions
+
+new() ->
+    Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+    Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+    % Some per-table overhead for the table metadata
+    Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+    #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Index = erlang:phash2(Term, tuple_size(Caches)),
+            #cache{} = element(1 + Index, Caches);
+        undefined ->
+            undefined
+    end.
+
+bump_usage(Tid, Key) ->
+    % We're updating the second field incrementing it by 1 and clamping it
+    % at ?MAX_PRIORITY. We don't set the default for the update_counter
+    % specifically to avoid creating bogus entries just from updating the
+    % counter, so expect the error:badarg here sometimes.
+    UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+reset_usage(Tid, Key) ->
+    % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+    % subtract that and clamp it at 0. Do not provide a default since if an
+    % entry is missing we don't want to creat a bogus from this operation.

Review Comment:
   Thank you



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298379535


##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,118 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+    {
+        foreach,
+        fun setup/0,
+        fun teardown/1,
+        [
+            ?TDEF_FE(t_created),
+            ?TDEF_FE(t_insert_and_lookup),
+            ?TDEF_FE(t_decay_and_removal_works, 10),
+            ?TDEF_FE(t_pid_cleanup_works, 10)
+        ]
+    }.
+
+setup() ->
+    Ctx = test_util:start_applications([config]),
+    couch_bt_engine_cache:create_tables(),
+    N = tuple_size(persistent_term:get(?PTERM_KEY)),
+    Pids = lists:map(
+        fun(I) ->
+            {ok, Pid} = couch_bt_engine_cache:start_link(I),
+            Pid
+        end,
+        lists:seq(1, N)
+    ),
+    {Ctx, Pids}.
+
+teardown({Ctx, [_ | _] = Pids}) ->
+    lists:foreach(
+        fun(Pid) ->
+            catch unlink(Pid),
+            exit(Pid, kill)
+        end,
+        Pids
+    ),
+    clear_tables(),
+    config:delete("bt_engine_cache", "max_size", false),
+    config:delete("bt_engine_cache", "leave_percent", false),
+    test_util:stop_applications(Ctx).
+
+clear_tables() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Fun = fun({cache, Tid, _}) -> ets:delete(Tid) end,
+            lists:foreach(Fun, tuple_to_list(Caches));
+        undefined ->
+            ok
+    end.
+
+t_created(_) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    N = erlang:system_info(schedulers),
+    ?assert(tuple_size(Caches) >= 1),
+    ?assertEqual(N, tuple_size(Caches)),
+    Stats = couch_bt_engine_cache:status(),
+    ?assertMatch(#{size := 0, memory := Mem} when is_integer(Mem), Stats).
+
+t_insert_and_lookup(_) ->
+    % Can only insert pids and positions
+    ?assertError(function_clause, couch_bt_engine_cache:insert(not_a_pid, 1, foo)),
+    ?assertError(function_clause, couch_bt_engine_cache:insert(self(), xyz, foo)),
+    ?assertMatch(#{size := 0}, couch_bt_engine_cache:status()),
+
+    ?assert(couch_bt_engine_cache:insert({pid, 42}, term)),
+    ?assertMatch(#{size := 1}, couch_bt_engine_cache:status()),
+
+    % Already inserted
+    ?assertNot(couch_bt_engine_cache:insert({pid, 42}, term)),
+
+    % Should be there when we look it up
+    ?assertEqual(term, couch_bt_engine_cache:lookup({pid, 42})),
+
+    % Other stuff we didn't insert should return undefined
+    ?assertEqual(undefined, couch_bt_engine_cache:lookup({pid, 43})).
+
+t_decay_and_removal_works(_) ->
+    config:set("bt_engine_cache", "leave_percent", "0", false),
+    Term = [foo, bar, baz, lists:seq(1, 100)],
+    [couch_bt_engine_cache:insert({pid, I}, Term) || I <- lists:seq(1, 10000)],
+    WaitFun = fun() ->
+        #{size := Size} = couch_bt_engine_cache:status(),
+        case Size > 0 of
+            true -> wait;
+            false -> ok
+        end
+    end,
+    test_util:wait(WaitFun, 7500),

Review Comment:
   For this test we override the default timeout in `?TDEF_FE(t_decay_and_removal_works, 10),` the extra 10 is the 10 seconds timeout. It takes a bit longer to run but that seems better than mocking more things.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3216239859

   > Nice work! Is there a break even point where things get worser? Can you run those tests on smaller clusters too?
   
   @big-r81 I found a 3 node cluster with 64Gb of ram each. Since it's a smaller cluster I ran with Q=16.
   
   ```
   const db = 'k6db'
   const q = '16'
   const batch_size = 1000;
   const doc_size = 64;
   const dbs = 1
   const docs = 1000000;
   ```
   
   With the same k6 script as above
   
   #### MAIN ####
   ```
       HTTP
       http_req_duration..............: avg=1.52s    min=4.5ms   med=1.51s    max=7.29s    p(90)=3.02s    p(95)=3.18s
         { expected_response:true }...: avg=1.52s    min=4.5ms   med=1.51s    max=7.29s    p(90)=3.02s    p(95)=3.18s
         { name:bulk_docs }...........: avg=90.65ms  min=72.39ms med=89.05ms  max=543.13ms p(90)=100.43ms p(95)=103.92ms
         { name:get_doc }.............: avg=2.15s    min=4.5ms   med=2.65s    max=7.29s    p(90)=3.26s    p(95)=3.39s
         { name:post_doc }............: avg=989.49ms min=13.98ms med=826.58ms max=3.35s    p(90)=2.05s    p(95)=2.33s
         { name:put_doc }.............: avg=1.44s    min=10.49ms med=1.68s    max=3.46s    p(90)=2.51s    p(95)=2.67s
       http_req_failed................: 0.17%   5672 out of 3237326
       http_reqs......................: 3237326 4353.793364/s
   
       EXECUTION
       iteration_duration.............: avg=5.59s    min=1.07s   med=6.41s    max=9.72s    p(90)=7.85s    p(95)=8.06s
       iterations.....................: 1078774 1450.81437/s
       vus............................: 0       min=0               max=10000
       vus_max........................: 10000   min=10000           max=10000
   
       NETWORK
       data_received..................: 2.1 GB  2.8 MB/s
       data_sent......................: 596 MB  801 kB/s
   
   running (12m23.6s), 00000/10000 VUs, 1078774 complete and 0 interrupted iterations
   ```
   
   #### PR ####
   ```
       HTTP
       http_req_duration..............: avg=1.1s     min=3.53ms  med=1.07s    max=4.71s    p(90)=2s      p(95)=2.16s
         { expected_response:true }...: avg=1.1s     min=3.53ms  med=1.07s    max=4.71s    p(90)=1.99s   p(95)=2.16s
         { name:bulk_docs }...........: avg=74.84ms  min=63.94ms med=72.57ms  max=393.01ms p(90)=82.32ms p(95)=85.67ms
         { name:get_doc }.............: avg=624.54ms min=3.53ms  med=664.09ms max=4.03s    p(90)=1.24s   p(95)=1.35s
         { name:post_doc }............: avg=1.12s    min=13.98ms med=1.01s    max=4.71s    p(90)=1.9s    p(95)=2.09s
         { name:put_doc }.............: avg=1.55s    min=17.55ms med=1.65s    max=4.67s    p(90)=2.18s   p(95)=2.29s
       http_req_failed................: 0.14%   5896 out of 4179620
       http_reqs......................: 4179620 5757.994289/s
   
       EXECUTION
       iteration_duration.............: avg=4.31s    min=1.09s   med=4.67s    max=8.41s    p(90)=5.25s   p(95)=5.41s
       iterations.....................: 1392872 1918.870381/s
       vus............................: 1699    min=0               max=10000
       vus_max........................: 10000   min=10000           max=10000
   
       NETWORK
       data_received..................: 2.7 GB  3.7 MB/s
       data_sent......................: 738 MB  1.0 MB/s
   
   running (12m05.9s), 00000/10000 VUs, 1392872 complete and 0 interrupted iterations
   ```
   ```
   {
     "full": {
       "value": 0,
       "type": "counter",
       "desc": "number of times bt_engine cache was full"
     },
     "hits": {
       "value": 10302337,
       "type": "counter",
       "desc": "number of bt_engine cache hits"
     },
     "misses": {
       "value": 2212648,
       "type": "counter",
       "desc": "number of bt_engine cache misses"
     }
   }
   ```
   
   It held up pretty well. The metric graphs show improvements in throughput, latency and rate of requests:
   
   <img width="836" height="665" alt="btree_3node_cluster" src="https://github.com/user-attachments/assets/10cda272-2e28-4cf9-afc9-8cfd439a6b84" />
   
   (The first grey bump is the bulk_load phase)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2308010224


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+    insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+    Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+    case get_cache(Key) of
+        #cache{tid = Tid, max_size = Max} ->
+            case ets:info(Tid, memory) < Max of
+                true ->
+                    case ets:insert_new(Tid, {Key, Priority1, Term}) of
+                        true ->
+                            true;
+                        false ->
+                            bump_usage(Tid, Key),
+                            false
+                    end;
+                false ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+                    false
+            end;
+        undefined ->
+            false
+    end.
+
+reset(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} -> reset_usage(Tid, Key);
+        undefined -> true
+    end.
+
+lookup(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} ->
+            case ets:lookup_element(Tid, Key, 3, undefined) of
+                undefined ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+                    undefined;
+                Term ->
+                    bump_usage(Tid, Key),
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+                    Term
+            end;
+        undefined ->
+            undefined
+    end.
+
+status() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+            {Sizes, Memory} = lists:unzip(SizeMem),
+            #{
+                size => lists:sum(Sizes),
+                memory => lists:sum(Memory)
+            };
+        undefined ->
+            #{}
+    end.
+
+% Supervisor helper functions
+
+create_tables() ->
+    BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+    persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+    [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+    Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+    #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+    {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    Cache = #cache{tid = Tid} = element(N, Caches),
+    ets:delete_all_objects(Tid),
+    loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+    decay(Tid),
+    Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+    clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+    remove_dead(Cache),
+    WaitLeft = max(10, Next - now_msec()),
+    timer:sleep(WaitLeft),
+    loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+    case now_msec() < Until of
+        true ->
+            case should_clean(Cache) of
+                true ->
+                    ets:match_delete(Tid, {'_', 0, '_'});
+                false ->
+                    timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+                    clean(Cache, Until)
+            end;
+        false ->
+            false
+    end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+    ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+    All = pids(Tid),
+    Alive = sets:filter(fun is_process_alive/1, All),
+    Dead = sets:subtract(All, Alive),
+    % In OTP 27+ use sets:map/2
+    Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+    sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+    Acc = couch_util:new_set(),
+    try
+        ets:foldl(fun pids_fold/2, Acc, Tid)
+    catch
+        error:badarg -> Acc
+    end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+    sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+    Acc.
+
+% Private helper functions
+
+new() ->
+    Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+    Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+    % Some per-table overhead for the table metadata
+    Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+    #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Index = erlang:phash2(Term, tuple_size(Caches)),
+            #cache{} = element(1 + Index, Caches);
+        undefined ->
+            undefined
+    end.
+
+bump_usage(Tid, Key) ->
+    % We're updating the second field incrementing it by 1 and clamping it
+    % at ?MAX_PRIORITY. We don't set the default for the update_counter
+    % specifically to avoid creating bogus entries just from updating the
+    % counter, so expect the error:badarg here sometimes.
+    UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+reset_usage(Tid, Key) ->
+    % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+    % subtract that and clamp it at 0. Do not provide a default since if an
+    % entry is missing we don't want to create a bogus one from this operation.
+    UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+info(#cache{tid = Tid}) ->
+    Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+    Size = ets:info(Tid, size),
+    {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+    Head = {'$1', '$2', '$3'},
+    Cond = [{'>', '$2', 0}],
+    Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+    ets:select_replace(Tid, [{Head, Cond, Result}]).

Review Comment:
   Worked out well I think, it's much clearer to read and understand the code. Hot code loading shouldn't be an issue.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307697680


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+    insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+    Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+    case get_cache(Key) of
+        #cache{tid = Tid, max_size = Max} ->
+            case ets:info(Tid, memory) < Max of
+                true ->
+                    case ets:insert_new(Tid, {Key, Priority1, Term}) of
+                        true ->
+                            true;
+                        false ->
+                            bump_usage(Tid, Key),
+                            false
+                    end;
+                false ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+                    false
+            end;
+        undefined ->
+            false
+    end.
+
+reset(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} -> reset_usage(Tid, Key);
+        undefined -> true
+    end.
+
+lookup(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} ->
+            case ets:lookup_element(Tid, Key, 3, undefined) of
+                undefined ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+                    undefined;
+                Term ->
+                    bump_usage(Tid, Key),
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+                    Term
+            end;
+        undefined ->
+            undefined
+    end.
+
+status() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+            {Sizes, Memory} = lists:unzip(SizeMem),
+            #{
+                size => lists:sum(Sizes),
+                memory => lists:sum(Memory)
+            };
+        undefined ->
+            #{}
+    end.
+
+% Supervisor helper functions
+
+create_tables() ->
+    BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+    persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+    [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+    Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+    #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+    {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    Cache = #cache{tid = Tid} = element(N, Caches),
+    ets:delete_all_objects(Tid),
+    loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+    decay(Tid),
+    Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+    clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+    remove_dead(Cache),
+    WaitLeft = max(10, Next - now_msec()),
+    timer:sleep(WaitLeft),
+    loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+    case now_msec() < Until of
+        true ->
+            case should_clean(Cache) of
+                true ->
+                    ets:match_delete(Tid, {'_', 0, '_'});
+                false ->
+                    timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+                    clean(Cache, Until)
+            end;
+        false ->
+            false
+    end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+    ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+    All = pids(Tid),
+    Alive = sets:filter(fun is_process_alive/1, All),
+    Dead = sets:subtract(All, Alive),
+    % In OTP 27+ use sets:map/2
+    Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+    sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+    Acc = couch_util:new_set(),
+    try
+        ets:foldl(fun pids_fold/2, Acc, Tid)
+    catch
+        error:badarg -> Acc
+    end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+    sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+    Acc.
+
+% Private helper functions
+
+new() ->
+    Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+    Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+    % Some per-table overhead for the table metadata
+    Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+    #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Index = erlang:phash2(Term, tuple_size(Caches)),
+            #cache{} = element(1 + Index, Caches);
+        undefined ->
+            undefined
+    end.
+
+bump_usage(Tid, Key) ->
+    % We're updating the second field incrementing it by 1 and clamping it
+    % at ?MAX_PRIORITY. We don't set the default for the update_counter
+    % specifically to avoid creating bogus entries just from updating the
+    % counter, so expect the error:badarg here sometimes.
+    UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+reset_usage(Tid, Key) ->
+    % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+    % subtract that and clamp it at 0. Do not provide a default since if an
+    % entry is missing we don't want to create a bogus one from this operation.
+    UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+info(#cache{tid = Tid}) ->
+    Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+    Size = ets:info(Tid, size),
+    {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+    Head = {'$1', '$2', '$3'},
+    Cond = [{'>', '$2', 0}],
+    Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+    ets:select_replace(Tid, [{Head, Cond, Result}]).

Review Comment:
   That would work, not bad idea, but typically we have not been using the fun2ms parse transform before (only in a few tests before). Also, would this interfere with hot code loading if we ever have to do it?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298191580


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning
+% them. This is to quickly react and make room for potentially new entries
+% being added without having to for a whole interval

Review Comment:
   missing verb `wait`
   
    being added without having to `wait` for a whole interval



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298182015


##########
rel/overlay/etc/default.ini:
##########
@@ -130,6 +130,26 @@ view_index_dir = {{view_index_dir}}
 ; downgrade or even open the databases in question.
 ;prohibit_downgrade = true
 
+[bt_engine_cache]
+; Memory used for btree engine cache. This is a cache for top levels of
+; database btrees (id tree, seq tree) and a few terms from the db header. Value
+; is in bytes.
+;max_size = 67108864
+;
+; Items not accessed in a while get are eventually evicted. However, if the
+; memory used is below this percentage, then even the unused items are left in
+; the cache. The trade-off here is when a new workload starts, it may find the
+; cache with some stale items during the first few seconds and not be able to
+; insert its entries in.
+;leave_percent = 30
+;
+; Cache database btree nodes up to this depth only. Depth starts at 1 at root,
+; then 2 the next level down, and so on. Only intermediate (pointer) nodes will
+; be cached, those are the nodes which point to other nodes, as opposed to leaf
+; key-value nodes, which hold revision trees. To disably db btree node caching

Review Comment:
   A typo `disable`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298529219


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning
+% them. This is to quickly react and make room for potentially new entries
+% being added without having to for a whole interval

Review Comment:
   I opted to re-write the section to make it a bit more clear



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3202005847

   Update the PR with these changes:
   
    * As a safety precaution add a `is_process_alive(Fd)` check after we fetch the item from the cache.  Behind the scenes that still sends a signal to the process so it will add some overhead but it seems safer. If the process is already dead, the cache returns `undefined` as if the item is missing and then cleans any other entries associated with that `Fd`. The caller will then try to fetch the term  from `couch_file` and crash like it would without the cache.
    
    * Parameterized the percentage of max size below which entries are left around in the cache. That is if the cache is mostly empty then some entries with 0 bump_counts are left around in there in case they may be needed again. The tradeoff here is that if a bunch of new work come suddenly it may find the cache full until the next cleanup cycles removes all the 0 entries. This will let us test some of the scenarios mentioned like what happens if most entries are already in the cache.
    
     * Added a 3rd `[couchdb, bt_engine_cache, full]` metric which is bumped when tried to insert into the cache but it was full.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307344394


##########
rel/overlay/etc/default.ini:
##########
@@ -130,6 +130,26 @@ view_index_dir = {{view_index_dir}}
 ; downgrade or even open the databases in question.
 ;prohibit_downgrade = true
 
+[bt_engine_cache]
+; Memory used for btree engine cache. This is a cache for top levels of
+; database btrees (id tree, seq tree) and a few terms from the db header. Value
+; is in bytes.
+;max_size = 67108864
+;
+; Items not accessed in a while are eventually evicted. However, if the memory
+; used is below this percentage, then even the unused items are left in the
+; cache. The trade-off here is when a new workload starts, it may find the
+; cache with some stale items during the first few seconds and not be able to
+; insert its entries in.
+;leave_percent = 30
+;
+; Cache database btree nodes up to this depth only. Depth starts at 1 at root,
+; then at 2 the next level down, and so on. Only intermediate (pointer) nodes
+; will be cached, those are the nodes which point to other nodes, as opposed to
+; leaf key-value nodes, which hold revision trees. To disabe db btree node

Review Comment:
   Typo `disable`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307436246


##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+    insert/2,
+    insert/3,
+    reset/1,
+    lookup/1,
+    status/0
+]).
+
+% Supervision and start API
+%
+-export([
+    create_tables/0,
+    sup_children/0,
+    start_link/1,
+    init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+    insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+    Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+    case get_cache(Key) of
+        #cache{tid = Tid, max_size = Max} ->
+            case ets:info(Tid, memory) < Max of
+                true ->
+                    case ets:insert_new(Tid, {Key, Priority1, Term}) of
+                        true ->
+                            true;
+                        false ->
+                            bump_usage(Tid, Key),
+                            false
+                    end;
+                false ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+                    false
+            end;
+        undefined ->
+            false
+    end.
+
+reset(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} -> reset_usage(Tid, Key);
+        undefined -> true
+    end.
+
+lookup(Key) ->
+    case get_cache(Key) of
+        #cache{tid = Tid} ->
+            case ets:lookup_element(Tid, Key, 3, undefined) of
+                undefined ->
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+                    undefined;
+                Term ->
+                    bump_usage(Tid, Key),
+                    couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+                    Term
+            end;
+        undefined ->
+            undefined
+    end.
+
+status() ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+            {Sizes, Memory} = lists:unzip(SizeMem),
+            #{
+                size => lists:sum(Sizes),
+                memory => lists:sum(Memory)
+            };
+        undefined ->
+            #{}
+    end.
+
+% Supervisor helper functions
+
+create_tables() ->
+    BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+    persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+    [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+    Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+    #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+    {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+    Caches = persistent_term:get(?PTERM_KEY),
+    Cache = #cache{tid = Tid} = element(N, Caches),
+    ets:delete_all_objects(Tid),
+    loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+    decay(Tid),
+    Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+    clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+    remove_dead(Cache),
+    WaitLeft = max(10, Next - now_msec()),
+    timer:sleep(WaitLeft),
+    loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+    case now_msec() < Until of
+        true ->
+            case should_clean(Cache) of
+                true ->
+                    ets:match_delete(Tid, {'_', 0, '_'});
+                false ->
+                    timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+                    clean(Cache, Until)
+            end;
+        false ->
+            false
+    end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+    ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+    All = pids(Tid),
+    Alive = sets:filter(fun is_process_alive/1, All),
+    Dead = sets:subtract(All, Alive),
+    % In OTP 27+ use sets:map/2
+    Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+    sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+    Acc = couch_util:new_set(),
+    try
+        ets:foldl(fun pids_fold/2, Acc, Tid)
+    catch
+        error:badarg -> Acc
+    end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+    sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+    Acc.
+
+% Private helper functions
+
+new() ->
+    Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+    Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+    % Some per-table overhead for the table metadata
+    Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+    #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+    case persistent_term:get(?PTERM_KEY, undefined) of
+        Caches when is_tuple(Caches) ->
+            Index = erlang:phash2(Term, tuple_size(Caches)),
+            #cache{} = element(1 + Index, Caches);
+        undefined ->
+            undefined
+    end.
+
+bump_usage(Tid, Key) ->
+    % We're updating the second field incrementing it by 1 and clamping it
+    % at ?MAX_PRIORITY. We don't set the default for the update_counter
+    % specifically to avoid creating bogus entries just from updating the
+    % counter, so expect the error:badarg here sometimes.
+    UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+reset_usage(Tid, Key) ->
+    % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+    % subtract that and clamp it at 0. Do not provide a default since if an
+    % entry is missing we don't want to create a bogus one from this operation.
+    UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+    try
+        ets:update_counter(Tid, Key, UpdateOp)
+    catch
+        error:badarg -> ok
+    end.
+
+info(#cache{tid = Tid}) ->
+    Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+    Size = ets:info(Tid, size),
+    {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+    Head = {'$1', '$2', '$3'},
+    Cond = [{'>', '$2', 0}],
+    Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+    ets:select_replace(Tid, [{Head, Cond, Result}]).
+
+shard_count() ->
+    erlang:system_info(schedulers).
+
+wait_interval(Interval) ->
+    Jitter = rand:uniform(max(1, round(Interval * 0.5))),

Review Comment:
   Interval is integer. How about replacing the `round(Interval * 0.5)` with `Interval bsr 1`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307537506


##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,117 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+%   http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+    {
+        foreach,
+        fun setup/0,
+        fun teardown/1,
+        [
+            ?TDEF_FE(t_created),
+            ?TDEF_FE(t_insert_and_lookup),
+            ?TDEF_FE(t_decay_and_removal_works, 10),
+            ?TDEF_FE(t_pid_cleanup_works, 10)
+        ]
+    }.
+
+setup() ->
+    Ctx = test_util:start_applications([config]),
+    couch_bt_engine_cache:create_tables(),
+    N = tuple_size(persistent_term:get(?PTERM_KEY)),

Review Comment:
   A nitpick. Here and after in the module you directly access PTERM_KEY. You could export the `couch_bt_engine_cache:list_caches().` or better `couch_bt_engine_cache:info().`, 
   
   ```erlang
   -module(couch_bt_engine_cache).
   
   -export([info/0]).
   
   info() ->
       case persistent_term:get(?PTERM_KEY, undefined) of
           Caches when is_tuple(Caches) ->
               lists:map(fun(#cache{tid = Tid, max_size = Max}) -> 
                    Info = maps:from_list(ets:i(Tid)),
                    Info#{
                      stats => #{
                         full => couch_stats:sample([couchdb, bt_engine_cache, full]),
                         misses => couch_stats:sample([couchdb, bt_engine_cache, misses]),
                         hits => couch_stats:sample([couchdb, bt_engine_cache, hits])
                      },
                      max_cache_size => Max
                   }
               end, tuple_to_list(Caches));
           undefined ->
               []
       end.
   ```
   
   Then we can add a call to this function into `couch_debug:bt_cache_info/0`.
   
   I think information provided by this function should be enough to write all tests in this module.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3233695567

   Great work. I have couple of nitpicks, otherwise looks really good. I apologize about my mistake to use another my account initially (@iilyak-ibm is also me). 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3209005577

   @jaydoane wondered what happened if we try depth = 3. With the same setup as above ran a few trials with a various cache depths 3, 2 and 1. Maybe we should go with depth = 3 as the new default?
   
   <img width="954" height="849" alt="cache_depths_experiment" src="https://github.com/user-attachments/assets/651013ab-b9e3-49df-84bb-28c4f1e4a0d9" />
   
   #### Depth 3 
   
   ```
    HTTP
       http_req_duration..............: avg=586.74ms min=3.14ms  med=673.8ms  max=5.66s    p(90)=1.16s    p(95)=1.28s
         { expected_response:true }...: avg=585.98ms min=3.14ms  med=673.07ms max=5.36s    p(90)=1.16s    p(95)=1.28s
         { name:bulk_docs }...........: avg=112.06ms min=88.56ms med=108.77ms max=422.51ms p(90)=117.81ms p(95)=120.51ms
         { name:get_doc }.............: avg=15.37ms  min=3.14ms  med=11.77ms  max=4.25s    p(90)=21.93ms  p(95)=24.55ms
         { name:post_doc }............: avg=794.25ms min=15.16ms med=771.39ms max=4.91s    p(90)=1.16s    p(95)=1.29s
         { name:put_doc }.............: avg=950.81ms min=39.64ms med=947.47ms max=5.66s    p(90)=1.28s    p(95)=1.38s
       http_req_failed................: 0.11%   7585 out of 6529402
       http_reqs......................: 6529402 8535.726214/s
   
       EXECUTION
       iteration_duration.............: avg=2.76s    min=1.13s   med=2.74s    max=8.37s    p(90)=3.21s    p(95)=3.37s
       iterations.....................: 2176133 2844.805006/s
       vus............................: 347     min=0               max=10000
       vus_max........................: 10000   min=10000           max=10000
   
       NETWORK
       data_received..................: 4.1 GB  5.3 MB/s
       data_sent......................: 1.1 GB  1.4 MB/s
   ```
   ```
   {
     "full": {
       "value": 0,
       "type": "counter",
       "desc": "number of times bt_engine cache was full"
     },
     "hits": {
       "value": 38216261,
       "type": "counter",
       "desc": "number of bt_engine cache hits"
     },
     "misses": {
       "value": 1476375,
       "type": "counter",
       "desc": "number of bt_engine cache misses"
     }
   }
   ```
   
   #### Depth 2
   
   ```
       HTTP
       http_req_duration..............: avg=735.09ms min=3.71ms  med=877.78ms max=3.4s     p(90)=1.41s    p(95)=1.55s
         { expected_response:true }...: avg=734.3ms  min=3.71ms  med=877.09ms max=3.2s     p(90)=1.41s    p(95)=1.55s
         { name:bulk_docs }...........: avg=109.51ms min=94.58ms med=106.51ms max=429.49ms p(90)=113.26ms p(95)=115.84ms
         { name:get_doc }.............: avg=15.45ms  min=3.71ms  med=12.45ms  max=2.05s    p(90)=25.08ms  p(95)=28.88ms
         { name:post_doc }............: avg=1.04s    min=13.54ms med=1.02s    max=3.2s     p(90)=1.45s    p(95)=1.59s
         { name:put_doc }.............: avg=1.14s    min=46.67ms med=1.12s    max=3.4s     p(90)=1.53s    p(95)=1.65s
       http_req_failed................: 0.10%   5948 out of 5622800
       http_reqs......................: 5622800 7371.456322/s
   
       EXECUTION
       iteration_duration.............: avg=3.2s     min=1.41s   med=3.18s    max=6.65s    p(90)=3.74s    p(95)=3.92s
       iterations.....................: 1873932 2456.713362/s
       vus............................: 106     min=0               max=10000
       vus_max........................: 10000   min=10000           max=10000
   
       NETWORK
       data_received..................: 3.5 GB  4.6 MB/s
       data_sent......................: 933 MB  1.2 MB/s
   ```
   ```
   {
     "full": {
       "value": 0,
       "type": "counter",
       "desc": "number of times bt_engine cache was full"
     },
     "hits": {
       "value": 24445315,
       "type": "counter",
       "desc": "number of bt_engine cache hits"
     },
     "misses": {
       "value": 367983,
       "type": "counter",
       "desc": "number of bt_engine cache misses"
     }
   }
   ```
   
   
   ### Depth 1
   
   ```
       HTTP
       http_req_duration..............: avg=897.76ms min=4.3ms   med=1.05s    max=4.06s    p(90)=1.75s    p(95)=1.93s
         { expected_response:true }...: avg=896.76ms min=4.3ms   med=1.05s    max=3.64s    p(90)=1.75s    p(95)=1.92s
         { name:bulk_docs }...........: avg=110.12ms min=88.8ms  med=107.17ms max=424.52ms p(90)=114.56ms p(95)=116.5ms
         { name:get_doc }.............: avg=21.44ms  min=4.3ms   med=16.46ms  max=3.55s    p(90)=35.67ms  p(95)=40.64ms
         { name:post_doc }............: avg=1.25s    min=22.69ms med=1.23s    max=3.62s    p(90)=1.77s    p(95)=1.94s
         { name:put_doc }.............: avg=1.41s    min=24.89ms med=1.4s     max=4.06s    p(90)=1.92s    p(95)=2.07s
       http_req_failed................: 0.10%   5285 out of 4881638
       http_reqs......................: 4881638 6394.596739/s
   
       EXECUTION
       iteration_duration.............: avg=3.69s    min=1.29s   med=3.67s    max=8.51s    p(90)=4.32s    p(95)=4.53s
       iterations.....................: 1626878 2131.093857/s
       vus............................: 3502    min=0               max=10000
       vus_max........................: 10000   min=10000           max=10000
   
       NETWORK
       data_received..................: 3.1 GB  4.0 MB/s
       data_sent......................: 824 MB  1.1 MB/s
   ```
   ```
   {
     "full": {
       "value": 0,
       "type": "counter",
       "desc": "number of times bt_engine cache was full"
     },
     "hits": {
       "value": 16728022,
       "type": "counter",
       "desc": "number of bt_engine cache hits"
     },
     "misses": {
       "value": 138036,
       "type": "counter",
       "desc": "number of bt_engine cache misses"
     }
   }
   ```
   
   Maybe we should go
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BTree engine term cache [couchdb]

Posted by "mikerhodes (via GitHub)" <gi...@apache.org>.
mikerhodes commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3211112536

   I don't think the eviction algorithm choice is un-smart. It's quite a nice least-used algorithm that looks to be quite scan-resistant due to the way it exponentially decays (this is useful against applications which "scan" by periodically open a lot of databases, rather than scan a single file on disk).
   
   The scan resistance property is why something like ets-lru doesn't make as much sense for this use-case. If the cache gets full, hopefully it's full of stuff we use a lot, rather than just what we opened recently. For those periodic cases, we don't churn the cache for them.
   
   [TinyLFU](https://dl.acm.org/doi/10.1145/3149371) is quite interesting in that it uses a sketch to decide whether to add something to the cache at all, which is also nice against the scan-databases use-case, but also seems quite a bit more complicated. I note that it also degrades by dividing by two.
   
   TinyLFU or something like it could be _really_ useful for things like the main LRU where we keep open files --- avoiding evicting things during periodic database scans would be good.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org