You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by "nickva (via GitHub)" <gi...@apache.org> on 2025/08/18 23:13:28 UTC
[PR] BTree engine term cache [couchdb]
nickva opened a new pull request, #5625:
URL: https://github.com/apache/couchdb/pull/5625
Cache the top b-tree nodes and header terms. Items are inserted into the cache on first use and then they have to be updated when accessed. If they are not updated frequently enough they are evicted.
In order to get or update any kv leaf nodes in the b-trees, we always have to read the top few kp nodes closer to the root. Even with parallel preads and data being in the page cache the cost of system calls, marshaling the term, going through the erlang IO system can add up.
The cache is limited in size. The size defaults to 64Mb and is configurable by the user. If the cache is full, no more entries will be added until more space frees up.
Since multiple processes are accessing the data concurrently, ets tables are sharded by the number of schedulers, not unlike how we shard our couch_server processes. Each ets table has an associated cleaner process which evicts unused entries. Cleaners traverse table entries and "decay" the usage counters exponentially, by repeatedly shifting the value to the right with the `bsr` op. Entries with a usage counter equal 0 are then removed.
In order to only insert the top nodes in the cache the couch_btree module's lookup and streaming functions were updated with a `depth` parameter. If that way we avoid thrashing node which are less likely to be reused through the cache: if the depth > 2 or the node is a kv_node it skips the cache altogether.
To get an idea on tuning the cache size for various workload there are two new metrics to count hits and misses:
```
% http $DB/_node/_local/_stats/couchdb/bt_engine_cache
{
"hits": {
"desc": "number of bt_engine cache hits",
"type": "counter",
"value": 233310
},
"misses": {
"desc": "number of bt_engine cache misses",
"type": "counter",
"value": 7386
}
}
```
Those are the metrics after running `fabric_bench:go(#{q=>8, doc_size=>small, docs=>100000})`.
A quick and dirty comparison with main:
```
> fabric_bench:go(#{q=>8, doc_size=>small, docs=>100000}).
*** Parameters
* batch_size : 1000
* doc_size : small
* docs : 100000
* individual_docs : 1000
* n : default
* q : 8
*** Environment
* Nodes : 1
* Bench ver. : 1
* N : 1
* Q : 8
* OS : unix/darwin
* Couch ver. : 3.5.0-d16fc1f
* Couch git sha: d16fc1f
* VM details : Erlang/OTP 26 [erts-14.2.5.11] [source] [64-bit] [smp:12:12] [ds:12:12:16] [async-threads:1]
*** Inserting 100000 docs
* Add 100000 docs, ok:100/accepted:0 (Hz): 22000
* Get random doc 100000X (Hz): 3800
* All docs (Hz): 130000
* All docs w/ include_docs (Hz): 48000
* Changes (Hz): 16000
* Single doc updates 1000X (Hz): 530
* Time to run all benchmarks (sec): 40
```
With the btree cache:
```
fabric_bench:go(#{q=>8, doc_size=>small, docs=>100000}).
*** Parameters
* batch_size : 1000
* doc_size : small
* docs : 100000
* individual_docs : 1000
* n : default
* q : 8
*** Environment
* Nodes : 1
* Bench ver. : 1
* N : 1
* Q : 8
* OS : unix/darwin
* Couch ver. : 3.5.0-d16fc1f
* Couch git sha: d16fc1f
* VM details : Erlang/OTP 26 [erts-14.2.5.11] [source] [64-bit] [smp:12:12] [ds:12:12:16] [async-threads:1]
*** Inserting 100000 docs
* Add 100000 docs, ok:100/accepted:0 (Hz): 24000
* Get random doc 100000X (Hz): 4400
* All docs (Hz): 140000
* All docs w/ include_docs (Hz): 49000
* Changes (Hz): 29000
* Single doc updates 1000X (Hz): 680
* Time to run all benchmarks (sec): 33
```
The idea to use a depth parameter and an ets table came from Paul J. Davis (@davisp).
<!-- Thank you for your contribution!
Please file this form by replacing the Markdown comments
with your text. If a section needs no action - remove it.
Also remember, that CouchDB uses the Review-Then-Commit (RTC) model
of code collaboration. Positive feedback is represented +1 from committers
and negative is a -1. The -1 also means veto, and needs to be addressed
to proceed. Once there are no objections, the PR can be merged by a
CouchDB committer.
See: http://couchdb.apache.org/bylaws.html#decisions for more info. -->
## Overview
<!-- Please give a short brief for the pull request,
what problem it solves or how it makes things better. -->
## Testing recommendations
<!-- Describe how we can test your changes.
Does it provide any behaviour that the end users
could notice? -->
## Related Issues or Pull Requests
<!-- If your changes affect multiple components in different
repositories please put links to those issues or pull requests here. -->
## Checklist
- [ ] Code is written and works correctly
- [ ] Changes are covered by tests
- [ ] Any new configurable parameters are documented in `rel/overlay/etc/default.ini`
- [ ] Documentation changes were made in the `src/docs` folder
- [ ] Documentation changes were backported (separated PR) to affected branches
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298380859
##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,118 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+ {
+ foreach,
+ fun setup/0,
+ fun teardown/1,
+ [
+ ?TDEF_FE(t_created),
+ ?TDEF_FE(t_insert_and_lookup),
+ ?TDEF_FE(t_decay_and_removal_works, 10),
+ ?TDEF_FE(t_pid_cleanup_works, 10)
+ ]
+ }.
+
+setup() ->
+ Ctx = test_util:start_applications([config]),
+ couch_bt_engine_cache:create_tables(),
+ N = tuple_size(persistent_term:get(?PTERM_KEY)),
+ Pids = lists:map(
+ fun(I) ->
+ {ok, Pid} = couch_bt_engine_cache:start_link(I),
+ Pid
+ end,
+ lists:seq(1, N)
+ ),
+ {Ctx, Pids}.
+
+teardown({Ctx, [_ | _] = Pids}) ->
+ lists:foreach(
+ fun(Pid) ->
+ catch unlink(Pid),
+ exit(Pid, kill)
+ end,
+ Pids
+ ),
+ clear_tables(),
+ config:delete("bt_engine_cache", "max_size", false),
+ config:delete("bt_engine_cache", "leave_percent", false),
+ test_util:stop_applications(Ctx).
+
+clear_tables() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Fun = fun({cache, Tid, _}) -> ets:delete(Tid) end,
+ lists:foreach(Fun, tuple_to_list(Caches));
+ undefined ->
+ ok
+ end.
+
+t_created(_) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ N = erlang:system_info(schedulers),
+ ?assert(tuple_size(Caches) >= 1),
+ ?assertEqual(N, tuple_size(Caches)),
+ Stats = couch_bt_engine_cache:status(),
+ ?assertMatch(#{size := 0, memory := Mem} when is_integer(Mem), Stats).
+
+t_insert_and_lookup(_) ->
+ % Can only insert pids and positions
+ ?assertError(function_clause, couch_bt_engine_cache:insert(not_a_pid, 1, foo)),
+ ?assertError(function_clause, couch_bt_engine_cache:insert(self(), xyz, foo)),
+ ?assertMatch(#{size := 0}, couch_bt_engine_cache:status()),
+
+ ?assert(couch_bt_engine_cache:insert({pid, 42}, term)),
+ ?assertMatch(#{size := 1}, couch_bt_engine_cache:status()),
+
+ % Already inserted
+ ?assertNot(couch_bt_engine_cache:insert({pid, 42}, term)),
+
+ % Should be there when we look it up
+ ?assertEqual(term, couch_bt_engine_cache:lookup({pid, 42})),
+
+ % Other stuff we didn't insert should return undefined
+ ?assertEqual(undefined, couch_bt_engine_cache:lookup({pid, 43})).
+
+t_decay_and_removal_works(_) ->
+ config:set("bt_engine_cache", "leave_percent", "0", false),
+ Term = [foo, bar, baz, lists:seq(1, 100)],
+ [couch_bt_engine_cache:insert({pid, I}, Term) || I <- lists:seq(1, 10000)],
+ WaitFun = fun() ->
+ #{size := Size} = couch_bt_engine_cache:status(),
+ case Size > 0 of
+ true -> wait;
+ false -> ok
+ end
+ end,
+ test_util:wait(WaitFun, 7500),
+ ?assertMatch(#{size := 0}, couch_bt_engine_cache:status()).
+
+t_pid_cleanup_works(_) ->
+ Pid = spawn(fun() -> timer:sleep(2000) end),
+ [couch_bt_engine_cache:insert({Pid, I}, baz) || I <- lists:seq(1, 1000)],
+ WaitFun = fun() ->
+ #{size := Size} = couch_bt_engine_cache:status(),
+ case Size > 0 of
+ true -> wait;
+ false -> ok
+ end
+ end,
+ test_util:wait(WaitFun, 7500),
Review Comment:
This one has a longer timeout set as well in `?TDEF_FE(t_pid_cleanup_works, 10)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307697680
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+ insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+ Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+ case get_cache(Key) of
+ #cache{tid = Tid, max_size = Max} ->
+ case ets:info(Tid, memory) < Max of
+ true ->
+ case ets:insert_new(Tid, {Key, Priority1, Term}) of
+ true ->
+ true;
+ false ->
+ bump_usage(Tid, Key),
+ false
+ end;
+ false ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+ false
+ end;
+ undefined ->
+ false
+ end.
+
+reset(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} -> reset_usage(Tid, Key);
+ undefined -> true
+ end.
+
+lookup(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} ->
+ case ets:lookup_element(Tid, Key, 3, undefined) of
+ undefined ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+ undefined;
+ Term ->
+ bump_usage(Tid, Key),
+ couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+ Term
+ end;
+ undefined ->
+ undefined
+ end.
+
+status() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+ {Sizes, Memory} = lists:unzip(SizeMem),
+ #{
+ size => lists:sum(Sizes),
+ memory => lists:sum(Memory)
+ };
+ undefined ->
+ #{}
+ end.
+
+% Supervisor helper functions
+
+create_tables() ->
+ BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+ persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+ [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+ Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+ #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+ {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ Cache = #cache{tid = Tid} = element(N, Caches),
+ ets:delete_all_objects(Tid),
+ loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+ decay(Tid),
+ Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+ clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+ remove_dead(Cache),
+ WaitLeft = max(10, Next - now_msec()),
+ timer:sleep(WaitLeft),
+ loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+ case now_msec() < Until of
+ true ->
+ case should_clean(Cache) of
+ true ->
+ ets:match_delete(Tid, {'_', 0, '_'});
+ false ->
+ timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+ clean(Cache, Until)
+ end;
+ false ->
+ false
+ end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+ ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+ All = pids(Tid),
+ Alive = sets:filter(fun is_process_alive/1, All),
+ Dead = sets:subtract(All, Alive),
+ % In OTP 27+ use sets:map/2
+ Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+ sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+ Acc = couch_util:new_set(),
+ try
+ ets:foldl(fun pids_fold/2, Acc, Tid)
+ catch
+ error:badarg -> Acc
+ end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+ sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+ Acc.
+
+% Private helper functions
+
+new() ->
+ Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+ Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+ % Some per-table overhead for the table metadata
+ Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+ #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Index = erlang:phash2(Term, tuple_size(Caches)),
+ #cache{} = element(1 + Index, Caches);
+ undefined ->
+ undefined
+ end.
+
+bump_usage(Tid, Key) ->
+ % We're updating the second field incrementing it by 1 and clamping it
+ % at ?MAX_PRIORITY. We don't set the default for the update_counter
+ % specifically to avoid creating bogus entries just from updating the
+ % counter, so expect the error:badarg here sometimes.
+ UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+reset_usage(Tid, Key) ->
+ % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+ % subtract that and clamp it at 0. Do not provide a default since if an
+ % entry is missing we don't want to create a bogus one from this operation.
+ UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+info(#cache{tid = Tid}) ->
+ Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+ Size = ets:info(Tid, size),
+ {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+ Head = {'$1', '$2', '$3'},
+ Cond = [{'>', '$2', 0}],
+ Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+ ets:select_replace(Tid, [{Head, Cond, Result}]).
Review Comment:
That would work, not bad idea, but we haven't been typically been using the fun2ms parse transforms before (only in a few tests before). Also, would this interfere with hot code loading if we ever have to do it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3199749004
Nice work.
I'd like to see a bigger benchmark before deciding but the performance gain is quite modest (except for changes) to warrant the extra complexity and failure modes (on the flipside it does validate our deliberate reliance on the OS page cache for this so far). I'd be curious to see the same benchmarks but with the code modified to not expire the cached items so aggressively (to find the upper bound to how much this could improve things). It would also be useful to know how deep the btrees are in question (or some other indication of the cache hit/miss ratio to give us context when assessing the throughput figures). As for why changes gets such a boost, that needs some thinking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298432070
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning
Review Comment:
Thank you. You're right that is confusing.
I think something like this might be more clear:
```
% How often cleaners check if the ets size increases. This is used in cases
% when initially, at the start the cleaning interval, ets table size is below
% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
% new set of requests come in soon after, but before the next clean-up interval
% they'll fail since the cache would be full. To be able to react quicker and
% make room, cleaners poll table sizes a bit more often.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3202798994
I pretty-fied a quick btree stats reporter I had since we were wondering what the tree looked like from above (this is q=8, 100k docs) just one shard copy:
```json
"sizes": {
"active": 5812423,
"external": 4040930,
"file": 7188696,
"id_tree": {
"1": {
"kp_node": {
"cnt": 1,
"max": 5,
"min": 5
}
},
"2": {
"kp_node": {
"cnt": 5,
"max": 21,
"min": 15
}
},
"3": {
"kp_node": {
"cnt": 91,
"max": 23,
"min": 11
}
},
"4": {
"kv_node": {
"cnt": 1450,
"max": 15,
"min": 1
}
}
},
"seq_tree": {
"1": {
"kp_node": {
"cnt": 1,
"max": 2,
"min": 2
}
},
"2": {
"kp_node": {
"cnt": 2,
"max": 26,
"min": 2
}
},
"3": {
"kp_node": {
"cnt": 28,
"max": 47,
"min": 6
}
},
"4": {
"kv_node": {
"cnt": 975,
"max": 15,
"min": 2
}
}
}
}
```
The key is depth, then node type, then `cnt` is the number of nodes at that level, `min` is the smallest node size (number of kvs/kps), `max` is the largest size.
It's not as shallow as we'd expect due to how complete_root works, and chunk_size is probably not the best any longer (doesn't count for compression). I was going to look into maybe having a different chunk size, or different per node type (kps get more), but that's for a different PR.
So caching the top 2 nodes makes sense, there are not that many and they bring the biggest benefit. Top 3 could also be an option but maybe start smaller at first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298114554
##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,118 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+ {
+ foreach,
+ fun setup/0,
+ fun teardown/1,
+ [
+ ?TDEF_FE(t_created),
+ ?TDEF_FE(t_insert_and_lookup),
+ ?TDEF_FE(t_decay_and_removal_works, 10),
+ ?TDEF_FE(t_pid_cleanup_works, 10)
+ ]
+ }.
+
+setup() ->
+ Ctx = test_util:start_applications([config]),
+ couch_bt_engine_cache:create_tables(),
+ N = tuple_size(persistent_term:get(?PTERM_KEY)),
+ Pids = lists:map(
+ fun(I) ->
+ {ok, Pid} = couch_bt_engine_cache:start_link(I),
+ Pid
+ end,
+ lists:seq(1, N)
+ ),
+ {Ctx, Pids}.
+
+teardown({Ctx, [_ | _] = Pids}) ->
+ lists:foreach(
+ fun(Pid) ->
+ catch unlink(Pid),
+ exit(Pid, kill)
+ end,
+ Pids
+ ),
+ clear_tables(),
+ config:delete("bt_engine_cache", "max_size", false),
+ config:delete("bt_engine_cache", "leave_percent", false),
+ test_util:stop_applications(Ctx).
+
+clear_tables() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Fun = fun({cache, Tid, _}) -> ets:delete(Tid) end,
+ lists:foreach(Fun, tuple_to_list(Caches));
+ undefined ->
+ ok
+ end.
+
+t_created(_) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ N = erlang:system_info(schedulers),
+ ?assert(tuple_size(Caches) >= 1),
+ ?assertEqual(N, tuple_size(Caches)),
+ Stats = couch_bt_engine_cache:status(),
+ ?assertMatch(#{size := 0, memory := Mem} when is_integer(Mem), Stats).
+
+t_insert_and_lookup(_) ->
+ % Can only insert pids and positions
Review Comment:
The comment and following two assertions are confusing. The `couch_bt_engine_cache:insert/{2, 3}` expects the key to be in the form `{pid(), pos_integer()}`, while comment imply that it can be `pid()`. Please update comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298135326
##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,118 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+ {
+ foreach,
+ fun setup/0,
+ fun teardown/1,
+ [
+ ?TDEF_FE(t_created),
+ ?TDEF_FE(t_insert_and_lookup),
+ ?TDEF_FE(t_decay_and_removal_works, 10),
+ ?TDEF_FE(t_pid_cleanup_works, 10)
+ ]
+ }.
+
+setup() ->
+ Ctx = test_util:start_applications([config]),
+ couch_bt_engine_cache:create_tables(),
+ N = tuple_size(persistent_term:get(?PTERM_KEY)),
+ Pids = lists:map(
+ fun(I) ->
+ {ok, Pid} = couch_bt_engine_cache:start_link(I),
+ Pid
+ end,
+ lists:seq(1, N)
+ ),
+ {Ctx, Pids}.
+
+teardown({Ctx, [_ | _] = Pids}) ->
+ lists:foreach(
+ fun(Pid) ->
+ catch unlink(Pid),
+ exit(Pid, kill)
+ end,
+ Pids
+ ),
+ clear_tables(),
+ config:delete("bt_engine_cache", "max_size", false),
+ config:delete("bt_engine_cache", "leave_percent", false),
+ test_util:stop_applications(Ctx).
+
+clear_tables() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Fun = fun({cache, Tid, _}) -> ets:delete(Tid) end,
+ lists:foreach(Fun, tuple_to_list(Caches));
+ undefined ->
+ ok
+ end.
+
+t_created(_) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ N = erlang:system_info(schedulers),
+ ?assert(tuple_size(Caches) >= 1),
+ ?assertEqual(N, tuple_size(Caches)),
+ Stats = couch_bt_engine_cache:status(),
+ ?assertMatch(#{size := 0, memory := Mem} when is_integer(Mem), Stats).
+
+t_insert_and_lookup(_) ->
+ % Can only insert pids and positions
+ ?assertError(function_clause, couch_bt_engine_cache:insert(not_a_pid, 1, foo)),
+ ?assertError(function_clause, couch_bt_engine_cache:insert(self(), xyz, foo)),
+ ?assertMatch(#{size := 0}, couch_bt_engine_cache:status()),
+
+ ?assert(couch_bt_engine_cache:insert({pid, 42}, term)),
+ ?assertMatch(#{size := 1}, couch_bt_engine_cache:status()),
+
+ % Already inserted
+ ?assertNot(couch_bt_engine_cache:insert({pid, 42}, term)),
+
+ % Should be there when we look it up
+ ?assertEqual(term, couch_bt_engine_cache:lookup({pid, 42})),
+
+ % Other stuff we didn't insert should return undefined
+ ?assertEqual(undefined, couch_bt_engine_cache:lookup({pid, 43})).
+
+t_decay_and_removal_works(_) ->
+ config:set("bt_engine_cache", "leave_percent", "0", false),
+ Term = [foo, bar, baz, lists:seq(1, 100)],
+ [couch_bt_engine_cache:insert({pid, I}, Term) || I <- lists:seq(1, 10000)],
+ WaitFun = fun() ->
+ #{size := Size} = couch_bt_engine_cache:status(),
+ case Size > 0 of
+ true -> wait;
+ false -> ok
+ end
+ end,
+ test_util:wait(WaitFun, 7500),
+ ?assertMatch(#{size := 0}, couch_bt_engine_cache:status()).
+
+t_pid_cleanup_works(_) ->
+ Pid = spawn(fun() -> timer:sleep(2000) end),
+ [couch_bt_engine_cache:insert({Pid, I}, baz) || I <- lists:seq(1, 1000)],
+ WaitFun = fun() ->
+ #{size := Size} = couch_bt_engine_cache:status(),
+ case Size > 0 of
+ true -> wait;
+ false -> ok
+ end
+ end,
+ test_util:wait(WaitFun, 7500),
Review Comment:
Timeout value is greater than 5 sec
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298132326
##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,118 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+ {
+ foreach,
+ fun setup/0,
+ fun teardown/1,
+ [
+ ?TDEF_FE(t_created),
+ ?TDEF_FE(t_insert_and_lookup),
+ ?TDEF_FE(t_decay_and_removal_works, 10),
+ ?TDEF_FE(t_pid_cleanup_works, 10)
+ ]
+ }.
+
+setup() ->
+ Ctx = test_util:start_applications([config]),
+ couch_bt_engine_cache:create_tables(),
+ N = tuple_size(persistent_term:get(?PTERM_KEY)),
+ Pids = lists:map(
+ fun(I) ->
+ {ok, Pid} = couch_bt_engine_cache:start_link(I),
+ Pid
+ end,
+ lists:seq(1, N)
+ ),
+ {Ctx, Pids}.
+
+teardown({Ctx, [_ | _] = Pids}) ->
+ lists:foreach(
+ fun(Pid) ->
+ catch unlink(Pid),
+ exit(Pid, kill)
+ end,
+ Pids
+ ),
+ clear_tables(),
+ config:delete("bt_engine_cache", "max_size", false),
+ config:delete("bt_engine_cache", "leave_percent", false),
+ test_util:stop_applications(Ctx).
+
+clear_tables() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Fun = fun({cache, Tid, _}) -> ets:delete(Tid) end,
+ lists:foreach(Fun, tuple_to_list(Caches));
+ undefined ->
+ ok
+ end.
+
+t_created(_) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ N = erlang:system_info(schedulers),
+ ?assert(tuple_size(Caches) >= 1),
+ ?assertEqual(N, tuple_size(Caches)),
+ Stats = couch_bt_engine_cache:status(),
+ ?assertMatch(#{size := 0, memory := Mem} when is_integer(Mem), Stats).
+
+t_insert_and_lookup(_) ->
+ % Can only insert pids and positions
+ ?assertError(function_clause, couch_bt_engine_cache:insert(not_a_pid, 1, foo)),
+ ?assertError(function_clause, couch_bt_engine_cache:insert(self(), xyz, foo)),
+ ?assertMatch(#{size := 0}, couch_bt_engine_cache:status()),
+
+ ?assert(couch_bt_engine_cache:insert({pid, 42}, term)),
+ ?assertMatch(#{size := 1}, couch_bt_engine_cache:status()),
+
+ % Already inserted
+ ?assertNot(couch_bt_engine_cache:insert({pid, 42}, term)),
+
+ % Should be there when we look it up
+ ?assertEqual(term, couch_bt_engine_cache:lookup({pid, 42})),
+
+ % Other stuff we didn't insert should return undefined
+ ?assertEqual(undefined, couch_bt_engine_cache:lookup({pid, 43})).
+
+t_decay_and_removal_works(_) ->
+ config:set("bt_engine_cache", "leave_percent", "0", false),
+ Term = [foo, bar, baz, lists:seq(1, 100)],
+ [couch_bt_engine_cache:insert({pid, I}, Term) || I <- lists:seq(1, 10000)],
+ WaitFun = fun() ->
+ #{size := Size} = couch_bt_engine_cache:status(),
+ case Size > 0 of
+ true -> wait;
+ false -> ok
+ end
+ end,
+ test_util:wait(WaitFun, 7500),
Review Comment:
The default timeout for a test in eunit is 5 seconds, the value you use for wait function is greater.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298189107
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning
Review Comment:
The sentence is unclear.
If no entries were cleaned, how often should we poll the ETS size to determine when to start cleaning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3200929782
> I find the 33% improvement in single doc reads is significant.
I think it's pretty good too and shows how much overhead even getting data from page cache can be.
> like you @rnewson I'm somewhat surprised by the `_changes` seeing an 80% increase in reads per sec (I also wonder why `_all_docs` is so much higher than `_changes` in the first place).
_changes for some reason is pretty variable in this benchmark for some reason, we'll have to investigate a bit why that is
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307427835
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+ insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+ Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+ case get_cache(Key) of
+ #cache{tid = Tid, max_size = Max} ->
+ case ets:info(Tid, memory) < Max of
+ true ->
+ case ets:insert_new(Tid, {Key, Priority1, Term}) of
+ true ->
+ true;
+ false ->
+ bump_usage(Tid, Key),
+ false
+ end;
+ false ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+ false
+ end;
+ undefined ->
+ false
+ end.
+
+reset(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} -> reset_usage(Tid, Key);
+ undefined -> true
+ end.
+
+lookup(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} ->
+ case ets:lookup_element(Tid, Key, 3, undefined) of
+ undefined ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+ undefined;
+ Term ->
+ bump_usage(Tid, Key),
+ couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+ Term
+ end;
+ undefined ->
+ undefined
+ end.
+
+status() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+ {Sizes, Memory} = lists:unzip(SizeMem),
+ #{
+ size => lists:sum(Sizes),
+ memory => lists:sum(Memory)
+ };
+ undefined ->
+ #{}
+ end.
+
+% Supervisor helper functions
+
+create_tables() ->
+ BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+ persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+ [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+ Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+ #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+ {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ Cache = #cache{tid = Tid} = element(N, Caches),
+ ets:delete_all_objects(Tid),
+ loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+ decay(Tid),
+ Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+ clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+ remove_dead(Cache),
+ WaitLeft = max(10, Next - now_msec()),
+ timer:sleep(WaitLeft),
+ loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+ case now_msec() < Until of
+ true ->
+ case should_clean(Cache) of
+ true ->
+ ets:match_delete(Tid, {'_', 0, '_'});
+ false ->
+ timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+ clean(Cache, Until)
+ end;
+ false ->
+ false
+ end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+ ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+ All = pids(Tid),
+ Alive = sets:filter(fun is_process_alive/1, All),
+ Dead = sets:subtract(All, Alive),
+ % In OTP 27+ use sets:map/2
+ Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+ sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+ Acc = couch_util:new_set(),
+ try
+ ets:foldl(fun pids_fold/2, Acc, Tid)
+ catch
+ error:badarg -> Acc
+ end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+ sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+ Acc.
+
+% Private helper functions
+
+new() ->
+ Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+ Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+ % Some per-table overhead for the table metadata
+ Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+ #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Index = erlang:phash2(Term, tuple_size(Caches)),
+ #cache{} = element(1 + Index, Caches);
+ undefined ->
+ undefined
+ end.
+
+bump_usage(Tid, Key) ->
+ % We're updating the second field incrementing it by 1 and clamping it
+ % at ?MAX_PRIORITY. We don't set the default for the update_counter
+ % specifically to avoid creating bogus entries just from updating the
+ % counter, so expect the error:badarg here sometimes.
+ UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+reset_usage(Tid, Key) ->
+ % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+ % subtract that and clamp it at 0. Do not provide a default since if an
+ % entry is missing we don't want to create a bogus one from this operation.
+ UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+info(#cache{tid = Tid}) ->
+ Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+ Size = ets:info(Tid, size),
+ {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+ Head = {'$1', '$2', '$3'},
+ Cond = [{'>', '$2', 0}],
+ Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+ ets:select_replace(Tid, [{Head, Cond, Result}]).
Review Comment:
I think matchspec would read better here.
```erlang
MS = ets:fun2ms(
fun({Key, Priority, Term}) when Priority > 0 ->
% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
{Key, Priority bsr 1, Term}
end),
ets:select_replace(Tid, MS).
```
The matchspec above produces the same query at compile time (if you would do `
-include_lib("stdlib/include/ms_transform.hrl").`).
```
8> ets:fun2ms(fun({Key, Priority, Term}) when Priority > 0 -> {Key, Priority bsr 1, Term} end).
[{{'$1','$2','$3'},
[{'>','$2',0}],
[{{'$1',{'bsr','$2',1},'$3'}}]}]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298376754
##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,118 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+ {
+ foreach,
+ fun setup/0,
+ fun teardown/1,
+ [
+ ?TDEF_FE(t_created),
+ ?TDEF_FE(t_insert_and_lookup),
+ ?TDEF_FE(t_decay_and_removal_works, 10),
+ ?TDEF_FE(t_pid_cleanup_works, 10)
+ ]
+ }.
+
+setup() ->
+ Ctx = test_util:start_applications([config]),
+ couch_bt_engine_cache:create_tables(),
+ N = tuple_size(persistent_term:get(?PTERM_KEY)),
+ Pids = lists:map(
+ fun(I) ->
+ {ok, Pid} = couch_bt_engine_cache:start_link(I),
+ Pid
+ end,
+ lists:seq(1, N)
+ ),
+ {Ctx, Pids}.
+
+teardown({Ctx, [_ | _] = Pids}) ->
+ lists:foreach(
+ fun(Pid) ->
+ catch unlink(Pid),
+ exit(Pid, kill)
+ end,
+ Pids
+ ),
+ clear_tables(),
+ config:delete("bt_engine_cache", "max_size", false),
+ config:delete("bt_engine_cache", "leave_percent", false),
+ test_util:stop_applications(Ctx).
+
+clear_tables() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Fun = fun({cache, Tid, _}) -> ets:delete(Tid) end,
+ lists:foreach(Fun, tuple_to_list(Caches));
+ undefined ->
+ ok
+ end.
+
+t_created(_) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ N = erlang:system_info(schedulers),
+ ?assert(tuple_size(Caches) >= 1),
+ ?assertEqual(N, tuple_size(Caches)),
+ Stats = couch_bt_engine_cache:status(),
+ ?assertMatch(#{size := 0, memory := Mem} when is_integer(Mem), Stats).
+
+t_insert_and_lookup(_) ->
+ % Can only insert pids and positions
Review Comment:
Good spot. Thank you, I switched between two implementation previously so this is a remnant of that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3216348604
One missing part of the testing was to see how the PR fares with multiple dbs when it hits the full cache limit, so I ran the benchmark again (with the latest version) with 1 database (Q=64, 1M docs) and then 100 dbs also with Q=64, 10k docs per db. Since the working set is larger and we hit the full cache a few times, it's less effective, but still shows an improvement both latency and throughput.
#### MAIN
##### 1 DB
```
HTTP
http_req_duration..............: avg=1.1s min=3.8ms med=1.12s max=3.97s p(90)=2.32s p(95)=2.52s
{ expected_response:true }...: avg=1.1s min=3.8ms med=1.12s max=3.97s p(90)=2.31s p(95)=2.52s
{ name:bulk_docs }...........: avg=194.82ms min=122.29ms med=194.57ms max=506.31ms p(90)=210.71ms p(95)=216.96ms
{ name:get_doc }.............: avg=39.48ms min=3.8ms med=38.29ms max=3.25s p(90)=50.8ms p(95)=54.52ms
{ name:post_doc }............: avg=1.37s min=39.94ms med=1.28s max=3.88s p(90)=2.15s p(95)=2.37s
{ name:put_doc }.............: avg=1.91s min=117.86ms med=1.99s max=3.97s p(90)=2.57s p(95)=2.73s
http_req_failed................: 0.13% 5492 out of 4172744
http_reqs......................: 4172744 4918.641943/s
EXECUTION
iteration_duration.............: avg=4.32s min=1.42s med=4.35s max=8.74s p(90)=4.97s p(95)=5.17s
iterations.....................: 1390580 1639.152824/s
vus............................: 3175 min=0 max=10000
vus_max........................: 10000 min=10000 max=10000
NETWORK
data_received..................: 2.7 GB 3.1 MB/s
data_sent......................: 738 MB 870 kB/s
running (14m08.4s), 00000/10000 VUs, 1390580 complete and 0 interrupted iterations
```
##### 100 DBs
```
HTTP
http_req_duration..............: avg=739.75ms min=3.41ms med=722.39ms max=4.29s p(90)=1.53s p(95)=1.74s
{ expected_response:true }...: avg=739.01ms min=5.91ms med=722.02ms max=4.28s p(90)=1.53s p(95)=1.74s
{ name:bulk_docs }...........: avg=176.97ms min=104.51ms med=180.09ms max=222.71ms p(90)=198ms p(95)=204.05ms
{ name:get_doc }.............: avg=72.49ms min=6.42ms med=72.87ms max=2.5s p(90)=90.76ms p(95)=97.98ms
{ name:post_doc }............: avg=1.01s min=5.91ms med=895.98ms max=4.28s p(90)=1.6s p(95)=1.8s
{ name:put_doc }.............: avg=1.13s min=57.11ms med=1.02s max=4.29s p(90)=1.71s p(95)=1.91s
http_req_failed................: 0.11% 6299 out of 5597273
http_reqs......................: 5597273 6640.279304/s
EXECUTION
iteration_duration.............: avg=3.22s min=1.7s med=3.15s max=7.57s p(90)=4.04s p(95)=4.33s
iterations.....................: 1865324 2212.911958/s
vus............................: 0 min=0 max=10000
vus_max........................: 10000 min=10000 max=10000
NETWORK
data_received..................: 3.5 GB 4.2 MB/s
data_sent......................: 980 MB 1.2 MB/s
running (14m02.9s), 00000/10000 VUs, 1865324 complete and 0 interrupted iterations
```
#### BTREE CACHE
##### 1 DB
```
HTTP
http_req_duration..............: avg=586.26ms min=3.41ms med=683.49ms max=3.62s p(90)=1.14s p(95)=1.27s
{ expected_response:true }...: avg=585.63ms min=3.41ms med=682.96ms max=3.04s p(90)=1.14s p(95)=1.27s
{ name:bulk_docs }...........: avg=118.95ms min=88.45ms med=115.99ms max=430.22ms p(90)=123.29ms p(95)=125.34ms
{ name:get_doc }.............: avg=12.71ms min=3.41ms med=10.05ms max=2.11s p(90)=19.59ms p(95)=22.72ms
{ name:post_doc }............: avg=832.66ms min=7.93ms med=808.87ms max=3.04s p(90)=1.19s p(95)=1.32s
{ name:put_doc }.............: avg=913.62ms min=45.74ms med=896.9ms max=3.62s p(90)=1.25s p(95)=1.36s
http_req_failed................: 0.09% 6333 out of 6530402
http_reqs......................: 6530402 8459.167081/s
EXECUTION
iteration_duration.............: avg=2.76s min=1.14s med=2.73s max=6.74s p(90)=3.24s p(95)=3.42s
iterations.....................: 2176466 2819.288849/s
vus............................: 262 min=0 max=10000
vus_max........................: 10000 min=10000 max=10000
NETWORK
data_received..................: 4.1 GB 5.3 MB/s
data_sent......................: 1.1 GB 1.4 MB/s
running (12m52.0s), 00000/10000 VUs, 2176466 complete and 0 interrupted iterations
```
```
{
"full": {
"value": 0,
"type": "counter",
"desc": "number of times bt_engine cache was full"
},
"hits": {
"value": 37521910,
"type": "counter",
"desc": "number of bt_engine cache hits"
},
"misses": {
"value": 1364431,
"type": "counter",
"desc": "number of bt_engine cache misses"
}
}
```
##### 100 DBs
```
HTTP
http_req_duration..............: avg=578.9ms min=2.82ms med=552.61ms max=3.93s p(90)=1.25s p(95)=1.44s
{ expected_response:true }...: avg=578.27ms min=3.34ms med=552.29ms max=3.82s p(90)=1.25s p(95)=1.44s
{ name:bulk_docs }...........: avg=118.11ms min=88.52ms med=117.76ms max=144.29ms p(90)=126.05ms p(95)=129.4ms
{ name:get_doc }.............: avg=39.67ms min=3.34ms med=38.55ms max=2.34s p(90)=53.12ms p(95)=58.21ms
{ name:post_doc }............: avg=798.7ms min=6.24ms med=668.4ms max=3.65s p(90)=1.31s p(95)=1.49s
{ name:put_doc }.............: avg=898.61ms min=6.29ms med=779.32ms max=3.93s p(90)=1.41s p(95)=1.59s
http_req_failed................: 0.10% 6988 out of 6583385
http_reqs......................: 6583385 8474.816169/s
EXECUTION
iteration_duration.............: avg=2.73s min=1.47s med=2.67s max=7.45s p(90)=3.45s p(95)=3.71s
iterations.....................: 2194028 2824.380462/s
vus............................: 0 min=0 max=10000
vus_max........................: 10000 min=10000 max=10000
NETWORK
data_received..................: 4.1 GB 5.3 MB/s
data_sent......................: 1.1 GB 1.5 MB/s
```
```
{
"full": {
"value": 726465,
"type": "counter",
"desc": "number of times bt_engine cache was full"
},
"hits": {
"value": 27097580,
"type": "counter",
"desc": "number of bt_engine cache hits"
},
"misses": {
"value": 14718983,
"type": "counter",
"desc": "number of bt_engine cache misses"
}
}
```
<img width="930" height="816" alt="multi_db_btree" src="https://github.com/user-attachments/assets/ff55f72c-edf9-4d39-abd0-d81292cd8121" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298234945
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning
+% them. This is to quickly react and make room for potentially new entries
+% being added without having to for a whole interval
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+ insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+ Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+ case get_cache(Key) of
+ #cache{tid = Tid, max_size = Max} ->
+ case ets:info(Tid, memory) < Max of
+ true ->
+ case ets:insert_new(Tid, {Key, Priority1, Term}) of
+ true ->
+ true;
+ false ->
+ bump_usage(Tid, Key),
+ false
+ end;
+ false ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+ false
+ end;
+ undefined ->
+ false
+ end.
+
+reset(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} -> reset_usage(Tid, Key);
+ undefined -> true
+ end.
+
+lookup(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} ->
+ case ets:lookup_element(Tid, Key, 3, undefined) of
+ undefined ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+ undefined;
+ Term ->
+ bump_usage(Tid, Key),
+ couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+ Term
+ end;
+ undefined ->
+ undefined
+ end.
+
+status() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+ {Sizes, Memory} = lists:unzip(SizeMem),
+ #{
+ size => lists:sum(Sizes),
+ memory => lists:sum(Memory)
+ };
+ undefined ->
+ #{}
+ end.
+
+% Supervisor helper functions
+
+create_tables() ->
+ BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+ persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+ [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+ Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+ #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+ {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ Cache = #cache{tid = Tid} = element(N, Caches),
+ ets:delete_all_objects(Tid),
+ loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+ decay(Tid),
+ Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+ clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+ remove_dead(Cache),
+ WaitLeft = max(10, Next - now_msec()),
+ timer:sleep(WaitLeft),
+ loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+ case now_msec() < Until of
+ true ->
+ case should_clean(Cache) of
+ true ->
+ ets:match_delete(Tid, {'_', 0, '_'});
+ false ->
+ timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+ clean(Cache, Until)
+ end;
+ false ->
+ false
+ end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+ ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+ All = pids(Tid),
+ Alive = sets:filter(fun is_process_alive/1, All),
+ Dead = sets:subtract(All, Alive),
+ % In OTP 27+ use sets:map/2
+ Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+ sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+ Acc = couch_util:new_set(),
+ try
+ ets:foldl(fun pids_fold/2, Acc, Tid)
+ catch
+ error:badarg -> Acc
+ end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+ sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+ Acc.
+
+% Private helper functions
+
+new() ->
+ Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+ Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+ % Some per-table overhead for the table metadata
+ Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+ #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Index = erlang:phash2(Term, tuple_size(Caches)),
+ #cache{} = element(1 + Index, Caches);
+ undefined ->
+ undefined
+ end.
+
+bump_usage(Tid, Key) ->
+ % We're updating the second field incrementing it by 1 and clamping it
+ % at ?MAX_PRIORITY. We don't set the default for the update_counter
+ % specifically to avoid creating bogus entries just from updating the
+ % counter, so expect the error:badarg here sometimes.
+ UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+reset_usage(Tid, Key) ->
+ % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
Review Comment:
Nice approach 💯
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "mikerhodes (via GitHub)" <gi...@apache.org>.
mikerhodes commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3200592575
> ...the performance gain is quite modest (except for changes) to warrant the extra complexity and failure modes (on the flipside it does validate our deliberate reliance on the OS page cache for this so far).
I find the 33% improvement in single doc reads is significant. It's definitely in line with, if not slightly greater than, what I'd expect for this kind of change. It's also where I'd expect to see the greatest benefit of this change --- like you @rnewson I'm somewhat surprised by the `_changes` seeing an 80% increase in reads per sec (I also wonder why `_all_docs` is so much higher).
One other interesting thing in general with these numbers is how many more docs we read per second with the `_all_docs` call --- 49k/s vs 4k/s. Presumably the random reads are going through an `r=2` read, whereas `_all_docs` include_docs reads are going to the local shard that was chosen for `_all_docs` using an implicit `r=1`? Feels like there's something important in that disparity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3202493282
I wonder also if supplying the fadvise hints as we read data might give a boost without adding an in-vm cache. (e.g, `WILLNEED` for non-leaf btree nodes and `DONTNEED` for all leaf data (doc bodies, attachment chunks).
for _all_docs and _changes traversals (which under the covers are walking the by_id and by_seq trees in their respective orders) I wonder why caching does help. We "should" read the btree optimally there, we're not skipping around the btree. there's no seek penalty these days, etc. It might be a code wart we could solve. Or, perhaps, we could cache in-process more safely/easily. repeated calls to pread_term with the same offset by the same process that has an open #db record will yield the same result each time (until a reopen).
I'd focus on benchmarking and more statistics, so we know what we're comparing. We'd want to know how many times we read the same term from disk in the current codebase versus how often we avoided it with the cache.
Finally, ets_lru exists and has both size and time expiration. why invent a new way?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298179724
##########
rel/overlay/etc/default.ini:
##########
@@ -130,6 +130,26 @@ view_index_dir = {{view_index_dir}}
; downgrade or even open the databases in question.
;prohibit_downgrade = true
+[bt_engine_cache]
+; Memory used for btree engine cache. This is a cache for top levels of
+; database btrees (id tree, seq tree) and a few terms from the db header. Value
+; is in bytes.
+;max_size = 67108864
+;
+; Items not accessed in a while get are eventually evicted. However, if the
+; memory used is below this percentage, then even the unused items are left in
+; the cache. The trade-off here is when a new workload starts, it may find the
+; cache with some stale items during the first few seconds and not be able to
+; insert its entries in.
+;leave_percent = 30
+;
+; Cache database btree nodes up to this depth only. Depth starts at 1 at root,
+; then 2 the next level down, and so on. Only intermediate (pointer) nodes will
Review Comment:
maybe add an `at`
Depth starts at 1 at root, then 2 *at* the next level down, and so on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307672524
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+ insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+ Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+ case get_cache(Key) of
+ #cache{tid = Tid, max_size = Max} ->
+ case ets:info(Tid, memory) < Max of
+ true ->
+ case ets:insert_new(Tid, {Key, Priority1, Term}) of
+ true ->
+ true;
+ false ->
+ bump_usage(Tid, Key),
+ false
+ end;
+ false ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+ false
+ end;
+ undefined ->
+ false
+ end.
+
+reset(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} -> reset_usage(Tid, Key);
+ undefined -> true
+ end.
+
+lookup(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} ->
+ case ets:lookup_element(Tid, Key, 3, undefined) of
+ undefined ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+ undefined;
+ Term ->
+ bump_usage(Tid, Key),
+ couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+ Term
+ end;
+ undefined ->
+ undefined
+ end.
+
+status() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+ {Sizes, Memory} = lists:unzip(SizeMem),
+ #{
+ size => lists:sum(Sizes),
+ memory => lists:sum(Memory)
+ };
+ undefined ->
+ #{}
+ end.
+
+% Supervisor helper functions
+
+create_tables() ->
+ BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+ persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+ [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+ Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+ #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+ {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ Cache = #cache{tid = Tid} = element(N, Caches),
+ ets:delete_all_objects(Tid),
+ loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+ decay(Tid),
+ Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+ clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+ remove_dead(Cache),
+ WaitLeft = max(10, Next - now_msec()),
+ timer:sleep(WaitLeft),
+ loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+ case now_msec() < Until of
+ true ->
+ case should_clean(Cache) of
+ true ->
+ ets:match_delete(Tid, {'_', 0, '_'});
+ false ->
+ timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+ clean(Cache, Until)
+ end;
+ false ->
+ false
+ end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+ ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+ All = pids(Tid),
+ Alive = sets:filter(fun is_process_alive/1, All),
+ Dead = sets:subtract(All, Alive),
+ % In OTP 27+ use sets:map/2
+ Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+ sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+ Acc = couch_util:new_set(),
+ try
+ ets:foldl(fun pids_fold/2, Acc, Tid)
+ catch
+ error:badarg -> Acc
+ end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+ sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+ Acc.
+
+% Private helper functions
+
+new() ->
+ Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+ Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+ % Some per-table overhead for the table metadata
+ Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+ #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Index = erlang:phash2(Term, tuple_size(Caches)),
+ #cache{} = element(1 + Index, Caches);
+ undefined ->
+ undefined
+ end.
+
+bump_usage(Tid, Key) ->
+ % We're updating the second field incrementing it by 1 and clamping it
+ % at ?MAX_PRIORITY. We don't set the default for the update_counter
+ % specifically to avoid creating bogus entries just from updating the
+ % counter, so expect the error:badarg here sometimes.
+ UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+reset_usage(Tid, Key) ->
+ % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+ % subtract that and clamp it at 0. Do not provide a default since if an
+ % entry is missing we don't want to create a bogus one from this operation.
+ UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+info(#cache{tid = Tid}) ->
+ Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+ Size = ets:info(Tid, size),
+ {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+ Head = {'$1', '$2', '$3'},
+ Cond = [{'>', '$2', 0}],
+ Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+ ets:select_replace(Tid, [{Head, Cond, Result}]).
+
+shard_count() ->
+ erlang:system_info(schedulers).
+
+wait_interval(Interval) ->
+ Jitter = rand:uniform(max(1, round(Interval * 0.5))),
Review Comment:
Good idea
##########
rel/overlay/etc/default.ini:
##########
@@ -130,6 +130,26 @@ view_index_dir = {{view_index_dir}}
; downgrade or even open the databases in question.
;prohibit_downgrade = true
+[bt_engine_cache]
+; Memory used for btree engine cache. This is a cache for top levels of
+; database btrees (id tree, seq tree) and a few terms from the db header. Value
+; is in bytes.
+;max_size = 67108864
+;
+; Items not accessed in a while are eventually evicted. However, if the memory
+; used is below this percentage, then even the unused items are left in the
+; cache. The trade-off here is when a new workload starts, it may find the
+; cache with some stale items during the first few seconds and not be able to
+; insert its entries in.
+;leave_percent = 30
+;
+; Cache database btree nodes up to this depth only. Depth starts at 1 at root,
+; then at 2 the next level down, and so on. Only intermediate (pointer) nodes
+; will be cached, those are the nodes which point to other nodes, as opposed to
+; leaf key-value nodes, which hold revision trees. To disabe db btree node
Review Comment:
thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva merged PR #5625:
URL: https://github.com/apache/couchdb/pull/5625
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3202941355
> Finally, ets_lru exists and has both size and time expiration. why invent a new way?
I looked at ets_lru but opted to go with something which has O(1) reads and inserts. And it's kind of a mix of a frequency based lru with a delayed time component, not a plain LRU. For an LRU the update would need to bump a sorted ets or list of some sort, while with a frequency counter we can rely on an ets:update_counter/3 call then let the scanners for each sharded ets do the work. I mostly thought of how we do the rate limiting tables for couch_replicator and the couch_server sharding.
> I'd focus on benchmarking and more statistics, so we know what we're comparing. We'd want to know how many times we read the same term from disk in the current codebase versus how often we avoided it with the cache.
Yeah, definitely will try to run more benchmarks. These are ones were run on my laptop, I'll see if I can get my hands on a real cluster. I think some of the stats there already tell a good story for instance for the benchmark above we get:
```
> % http $DB/_node/_local/_stats/couchdb/bt_engine_cache
{
"full": {
"desc": "number of times bt_engine cache was full",
"type": "counter",
"value": 0
},
"hits": {
"desc": "number of bt_engine cache hits",
"type": "counter",
"value": 233296
},
"misses": {
"desc": "number of bt_engine cache misses",
"type": "counter",
"value": 7343
}
}
```
That is caching level 2 btree tops we already get 233296 kp_node hits, that's how many of them we can fetch from the cache not from the disk.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3212432680
I made two more improvements inspired by some discussions with Mike and Bob:
* Added a priority field to the insert call. So the depth 1 nodes get a higher initial priority. Priority higher than 1 is basically the initial usage count, so the entry would survive a few more cleanup intervals.
* When b-trees are updated there is an effect that a root node that was in use would now become abandoned suddenly. To help evict it faster, since we know how the b-tree get updated, we can reset its usage count to 0 (not remove it because some clients may still be reading from the old snapshot, otherwise the entry stays at least until the next cleanup interval but no more than that)
* If the cache is not very full previously we just leave the entries in there case they may be reused if there is no pressure on the cache. However with a longer interval of 1-2 seconds a new set of updates would be prevented from using the cache for a bit. So the change is that now if we didn't do any cleanup (we're half empty) then instead of sleeping for 1-2 seconds we checking the cache size every few hundred milliseconds, so we can react faster to new entries wanting to use the cache
* To avoid the cache users all calling is_process_alive(Fd), which is still a process signal send/receive I made the cleaner processes do it periodically. They gather the processes in a set then delete the dead ones, that gets the send/recv signal path out of main data path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298179724
##########
rel/overlay/etc/default.ini:
##########
@@ -130,6 +130,26 @@ view_index_dir = {{view_index_dir}}
; downgrade or even open the databases in question.
;prohibit_downgrade = true
+[bt_engine_cache]
+; Memory used for btree engine cache. This is a cache for top levels of
+; database btrees (id tree, seq tree) and a few terms from the db header. Value
+; is in bytes.
+;max_size = 67108864
+;
+; Items not accessed in a while get are eventually evicted. However, if the
+; memory used is below this percentage, then even the unused items are left in
+; the cache. The trade-off here is when a new workload starts, it may find the
+; cache with some stale items during the first few seconds and not be able to
+; insert its entries in.
+;leave_percent = 30
+;
+; Cache database btree nodes up to this depth only. Depth starts at 1 at root,
+; then 2 the next level down, and so on. Only intermediate (pointer) nodes will
Review Comment:
maybe add an `at`
Depth starts at 1 at root, then 2 `at` the next level down, and so on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298168947
##########
rel/overlay/etc/default.ini:
##########
@@ -130,6 +130,26 @@ view_index_dir = {{view_index_dir}}
; downgrade or even open the databases in question.
;prohibit_downgrade = true
+[bt_engine_cache]
+; Memory used for btree engine cache. This is a cache for top levels of
+; database btrees (id tree, seq tree) and a few terms from the db header. Value
+; is in bytes.
+;max_size = 67108864
+;
+; Items not accessed in a while get are eventually evicted. However, if the
Review Comment:
I think the verb `get` is not needed there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2308008760
##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,117 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+ {
+ foreach,
+ fun setup/0,
+ fun teardown/1,
+ [
+ ?TDEF_FE(t_created),
+ ?TDEF_FE(t_insert_and_lookup),
+ ?TDEF_FE(t_decay_and_removal_works, 10),
+ ?TDEF_FE(t_pid_cleanup_works, 10)
+ ]
+ }.
+
+setup() ->
+ Ctx = test_util:start_applications([config]),
+ couch_bt_engine_cache:create_tables(),
+ N = tuple_size(persistent_term:get(?PTERM_KEY)),
Review Comment:
Worked out pretty well. I also renamed it from `status/0` to `info/0` like you suggested, that's a more consistent name.
```
> couch_bt_engine_cache:info().
#{memory => 23103896,size => 11618,full => 0,hits => 4734895,
misses => 784110,max_memory => 71204864,shard_count => 16}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3234459217
> Great work. I have couple of nitpicks, otherwise looks really good. I apologize about my mistake to use another my account initially (@iilyak-ibm is also me).
No worries at all. Thank you for taking a look and for the nice suggestions. I updated the PR accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3208291841
I found a 12 node cluster (80 cpus, 128GB ram per node) and ran a high concurrency benchmark with 10k virtual users reading, updating and creating documents.
To generate the load used k6 with this script: https://gist.github.com/nickva/d5312248635534315e76d25b31a86faa
- starts with a 1M docs
- then 10k users connect to the cluster and each do:
* read a random document from the initial 1M
* update the document
* insert a new document (with a random uuid)
- run for 10 minutes after initial setup and report stats
Results for main:
```
HTTP
http_req_duration..............: avg=1.1s min=5.01ms med=1.09s max=3.81s p(90)=2.32s p(95)=2.52s
{ expected_response:true }...: avg=1.1s min=5.01ms med=1.09s max=3.81s p(90)=2.32s p(95)=2.52s
{ name:bulk_docs }...........: avg=188.58ms min=124.78ms med=188.34ms max=517.28ms p(90)=204.56ms p(95)=208.42ms
{ name:get_doc }.............: avg=40.97ms min=5.01ms med=39.47ms max=3.49s p(90)=52.12ms p(95)=55.7ms
{ name:post_doc }............: avg=1.34s min=29.05ms med=1.25s max=3.81s p(90)=2.13s p(95)=2.35s
{ name:put_doc }.............: avg=1.92s min=75.03ms med=2.01s max=3.81s p(90)=2.58s p(95)=2.73s
http_req_failed................: 0.13% 5617 out of 4183555
http_reqs......................: 4183555 4969.270526/s
EXECUTION
iteration_duration.............: avg=4.31s min=1.31s med=4.34s max=8.35s p(90)=4.94s p(95)=5.12s
iterations.....................: 1394184 1656.026384/s
vus............................: 390 min=0 max=10000
vus_max........................: 10000 min=10000 max=10000
NETWORK
data_received..................: 2.6 GB 3.1 MB/s
data_sent......................: 721 MB 856 kB/s
running (14m01.9s), 00000/10000 VUs, 1394184 complete and 0 interrupted iterations
```
Results with the bt cache
```
HTTP
http_req_duration..............: avg=743.25ms min=3.62ms med=896.05ms max=3.4s p(90)=1.41s p(95)=1.56s
{ expected_response:true }...: avg=742.56ms min=3.62ms med=895.55ms max=3.03s p(90)=1.41s p(95)=1.56s
{ name:bulk_docs }...........: avg=112.65ms min=93.67ms med=109.18ms max=424.51ms p(90)=117.06ms p(95)=119.81ms
{ name:get_doc }.............: avg=13.6ms min=3.62ms med=11.68ms max=2.35s p(90)=15.95ms p(95)=17.6ms
{ name:post_doc }............: avg=1.09s min=16.85ms med=1.06s max=2.98s p(90)=1.49s p(95)=1.64s
{ name:put_doc }.............: avg=1.11s min=117.82ms med=1.08s max=3.4s p(90)=1.51s p(95)=1.65s
http_req_failed................: 0.09% 5148 out of 5580982
http_reqs......................: 5580982 7284.775804/s
EXECUTION
iteration_duration.............: avg=3.23s min=1.63s med=3.19s max=6.92s p(90)=3.8s p(95)=3.99s
iterations.....................: 1859993 2427.822201/s
vus............................: 959 min=0 max=10000
vus_max........................: 10000 min=10000 max=10000
NETWORK
data_received..................: 3.5 GB 4.5 MB/s
data_sent......................: 927 MB 1.2 MB/s
running (12m46.1s), 00000/10000 VUs, 1859993 complete and 0 interrupted iterations
```
<img width="792" height="1279" alt="btree_bench" src="https://github.com/user-attachments/assets/89bebf02-114d-4b67-84b1-e1f65a142bc8" />
Based on the metrics:
* We can do 9.6k vs http 7.3k requests / sec in favor of the btree cache.
* In the read/write metrics the first grey bump is the initial bulk load. We can do 18k vs 12k inserts /sec in favor of the cache. During the main part of the benchmark 10k db reads vs 8k db reads also in favor of the btree-cache.
* In the latency graph all latencies, but especially the higher ones p90, p99 are improved (1.4 sec vs 2.2 sec).
* Memory usage is a bit better with the cache probably because we pile up more client processes.
After the benchmark ran I queried the stats and got:
```
curl http://$node/_stats/couchdb/bt_engine_cache
{
"full": {
"value": 0,
"type": "counter",
"desc": "number of times bt_engine cache was full"
},
"hits": {
"value": 24371805,
"type": "counter",
"desc": "number of bt_engine cache hits"
},
"misses": {
"value": 374470,
"type": "counter",
"desc": "number of bt_engine cache misses"
}
}
```
We saved 24M btree block reads and didn't hit the full limit/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "big-r81 (via GitHub)" <gi...@apache.org>.
big-r81 commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3209048118
Is there a break even point where things get worser? Can you run those tests on smaller clusters too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3209109828
> Nice work! Is there a break even point where things get worser? Can you run those tests on smaller clusters too?
I'll try to find a smaller cluster but sometime setting it up my not work as well as our dedicated perf cluster. I could run some more benchmarks on my ubuntu laptop, that could be an example of a small machine.
> Is there a break even point where things get worser?
I'd imagine if we shuffled every node through the cache, things would get worse. As we'd fill it up with nodes used only a times. We see the distribution of counts in one of the above comments, at level 2 or 3 we may have only 40-50 kp nodes and one level lower there are thousands. The `full` metric can sort of help determine if the cache size may need to be bumped or if the depth may need to be reduced to make it more effective.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3206587494
fair comment on ets_lru. this seems a good thing to pursue given the hit rate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2308265489
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+ insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+ Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+ case get_cache(Key) of
+ #cache{tid = Tid, max_size = Max} ->
+ case ets:info(Tid, memory) < Max of
+ true ->
+ case ets:insert_new(Tid, {Key, Priority1, Term}) of
+ true ->
+ true;
+ false ->
+ bump_usage(Tid, Key),
+ false
+ end;
+ false ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+ false
+ end;
+ undefined ->
+ false
+ end.
+
+reset(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} -> reset_usage(Tid, Key);
+ undefined -> true
+ end.
+
+lookup(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} ->
+ case ets:lookup_element(Tid, Key, 3, undefined) of
+ undefined ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+ undefined;
+ Term ->
+ bump_usage(Tid, Key),
+ couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+ Term
+ end;
+ undefined ->
+ undefined
+ end.
+
+status() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+ {Sizes, Memory} = lists:unzip(SizeMem),
+ #{
+ size => lists:sum(Sizes),
+ memory => lists:sum(Memory)
+ };
+ undefined ->
+ #{}
+ end.
+
+% Supervisor helper functions
+
+create_tables() ->
+ BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+ persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+ [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+ Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+ #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+ {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ Cache = #cache{tid = Tid} = element(N, Caches),
+ ets:delete_all_objects(Tid),
+ loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+ decay(Tid),
+ Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+ clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+ remove_dead(Cache),
+ WaitLeft = max(10, Next - now_msec()),
+ timer:sleep(WaitLeft),
+ loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+ case now_msec() < Until of
+ true ->
+ case should_clean(Cache) of
+ true ->
+ ets:match_delete(Tid, {'_', 0, '_'});
+ false ->
+ timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+ clean(Cache, Until)
+ end;
+ false ->
+ false
+ end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+ ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+ All = pids(Tid),
+ Alive = sets:filter(fun is_process_alive/1, All),
+ Dead = sets:subtract(All, Alive),
+ % In OTP 27+ use sets:map/2
+ Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+ sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+ Acc = couch_util:new_set(),
+ try
+ ets:foldl(fun pids_fold/2, Acc, Tid)
+ catch
+ error:badarg -> Acc
+ end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+ sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+ Acc.
+
+% Private helper functions
+
+new() ->
+ Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+ Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+ % Some per-table overhead for the table metadata
+ Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+ #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Index = erlang:phash2(Term, tuple_size(Caches)),
+ #cache{} = element(1 + Index, Caches);
+ undefined ->
+ undefined
+ end.
+
+bump_usage(Tid, Key) ->
+ % We're updating the second field incrementing it by 1 and clamping it
+ % at ?MAX_PRIORITY. We don't set the default for the update_counter
+ % specifically to avoid creating bogus entries just from updating the
+ % counter, so expect the error:badarg here sometimes.
+ UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+reset_usage(Tid, Key) ->
+ % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+ % subtract that and clamp it at 0. Do not provide a default since if an
+ % entry is missing we don't want to create a bogus one from this operation.
+ UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+info(#cache{tid = Tid}) ->
+ Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+ Size = ets:info(Tid, size),
+ {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+ Head = {'$1', '$2', '$3'},
+ Cond = [{'>', '$2', 0}],
+ Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+ ets:select_replace(Tid, [{Head, Cond, Result}]).
Review Comment:
> Hot code loading shouldn't be an issue.
It will not be an issue since the beam file with fun2ms and without is identical.
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+ insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+ Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+ case get_cache(Key) of
+ #cache{tid = Tid, max_size = Max} ->
+ case ets:info(Tid, memory) < Max of
+ true ->
+ case ets:insert_new(Tid, {Key, Priority1, Term}) of
+ true ->
+ true;
+ false ->
+ bump_usage(Tid, Key),
+ false
+ end;
+ false ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+ false
+ end;
+ undefined ->
+ false
+ end.
+
+reset(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} -> reset_usage(Tid, Key);
+ undefined -> true
+ end.
+
+lookup(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} ->
+ case ets:lookup_element(Tid, Key, 3, undefined) of
+ undefined ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+ undefined;
+ Term ->
+ bump_usage(Tid, Key),
+ couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+ Term
+ end;
+ undefined ->
+ undefined
+ end.
+
+status() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+ {Sizes, Memory} = lists:unzip(SizeMem),
+ #{
+ size => lists:sum(Sizes),
+ memory => lists:sum(Memory)
+ };
+ undefined ->
+ #{}
+ end.
+
+% Supervisor helper functions
+
+create_tables() ->
+ BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+ persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+ [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+ Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+ #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+ {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ Cache = #cache{tid = Tid} = element(N, Caches),
+ ets:delete_all_objects(Tid),
+ loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+ decay(Tid),
+ Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+ clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+ remove_dead(Cache),
+ WaitLeft = max(10, Next - now_msec()),
+ timer:sleep(WaitLeft),
+ loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+ case now_msec() < Until of
+ true ->
+ case should_clean(Cache) of
+ true ->
+ ets:match_delete(Tid, {'_', 0, '_'});
+ false ->
+ timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+ clean(Cache, Until)
+ end;
+ false ->
+ false
+ end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+ ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+ All = pids(Tid),
+ Alive = sets:filter(fun is_process_alive/1, All),
+ Dead = sets:subtract(All, Alive),
+ % In OTP 27+ use sets:map/2
+ Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+ sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+ Acc = couch_util:new_set(),
+ try
+ ets:foldl(fun pids_fold/2, Acc, Tid)
+ catch
+ error:badarg -> Acc
+ end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+ sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+ Acc.
+
+% Private helper functions
+
+new() ->
+ Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+ Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+ % Some per-table overhead for the table metadata
+ Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+ #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Index = erlang:phash2(Term, tuple_size(Caches)),
+ #cache{} = element(1 + Index, Caches);
+ undefined ->
+ undefined
+ end.
+
+bump_usage(Tid, Key) ->
+ % We're updating the second field incrementing it by 1 and clamping it
+ % at ?MAX_PRIORITY. We don't set the default for the update_counter
+ % specifically to avoid creating bogus entries just from updating the
+ % counter, so expect the error:badarg here sometimes.
+ UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+reset_usage(Tid, Key) ->
+ % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+ % subtract that and clamp it at 0. Do not provide a default since if an
+ % entry is missing we don't want to create a bogus one from this operation.
+ UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+info(#cache{tid = Tid}) ->
+ Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+ Size = ets:info(Tid, size),
+ {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+ Head = {'$1', '$2', '$3'},
+ Cond = [{'>', '$2', 0}],
+ Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+ ets:select_replace(Tid, [{Head, Cond, Result}]).
Review Comment:
> Hot code loading shouldn't be an issue.
It will not be an issue since the beam file with fun2ms and without is identical.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307671142
##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,117 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+ {
+ foreach,
+ fun setup/0,
+ fun teardown/1,
+ [
+ ?TDEF_FE(t_created),
+ ?TDEF_FE(t_insert_and_lookup),
+ ?TDEF_FE(t_decay_and_removal_works, 10),
+ ?TDEF_FE(t_pid_cleanup_works, 10)
+ ]
+ }.
+
+setup() ->
+ Ctx = test_util:start_applications([config]),
+ couch_bt_engine_cache:create_tables(),
+ N = tuple_size(persistent_term:get(?PTERM_KEY)),
Review Comment:
That could work, good idea!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3200915315
> I'd like to see a bigger benchmark before deciding but the performance gain is quite modest (except for changes) to warrant the extra complexity and failure modes (on the flipside it does validate our deliberate reliance on the OS page cache for this so far).
Definitely would need a better benchmark, this was a quick and dirty one mainly to mainly show how many reads we saved with a mix of operations on 100k docs. Since I didn't do any page cache evicting all the block were in the page cache, so it validates our use of the page but it also shows that there is quite an overhead in going through all the IO layers. And since we're aware of the b-tree/header structure at this layer we can make better decisions on what to cache.
> I'd be curious to see the same benchmarks but with the code modified to not expire the cached items so aggressively (to find the upper bound to how much this could improve things).
Maybe we can make the "don't clean up if fill ratio is below 1/4" setting configurable, and with a large enough cache and a value like 3/4 we could look at the effects.
> It would also be useful to know how deep the btrees are in question (or some other indication of the cache hit/miss ratio to give us context when assessing the throughput figures). As for why changes gets such a boost, that needs some thinking.
Good idea, maybe we'd want a diagnostic get depth traversal API to get an idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3209915418
these results are really good, so I'm ok with the added complexity.
As for what depth to cache to; we are forced to choose a level because of the eviction mechanism (which was chosen for speed of execution I think). If, instead, we evicted more intelligently, we could just use a memory bound (which ops could move up/down if necessary). that is, try to evict non-leaf nodes we haven't hit recently. The Java caching library Caffeine operates this way and has some research on how optimal it is (https://github.com/ben-manes/caffeine/wiki/Efficiency : Window TinyLfu). An eviction scheme like this takes the guesswork out of things, and would basically tune itself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298163717
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning
+% them. This is to quickly react and make room for potentially new entries
+% being added without having to for a whole interval
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+ insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+ Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+ case get_cache(Key) of
+ #cache{tid = Tid, max_size = Max} ->
+ case ets:info(Tid, memory) < Max of
+ true ->
+ case ets:insert_new(Tid, {Key, Priority1, Term}) of
+ true ->
+ true;
+ false ->
+ bump_usage(Tid, Key),
+ false
+ end;
+ false ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+ false
+ end;
+ undefined ->
+ false
+ end.
+
+reset(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} -> reset_usage(Tid, Key);
+ undefined -> true
+ end.
+
+lookup(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} ->
+ case ets:lookup_element(Tid, Key, 3, undefined) of
+ undefined ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+ undefined;
+ Term ->
+ bump_usage(Tid, Key),
+ couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+ Term
+ end;
+ undefined ->
+ undefined
+ end.
+
+status() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+ {Sizes, Memory} = lists:unzip(SizeMem),
+ #{
+ size => lists:sum(Sizes),
+ memory => lists:sum(Memory)
+ };
+ undefined ->
+ #{}
+ end.
+
+% Supervisor helper functions
+
+create_tables() ->
+ BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+ persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+ [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+ Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+ #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+ {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ Cache = #cache{tid = Tid} = element(N, Caches),
+ ets:delete_all_objects(Tid),
+ loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+ decay(Tid),
+ Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+ clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+ remove_dead(Cache),
+ WaitLeft = max(10, Next - now_msec()),
+ timer:sleep(WaitLeft),
+ loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+ case now_msec() < Until of
+ true ->
+ case should_clean(Cache) of
+ true ->
+ ets:match_delete(Tid, {'_', 0, '_'});
+ false ->
+ timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+ clean(Cache, Until)
+ end;
+ false ->
+ false
+ end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+ ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+ All = pids(Tid),
+ Alive = sets:filter(fun is_process_alive/1, All),
+ Dead = sets:subtract(All, Alive),
+ % In OTP 27+ use sets:map/2
+ Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+ sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+ Acc = couch_util:new_set(),
+ try
+ ets:foldl(fun pids_fold/2, Acc, Tid)
+ catch
+ error:badarg -> Acc
+ end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+ sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+ Acc.
+
+% Private helper functions
+
+new() ->
+ Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+ Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+ % Some per-table overhead for the table metadata
+ Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+ #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Index = erlang:phash2(Term, tuple_size(Caches)),
+ #cache{} = element(1 + Index, Caches);
+ undefined ->
+ undefined
+ end.
+
+bump_usage(Tid, Key) ->
+ % We're updating the second field incrementing it by 1 and clamping it
+ % at ?MAX_PRIORITY. We don't set the default for the update_counter
+ % specifically to avoid creating bogus entries just from updating the
+ % counter, so expect the error:badarg here sometimes.
+ UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+reset_usage(Tid, Key) ->
+ % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+ % subtract that and clamp it at 0. Do not provide a default since if an
+ % entry is missing we don't want to creat a bogus from this operation.
Review Comment:
A typo `creat` -> `create`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298383419
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning
+% them. This is to quickly react and make room for potentially new entries
+% being added without having to for a whole interval
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+ insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+ Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+ case get_cache(Key) of
+ #cache{tid = Tid, max_size = Max} ->
+ case ets:info(Tid, memory) < Max of
+ true ->
+ case ets:insert_new(Tid, {Key, Priority1, Term}) of
+ true ->
+ true;
+ false ->
+ bump_usage(Tid, Key),
+ false
+ end;
+ false ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+ false
+ end;
+ undefined ->
+ false
+ end.
+
+reset(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} -> reset_usage(Tid, Key);
+ undefined -> true
+ end.
+
+lookup(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} ->
+ case ets:lookup_element(Tid, Key, 3, undefined) of
+ undefined ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+ undefined;
+ Term ->
+ bump_usage(Tid, Key),
+ couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+ Term
+ end;
+ undefined ->
+ undefined
+ end.
+
+status() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+ {Sizes, Memory} = lists:unzip(SizeMem),
+ #{
+ size => lists:sum(Sizes),
+ memory => lists:sum(Memory)
+ };
+ undefined ->
+ #{}
+ end.
+
+% Supervisor helper functions
+
+create_tables() ->
+ BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+ persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+ [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+ Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+ #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+ {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ Cache = #cache{tid = Tid} = element(N, Caches),
+ ets:delete_all_objects(Tid),
+ loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+ decay(Tid),
+ Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+ clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+ remove_dead(Cache),
+ WaitLeft = max(10, Next - now_msec()),
+ timer:sleep(WaitLeft),
+ loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+ case now_msec() < Until of
+ true ->
+ case should_clean(Cache) of
+ true ->
+ ets:match_delete(Tid, {'_', 0, '_'});
+ false ->
+ timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+ clean(Cache, Until)
+ end;
+ false ->
+ false
+ end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+ ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+ All = pids(Tid),
+ Alive = sets:filter(fun is_process_alive/1, All),
+ Dead = sets:subtract(All, Alive),
+ % In OTP 27+ use sets:map/2
+ Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+ sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+ Acc = couch_util:new_set(),
+ try
+ ets:foldl(fun pids_fold/2, Acc, Tid)
+ catch
+ error:badarg -> Acc
+ end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+ sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+ Acc.
+
+% Private helper functions
+
+new() ->
+ Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+ Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+ % Some per-table overhead for the table metadata
+ Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+ #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Index = erlang:phash2(Term, tuple_size(Caches)),
+ #cache{} = element(1 + Index, Caches);
+ undefined ->
+ undefined
+ end.
+
+bump_usage(Tid, Key) ->
+ % We're updating the second field incrementing it by 1 and clamping it
+ % at ?MAX_PRIORITY. We don't set the default for the update_counter
+ % specifically to avoid creating bogus entries just from updating the
+ % counter, so expect the error:badarg here sometimes.
+ UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+reset_usage(Tid, Key) ->
+ % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+ % subtract that and clamp it at 0. Do not provide a default since if an
+ % entry is missing we don't want to creat a bogus from this operation.
Review Comment:
Thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298379535
##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,118 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+ {
+ foreach,
+ fun setup/0,
+ fun teardown/1,
+ [
+ ?TDEF_FE(t_created),
+ ?TDEF_FE(t_insert_and_lookup),
+ ?TDEF_FE(t_decay_and_removal_works, 10),
+ ?TDEF_FE(t_pid_cleanup_works, 10)
+ ]
+ }.
+
+setup() ->
+ Ctx = test_util:start_applications([config]),
+ couch_bt_engine_cache:create_tables(),
+ N = tuple_size(persistent_term:get(?PTERM_KEY)),
+ Pids = lists:map(
+ fun(I) ->
+ {ok, Pid} = couch_bt_engine_cache:start_link(I),
+ Pid
+ end,
+ lists:seq(1, N)
+ ),
+ {Ctx, Pids}.
+
+teardown({Ctx, [_ | _] = Pids}) ->
+ lists:foreach(
+ fun(Pid) ->
+ catch unlink(Pid),
+ exit(Pid, kill)
+ end,
+ Pids
+ ),
+ clear_tables(),
+ config:delete("bt_engine_cache", "max_size", false),
+ config:delete("bt_engine_cache", "leave_percent", false),
+ test_util:stop_applications(Ctx).
+
+clear_tables() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Fun = fun({cache, Tid, _}) -> ets:delete(Tid) end,
+ lists:foreach(Fun, tuple_to_list(Caches));
+ undefined ->
+ ok
+ end.
+
+t_created(_) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ N = erlang:system_info(schedulers),
+ ?assert(tuple_size(Caches) >= 1),
+ ?assertEqual(N, tuple_size(Caches)),
+ Stats = couch_bt_engine_cache:status(),
+ ?assertMatch(#{size := 0, memory := Mem} when is_integer(Mem), Stats).
+
+t_insert_and_lookup(_) ->
+ % Can only insert pids and positions
+ ?assertError(function_clause, couch_bt_engine_cache:insert(not_a_pid, 1, foo)),
+ ?assertError(function_clause, couch_bt_engine_cache:insert(self(), xyz, foo)),
+ ?assertMatch(#{size := 0}, couch_bt_engine_cache:status()),
+
+ ?assert(couch_bt_engine_cache:insert({pid, 42}, term)),
+ ?assertMatch(#{size := 1}, couch_bt_engine_cache:status()),
+
+ % Already inserted
+ ?assertNot(couch_bt_engine_cache:insert({pid, 42}, term)),
+
+ % Should be there when we look it up
+ ?assertEqual(term, couch_bt_engine_cache:lookup({pid, 42})),
+
+ % Other stuff we didn't insert should return undefined
+ ?assertEqual(undefined, couch_bt_engine_cache:lookup({pid, 43})).
+
+t_decay_and_removal_works(_) ->
+ config:set("bt_engine_cache", "leave_percent", "0", false),
+ Term = [foo, bar, baz, lists:seq(1, 100)],
+ [couch_bt_engine_cache:insert({pid, I}, Term) || I <- lists:seq(1, 10000)],
+ WaitFun = fun() ->
+ #{size := Size} = couch_bt_engine_cache:status(),
+ case Size > 0 of
+ true -> wait;
+ false -> ok
+ end
+ end,
+ test_util:wait(WaitFun, 7500),
Review Comment:
For this test we override the default timeout in `?TDEF_FE(t_decay_and_removal_works, 10),` the extra 10 is the 10 seconds timeout. It takes a bit longer to run but that seems better than mocking more things.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3216239859
> Nice work! Is there a break even point where things get worser? Can you run those tests on smaller clusters too?
@big-r81 I found a 3 node cluster with 64Gb of ram each. Since it's a smaller cluster I ran with Q=16.
```
const db = 'k6db'
const q = '16'
const batch_size = 1000;
const doc_size = 64;
const dbs = 1
const docs = 1000000;
```
With the same k6 script as above
#### MAIN ####
```
HTTP
http_req_duration..............: avg=1.52s min=4.5ms med=1.51s max=7.29s p(90)=3.02s p(95)=3.18s
{ expected_response:true }...: avg=1.52s min=4.5ms med=1.51s max=7.29s p(90)=3.02s p(95)=3.18s
{ name:bulk_docs }...........: avg=90.65ms min=72.39ms med=89.05ms max=543.13ms p(90)=100.43ms p(95)=103.92ms
{ name:get_doc }.............: avg=2.15s min=4.5ms med=2.65s max=7.29s p(90)=3.26s p(95)=3.39s
{ name:post_doc }............: avg=989.49ms min=13.98ms med=826.58ms max=3.35s p(90)=2.05s p(95)=2.33s
{ name:put_doc }.............: avg=1.44s min=10.49ms med=1.68s max=3.46s p(90)=2.51s p(95)=2.67s
http_req_failed................: 0.17% 5672 out of 3237326
http_reqs......................: 3237326 4353.793364/s
EXECUTION
iteration_duration.............: avg=5.59s min=1.07s med=6.41s max=9.72s p(90)=7.85s p(95)=8.06s
iterations.....................: 1078774 1450.81437/s
vus............................: 0 min=0 max=10000
vus_max........................: 10000 min=10000 max=10000
NETWORK
data_received..................: 2.1 GB 2.8 MB/s
data_sent......................: 596 MB 801 kB/s
running (12m23.6s), 00000/10000 VUs, 1078774 complete and 0 interrupted iterations
```
#### PR ####
```
HTTP
http_req_duration..............: avg=1.1s min=3.53ms med=1.07s max=4.71s p(90)=2s p(95)=2.16s
{ expected_response:true }...: avg=1.1s min=3.53ms med=1.07s max=4.71s p(90)=1.99s p(95)=2.16s
{ name:bulk_docs }...........: avg=74.84ms min=63.94ms med=72.57ms max=393.01ms p(90)=82.32ms p(95)=85.67ms
{ name:get_doc }.............: avg=624.54ms min=3.53ms med=664.09ms max=4.03s p(90)=1.24s p(95)=1.35s
{ name:post_doc }............: avg=1.12s min=13.98ms med=1.01s max=4.71s p(90)=1.9s p(95)=2.09s
{ name:put_doc }.............: avg=1.55s min=17.55ms med=1.65s max=4.67s p(90)=2.18s p(95)=2.29s
http_req_failed................: 0.14% 5896 out of 4179620
http_reqs......................: 4179620 5757.994289/s
EXECUTION
iteration_duration.............: avg=4.31s min=1.09s med=4.67s max=8.41s p(90)=5.25s p(95)=5.41s
iterations.....................: 1392872 1918.870381/s
vus............................: 1699 min=0 max=10000
vus_max........................: 10000 min=10000 max=10000
NETWORK
data_received..................: 2.7 GB 3.7 MB/s
data_sent......................: 738 MB 1.0 MB/s
running (12m05.9s), 00000/10000 VUs, 1392872 complete and 0 interrupted iterations
```
```
{
"full": {
"value": 0,
"type": "counter",
"desc": "number of times bt_engine cache was full"
},
"hits": {
"value": 10302337,
"type": "counter",
"desc": "number of bt_engine cache hits"
},
"misses": {
"value": 2212648,
"type": "counter",
"desc": "number of bt_engine cache misses"
}
}
```
It held up pretty well. The metric graphs show improvements in throughput, latency and rate of requests:
<img width="836" height="665" alt="btree_3node_cluster" src="https://github.com/user-attachments/assets/10cda272-2e28-4cf9-afc9-8cfd439a6b84" />
(The first grey bump is the bulk_load phase)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2308010224
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+ insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+ Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+ case get_cache(Key) of
+ #cache{tid = Tid, max_size = Max} ->
+ case ets:info(Tid, memory) < Max of
+ true ->
+ case ets:insert_new(Tid, {Key, Priority1, Term}) of
+ true ->
+ true;
+ false ->
+ bump_usage(Tid, Key),
+ false
+ end;
+ false ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+ false
+ end;
+ undefined ->
+ false
+ end.
+
+reset(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} -> reset_usage(Tid, Key);
+ undefined -> true
+ end.
+
+lookup(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} ->
+ case ets:lookup_element(Tid, Key, 3, undefined) of
+ undefined ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+ undefined;
+ Term ->
+ bump_usage(Tid, Key),
+ couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+ Term
+ end;
+ undefined ->
+ undefined
+ end.
+
+status() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+ {Sizes, Memory} = lists:unzip(SizeMem),
+ #{
+ size => lists:sum(Sizes),
+ memory => lists:sum(Memory)
+ };
+ undefined ->
+ #{}
+ end.
+
+% Supervisor helper functions
+
+create_tables() ->
+ BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+ persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+ [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+ Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+ #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+ {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ Cache = #cache{tid = Tid} = element(N, Caches),
+ ets:delete_all_objects(Tid),
+ loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+ decay(Tid),
+ Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+ clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+ remove_dead(Cache),
+ WaitLeft = max(10, Next - now_msec()),
+ timer:sleep(WaitLeft),
+ loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+ case now_msec() < Until of
+ true ->
+ case should_clean(Cache) of
+ true ->
+ ets:match_delete(Tid, {'_', 0, '_'});
+ false ->
+ timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+ clean(Cache, Until)
+ end;
+ false ->
+ false
+ end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+ ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+ All = pids(Tid),
+ Alive = sets:filter(fun is_process_alive/1, All),
+ Dead = sets:subtract(All, Alive),
+ % In OTP 27+ use sets:map/2
+ Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+ sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+ Acc = couch_util:new_set(),
+ try
+ ets:foldl(fun pids_fold/2, Acc, Tid)
+ catch
+ error:badarg -> Acc
+ end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+ sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+ Acc.
+
+% Private helper functions
+
+new() ->
+ Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+ Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+ % Some per-table overhead for the table metadata
+ Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+ #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Index = erlang:phash2(Term, tuple_size(Caches)),
+ #cache{} = element(1 + Index, Caches);
+ undefined ->
+ undefined
+ end.
+
+bump_usage(Tid, Key) ->
+ % We're updating the second field incrementing it by 1 and clamping it
+ % at ?MAX_PRIORITY. We don't set the default for the update_counter
+ % specifically to avoid creating bogus entries just from updating the
+ % counter, so expect the error:badarg here sometimes.
+ UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+reset_usage(Tid, Key) ->
+ % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+ % subtract that and clamp it at 0. Do not provide a default since if an
+ % entry is missing we don't want to create a bogus one from this operation.
+ UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+info(#cache{tid = Tid}) ->
+ Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+ Size = ets:info(Tid, size),
+ {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+ Head = {'$1', '$2', '$3'},
+ Cond = [{'>', '$2', 0}],
+ Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+ ets:select_replace(Tid, [{Head, Cond, Result}]).
Review Comment:
Worked out well I think, it's much clearer to read and understand the code. Hot code loading shouldn't be an issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307697680
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+ insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+ Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+ case get_cache(Key) of
+ #cache{tid = Tid, max_size = Max} ->
+ case ets:info(Tid, memory) < Max of
+ true ->
+ case ets:insert_new(Tid, {Key, Priority1, Term}) of
+ true ->
+ true;
+ false ->
+ bump_usage(Tid, Key),
+ false
+ end;
+ false ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+ false
+ end;
+ undefined ->
+ false
+ end.
+
+reset(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} -> reset_usage(Tid, Key);
+ undefined -> true
+ end.
+
+lookup(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} ->
+ case ets:lookup_element(Tid, Key, 3, undefined) of
+ undefined ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+ undefined;
+ Term ->
+ bump_usage(Tid, Key),
+ couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+ Term
+ end;
+ undefined ->
+ undefined
+ end.
+
+status() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+ {Sizes, Memory} = lists:unzip(SizeMem),
+ #{
+ size => lists:sum(Sizes),
+ memory => lists:sum(Memory)
+ };
+ undefined ->
+ #{}
+ end.
+
+% Supervisor helper functions
+
+create_tables() ->
+ BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+ persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+ [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+ Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+ #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+ {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ Cache = #cache{tid = Tid} = element(N, Caches),
+ ets:delete_all_objects(Tid),
+ loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+ decay(Tid),
+ Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+ clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+ remove_dead(Cache),
+ WaitLeft = max(10, Next - now_msec()),
+ timer:sleep(WaitLeft),
+ loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+ case now_msec() < Until of
+ true ->
+ case should_clean(Cache) of
+ true ->
+ ets:match_delete(Tid, {'_', 0, '_'});
+ false ->
+ timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+ clean(Cache, Until)
+ end;
+ false ->
+ false
+ end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+ ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+ All = pids(Tid),
+ Alive = sets:filter(fun is_process_alive/1, All),
+ Dead = sets:subtract(All, Alive),
+ % In OTP 27+ use sets:map/2
+ Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+ sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+ Acc = couch_util:new_set(),
+ try
+ ets:foldl(fun pids_fold/2, Acc, Tid)
+ catch
+ error:badarg -> Acc
+ end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+ sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+ Acc.
+
+% Private helper functions
+
+new() ->
+ Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+ Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+ % Some per-table overhead for the table metadata
+ Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+ #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Index = erlang:phash2(Term, tuple_size(Caches)),
+ #cache{} = element(1 + Index, Caches);
+ undefined ->
+ undefined
+ end.
+
+bump_usage(Tid, Key) ->
+ % We're updating the second field incrementing it by 1 and clamping it
+ % at ?MAX_PRIORITY. We don't set the default for the update_counter
+ % specifically to avoid creating bogus entries just from updating the
+ % counter, so expect the error:badarg here sometimes.
+ UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+reset_usage(Tid, Key) ->
+ % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+ % subtract that and clamp it at 0. Do not provide a default since if an
+ % entry is missing we don't want to create a bogus one from this operation.
+ UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+info(#cache{tid = Tid}) ->
+ Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+ Size = ets:info(Tid, size),
+ {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+ Head = {'$1', '$2', '$3'},
+ Cond = [{'>', '$2', 0}],
+ Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+ ets:select_replace(Tid, [{Head, Cond, Result}]).
Review Comment:
That would work, not bad idea, but typically we have not been using the fun2ms parse transform before (only in a few tests before). Also, would this interfere with hot code loading if we ever have to do it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298191580
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning
+% them. This is to quickly react and make room for potentially new entries
+% being added without having to for a whole interval
Review Comment:
missing verb `wait`
being added without having to `wait` for a whole interval
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak-ibm (via GitHub)" <gi...@apache.org>.
iilyak-ibm commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298182015
##########
rel/overlay/etc/default.ini:
##########
@@ -130,6 +130,26 @@ view_index_dir = {{view_index_dir}}
; downgrade or even open the databases in question.
;prohibit_downgrade = true
+[bt_engine_cache]
+; Memory used for btree engine cache. This is a cache for top levels of
+; database btrees (id tree, seq tree) and a few terms from the db header. Value
+; is in bytes.
+;max_size = 67108864
+;
+; Items not accessed in a while get are eventually evicted. However, if the
+; memory used is below this percentage, then even the unused items are left in
+; the cache. The trade-off here is when a new workload starts, it may find the
+; cache with some stale items during the first few seconds and not be able to
+; insert its entries in.
+;leave_percent = 30
+;
+; Cache database btree nodes up to this depth only. Depth starts at 1 at root,
+; then 2 the next level down, and so on. Only intermediate (pointer) nodes will
+; be cached, those are the nodes which point to other nodes, as opposed to leaf
+; key-value nodes, which hold revision trees. To disably db btree node caching
Review Comment:
A typo `disable`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2298529219
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,251 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% If we didn't clean any entries how often to poll ets size to start cleaning
+% them. This is to quickly react and make room for potentially new entries
+% being added without having to for a whole interval
Review Comment:
I opted to re-write the section to make it a bit more clear
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3202005847
Update the PR with these changes:
* As a safety precaution add a `is_process_alive(Fd)` check after we fetch the item from the cache. Behind the scenes that still sends a signal to the process so it will add some overhead but it seems safer. If the process is already dead, the cache returns `undefined` as if the item is missing and then cleans any other entries associated with that `Fd`. The caller will then try to fetch the term from `couch_file` and crash like it would without the cache.
* Parameterized the percentage of max size below which entries are left around in the cache. That is if the cache is mostly empty then some entries with 0 bump_counts are left around in there in case they may be needed again. The tradeoff here is that if a bunch of new work come suddenly it may find the cache full until the next cleanup cycles removes all the 0 entries. This will let us test some of the scenarios mentioned like what happens if most entries are already in the cache.
* Added a 3rd `[couchdb, bt_engine_cache, full]` metric which is bumped when tried to insert into the cache but it was full.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307344394
##########
rel/overlay/etc/default.ini:
##########
@@ -130,6 +130,26 @@ view_index_dir = {{view_index_dir}}
; downgrade or even open the databases in question.
;prohibit_downgrade = true
+[bt_engine_cache]
+; Memory used for btree engine cache. This is a cache for top levels of
+; database btrees (id tree, seq tree) and a few terms from the db header. Value
+; is in bytes.
+;max_size = 67108864
+;
+; Items not accessed in a while are eventually evicted. However, if the memory
+; used is below this percentage, then even the unused items are left in the
+; cache. The trade-off here is when a new workload starts, it may find the
+; cache with some stale items during the first few seconds and not be able to
+; insert its entries in.
+;leave_percent = 30
+;
+; Cache database btree nodes up to this depth only. Depth starts at 1 at root,
+; then at 2 the next level down, and so on. Only intermediate (pointer) nodes
+; will be cached, those are the nodes which point to other nodes, as opposed to
+; leaf key-value nodes, which hold revision trees. To disabe db btree node
Review Comment:
Typo `disable`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307436246
##########
src/couch/src/couch_bt_engine_cache.erl:
##########
@@ -0,0 +1,254 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache).
+
+% Main API
+%
+-export([
+ insert/2,
+ insert/3,
+ reset/1,
+ lookup/1,
+ status/0
+]).
+
+% Supervision and start API
+%
+-export([
+ create_tables/0,
+ sup_children/0,
+ start_link/1,
+ init/1
+]).
+
+-define(DEFAULT_SIZE, 67108864).
+-define(DEFAULT_LEAVE_PERCENT, 30).
+-define(INTERVAL_MSEC, 3000).
+% How often cleaners check if the ets size increases. This is used in cases
+% when initially, at the start the cleaning interval, ets table size is below
+% "leave percent". Then we skip cleaning all the 0 usage entries. However, if a
+% new set of requests come in soon after, but before the next clean-up interval
+% they'll fail since the cache would be full. To be able to react quicker and
+% make room, cleaners poll table sizes a bit more often.
+-define(CLEANUP_INTERVAL_MSEC, 100).
+% 1 bsl 58, power of 2 that's still an immediate integer
+-define(MAX_PRIORITY, 288230376151711744).
+-define(PTERM_KEY, {?MODULE, caches}).
+
+-record(cache, {tid, max_size}).
+
+% Main API functions
+
+insert(Key, Term) ->
+ insert(Key, Term, 1).
+
+insert(Key, Term, Priority) when is_integer(Priority) ->
+ Priority1 = min(?MAX_PRIORITY, max(0, Priority)),
+ case get_cache(Key) of
+ #cache{tid = Tid, max_size = Max} ->
+ case ets:info(Tid, memory) < Max of
+ true ->
+ case ets:insert_new(Tid, {Key, Priority1, Term}) of
+ true ->
+ true;
+ false ->
+ bump_usage(Tid, Key),
+ false
+ end;
+ false ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, full]),
+ false
+ end;
+ undefined ->
+ false
+ end.
+
+reset(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} -> reset_usage(Tid, Key);
+ undefined -> true
+ end.
+
+lookup(Key) ->
+ case get_cache(Key) of
+ #cache{tid = Tid} ->
+ case ets:lookup_element(Tid, Key, 3, undefined) of
+ undefined ->
+ couch_stats:increment_counter([couchdb, bt_engine_cache, misses]),
+ undefined;
+ Term ->
+ bump_usage(Tid, Key),
+ couch_stats:increment_counter([couchdb, bt_engine_cache, hits]),
+ Term
+ end;
+ undefined ->
+ undefined
+ end.
+
+status() ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ SizeMem = [info(C) || C <- tuple_to_list(Caches)],
+ {Sizes, Memory} = lists:unzip(SizeMem),
+ #{
+ size => lists:sum(Sizes),
+ memory => lists:sum(Memory)
+ };
+ undefined ->
+ #{}
+ end.
+
+% Supervisor helper functions
+
+create_tables() ->
+ BtCaches = [new() || _ <- lists:seq(1, shard_count())],
+ persistent_term:put(?PTERM_KEY, list_to_tuple(BtCaches)).
+
+sup_children() ->
+ [sup_child(I) || I <- lists:seq(1, shard_count())].
+
+sup_child(N) ->
+ Name = list_to_atom("couch_bt_engine_cache_" ++ integer_to_list(N)),
+ #{id => Name, start => {?MODULE, start_link, [N]}, shutdown => brutal_kill}.
+
+% Process start and main loop
+
+start_link(N) when is_integer(N) ->
+ {ok, proc_lib:spawn_link(?MODULE, init, [N])}.
+
+init(N) ->
+ Caches = persistent_term:get(?PTERM_KEY),
+ Cache = #cache{tid = Tid} = element(N, Caches),
+ ets:delete_all_objects(Tid),
+ loop(Cache).
+
+loop(#cache{tid = Tid} = Cache) ->
+ decay(Tid),
+ Next = now_msec() + wait_interval(?INTERVAL_MSEC),
+ clean(Cache, Next - ?CLEANUP_INTERVAL_MSEC),
+ remove_dead(Cache),
+ WaitLeft = max(10, Next - now_msec()),
+ timer:sleep(WaitLeft),
+ loop(Cache).
+
+% Clean unused procs. If we haven't cleaned any keep polling at a higher
+% rate so we react quicker if a new set of entries are added.
+%
+clean(#cache{tid = Tid} = Cache, Until) ->
+ case now_msec() < Until of
+ true ->
+ case should_clean(Cache) of
+ true ->
+ ets:match_delete(Tid, {'_', 0, '_'});
+ false ->
+ timer:sleep(wait_interval(?CLEANUP_INTERVAL_MSEC)),
+ clean(Cache, Until)
+ end;
+ false ->
+ false
+ end.
+
+should_clean(#cache{tid = Tid, max_size = Max}) ->
+ ets:info(Tid, memory) >= Max * leave_percent() / 100.
+
+remove_dead(#cache{tid = Tid}) ->
+ All = pids(Tid),
+ Alive = sets:filter(fun is_process_alive/1, All),
+ Dead = sets:subtract(All, Alive),
+ % In OTP 27+ use sets:map/2
+ Fun = fun(Pid, _) -> ets:match_delete(Tid, {{Pid, '_'}, '_', '_'}) end,
+ sets:fold(Fun, true, Dead).
+
+pids(Tid) ->
+ Acc = couch_util:new_set(),
+ try
+ ets:foldl(fun pids_fold/2, Acc, Tid)
+ catch
+ error:badarg -> Acc
+ end.
+
+pids_fold({{Pid, _}, _, _}, Acc) when is_pid(Pid) ->
+ sets:add_element(Pid, Acc);
+pids_fold({_, _, _}, Acc) ->
+ Acc.
+
+% Private helper functions
+
+new() ->
+ Opts = [public, {write_concurrency, true}, {read_concurrency, true}],
+ Max0 = round(max_size() / erlang:system_info(wordsize) / shard_count()),
+ % Some per-table overhead for the table metadata
+ Max = Max0 + round((100 * 1024 / erlang:system_info(wordsize))),
+ #cache{tid = ets:new(?MODULE, Opts), max_size = Max}.
+
+get_cache(Term) ->
+ case persistent_term:get(?PTERM_KEY, undefined) of
+ Caches when is_tuple(Caches) ->
+ Index = erlang:phash2(Term, tuple_size(Caches)),
+ #cache{} = element(1 + Index, Caches);
+ undefined ->
+ undefined
+ end.
+
+bump_usage(Tid, Key) ->
+ % We're updating the second field incrementing it by 1 and clamping it
+ % at ?MAX_PRIORITY. We don't set the default for the update_counter
+ % specifically to avoid creating bogus entries just from updating the
+ % counter, so expect the error:badarg here sometimes.
+ UpdateOp = {2, 1, ?MAX_PRIORITY, ?MAX_PRIORITY},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+reset_usage(Tid, Key) ->
+ % Reset the value of the usage to 0. Since max value is ?MAX_PRIORITY,
+ % subtract that and clamp it at 0. Do not provide a default since if an
+ % entry is missing we don't want to create a bogus one from this operation.
+ UpdateOp = {2, -?MAX_PRIORITY, 0, 0},
+ try
+ ets:update_counter(Tid, Key, UpdateOp)
+ catch
+ error:badarg -> ok
+ end.
+
+info(#cache{tid = Tid}) ->
+ Memory = ets:info(Tid, memory) * erlang:system_info(wordsize),
+ Size = ets:info(Tid, size),
+ {Size, Memory}.
+
+% bsr bit-shifts right so we divide by 2, then 4, 8, etc.
+%
+decay(Tid) ->
+ Head = {'$1', '$2', '$3'},
+ Cond = [{'>', '$2', 0}],
+ Result = [{{'$1', {'bsr', '$2', 1}, '$3'}}],
+ ets:select_replace(Tid, [{Head, Cond, Result}]).
+
+shard_count() ->
+ erlang:system_info(schedulers).
+
+wait_interval(Interval) ->
+ Jitter = rand:uniform(max(1, round(Interval * 0.5))),
Review Comment:
Interval is integer. How about replacing the `round(Interval * 0.5)` with `Interval bsr 1`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#discussion_r2307537506
##########
src/couch/test/eunit/couch_bt_engine_cache_test.erl:
##########
@@ -0,0 +1,117 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_bt_engine_cache_test).
+
+-include_lib("couch/include/couch_eunit.hrl").
+
+-define(PTERM_KEY, {couch_bt_engine_cache, caches}).
+
+couch_bt_engine_cache_test_() ->
+ {
+ foreach,
+ fun setup/0,
+ fun teardown/1,
+ [
+ ?TDEF_FE(t_created),
+ ?TDEF_FE(t_insert_and_lookup),
+ ?TDEF_FE(t_decay_and_removal_works, 10),
+ ?TDEF_FE(t_pid_cleanup_works, 10)
+ ]
+ }.
+
+setup() ->
+ Ctx = test_util:start_applications([config]),
+ couch_bt_engine_cache:create_tables(),
+ N = tuple_size(persistent_term:get(?PTERM_KEY)),
Review Comment:
A nitpick. Here and after in the module you directly access PTERM_KEY. You could export the `couch_bt_engine_cache:list_caches().` or better `couch_bt_engine_cache:info().`,
```erlang
-module(couch_bt_engine_cache).
-export([info/0]).
info() ->
case persistent_term:get(?PTERM_KEY, undefined) of
Caches when is_tuple(Caches) ->
lists:map(fun(#cache{tid = Tid, max_size = Max}) ->
Info = maps:from_list(ets:i(Tid)),
Info#{
stats => #{
full => couch_stats:sample([couchdb, bt_engine_cache, full]),
misses => couch_stats:sample([couchdb, bt_engine_cache, misses]),
hits => couch_stats:sample([couchdb, bt_engine_cache, hits])
},
max_cache_size => Max
}
end, tuple_to_list(Caches));
undefined ->
[]
end.
```
Then we can add a call to this function into `couch_debug:bt_cache_info/0`.
I think information provided by this function should be enough to write all tests in this module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3233695567
Great work. I have couple of nitpicks, otherwise looks really good. I apologize about my mistake to use another my account initially (@iilyak-ibm is also me).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3209005577
@jaydoane wondered what happened if we try depth = 3. With the same setup as above ran a few trials with a various cache depths 3, 2 and 1. Maybe we should go with depth = 3 as the new default?
<img width="954" height="849" alt="cache_depths_experiment" src="https://github.com/user-attachments/assets/651013ab-b9e3-49df-84bb-28c4f1e4a0d9" />
#### Depth 3
```
HTTP
http_req_duration..............: avg=586.74ms min=3.14ms med=673.8ms max=5.66s p(90)=1.16s p(95)=1.28s
{ expected_response:true }...: avg=585.98ms min=3.14ms med=673.07ms max=5.36s p(90)=1.16s p(95)=1.28s
{ name:bulk_docs }...........: avg=112.06ms min=88.56ms med=108.77ms max=422.51ms p(90)=117.81ms p(95)=120.51ms
{ name:get_doc }.............: avg=15.37ms min=3.14ms med=11.77ms max=4.25s p(90)=21.93ms p(95)=24.55ms
{ name:post_doc }............: avg=794.25ms min=15.16ms med=771.39ms max=4.91s p(90)=1.16s p(95)=1.29s
{ name:put_doc }.............: avg=950.81ms min=39.64ms med=947.47ms max=5.66s p(90)=1.28s p(95)=1.38s
http_req_failed................: 0.11% 7585 out of 6529402
http_reqs......................: 6529402 8535.726214/s
EXECUTION
iteration_duration.............: avg=2.76s min=1.13s med=2.74s max=8.37s p(90)=3.21s p(95)=3.37s
iterations.....................: 2176133 2844.805006/s
vus............................: 347 min=0 max=10000
vus_max........................: 10000 min=10000 max=10000
NETWORK
data_received..................: 4.1 GB 5.3 MB/s
data_sent......................: 1.1 GB 1.4 MB/s
```
```
{
"full": {
"value": 0,
"type": "counter",
"desc": "number of times bt_engine cache was full"
},
"hits": {
"value": 38216261,
"type": "counter",
"desc": "number of bt_engine cache hits"
},
"misses": {
"value": 1476375,
"type": "counter",
"desc": "number of bt_engine cache misses"
}
}
```
#### Depth 2
```
HTTP
http_req_duration..............: avg=735.09ms min=3.71ms med=877.78ms max=3.4s p(90)=1.41s p(95)=1.55s
{ expected_response:true }...: avg=734.3ms min=3.71ms med=877.09ms max=3.2s p(90)=1.41s p(95)=1.55s
{ name:bulk_docs }...........: avg=109.51ms min=94.58ms med=106.51ms max=429.49ms p(90)=113.26ms p(95)=115.84ms
{ name:get_doc }.............: avg=15.45ms min=3.71ms med=12.45ms max=2.05s p(90)=25.08ms p(95)=28.88ms
{ name:post_doc }............: avg=1.04s min=13.54ms med=1.02s max=3.2s p(90)=1.45s p(95)=1.59s
{ name:put_doc }.............: avg=1.14s min=46.67ms med=1.12s max=3.4s p(90)=1.53s p(95)=1.65s
http_req_failed................: 0.10% 5948 out of 5622800
http_reqs......................: 5622800 7371.456322/s
EXECUTION
iteration_duration.............: avg=3.2s min=1.41s med=3.18s max=6.65s p(90)=3.74s p(95)=3.92s
iterations.....................: 1873932 2456.713362/s
vus............................: 106 min=0 max=10000
vus_max........................: 10000 min=10000 max=10000
NETWORK
data_received..................: 3.5 GB 4.6 MB/s
data_sent......................: 933 MB 1.2 MB/s
```
```
{
"full": {
"value": 0,
"type": "counter",
"desc": "number of times bt_engine cache was full"
},
"hits": {
"value": 24445315,
"type": "counter",
"desc": "number of bt_engine cache hits"
},
"misses": {
"value": 367983,
"type": "counter",
"desc": "number of bt_engine cache misses"
}
}
```
### Depth 1
```
HTTP
http_req_duration..............: avg=897.76ms min=4.3ms med=1.05s max=4.06s p(90)=1.75s p(95)=1.93s
{ expected_response:true }...: avg=896.76ms min=4.3ms med=1.05s max=3.64s p(90)=1.75s p(95)=1.92s
{ name:bulk_docs }...........: avg=110.12ms min=88.8ms med=107.17ms max=424.52ms p(90)=114.56ms p(95)=116.5ms
{ name:get_doc }.............: avg=21.44ms min=4.3ms med=16.46ms max=3.55s p(90)=35.67ms p(95)=40.64ms
{ name:post_doc }............: avg=1.25s min=22.69ms med=1.23s max=3.62s p(90)=1.77s p(95)=1.94s
{ name:put_doc }.............: avg=1.41s min=24.89ms med=1.4s max=4.06s p(90)=1.92s p(95)=2.07s
http_req_failed................: 0.10% 5285 out of 4881638
http_reqs......................: 4881638 6394.596739/s
EXECUTION
iteration_duration.............: avg=3.69s min=1.29s med=3.67s max=8.51s p(90)=4.32s p(95)=4.53s
iterations.....................: 1626878 2131.093857/s
vus............................: 3502 min=0 max=10000
vus_max........................: 10000 min=10000 max=10000
NETWORK
data_received..................: 3.1 GB 4.0 MB/s
data_sent......................: 824 MB 1.1 MB/s
```
```
{
"full": {
"value": 0,
"type": "counter",
"desc": "number of times bt_engine cache was full"
},
"hits": {
"value": 16728022,
"type": "counter",
"desc": "number of bt_engine cache hits"
},
"misses": {
"value": 138036,
"type": "counter",
"desc": "number of bt_engine cache misses"
}
}
```
Maybe we should go
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BTree engine term cache [couchdb]
Posted by "mikerhodes (via GitHub)" <gi...@apache.org>.
mikerhodes commented on PR #5625:
URL: https://github.com/apache/couchdb/pull/5625#issuecomment-3211112536
I don't think the eviction algorithm choice is un-smart. It's quite a nice least-used algorithm that looks to be quite scan-resistant due to the way it exponentially decays (this is useful against applications which "scan" by periodically open a lot of databases, rather than scan a single file on disk).
The scan resistance property is why something like ets-lru doesn't make as much sense for this use-case. If the cache gets full, hopefully it's full of stuff we use a lot, rather than just what we opened recently. For those periodic cases, we don't churn the cache for them.
[TinyLFU](https://dl.acm.org/doi/10.1145/3149371) is quite interesting in that it uses a sketch to decide whether to add something to the cache at all, which is also nice against the scan-databases use-case, but also seems quite a bit more complicated. I note that it also degrades by dividing by two.
TinyLFU or something like it could be _really_ useful for things like the main LRU where we keep open files --- avoiding evicting things during periodic database scans would be good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org