You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by "chewbranca (via GitHub)" <gi...@apache.org> on 2024/09/03 00:09:15 UTC
[PR] Couch stats resource tracker v2 [couchdb]
chewbranca opened a new pull request, #5213:
URL: https://github.com/apache/couchdb/pull/5213
# Couch Stats Resource Tracker
This is a rework of PR: https://github.com/apache/couchdb/pull/4812
## Overview and Motivation
Couch Stats Resource Tracker (CSRT) is a new engine for tracking the amount of
resource operations induced by processes within CouchDB's Erlang VM. This PR
specifically targets coordinator processes and RPC workers induced by said
coordinator processes, but the underlying stats collection framework is designed
to be usable by anything consuming resources in CouchDB, such that we can extend
this out to background jobs like indexing, compaction, and replication. The long
term stretch goal is to be able to account for all system level activity induced
by CouchDB, but the practical goal is to be able to understand where and why
most of the resources in the system are being utilized, at a request/job level
granularity.
This PR is primarily motivated by the current lack of visibility into
identifying _what_ operations are inducing large quantities of work. We have
node/cluster level visibility into the amounts of IO operations being induced,
but we lack the ability to identify what request/operation induced that
workload.
This is especially problematic when dealing with large fold operations that do
local filtering (eg filtered `_changes` feeds or filtered `_find` queries)
because these operations lack the normal coordinator induced rate limiting that
results naturally from funneling individual results back to the coordinator node
to sort and process. In the local filtering case, we essentially do a direct
fold over the shard and invoke a filter function on that doc to find a matching
result, but in the event the docs fail to match this is essentially a local
tight loop over the shard that when run in parallel can easily dominate IO
operations. The `_find` logic has been extended to generate `report`'s to log
and identify the heavy hitter requests, especially the degenerative `_find`
queries that do a full database scan and find zero results to return.
## Approach in this PR and differences from mango reports
This PR takes the idea of the mango reports and creates a unified framework for
tracking these statistics in real time allowing for global querying of node and
cluster level resource usage in the live processes. This PR reflexes an
opinionated deviation from the approach in the mango reports, and instead of
introducing new stats tracking, it proposes the core approach of:
> Any stat worth tracking for reporting system usage is clearly worth tracking
> properly as a proper CouchDB metric.
So instead of embedding new stats like in the mango find reports, this system
hooks into the `couch_stats:increment_counter` logic to piggy back off of the
stats being collected in real time by the process doing the work, and then
funnels those updates into an ets table keyed off of the local process, and
joined at a cluster level by the coordinator ref, allowing for cluster level
aggregation of individual http requests, live. These tracked stats are forwarded
back to the coordinator process by way of embedding in the `rexi` RPC messages
such that long running find queries and other heavy weight processes can be
identified and tracked.
We then log a report detailing the total resources induced by the http request
so we can retroactively identify which requests are consuming the most
resources. The reporting by default is configured to only happen at the
coordinator level, but if you're able to handle the log volume it can be enabled
for all workers too. Down the road a nice feature would be supporting writing
reports directly to ClickHouse, some binary format, or even just a terse text
format to allow for increased report volumes; currently high throughput report
generation for coordinator and all rpc workers on high Q databases is
substantial in data volume, but there's much room for optimization given the
verbose nature of the current reports and how well they gzip up.
## New metrics introduced
As a result of the above mentioned philosophy of properly tracking the stats
worth tracking, this PR introduces a handful of new stats, predominantly in one
of two forms, listed below. I've also included sample screenshots of the new
metrics plotted during a 30 minute benchmark run that started with an empty
cluster and aggressively created new databases/indexes/documents while growing
worker count progressively during that run. All http traffic was ceased after 30
minutes, and you can clearly see the phase change in operations when that
happened.
1) Core stats for previously missing metrics
eg new stats for counting `couch_btree` reads and writes on kp/kv nodes
<img width="1422" alt="Screenshot 2024-08-23 at 5 53 42 PM" src="https://github.com/user-attachments/assets/523ca54f-ad1c-4a04-8b45-de1c2af158e8">
2) RPC work induced
The idea here is that we should be tracking 1-3 things for all induced RPC work:
1) RPC worker spawned type
2) RPC worker docs processed
3) RPC worker docs returned
Item 1) is the primary item for core RPC work, this allows us to see the volume
of RPC workers spawned over time per node. Items 2) and 3) are specific to
aggregate operations, with 3) specific to aggregate operations that can perform
local filtering.
The idea is that we can a) see the types of work being induced on nodes over
time, observe how much documents are being processed by the aggregate worker
operations, and then b) directly observe major discrepancies between docs
processed and docs returned, as that's indicative of a missing index or poorly
designed workflows.
Here's a full cluster view of all nodes rpc traffic:
<img width="1425" alt="Screenshot 2024-08-23 at 5 53 23 PM" src="https://github.com/user-attachments/assets/79fc7f6e-143b-4a63-a518-c43297683733">
In the case of our benchmark above, the workload was evenly distributed so all
nodes performed similarly. This is a lot of data, but can easily be aggregated
by node or type to identify non-homogeneous workloads. Here's a simpler view
showing per node RPC workloads:
<img width="1423" alt="Screenshot 2024-08-23 at 5 53 32 PM" src="https://github.com/user-attachments/assets/f93ed420-66a1-4ce4-a86b-aec8632b6d7e">
## Tracking table for accumulating and querying metrics
The central tracking table is a global ets table utilizing `read_concurrency`,
`write_concurrency`, and `distributed_counters`, which results in an
impressively performant global table in which all processes update their local
stats. Writes are isolated to the process doing the work, so there is no
contention of parallel writes to the same key. Aggregations are performed
against the full ets table, but updates are constrained to a given key are
constrained to the corresponding worker process.
### Previous design that failed to scale
In previous PR I attempted to utilize a singular `gen_server` for monitoring the
processes and performing some cleanup operations. This was optimized down to
_only_ being a dedicated server doing `handle_call({monitor, Pid},..) ->
monitor_pid(), {reply, ok, State}). handle_info({DOWN, ..., REF, ...}) ->
ets:delete(maps:get(Ref, RefToPid))` and that was insufficient to handle the
load. I tried various approaches but I was able to melt a singular `gen_server`
easily. It's necessary to have a process monitor outside of the local process
because coordinator/worker processes can and will get killed mid operation,
therefore `after` clause/function based approaches are insufficient.
Even with that minimal of a workload, I was able to melt the `gen_server`:
<img width="569" alt="Screenshot 2024-04-04 at 3 45 15 PM" src="https://github.com/user-attachments/assets/22d64327-f4ee-4ef8-a71c-48c3126c8099">
and that's with it _really_ doing a minimum workload:
<img width="736" alt="Screenshot 2024-04-09 at 5 19 37 PM" src="https://github.com/user-attachments/assets/d70f99c5-a984-4b35-aa80-aaee34d07e80">
This was my final attempt to make a singular `gen_server` architecture, but with
80 core nodes I'm now fully convinced it's no longer viable to do singular
`gen_server` systems in hot code paths and we must take more distributed
approaches, either by way of sharding the servers or fully distributed.
### Distributed tracker appraoch in CSRT v2
In the case of CSRT, I engaged a fully distributed approach that spawns a
dedicated monitor process when a CSRT context is created by a coordinator or
worker. This monitor process handles the lifetime of a given entry in the ets
table so that we delete the worker entry when the worker is done. This dedicated
monitor process also generates the report afterwards. Switching to the dedicated
monitor approach eliminated the scaling issues I encountered, and the current
architecture is able to readily handle max throughput load.
The CSRT context is created in the coordinator process directly in
`chttpd:process_request_int`, and in the worker process directly in the
spawned process's initialization of `rexi_server:init_p`. The context is
basically just `erlang:put(?CONTEXT_MARKER, {self(), make_ref()})` which is then
the ets key used for tracking the coordinator process while it handles the given
request.
The `make_ref()` ensures that the coordinator processes that are reused in the
Mochiweb worker pool distinguish between individual http requests. More
generally, this allows a long lived process to isolate subsets of its own work.
This is essential if we want to add the ability to funnel the CSRT context
through IOQ/couch_file/couch_mrview/etc to accumulate
### A note on PidRef vs nonce for identification
Currently we don't funnel the coordinator's PidRef across the wire and instead
rely on the `nonce` as a global aggregator key, and then the coordinator
aggregations happen directly when the RPC responses are received and the deltas
are extracted. We could add this fairly easily in `rexi:cast_ref`, but I do
wonder if we'd be better off skipping the ref entirely and instead using
`{self(), Nonce}` as the key given we already funnel it around. That won't work
for background operations, so we'd need a `make_ref()` fallback for tracking
those jobs, but I do like the idea of consolidating down to using the `nonce`
given it's already the unique reference to the request inducing the workload,
and we already send it over the wire ubiquitously for all coordinator/worker
operations through `rexi:cast_ref`.
### Context creation and lifecycle
We create the initial context in
`chttpd:process_request_int`/`rexi_server:init_p` for the coordinator/workers,
respectively, and then we progressively fill in the details for things like
dbname/username/handler_fun so that we can track those data points naturally as
they arise in the request codepath, for example adding the chttp_db handler when
entering those functions, or setting the username after `chttpd_auth:authorize`
returns. Similarly, in `fabric_rpc` we piggy back off of
`fabric_rpc:set_io_priority` called by every callback function to cleanly set
the dbname involved in the RPC request. We could also extend this to track the
ddoc/index involved, if any.
The idea is to make it easy for the local process to update its global tracked
state at the appropriate points in the codebase so we can iteratively extend out
the tracking throughout the codebase. Background indexing and compaction are
apt targets for wiring in CSRT and we could even extend the `/_active_tasks`
jobs to include status about resource usage.
When we initiate the context, for workers, coordinators, or any future job
types, we spawn a dedicated tracking monitor process that sits by idly until it
gets a `stop` message from normal lifecycle termination, or it gets a `DOWN`
message from the process doing work. In either case, the tracker process cleans
up the corresponding ets table entry (the only form of two processes writing to
the same key in the ets tracker, but handed off with no interweaving) and then
conditionally generates a `report` to log the work induced.
The default, when CSRT is enabled, is to log a report for the coordinator
process totalling the tracked work induced by the RPC workers to fulfill the
given request. It's configurable to also log workers, and there's some
rudimentary filtering capabilities to allow for logging of only a specific rpc
worker type, but this could be improved upon considerably. In general, the
volume of these logs can be sizable, for example a singular http view request
against a Q=64 database on a healthy cluster induces 196 RPC workers, all
inducing their own volume of work and potentially logging a report. A compact
form or additional filtering capabilities to log interesting reports would be
beneficial.
## Status and next steps
Overall I'm happy with the performance of the core tracking system, the modern
ETS improvements with distributed counters on top of atomic increments are
_really_ impressive! I had the test suite fully passing recently but I've done a
decent bit of cleanup and restructuring recently so I haven't checked out a full
CI run in a minute, I'll see how it looks on this PR and address anything that
comes up. I'd like to start getting a review here and see what folks think, I
think the structure of the code is in a good place to discuss and get feedback
on. A few next steps to do:
- [ ] add more tests
- [ ] add Dialyzer specs `couch_stats_resource_tracker.erl` at the very least
- [ ] fix `time` tracking, the `tnow()` is not a `positive_integer()`
- [ ] add some standard query functions using ets:fun2ms
- The parse transform makes these more challenging to handle dynamically, so
let's add a handful of standard performant functions, eg:
- sort_by({dbname, shard, user}, ioq_calls)
- sort_by(user, ioq_calls)
- sort_by({user, request_type}, docs_processed)
- sort_by({request_type, user}, docs_processed)
- sort_by(user, get_kv_nodes)
- etc
- [ ] think about additional filtering on logs:
- skip report fields with zero values?
- set minimum thresholds for reports?
- allow skipping of some coordinator reports? eg welcome handler
- [ ] design on metrics namespacing from `rexi_server:init_p`
- eg `should_increment([M, F, spawned])` vs
- `should_increment([rexi_rpc, M, F, spawned]`
- or `should_increment([couchdb, rpc_worker, M, F, spawned]`
## Sample report
Filtered changes http request with `?include_docs=true` and JS filter:
```
[report] 2024-09-02T23:46:08.175264Z node1@127.0.0.1 <0.1012.0> -------- [csrt-pid-usage-lifetime changes_returned=1528 db_open=7 dbname="foo" docs_read=1528 from="null" get_kp_nodes=14 get_kv_nodes=180 ioq_calls=3254 js_filter=1528 js_filter_error=0 js_filtered_docs=1528 mfa="null" nonce="bce5d2ce6e" path="/foo/_changes" pid_ref="<0.965.0>:#Ref<0.2614010930.743440385.194505>" rows_read=1528 started_at=-576460745696 type="coordinator:GET:fun chttpd_db:handle_changes_req/2" updated_at=-576460745696 username="adm"]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743948497
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
Review Comment:
If we're stopping probably not worth bothering demonitoring
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1800224330
##########
src/chttpd/src/chttpd_misc.erl:
##########
@@ -224,6 +225,110 @@ handle_task_status_req(#httpd{method = 'GET'} = Req) ->
handle_task_status_req(Req) ->
send_method_not_allowed(Req, "GET,HEAD").
+handle_resource_status_req(#httpd{method = 'POST'} = Req) ->
+ ok = chttpd:verify_is_server_admin(Req),
+ chttpd:validate_ctype(Req, "application/json"),
+ {Props} = chttpd:json_body_obj(Req),
+ Action = proplists:get_value(<<"action">>, Props),
+ Key = proplists:get_value(<<"key">>, Props),
+ Val = proplists:get_value(<<"val">>, Props),
+
+ CountBy = fun couch_stats_resource_tracker:count_by/1,
+ GroupBy = fun couch_stats_resource_tracker:group_by/2,
+ SortedBy1 = fun couch_stats_resource_tracker:sorted_by/1,
+ SortedBy2 = fun couch_stats_resource_tracker:sorted_by/2,
+ ConvertEle = fun(K) -> list_to_existing_atom(binary_to_list(K)) end,
Review Comment:
Good call, fixed in 48db7908b.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1771870678
##########
src/couch/include/couch_db.hrl:
##########
@@ -53,6 +53,8 @@
-define(INTERACTIVE_EDIT, interactive_edit).
-define(REPLICATED_CHANGES, replicated_changes).
+-define(LOG_UNEXPECTED_MSG(Msg), couch_log:warning("[~p:~p:~p/~p]{~p[~p]} Unexpected message: ~w", [?MODULE, ?LINE, ?FUNCTION_NAME, ?FUNCTION_ARITY, self(), element(2, process_info(self(), message_queue_len)), Msg])).
Review Comment:
I am surprised linter doesn't complain about such long line.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1800230295
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
+ case conf_get("log_pid_usage_report", "coordinator") of
+ "coordinator" ->
+ coordinator;
+ "true" ->
+ true;
+ _ ->
+ false
+ end.
+
+should_log(undefined) ->
+ false;
+should_log(#rctx{}=Rctx) ->
+ should_log(Rctx, logging_enabled()).
+
+should_log(undefined, _) ->
+ false;
+should_log(#rctx{}, true) ->
+ true;
+should_log(#rctx{}, false) ->
+ false;
+should_log(#rctx{type = {coordinator, _, _}}, coordinator) ->
+ true;
+should_log(#rctx{type = {worker, fabric_rpc, FName}}, _) ->
+ case conf_get("log_fabric_rpc") of
+ "true" ->
+ true;
+ undefined ->
+ false;
+ Name ->
+ Name =:= atom_to_list(FName)
+ end;
+should_log(#rctx{}, _) ->
+ false.
+
+%%
+%% gen_server callbacks
+%%
+
+start_link() ->
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+
+init([]) ->
+ ets:new(?MODULE, [
+ named_table,
+ public,
+ {decentralized_counters, true},
+ {write_concurrency, true},
+ {read_concurrency, true},
+ {keypos, #rctx.pid_ref}
+ ]),
+ {ok, #st{}}.
+
+handle_call(fetch, _from, #st{} = St) ->
+ {reply, {ok, St}, St};
+handle_call({call_search, _}, _From, St) ->
+ %% TODO: provide isolated search queries here
+ {reply, ok, St};
+handle_call(Msg, _From, St) ->
+ {stop, {unknown_call, Msg}, St}.
+
+handle_cast(Msg, St) ->
+ {stop, {unknown_cast, Msg}, St}.
+
+handle_info(Msg, St) ->
+ {stop, {unknown_info, Msg}, St}.
+
+terminate(_Reason, _St) ->
+ ok.
+
+code_change(_OldVsn, St, _Extra) ->
+ {ok, St}.
Review Comment:
Fixed in de9a43047.
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
+ case conf_get("log_pid_usage_report", "coordinator") of
+ "coordinator" ->
+ coordinator;
+ "true" ->
+ true;
+ _ ->
+ false
+ end.
+
+should_log(undefined) ->
+ false;
+should_log(#rctx{}=Rctx) ->
+ should_log(Rctx, logging_enabled()).
+
+should_log(undefined, _) ->
+ false;
+should_log(#rctx{}, true) ->
+ true;
+should_log(#rctx{}, false) ->
+ false;
+should_log(#rctx{type = {coordinator, _, _}}, coordinator) ->
+ true;
+should_log(#rctx{type = {worker, fabric_rpc, FName}}, _) ->
+ case conf_get("log_fabric_rpc") of
+ "true" ->
+ true;
+ undefined ->
+ false;
+ Name ->
+ Name =:= atom_to_list(FName)
+ end;
+should_log(#rctx{}, _) ->
+ false.
+
+%%
+%% gen_server callbacks
+%%
+
+start_link() ->
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+
+init([]) ->
+ ets:new(?MODULE, [
+ named_table,
+ public,
+ {decentralized_counters, true},
+ {write_concurrency, true},
+ {read_concurrency, true},
+ {keypos, #rctx.pid_ref}
+ ]),
+ {ok, #st{}}.
+
+handle_call(fetch, _from, #st{} = St) ->
+ {reply, {ok, St}, St};
+handle_call({call_search, _}, _From, St) ->
+ %% TODO: provide isolated search queries here
+ {reply, ok, St};
+handle_call(Msg, _From, St) ->
+ {stop, {unknown_call, Msg}, St}.
+
+handle_cast(Msg, St) ->
+ {stop, {unknown_cast, Msg}, St}.
+
+handle_info(Msg, St) ->
+ {stop, {unknown_info, Msg}, St}.
Review Comment:
Fixed in de9a43047.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1807073259
##########
src/couch_stats/src/couch_stats.erl:
##########
@@ -49,6 +55,11 @@ increment_counter(Name) ->
-spec increment_counter(any(), pos_integer()) -> response().
increment_counter(Name, Value) ->
+ %% Should maybe_track_local happen before or after notify?
+ %% If after, only currently tracked metrics declared in the app's
+ %% stats_description.cfg will be trackable locally. Pros/cons.
+ %io:format("NOTIFY_EXISTING_METRIC: ~p || ~p || ~p~n", [Name, Op, Type]),
+ ok = maybe_track_local_counter(Name, Value),
Review Comment:
An essential part of the work done in this PR is to ensure that this is as minimally impactful as possible. We add ets config lookup to check if it's enabled, and then we perform the same type of simple integer counter bump utilizing `ets:update_counter` with `distributed_counters` enabled and _no_ concurrent writes to the same key, all writes to any given key are _only_ written to from the process tracking its own work, and then finally the tracker process will delete the entry from ets when the process dies and closes the context.
@nickva you did have me change my hard coded atom function to one utilizing maps (although I did mange to use static maps), so I'll make sure to do some more testing with the final version to compare. I plan on doing some comparisons between the start of this PR and the final version, to make sure there weren't any regressions induced by the changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743946496
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
Review Comment:
`_0DPid` looks a bit odd just like `_Reason0` can just use `Pid` to match exactly or `_Pid` or just `_`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1744027266
##########
src/rexi/src/rexi_utils.erl:
##########
@@ -95,6 +107,49 @@ process_message(RefList, Keypos, Fun, Acc0, TimeoutRef, PerMsgTO) ->
end;
{rexi_DOWN, _, _, _} = Msg ->
Fun(Msg, nil, Acc0)
- after PerMsgTO ->
- {timeout, Acc0}
+ end.
+
+add_delta({A}, Delta) -> {A, Delta};
+add_delta({A, B}, Delta) -> {A, B, Delta};
+add_delta({A, B, C}, Delta) -> {A, B, C, Delta};
+add_delta({A, B, C, D}, Delta) -> {A, B, C, D, Delta};
+add_delta({A, B, C, D, E}, Delta) -> {A, B, C, D, E, Delta};
+add_delta({A, B, C, D, E, F}, Delta) -> {A, B, C, D, E, F, Delta};
+add_delta({A, B, C, D, E, F, G}, Delta) -> {A, B, C, D, E, F, G, Delta};
Review Comment:
At first sight this looks a bit awkward. Do we have any `add_delta({A, B, C, D, E, F}, Delta) -> {A, B, C, D, E, F, Delta};` message sent to rexi?
It might be better to be explicit about what message rexi will actually get. We may have to do a two-stage PRs. One to prep the receivers to accept both old and new patterns, then another one to start sending the new messages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1816982151
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
Review Comment:
I looked briefly into bif implementation. It is nice:
- hash map internally
- the term sharing is preserved for complex containers (tuples, lists)
- the literals like numbers and strings are copied
- only one allocation during `get` (to create reference for GC)
My conclusion is - it should be way faster than `ets` or `gen_server` based approach. When updates are rare. Updates causing locking of a data structure so all concurrent reads are on hold.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1750678108
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
Review Comment:
This can cause confusion. The `_enabled` suffix reads like a boolean, however this function returns three different values. I am already confused. When I reviewed the function above this one it looked fine, however now I need to go back and think more
```
is_logging_enabled() ->
logging_enabled() =/= false.
```
What is the intent ^^^? Do we want to enable logging regardless of it being `coordinator` or `true`?
In such cases named variants are better.
```erlang
tracker_subject() ->
case conf_get("log_pid_usage_report", "coordinator") of
"coordinator" ->
coordinator;
"true" ->
all;
_ ->
no_tracking
end.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743865655
##########
src/chttpd/src/chttpd_misc.erl:
##########
@@ -224,6 +225,110 @@ handle_task_status_req(#httpd{method = 'GET'} = Req) ->
handle_task_status_req(Req) ->
send_method_not_allowed(Req, "GET,HEAD").
+handle_resource_status_req(#httpd{method = 'POST'} = Req) ->
+ ok = chttpd:verify_is_server_admin(Req),
+ chttpd:validate_ctype(Req, "application/json"),
+ {Props} = chttpd:json_body_obj(Req),
+ Action = proplists:get_value(<<"action">>, Props),
+ Key = proplists:get_value(<<"key">>, Props),
+ Val = proplists:get_value(<<"val">>, Props),
+
+ CountBy = fun couch_stats_resource_tracker:count_by/1,
+ GroupBy = fun couch_stats_resource_tracker:group_by/2,
+ SortedBy1 = fun couch_stats_resource_tracker:sorted_by/1,
+ SortedBy2 = fun couch_stats_resource_tracker:sorted_by/2,
+ ConvertEle = fun(K) -> list_to_existing_atom(binary_to_list(K)) end,
+ ConvertList = fun(L) -> [ConvertEle(E) || E <- L] end,
+ ToJson = fun couch_stats_resource_tracker:term_to_flat_json/1,
+ JsonKeys = fun(PL) -> [[ToJson(K), V] || {K, V} <- PL] end,
+
+ Fun = case {Action, Key, Val} of
+ {<<"count_by">>, Keys, undefined} when is_list(Keys) ->
+ Keys1 = [ConvertEle(K) || K <- Keys],
+ fun() -> CountBy(Keys1) end;
+ {<<"count_by">>, Key, undefined} ->
+ Key1 = ConvertEle(Key),
+ fun() -> CountBy(Key1) end;
+ {<<"group_by">>, Keys, Vals} when is_list(Keys) andalso is_list(Vals) ->
+ Keys1 = ConvertList(Keys),
+ Vals1 = ConvertList(Vals),
+ fun() -> GroupBy(Keys1, Vals1) end;
+ {<<"group_by">>, Key, Vals} when is_list(Vals) ->
+ Key1 = ConvertEle(Key),
+ Vals1 = ConvertList(Vals),
+ fun() -> GroupBy(Key1, Vals1) end;
+ {<<"group_by">>, Keys, Val} when is_list(Keys) ->
+ Keys1 = ConvertList(Keys),
+ Val1 = ConvertEle(Val),
+ fun() -> GroupBy(Keys1, Val1) end;
+ {<<"group_by">>, Key, Val} ->
+ Key1 = ConvertEle(Key),
+ Val1 = ConvertList(Val),
+ fun() -> GroupBy(Key1, Val1) end;
+
+ {<<"sorted_by">>, Key, undefined} ->
+ Key1 = ConvertEle(Key),
+ fun() -> JsonKeys(SortedBy1(Key1)) end;
+ {<<"sorted_by">>, Keys, undefined} when is_list(Keys) ->
+ Keys1 = [ConvertEle(K) || K <- Keys],
+ fun() -> JsonKeys(SortedBy1(Keys1)) end;
+ {<<"sorted_by">>, Keys, Vals} when is_list(Keys) andalso is_list(Vals) ->
+ Keys1 = ConvertList(Keys),
+ Vals1 = ConvertList(Vals),
+ fun() -> JsonKeys(SortedBy2(Keys1, Vals1)) end;
+ {<<"sorted_by">>, Key, Vals} when is_list(Vals) ->
+ Key1 = ConvertEle(Key),
+ Vals1 = ConvertList(Vals),
+ fun() -> JsonKeys(SortedBy2(Key1, Vals1)) end;
+ {<<"sorted_by">>, Keys, Val} when is_list(Keys) ->
+ Keys1 = ConvertList(Keys),
+ Val1 = ConvertEle(Val),
+ fun() -> JsonKeys(SortedBy2(Keys1, Val1)) end;
+ {<<"sorted_by">>, Key, Val} ->
+ Key1 = ConvertEle(Key),
+ Val1 = ConvertList(Val),
+ fun() -> JsonKeys(SortedBy2(Key1, Val1)) end;
+ _ ->
+ throw({badrequest, invalid_resource_request})
+ end,
+
+ Fun1 = fun() ->
+ case Fun() of
+ Map when is_map(Map) ->
+ {maps:fold(
+ fun
+ (_K,0,A) -> A; %% TODO: Skip 0 value entries?
+ (K,V,A) -> [{ToJson(K), V} | A]
+ end,
+ [], Map)};
+ List when is_list(List) ->
+ List
+ end
+ end,
+
+ {Resp, _Bad} = rpc:multicall(erlang, apply, [
+ fun() ->
+ {node(), Fun1()}
+ end,
+ []
+ ]),
+ %%io:format("{CSRT}***** GOT RESP: ~p~n", [Resp]),
+ send_json(Req, {Resp});
+handle_resource_status_req(#httpd{method = 'GET'} = Req) ->
+ ok = chttpd:verify_is_server_admin(Req),
+ {Resp, Bad} = rpc:multicall(erlang, apply, [
+ fun() ->
+ {node(), couch_stats_resource_tracker:active()}
Review Comment:
Instead of having to run in a closure just to get node() in the response, consider using an explicit list of nodes `[node() | nodes()]` with a plain `[M, F, A]` and then zipping over the nodes in the response. It's a bit longer but it's less fragile.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743857423
##########
src/chttpd/src/chttpd_misc.erl:
##########
@@ -224,6 +225,110 @@ handle_task_status_req(#httpd{method = 'GET'} = Req) ->
handle_task_status_req(Req) ->
send_method_not_allowed(Req, "GET,HEAD").
+handle_resource_status_req(#httpd{method = 'POST'} = Req) ->
+ ok = chttpd:verify_is_server_admin(Req),
+ chttpd:validate_ctype(Req, "application/json"),
+ {Props} = chttpd:json_body_obj(Req),
+ Action = proplists:get_value(<<"action">>, Props),
+ Key = proplists:get_value(<<"key">>, Props),
+ Val = proplists:get_value(<<"val">>, Props),
+
+ CountBy = fun couch_stats_resource_tracker:count_by/1,
+ GroupBy = fun couch_stats_resource_tracker:group_by/2,
+ SortedBy1 = fun couch_stats_resource_tracker:sorted_by/1,
+ SortedBy2 = fun couch_stats_resource_tracker:sorted_by/2,
+ ConvertEle = fun(K) -> list_to_existing_atom(binary_to_list(K)) end,
+ ConvertList = fun(L) -> [ConvertEle(E) || E <- L] end,
+ ToJson = fun couch_stats_resource_tracker:term_to_flat_json/1,
+ JsonKeys = fun(PL) -> [[ToJson(K), V] || {K, V} <- PL] end,
+
+ Fun = case {Action, Key, Val} of
+ {<<"count_by">>, Keys, undefined} when is_list(Keys) ->
+ Keys1 = [ConvertEle(K) || K <- Keys],
+ fun() -> CountBy(Keys1) end;
+ {<<"count_by">>, Key, undefined} ->
+ Key1 = ConvertEle(Key),
+ fun() -> CountBy(Key1) end;
+ {<<"group_by">>, Keys, Vals} when is_list(Keys) andalso is_list(Vals) ->
+ Keys1 = ConvertList(Keys),
+ Vals1 = ConvertList(Vals),
+ fun() -> GroupBy(Keys1, Vals1) end;
+ {<<"group_by">>, Key, Vals} when is_list(Vals) ->
+ Key1 = ConvertEle(Key),
+ Vals1 = ConvertList(Vals),
+ fun() -> GroupBy(Key1, Vals1) end;
+ {<<"group_by">>, Keys, Val} when is_list(Keys) ->
+ Keys1 = ConvertList(Keys),
+ Val1 = ConvertEle(Val),
+ fun() -> GroupBy(Keys1, Val1) end;
+ {<<"group_by">>, Key, Val} ->
+ Key1 = ConvertEle(Key),
+ Val1 = ConvertList(Val),
+ fun() -> GroupBy(Key1, Val1) end;
+
+ {<<"sorted_by">>, Key, undefined} ->
+ Key1 = ConvertEle(Key),
+ fun() -> JsonKeys(SortedBy1(Key1)) end;
+ {<<"sorted_by">>, Keys, undefined} when is_list(Keys) ->
+ Keys1 = [ConvertEle(K) || K <- Keys],
+ fun() -> JsonKeys(SortedBy1(Keys1)) end;
+ {<<"sorted_by">>, Keys, Vals} when is_list(Keys) andalso is_list(Vals) ->
+ Keys1 = ConvertList(Keys),
+ Vals1 = ConvertList(Vals),
+ fun() -> JsonKeys(SortedBy2(Keys1, Vals1)) end;
+ {<<"sorted_by">>, Key, Vals} when is_list(Vals) ->
+ Key1 = ConvertEle(Key),
+ Vals1 = ConvertList(Vals),
+ fun() -> JsonKeys(SortedBy2(Key1, Vals1)) end;
+ {<<"sorted_by">>, Keys, Val} when is_list(Keys) ->
+ Keys1 = ConvertList(Keys),
+ Val1 = ConvertEle(Val),
+ fun() -> JsonKeys(SortedBy2(Keys1, Val1)) end;
+ {<<"sorted_by">>, Key, Val} ->
+ Key1 = ConvertEle(Key),
+ Val1 = ConvertList(Val),
+ fun() -> JsonKeys(SortedBy2(Key1, Val1)) end;
+ _ ->
+ throw({badrequest, invalid_resource_request})
+ end,
+
+ Fun1 = fun() ->
+ case Fun() of
+ Map when is_map(Map) ->
+ {maps:fold(
+ fun
+ (_K,0,A) -> A; %% TODO: Skip 0 value entries?
+ (K,V,A) -> [{ToJson(K), V} | A]
+ end,
+ [], Map)};
+ List when is_list(List) ->
+ List
+ end
+ end,
+
+ {Resp, _Bad} = rpc:multicall(erlang, apply, [
+ fun() ->
+ {node(), Fun1()}
+ end,
+ []
+ ]),
+ %%io:format("{CSRT}***** GOT RESP: ~p~n", [Resp]),
Review Comment:
Left-over debug comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743878586
##########
src/couch/priv/stats_descriptions.cfg:
##########
@@ -418,10 +422,47 @@
{type, counter},
{desc, <<"number of other requests">>}
]}.
+{[couchdb, query_server, js_filter], [
+ {type, counter},
+ {desc, <<"number of JS filter invocations">>}
+]}.
+{[couchdb, query_server, js_filtered_docs], [
+ {type, counter},
+ {desc, <<"number of docs filtered through JS invocations">>}
+]}.
+{[couchdb, query_server, js_filter_error], [
+ {type, counter},
+ {desc, <<"number of JS filter invocation errors">>}
+]}.
{[couchdb, legacy_checksums], [
{type, counter},
{desc, <<"number of legacy checksums found in couch_file instances">>}
]}.
+{[couchdb, btree, folds], [
+ {type, counter},
+ {desc, <<"number of couch btree kv fold callback invocations">>}
+]}.
+{[couchdb, btree, get_node, kp_node], [
+ {type, counter},
+ {desc, <<"number of couch btree kp_nodes read">>}
+]}.
+{[couchdb, btree, get_node, kv_node], [
+ {type, counter},
+ {desc, <<"number of couch btree kv_nodes read">>}
+]}.
Review Comment:
Wonder if this is a bit too low level? Or rather, btrees are used for view and db files. It's not clear which btrees the metrics refer to.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743887117
##########
src/couch/src/couch_query_servers.erl:
##########
@@ -542,6 +542,8 @@ filter_docs(Req, Db, DDoc, FName, Docs) ->
{ok, filter_docs_int(Db, DDoc, FName, JsonReq, JsonDocs)}
catch
throw:{os_process_error, {exit_status, 1}} ->
+ %% TODO: wire in csrt tracking
+ couch_stats:increment_counter([couchdb, query_server, js_filter_error]),
Review Comment:
We already have a general counter for os process errors, normal exits and error exits. Probably don't need an explicit one just for filters?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743939518
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
+ case conf_get("log_pid_usage_report", "coordinator") of
+ "coordinator" ->
+ coordinator;
+ "true" ->
+ true;
+ _ ->
+ false
+ end.
+
+should_log(undefined) ->
+ false;
+should_log(#rctx{}=Rctx) ->
+ should_log(Rctx, logging_enabled()).
+
+should_log(undefined, _) ->
+ false;
+should_log(#rctx{}, true) ->
+ true;
+should_log(#rctx{}, false) ->
+ false;
+should_log(#rctx{type = {coordinator, _, _}}, coordinator) ->
+ true;
+should_log(#rctx{type = {worker, fabric_rpc, FName}}, _) ->
+ case conf_get("log_fabric_rpc") of
+ "true" ->
+ true;
+ undefined ->
+ false;
+ Name ->
+ Name =:= atom_to_list(FName)
+ end;
+should_log(#rctx{}, _) ->
+ false.
+
+%%
+%% gen_server callbacks
+%%
+
+start_link() ->
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+
+init([]) ->
+ ets:new(?MODULE, [
+ named_table,
+ public,
+ {decentralized_counters, true},
+ {write_concurrency, true},
+ {read_concurrency, true},
+ {keypos, #rctx.pid_ref}
+ ]),
+ {ok, #st{}}.
+
+handle_call(fetch, _from, #st{} = St) ->
+ {reply, {ok, St}, St};
+handle_call({call_search, _}, _From, St) ->
+ %% TODO: provide isolated search queries here
+ {reply, ok, St};
+handle_call(Msg, _From, St) ->
+ {stop, {unknown_call, Msg}, St}.
+
+handle_cast(Msg, St) ->
+ {stop, {unknown_cast, Msg}, St}.
+
+handle_info(Msg, St) ->
+ {stop, {unknown_info, Msg}, St}.
Review Comment:
I think this is option as well, may have to double-check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743915676
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
Review Comment:
Avoid using one-line functions. It's shorter, but it's unusual for our code base. They may work for a larger table of values sometimes but for these cases we put the body on a new line.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743927693
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
Review Comment:
Seeing that we have to do this both for incrementing and to_json wonder if it's not simpler to just keep a map with keys as map keys.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743939518
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
+ case conf_get("log_pid_usage_report", "coordinator") of
+ "coordinator" ->
+ coordinator;
+ "true" ->
+ true;
+ _ ->
+ false
+ end.
+
+should_log(undefined) ->
+ false;
+should_log(#rctx{}=Rctx) ->
+ should_log(Rctx, logging_enabled()).
+
+should_log(undefined, _) ->
+ false;
+should_log(#rctx{}, true) ->
+ true;
+should_log(#rctx{}, false) ->
+ false;
+should_log(#rctx{type = {coordinator, _, _}}, coordinator) ->
+ true;
+should_log(#rctx{type = {worker, fabric_rpc, FName}}, _) ->
+ case conf_get("log_fabric_rpc") of
+ "true" ->
+ true;
+ undefined ->
+ false;
+ Name ->
+ Name =:= atom_to_list(FName)
+ end;
+should_log(#rctx{}, _) ->
+ false.
+
+%%
+%% gen_server callbacks
+%%
+
+start_link() ->
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+
+init([]) ->
+ ets:new(?MODULE, [
+ named_table,
+ public,
+ {decentralized_counters, true},
+ {write_concurrency, true},
+ {read_concurrency, true},
+ {keypos, #rctx.pid_ref}
+ ]),
+ {ok, #st{}}.
+
+handle_call(fetch, _from, #st{} = St) ->
+ {reply, {ok, St}, St};
+handle_call({call_search, _}, _From, St) ->
+ %% TODO: provide isolated search queries here
+ {reply, ok, St};
+handle_call(Msg, _From, St) ->
+ {stop, {unknown_call, Msg}, St}.
+
+handle_cast(Msg, St) ->
+ {stop, {unknown_cast, Msg}, St}.
+
+handle_info(Msg, St) ->
+ {stop, {unknown_info, Msg}, St}.
Review Comment:
I think this is optional as well, may have to double-check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743943487
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
+ case conf_get("log_pid_usage_report", "coordinator") of
+ "coordinator" ->
+ coordinator;
+ "true" ->
+ true;
+ _ ->
+ false
+ end.
+
+should_log(undefined) ->
+ false;
+should_log(#rctx{}=Rctx) ->
+ should_log(Rctx, logging_enabled()).
+
+should_log(undefined, _) ->
+ false;
+should_log(#rctx{}, true) ->
+ true;
+should_log(#rctx{}, false) ->
+ false;
+should_log(#rctx{type = {coordinator, _, _}}, coordinator) ->
+ true;
+should_log(#rctx{type = {worker, fabric_rpc, FName}}, _) ->
+ case conf_get("log_fabric_rpc") of
+ "true" ->
+ true;
+ undefined ->
+ false;
+ Name ->
+ Name =:= atom_to_list(FName)
+ end;
+should_log(#rctx{}, _) ->
+ false.
+
+%%
+%% gen_server callbacks
+%%
+
+start_link() ->
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+
+init([]) ->
+ ets:new(?MODULE, [
+ named_table,
+ public,
+ {decentralized_counters, true},
+ {write_concurrency, true},
Review Comment:
Consider using `auto` here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743976839
##########
src/rexi/src/rexi_server.erl:
##########
@@ -158,12 +163,15 @@ init_p(From, {M, F, A}, Nonce) ->
]
),
exit(#error{
+ delta = couch_stats_resource_tracker:make_delta(),
timestamp = os:timestamp(),
reason = {Class, Reason},
mfa = {M, F, A},
nonce = Nonce,
stack = Stack
})
+ after
+ couch_stats_resource_tracker:destroy_context()
Review Comment:
If we're handling all the exit cases do we need an extra `after` clause?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1744027266
##########
src/rexi/src/rexi_utils.erl:
##########
@@ -95,6 +107,49 @@ process_message(RefList, Keypos, Fun, Acc0, TimeoutRef, PerMsgTO) ->
end;
{rexi_DOWN, _, _, _} = Msg ->
Fun(Msg, nil, Acc0)
- after PerMsgTO ->
- {timeout, Acc0}
+ end.
+
+add_delta({A}, Delta) -> {A, Delta};
+add_delta({A, B}, Delta) -> {A, B, Delta};
+add_delta({A, B, C}, Delta) -> {A, B, C, Delta};
+add_delta({A, B, C, D}, Delta) -> {A, B, C, D, Delta};
+add_delta({A, B, C, D, E}, Delta) -> {A, B, C, D, E, Delta};
+add_delta({A, B, C, D, E, F}, Delta) -> {A, B, C, D, E, F, Delta};
+add_delta({A, B, C, D, E, F, G}, Delta) -> {A, B, C, D, E, F, G, Delta};
Review Comment:
At first sight this looks a bit awkward. Do we have any `add_delta({A, B, C, D, E, F}, Delta) -> {A, B, C, D, E, F, Delta};` messages sent to rexi?
It might be better to be explicit about what message rexi will actually get. We may have to do a two-stage PRs. One to prep the receivers to accept both old and new patterns, then another one to start sending the new messages.
##########
src/rexi/src/rexi_utils.erl:
##########
@@ -95,6 +107,49 @@ process_message(RefList, Keypos, Fun, Acc0, TimeoutRef, PerMsgTO) ->
end;
{rexi_DOWN, _, _, _} = Msg ->
Fun(Msg, nil, Acc0)
- after PerMsgTO ->
- {timeout, Acc0}
+ end.
+
+add_delta({A}, Delta) -> {A, Delta};
+add_delta({A, B}, Delta) -> {A, B, Delta};
+add_delta({A, B, C}, Delta) -> {A, B, C, Delta};
+add_delta({A, B, C, D}, Delta) -> {A, B, C, D, Delta};
+add_delta({A, B, C, D, E}, Delta) -> {A, B, C, D, E, Delta};
+add_delta({A, B, C, D, E, F}, Delta) -> {A, B, C, D, E, F, Delta};
+add_delta({A, B, C, D, E, F, G}, Delta) -> {A, B, C, D, E, F, G, Delta};
Review Comment:
At first sight this looks a bit awkward. Do we have any `add_delta({A, B, C, D, E, F}, Delta) -> {A, B, C, D, E, F, Delta};` messages sent to rexi?
It might be better to be explicit about what message rexi will actually get. We may have to do a two-stage PR: One to prep the receivers to accept both old and new patterns, then another one to start sending the new messages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1800227379
##########
src/couch/priv/stats_descriptions.cfg:
##########
@@ -418,10 +422,47 @@
{type, counter},
{desc, <<"number of other requests">>}
]}.
+{[couchdb, query_server, js_filter], [
+ {type, counter},
+ {desc, <<"number of JS filter invocations">>}
+]}.
Review Comment:
Ahhh good catch, I didn't see those were hidden in https://github.com/apache/couchdb/blob/main/src/couch/src/couch_os_process.erl#L257-L259
I've reworked this in `4db08c9f2` to use the existing metrics and only add a new metric `[couchdb, query_server, volume, ddoc_filter]` for counting the number of docs funneled through the filter. I also reworked the logic so that volume metric is captured in the same module and style as the `bump_time_stat` function above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1800228448
##########
src/couch/src/couch_query_servers.erl:
##########
@@ -542,6 +542,8 @@ filter_docs(Req, Db, DDoc, FName, Docs) ->
{ok, filter_docs_int(Db, DDoc, FName, JsonReq, JsonDocs)}
catch
throw:{os_process_error, {exit_status, 1}} ->
+ %% TODO: wire in csrt tracking
+ couch_stats:increment_counter([couchdb, query_server, js_filter_error]),
Review Comment:
Fixed in 4db08c9f2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1744023860
##########
src/rexi/src/rexi_utils.erl:
##########
@@ -60,6 +62,16 @@ process_mailbox(RefList, Keypos, Fun, Acc0, TimeoutRef, PerMsgTO) ->
process_message(RefList, Keypos, Fun, Acc0, TimeoutRef, PerMsgTO) ->
receive
+ Msg ->
Review Comment:
This changes the rexi message receive pattern by accepting all messages, while previously we left unknown ones in the mailbox. We may crash in `case Payload` if we get an unknown message, while previously we left it in the mailbox.
It might be better if we explicitly receive only the messages we expect and not all of them. That was actually the impetus for `{rexi, ...}` message patterns. There we bugs related to rexi utils receiving 2 items tuples and ignoring them. That effort only went half-way by adding clauses to receive here but nowhere else. Perhaps we can resurrect it and enhance those {rexi, ... } message to take an extra map with options where delta or other context can do into?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743901365
##########
src/couch_stats/src/couch_stats.erl:
##########
@@ -100,6 +111,25 @@ stats() ->
now_sec() ->
erlang:monotonic_time(second).
+%% Only potentially track positive increments to counters
+-spec maybe_track_local_counter(any(), any()) -> ok.
+maybe_track_local_counter(Name, Val) when is_integer(Val) andalso Val > 0 ->
+ %%io:format("maybe_track_local[~p]: ~p~n", [Val, Name]),
Review Comment:
debug left-over?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1807080453
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
+ case conf_get("log_pid_usage_report", "coordinator") of
+ "coordinator" ->
+ coordinator;
+ "true" ->
+ true;
+ _ ->
+ false
+ end.
+
+should_log(undefined) ->
+ false;
+should_log(#rctx{}=Rctx) ->
+ should_log(Rctx, logging_enabled()).
+
+should_log(undefined, _) ->
+ false;
+should_log(#rctx{}, true) ->
+ true;
+should_log(#rctx{}, false) ->
+ false;
+should_log(#rctx{type = {coordinator, _, _}}, coordinator) ->
+ true;
+should_log(#rctx{type = {worker, fabric_rpc, FName}}, _) ->
+ case conf_get("log_fabric_rpc") of
+ "true" ->
+ true;
+ undefined ->
+ false;
+ Name ->
+ Name =:= atom_to_list(FName)
+ end;
+should_log(#rctx{}, _) ->
+ false.
+
+%%
+%% gen_server callbacks
+%%
+
+start_link() ->
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+
+init([]) ->
+ ets:new(?MODULE, [
+ named_table,
+ public,
+ {decentralized_counters, true},
+ {write_concurrency, true},
+ {read_concurrency, true},
+ {keypos, #rctx.pid_ref}
+ ]),
+ {ok, #st{}}.
+
+handle_call(fetch, _from, #st{} = St) ->
+ {reply, {ok, St}, St};
+handle_call({call_search, _}, _From, St) ->
+ %% TODO: provide isolated search queries here
+ {reply, ok, St};
Review Comment:
Yeah, I left this in as a stub as I think it's likely we want to regulate the concurrency of ets fold operations on the internal CSRT table. Doing various ets operations induces https://www.erlang.org/docs/23/man/ets#safe_fixtable-2 which in theory means that given a constant volume of search queries the ets table will never be able to garbage collect. So far I've done my testing with a small amount of search operations being performed, but I'd like to do more load testing where we increase the magnitude of ets operations induced and see if we need to serialize these operations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1815841142
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
Review Comment:
Thanks for the feedback in this comment and the above comment around filtering by types. I was curious how ets would handle nested record types but everything works fine, so I've reworked things to use defined record types in 4c091bcb5 and it cleaned things up nicely.
The only awkward bit there is that it complicates finding _all_ workers and the coordinator in a single query, as I'm not sure match specifications will allow you to match against two different records. I've still got the `nonce` value in the base `#rctx{}` record so all related (CSRT tracked) processes to a given http request can be found by way of matching on `#rctx{nonce=Nonce}`, and then if you know the `PidRef` or a coordinator, you can find all related workers by way of filtering with: `#rctx{type=#rpc_worker{from=PidRef}}`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1811599135
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
Review Comment:
> What is the intent ^^^? Do we want to enable logging regardless of it being coordinator or true?
The intent here is two fold:
First, the basic intent is to provide a mechanism to selectively enable the report logging, as enabling full logging for all induced RPC workers is useful information but can be sizable given the current verbose output format, so I think we need to have some approaches for selectively enabling desired output. I don't think my current approach here is sufficient, and you can see in `should_log` I've got some similar logic around selectively enabling logging for particular `fabric_rpc` worker types, but both of these approaches are one dimensional and leave much to be desired. Feedback welcome on approaches here, but I'm currently contemplating an `ets:fun2ms` based conditional logger, allowing us to utilize match specs as a mechanism for declaring logging filters. This would make it easy to have some default logging filters declared, yet allow for custom filters to easily be declared and registered in a performant manner (although `local.ini` based definitions might be tricky). The
n when the final record is loaded out of ets, we can match it directly against the match funs with https://www.erlang.org/docs/23/man/ets#match_spec_run-2 and use the logic for querying and log filtering.
Secondly, the idea is to short circuit out of logging as soon as possible to avoid a) loading the record and b) the additional logic checks and ets lookups around worker specific filtering (eg enabling logging for _only_ `{fabric_rpc, map_doc}` workers on a specific node). That's why the logic here is a somewhat convoluted, there's a bit of a chicken and egg situation of determining whether or not to log without loading the record from ets to, for instance, a) check if it's a worker type, and b) that worker is of type `{worker, fabric_rpc, map_doc}`.
Again, this type of filtering again would be nicely expressed in terms of match funs, but pragmatically speaking we could do the first level filtering by way of a `local.ini` section `[couch_stats_resource_tracker.should_log]` and entries like `fabric_rpc.map_doc = true` to construct a map in a gen_server of workers to log. This would be useful, but as earlier is still fairly one dimensional in that it makes it complicated to express complex filters, like reduce workers on a particular database, let alone more complex queries like "find all rpc workers against $shard for $ddoc with docs loaded greater than 50 and IOQ calls greater than 700".
The `ets:match_spec_run` occurred to me while writing this up, so I'll explore that a bit more, but feedback welcome here too, as I agree the logging enablement and filtering here is not clean or sufficient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743938624
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
+ case conf_get("log_pid_usage_report", "coordinator") of
+ "coordinator" ->
+ coordinator;
+ "true" ->
+ true;
+ _ ->
+ false
+ end.
+
+should_log(undefined) ->
+ false;
+should_log(#rctx{}=Rctx) ->
+ should_log(Rctx, logging_enabled()).
+
+should_log(undefined, _) ->
+ false;
+should_log(#rctx{}, true) ->
+ true;
+should_log(#rctx{}, false) ->
+ false;
+should_log(#rctx{type = {coordinator, _, _}}, coordinator) ->
+ true;
+should_log(#rctx{type = {worker, fabric_rpc, FName}}, _) ->
+ case conf_get("log_fabric_rpc") of
+ "true" ->
+ true;
+ undefined ->
+ false;
+ Name ->
+ Name =:= atom_to_list(FName)
+ end;
+should_log(#rctx{}, _) ->
+ false.
+
+%%
+%% gen_server callbacks
+%%
+
+start_link() ->
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+
+init([]) ->
+ ets:new(?MODULE, [
+ named_table,
+ public,
+ {decentralized_counters, true},
+ {write_concurrency, true},
+ {read_concurrency, true},
+ {keypos, #rctx.pid_ref}
+ ]),
+ {ok, #st{}}.
+
+handle_call(fetch, _from, #st{} = St) ->
+ {reply, {ok, St}, St};
+handle_call({call_search, _}, _From, St) ->
+ %% TODO: provide isolated search queries here
+ {reply, ok, St};
Review Comment:
This doesn't do anything currently
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743973871
##########
src/rexi/src/rexi_monitor.erl:
##########
@@ -35,6 +35,7 @@ start(Procs) ->
%% messages from our mailbox.
-spec stop(pid()) -> ok.
stop(MonitoringPid) ->
+ unlink(MonitoringPid),
Review Comment:
Wonder why we have to do this? Was there a crash that happened with stats tracking? We're exiting the monitor pid anyway. If this is not related to the stats tracking might be a good one for a separate PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743845878
##########
src/chttpd/src/chttpd_misc.erl:
##########
@@ -224,6 +225,110 @@ handle_task_status_req(#httpd{method = 'GET'} = Req) ->
handle_task_status_req(Req) ->
send_method_not_allowed(Req, "GET,HEAD").
+handle_resource_status_req(#httpd{method = 'POST'} = Req) ->
+ ok = chttpd:verify_is_server_admin(Req),
+ chttpd:validate_ctype(Req, "application/json"),
+ {Props} = chttpd:json_body_obj(Req),
+ Action = proplists:get_value(<<"action">>, Props),
+ Key = proplists:get_value(<<"key">>, Props),
+ Val = proplists:get_value(<<"val">>, Props),
+
+ CountBy = fun couch_stats_resource_tracker:count_by/1,
+ GroupBy = fun couch_stats_resource_tracker:group_by/2,
+ SortedBy1 = fun couch_stats_resource_tracker:sorted_by/1,
+ SortedBy2 = fun couch_stats_resource_tracker:sorted_by/2,
+ ConvertEle = fun(K) -> list_to_existing_atom(binary_to_list(K)) end,
+ ConvertList = fun(L) -> [ConvertEle(E) || E <- L] end,
+ ToJson = fun couch_stats_resource_tracker:term_to_flat_json/1,
+ JsonKeys = fun(PL) -> [[ToJson(K), V] || {K, V} <- PL] end,
+
+ Fun = case {Action, Key, Val} of
+ {<<"count_by">>, Keys, undefined} when is_list(Keys) ->
+ Keys1 = [ConvertEle(K) || K <- Keys],
+ fun() -> CountBy(Keys1) end;
+ {<<"count_by">>, Key, undefined} ->
+ Key1 = ConvertEle(Key),
+ fun() -> CountBy(Key1) end;
+ {<<"group_by">>, Keys, Vals} when is_list(Keys) andalso is_list(Vals) ->
+ Keys1 = ConvertList(Keys),
+ Vals1 = ConvertList(Vals),
+ fun() -> GroupBy(Keys1, Vals1) end;
+ {<<"group_by">>, Key, Vals} when is_list(Vals) ->
+ Key1 = ConvertEle(Key),
+ Vals1 = ConvertList(Vals),
+ fun() -> GroupBy(Key1, Vals1) end;
+ {<<"group_by">>, Keys, Val} when is_list(Keys) ->
+ Keys1 = ConvertList(Keys),
+ Val1 = ConvertEle(Val),
+ fun() -> GroupBy(Keys1, Val1) end;
+ {<<"group_by">>, Key, Val} ->
+ Key1 = ConvertEle(Key),
+ Val1 = ConvertList(Val),
+ fun() -> GroupBy(Key1, Val1) end;
+
+ {<<"sorted_by">>, Key, undefined} ->
+ Key1 = ConvertEle(Key),
+ fun() -> JsonKeys(SortedBy1(Key1)) end;
+ {<<"sorted_by">>, Keys, undefined} when is_list(Keys) ->
+ Keys1 = [ConvertEle(K) || K <- Keys],
+ fun() -> JsonKeys(SortedBy1(Keys1)) end;
+ {<<"sorted_by">>, Keys, Vals} when is_list(Keys) andalso is_list(Vals) ->
+ Keys1 = ConvertList(Keys),
+ Vals1 = ConvertList(Vals),
+ fun() -> JsonKeys(SortedBy2(Keys1, Vals1)) end;
+ {<<"sorted_by">>, Key, Vals} when is_list(Vals) ->
+ Key1 = ConvertEle(Key),
+ Vals1 = ConvertList(Vals),
+ fun() -> JsonKeys(SortedBy2(Key1, Vals1)) end;
+ {<<"sorted_by">>, Keys, Val} when is_list(Keys) ->
+ Keys1 = ConvertList(Keys),
+ Val1 = ConvertEle(Val),
+ fun() -> JsonKeys(SortedBy2(Keys1, Val1)) end;
+ {<<"sorted_by">>, Key, Val} ->
+ Key1 = ConvertEle(Key),
+ Val1 = ConvertList(Val),
+ fun() -> JsonKeys(SortedBy2(Key1, Val1)) end;
+ _ ->
+ throw({badrequest, invalid_resource_request})
+ end,
+
+ Fun1 = fun() ->
+ case Fun() of
+ Map when is_map(Map) ->
+ {maps:fold(
+ fun
+ (_K,0,A) -> A; %% TODO: Skip 0 value entries?
+ (K,V,A) -> [{ToJson(K), V} | A]
+ end,
+ [], Map)};
+ List when is_list(List) ->
+ List
+ end
+ end,
+
+ {Resp, _Bad} = rpc:multicall(erlang, apply, [
Review Comment:
erpc might be a better option https://www.erlang.org/doc/apps/kernel/erpc.html#multicall/3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743956426
##########
src/fabric/priv/stats_descriptions.cfg:
##########
@@ -26,3 +26,53 @@
{type, counter},
{desc, <<"number of write quorum errors">>}
]}.
+
+
+%% fabric_rpc worker stats
+%% TODO: decide on which naming scheme:
+%% {[fabric_rpc, get_all_security, spawned], [
+%% {[fabric_rpc, spawned, get_all_security], [
+{[fabric_rpc, get_all_security, spawned], [
Review Comment:
For non-streaming requests the spawn seem redundant. For instance we don't have get_all_security spawns, then row sends. Could just have them all as `fabric_rpc.$call`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1807075010
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
+ case conf_get("log_pid_usage_report", "coordinator") of
+ "coordinator" ->
+ coordinator;
+ "true" ->
+ true;
+ _ ->
+ false
+ end.
+
+should_log(undefined) ->
+ false;
+should_log(#rctx{}=Rctx) ->
+ should_log(Rctx, logging_enabled()).
+
+should_log(undefined, _) ->
+ false;
+should_log(#rctx{}, true) ->
+ true;
+should_log(#rctx{}, false) ->
+ false;
+should_log(#rctx{type = {coordinator, _, _}}, coordinator) ->
+ true;
+should_log(#rctx{type = {worker, fabric_rpc, FName}}, _) ->
+ case conf_get("log_fabric_rpc") of
+ "true" ->
+ true;
+ undefined ->
+ false;
+ Name ->
+ Name =:= atom_to_list(FName)
+ end;
+should_log(#rctx{}, _) ->
+ false.
+
+%%
+%% gen_server callbacks
+%%
+
+start_link() ->
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+
+init([]) ->
+ ets:new(?MODULE, [
+ named_table,
+ public,
+ {decentralized_counters, true},
+ {write_concurrency, true},
Review Comment:
Not sure about non-smp builds, but I guess I don't see if OTP25+ versions we wouldn't use `{write_concurency, auto}`.
This is just from taking the Erlang docs at their word here:
> {decentralized_counters,boolean()}(Since OTP 23.0) Performance tuning. Defaults to true for all tables with the write_concurrency option set to auto.
Are you thinking maybe they didn't document it properly and we should set both anyway? We should take a look a their ets source then, to verify.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1807069156
##########
src/fabric/priv/stats_descriptions.cfg:
##########
@@ -26,3 +26,53 @@
{type, counter},
{desc, <<"number of write quorum errors">>}
]}.
+
+
+%% fabric_rpc worker stats
+%% TODO: decide on which naming scheme:
+%% {[fabric_rpc, get_all_security, spawned], [
+%% {[fabric_rpc, spawned, get_all_security], [
+{[fabric_rpc, get_all_security, spawned], [
Review Comment:
Another note, `spawned` is ubiquitous as these processes are all being spawned in the same manner that we track directly, but the `processed` vs `returned` vs `etc` is far more operation specific and we should look at what other types of data are important to track here, for instance, what would be pertinent data to track out of Clouseau/Nouveau requests? Similarly, even singular doc API operations might have relevant magnitudes, eg we should probably track the number of doc revisions the worker had to load to complete the request.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1814085042
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
Review Comment:
Alright, I hacked together a `csrt_logger` module in 17021155d that provides two extended mechanisms for dynamically enabling logging, in both a simpler and more complex manner. I just hacked this together, there's no compiler warnings but I haven't actually run it yet or anything wild like that.
The first dynamic logging approach utilizes @iilyak's idea for dedicated type records, initially focusing on `#worker{}` type here https://github.com/apache/couchdb/commit/17021155db6ae0495e015d7571469a36c426b894#diff-ae4228aa19a3865ec48888021914ba2db0cf64dca17138497c8db885222f4699R77-R90 eg `#rpc_worker{mod=fabric_rpc, func=map_view}`. We then translate that to a key string like `worker.fabric_rpc.map_view` so we can then do `config:get_boolean("csrt.should_log", Key, true)` to have a direct toggle switch for the core default rpc operations we want to log, and we could use a similar mechanism to dynamically enable logging for additional rpc endpoints we don't enable by default, eg looking at dreyfus_rpc or mem3_rpc.
We generate that mapping in a gen_server so that we can easily do direct atom comparisons on module/function names rather than have to repeatedly do string conversion and comparisons, the strings are only changed at configuration initialization/update time and not used in the should log checks. We can then check to see if that module worker type is tracked directly https://github.com/apache/couchdb/commit/17021155db6ae0495e015d7571469a36c426b894#diff-ae4228aa19a3865ec48888021914ba2db0cf64dca17138497c8db885222f4699R220-R221
The next mechanism and core idea with this experimental hack in the commit is to create a way to dynamically register match spec funs to evaluate against the stream of reports. The idea here is to `ets:match_spec_compile` to generate a compiled match_spec we can store a reference to in the gen_server, then we can dynamically enable filtered logging by way of declaring and registering match_specs. For example, here's our matcher registration handling https://github.com/apache/couchdb/commit/17021155db6ae0495e015d7571469a36c426b894#diff-ae4228aa19a3865ec48888021914ba2db0cf64dca17138497c8db885222f4699R171-R174 along with the compilation https://github.com/apache/couchdb/commit/17021155db6ae0495e015d7571469a36c426b894#diff-ae4228aa19a3865ec48888021914ba2db0cf64dca17138497c8db885222f4699R209-R218 and then we can determine if we should logic the particular `#rctx{}` by way of checking if it matches any of the registered filteres here https://github.com/apache/couchdb/commit/17021155db6a
e0495e015d7571469a36c426b894#diff-ae4228aa19a3865ec48888021914ba2db0cf64dca17138497c8db885222f4699R133-R142
For example, here's a function to dynamically generate a matchspec parameterized by username:
```erlang
(node1@127.0.0.1)4> F = fun(User) -> ets:fun2ms(fun(#rctx{username=User0}=R) when User =:= User0 -> R end) end.
#Fun<erl_eval.42.3316493>
(node1@127.0.0.1)5> F(bar).
[{#rctx{started_at = '_',updated_at = '_',pid_ref = '_',
mfa = '_',nonce = '_',from = '_',type = '_',dbname = '_',
username = '$1',path = '_',db_open = '_',docs_read = '_',
rows_read = '_',changes_processed = '_',
changes_returned = '_',ioq_calls = '_',io_bytes_read = '_',
io_bytes_written = '_',js_evals = '_',js_filter = '_',
js_filtered_docs = '_',mango_eval_match = '_',
get_kv_node = '_',get_kp_node = '_',write_kv_node = '_',
write_kp_node = '_'},
[{'=:=',{const,bar},'$1'}],
['$_']}]
```
Then we can compile that match spec and store the reference in the gen_server, and check to see if any of the compiled matchers match the given `#rctx{}` record. This opens up considerable options for doing dynamic filtered logging of a subset of the csrt report stream, although if we adopt a model like this I think this would be great to extend out to the main logging systems as well, so for instance we could dynamically enable debug logs for a particular database shard. I think it might be trickier to dynamically define relations, eg `username=Foo and docs_opened>100`, but it _is_ easy to predefine a series of those types of functions for common use cases like filter on user and doc magnitude, or user and db shard plus magnitude, or whatever. Perhaps there's some other clever tricks we can do here too, it's a little tricky with having to define the match record within the macro expansion call, but as I demonstrated above we can dynamically generate parameterized match specs that
are then compiled and shareable, so I think there's a lot of potential with this current approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743932192
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
Review Comment:
Commented debug line?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743857023
##########
src/chttpd/src/chttpd_misc.erl:
##########
@@ -224,6 +225,110 @@ handle_task_status_req(#httpd{method = 'GET'} = Req) ->
handle_task_status_req(Req) ->
send_method_not_allowed(Req, "GET,HEAD").
+handle_resource_status_req(#httpd{method = 'POST'} = Req) ->
+ ok = chttpd:verify_is_server_admin(Req),
+ chttpd:validate_ctype(Req, "application/json"),
+ {Props} = chttpd:json_body_obj(Req),
+ Action = proplists:get_value(<<"action">>, Props),
+ Key = proplists:get_value(<<"key">>, Props),
+ Val = proplists:get_value(<<"val">>, Props),
Review Comment:
Since we're introducing a new API add some docs or some description (examples) how it works, and the intent behind it. What would CouchDB users use it for, how it is different than metrics, and active tasks, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743857023
##########
src/chttpd/src/chttpd_misc.erl:
##########
@@ -224,6 +225,110 @@ handle_task_status_req(#httpd{method = 'GET'} = Req) ->
handle_task_status_req(Req) ->
send_method_not_allowed(Req, "GET,HEAD").
+handle_resource_status_req(#httpd{method = 'POST'} = Req) ->
+ ok = chttpd:verify_is_server_admin(Req),
+ chttpd:validate_ctype(Req, "application/json"),
+ {Props} = chttpd:json_body_obj(Req),
+ Action = proplists:get_value(<<"action">>, Props),
+ Key = proplists:get_value(<<"key">>, Props),
+ Val = proplists:get_value(<<"val">>, Props),
Review Comment:
Since we're introducing a new API add some docs or some description (examples) how it works, and the intent behind it. What would CouchDB users use it, how it is different than metrics, and active tasks, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743906783
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
Review Comment:
Consider just doing one per-line for consistency with most of the code-base.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1819287766
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
Review Comment:
I think the second approach (`persistent_term:put(all_the_filters, maps:put(....)`) would be better. Since as you've mentioned we would need to retrieve all the filters anyway. My only suggestion would be to follow the recommendation from the docs and name the key as `{?MODULE, all_the_filters}`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1807052709
##########
src/fabric/priv/stats_descriptions.cfg:
##########
@@ -26,3 +26,53 @@
{type, counter},
{desc, <<"number of write quorum errors">>}
]}.
+
+
+%% fabric_rpc worker stats
+%% TODO: decide on which naming scheme:
+%% {[fabric_rpc, get_all_security, spawned], [
+%% {[fabric_rpc, spawned, get_all_security], [
+{[fabric_rpc, get_all_security, spawned], [
Review Comment:
It's `spawned` because it directly corresponds to `rexi_server` spawning the RPC worker processes by way of `rexi_server:init_p` here https://github.com/apache/couchdb/blob/couch-stats-resource-tracker-v2/src/rexi/src/rexi_server.erl#L143 and we handle that here https://github.com/apache/couchdb/blob/couch-stats-resource-tracker-v2/src/couch_stats/src/couch_stats.erl#L126-L131.
The core goal here is to track the volume or RPC work being spawned on the system, the amount of data they're processing, and the amount of data returned, as appropriate. All RPC workers are spawned processes, and tracking the rate of arrival is an important metric we're missing, so we track `spawned` for the core API operations (the should_track logic keeps it to a targeted subset of API operations to focus on commonly useful metrics, but it's arguable we should extend this tracking to _all_ induced RPC work). Many of the API operations stream out many rows, so we want to be able to track the volume of `returned` rows/data, and then similarly, some of the API operations filter rows and return a subset of the data.
Currently it's way too difficult to correlate a CouchDB node's IOQ usage with the underlying database operations being performed. Seeing the rate of incoming work, the amount of data processed, and the amount returned opens up a lot of visibility on that front, especially around things like filtered operations where we haven't had metrics around the skipped work that is tricky to introspect.
If you're comment is more around the naming hierarchy, my approach was to have a consistent naming hierarchy with `[fabric_rpc, $call, spawned]` for all the tracked RPC workers, and then similarly with `processed` or `returned`, I figured this naming hierarchy works well with various metrics stacks for doing things like `graph([fabric_rpc, *, spawned])` to easily see the rate of induced workers, like I did in the Graphite links in the top description of the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1816971852
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
Review Comment:
> The idea here is to ets:match_spec_compile to generate a compiled match_spec we can store a reference to in the gen_server, then we can dynamically enable filtered logging by way of declaring and registering match_specs.
A combination of ` ets:match_spec_compile` and `ets:match_spec_run` is a very neat idea. Did you consider https://www.erlang.org/doc/apps/erts/persistent_term.html instead of gen_server? I wish persistent_term had `persistent_term:from_map/1`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1800233208
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
Review Comment:
Yeah, I needed a way to map the metrics to the actual field names of the record given we don't have that at compile time. I didn't want to have to instantiate a new map everytime `inc` got called, so I ended up using a static map for the named translations, and I added some basic tests to assert the fields are as expected in 1d3443232.
I also simplified `to_json` down to a simple record to map translation in ce91a4008.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1817430071
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
Review Comment:
> previous code where shape of worker and coordinator was the same (a tuple with three elements). You invalidated that assumption by adding extra fields into each records.
Yeah, it kind of fell out naturally from switching the types to records, for instance, `path` is clearly only relevant for the http coordinator, but we stored that for all `#rctx{}` entries, and similarly, the RPC side has a `from` context tracking the coordinator.
I think there's a balance with keeping common things like `nonce`, `dbname`, `username`, `etc` in the top level record for easy global querying, while storing context specific information in the type records. For example, this will make it easy to add additional worker types for indexing, compaction, replication, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743904374
##########
src/couch_stats/src/couch_stats.erl:
##########
@@ -49,6 +55,11 @@ increment_counter(Name) ->
-spec increment_counter(any(), pos_integer()) -> response().
increment_counter(Name, Value) ->
+ %% Should maybe_track_local happen before or after notify?
+ %% If after, only currently tracked metrics declared in the app's
+ %% stats_description.cfg will be trackable locally. Pros/cons.
+ %io:format("NOTIFY_EXISTING_METRIC: ~p || ~p || ~p~n", [Name, Op, Type]),
+ ok = maybe_track_local_counter(Name, Value),
Review Comment:
If this is enabled for all requests, a bit worried about the performance here as we add an ets operation over a simple integer counter bump. Was there any noticeable performance impact from it, during perf runs?
##########
src/couch_stats/src/couch_stats.erl:
##########
@@ -49,6 +55,11 @@ increment_counter(Name) ->
-spec increment_counter(any(), pos_integer()) -> response().
increment_counter(Name, Value) ->
+ %% Should maybe_track_local happen before or after notify?
+ %% If after, only currently tracked metrics declared in the app's
+ %% stats_description.cfg will be trackable locally. Pros/cons.
+ %io:format("NOTIFY_EXISTING_METRIC: ~p || ~p || ~p~n", [Name, Op, Type]),
Review Comment:
Left-over debug statement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743839175
##########
src/chttpd/src/chttpd_misc.erl:
##########
@@ -224,6 +225,110 @@ handle_task_status_req(#httpd{method = 'GET'} = Req) ->
handle_task_status_req(Req) ->
send_method_not_allowed(Req, "GET,HEAD").
+handle_resource_status_req(#httpd{method = 'POST'} = Req) ->
+ ok = chttpd:verify_is_server_admin(Req),
+ chttpd:validate_ctype(Req, "application/json"),
+ {Props} = chttpd:json_body_obj(Req),
+ Action = proplists:get_value(<<"action">>, Props),
+ Key = proplists:get_value(<<"key">>, Props),
+ Val = proplists:get_value(<<"val">>, Props),
+
+ CountBy = fun couch_stats_resource_tracker:count_by/1,
+ GroupBy = fun couch_stats_resource_tracker:group_by/2,
+ SortedBy1 = fun couch_stats_resource_tracker:sorted_by/1,
+ SortedBy2 = fun couch_stats_resource_tracker:sorted_by/2,
+ ConvertEle = fun(K) -> list_to_existing_atom(binary_to_list(K)) end,
Review Comment:
https://www.erlang.org/doc/apps/erts/erlang.html#binary_to_existing_atom/1 might work here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1750679611
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
Review Comment:
Please add a comment here that `is_logging_enabled()` suppose to be `true` for both `true` and `coordinator`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1800230361
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
+ case conf_get("log_pid_usage_report", "coordinator") of
+ "coordinator" ->
+ coordinator;
+ "true" ->
+ true;
+ _ ->
+ false
+ end.
+
+should_log(undefined) ->
+ false;
+should_log(#rctx{}=Rctx) ->
+ should_log(Rctx, logging_enabled()).
+
+should_log(undefined, _) ->
+ false;
+should_log(#rctx{}, true) ->
+ true;
+should_log(#rctx{}, false) ->
+ false;
+should_log(#rctx{type = {coordinator, _, _}}, coordinator) ->
+ true;
+should_log(#rctx{type = {worker, fabric_rpc, FName}}, _) ->
+ case conf_get("log_fabric_rpc") of
+ "true" ->
+ true;
+ undefined ->
+ false;
+ Name ->
+ Name =:= atom_to_list(FName)
+ end;
+should_log(#rctx{}, _) ->
+ false.
+
+%%
+%% gen_server callbacks
+%%
+
+start_link() ->
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+
+init([]) ->
+ ets:new(?MODULE, [
+ named_table,
+ public,
+ {decentralized_counters, true},
+ {write_concurrency, true},
+ {read_concurrency, true},
+ {keypos, #rctx.pid_ref}
+ ]),
+ {ok, #st{}}.
+
+handle_call(fetch, _from, #st{} = St) ->
+ {reply, {ok, St}, St};
+handle_call({call_search, _}, _From, St) ->
+ %% TODO: provide isolated search queries here
+ {reply, ok, St};
+handle_call(Msg, _From, St) ->
+ {stop, {unknown_call, Msg}, St}.
+
+handle_cast(Msg, St) ->
+ {stop, {unknown_cast, Msg}, St}.
+
+handle_info(Msg, St) ->
+ {stop, {unknown_info, Msg}, St}.
+
+terminate(_Reason, _St) ->
+ ok.
Review Comment:
Fixed in de9a43047.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1800230079
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
Review Comment:
Fixed in 5ac8a91a4.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1800229643
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
Review Comment:
Fixed in 46fba8e0f.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1807075010
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
+ case conf_get("log_pid_usage_report", "coordinator") of
+ "coordinator" ->
+ coordinator;
+ "true" ->
+ true;
+ _ ->
+ false
+ end.
+
+should_log(undefined) ->
+ false;
+should_log(#rctx{}=Rctx) ->
+ should_log(Rctx, logging_enabled()).
+
+should_log(undefined, _) ->
+ false;
+should_log(#rctx{}, true) ->
+ true;
+should_log(#rctx{}, false) ->
+ false;
+should_log(#rctx{type = {coordinator, _, _}}, coordinator) ->
+ true;
+should_log(#rctx{type = {worker, fabric_rpc, FName}}, _) ->
+ case conf_get("log_fabric_rpc") of
+ "true" ->
+ true;
+ undefined ->
+ false;
+ Name ->
+ Name =:= atom_to_list(FName)
+ end;
+should_log(#rctx{}, _) ->
+ false.
+
+%%
+%% gen_server callbacks
+%%
+
+start_link() ->
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+
+init([]) ->
+ ets:new(?MODULE, [
+ named_table,
+ public,
+ {decentralized_counters, true},
+ {write_concurrency, true},
Review Comment:
Not sure about non-smp builds, but I guess I don't see why for OTP25+ versions we wouldn't use `{write_concurency, auto}`.
This is just from taking the Erlang docs at their word here:
> {decentralized_counters,boolean()}(Since OTP 23.0) Performance tuning. Defaults to true for all tables with the write_concurrency option set to auto.
Are you thinking maybe they didn't document it properly and we should set both anyway? We should take a look a their ets source then, to verify.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1817427800
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
Review Comment:
Ahhhhh that's an interesting idea with using persistent_term, I need to remember that's a thing now.
The purpose of the gen_server was two fold: 1) to have a way of registering and preserving the filters; 2) to have a mechanism for getting the registered filters applied to the reports as the the CSRT contexts are cleaned up.
There's various ways to do 1), but for 2) my experiment above used a gen_server so that it could store the filters and then have the logs funneled through the gen_server, which likely should use sharded gen_servers like we've done elsewhere.
This isn't strictly necessary though, as the current PR approach uses the dedicated monitor process to generate the report https://github.com/apache/couchdb/blob/couch-stats-resource-tracker-v2/src/couch_stats/src/couch_stats_resource_tracker.erl#L734 and avoids needing dedicated servers to filter the operations. The tricky bit was about how to get the registered filters down to the monitor processes so they could determine locally whether to log the report or not.
The use of persistent terms is an interesting approach for declaring the filter matchers, especially since the output `ets:match_spec_compile` is a reference to an internal representation, so fetching the set of references across the various workers and coordinators to then apply locally seems worth exploring more, especially since it allows us to stick with the decentralized monitor processes rather than creating new high throughput gen_server's.
We still have the aspect of discovery, I'm curious what you had in mind on this front @iilyak, but from the persistent term API unless we use some other mechanism for dynamically declaring the set of enabled filters, we'll have to do something with an expected naming scheme and iterate over the entries. Maybe something like
```erlang
register_filter(Name, CompMatchSpec) when is_binary(Name0) ->
Name = <<"csrt-logging-filter::", Name0/binary>>,
persistent_term:put(Name, CompMatchSpec).
get_active_filters() ->
lists:filter(fun (<<"csrt-logging-filter::", _/Binary>>) -> true; (_) -> false end, persistent_term:get()).
```
Basically using https://www.erlang.org/doc/apps/erts/persistent_term.html#get/0 to iterate over all declared persistent terms and then looking for an expected prefix naming scheme. Did you have something else in mind? I suppose we could also have a gen_server for handling the state and declarations, but there still needs to be some mechanism for publishing what is active. That might also be useful if we want to have the ability to register TTL filtering, but I'm not convinced that's worth complicating the design over. I'm curious what thoughts you had on this front, how would a given CSRT monitor process query the registered filters?
Hmmm... I suppose we could make it even simpler and just do `persistent_term:put(all_the_filters, #{...}` and then just store and fetch a map of those entries. That might be a decent idea, we need to iterate through _all_ the filters so we might as well grab them in one query. That also avoids having to do the dynamic naming sequences. Then our earlier functions become:
```erlang
register_filter(Name, CompMatchSpec) ->
Filters = persistent_term:get(all_the_filters, #{}),
persistent_term:put(all_the_filters, maps:put(Name, CompMatchSpec, Filters)).
get_active_filters() ->
persistent_term:get(all_the_filters, #{}).
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1816947917
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
Review Comment:
My suggestion was based on the previous code where shape of worker and coordinator was the same (a tuple with three elements). You invalidated that assumption by adding extra fields into each records. I didn't thought about this possibility.
Technically you can match on two different shapes using something like
```
ets:fun2ms(fun
(#rctx{type = #coordinator{}} = R) -> R;
(#rctx{type = #rpc_worker{}} = R) -> R
end).
```
However this would not be efficient. Since it is translated into this spec:
```
2> ets:fun2ms(fun
2> (#rctx{type = #coordinator{}} = R) -> R;
2> (#rctx{type = #rpc_worker{}} = R) -> R
2> end).
[{#rctx{started_at = '_',updated_at = '_',exited_at = '_',
pid_ref = '_',mon_ref = '_',mfa = '_',nonce = '_',
from = '_',
type = #coordinator{mod = '_',func = '_',method = '_',
path = '_'},
state = '_',dbname = '_',username = '_',db_open = '_',
docs_read = '_',rows_read = '_',btree_folds = '_',
changes_processed = '_',changes_returned = '_',
ioq_calls = '_',io_bytes_read = '_',io_bytes_written = '_',
js_evals = '_',js_filter = '_',js_filter_error = '_',
js_filtered_docs = '_',mango_eval_match = '_',...},
[],
['$_']},
{#rctx{started_at = '_',updated_at = '_',exited_at = '_',
pid_ref = '_',mon_ref = '_',mfa = '_',nonce = '_',
from = '_',
type = #rpc_worker{mod = '_',func = '_',from = '_'},
state = '_',dbname = '_',username = '_',db_open = '_',
docs_read = '_',rows_read = '_',btree_folds = '_',
changes_processed = '_',changes_returned = '_',
ioq_calls = '_',io_bytes_read = '_',io_bytes_written = '_',
js_evals = '_',js_filter = '_',js_filter_error = '_',
js_filtered_docs = '_',...},
[],
['$_']}]
```
However in your specific case the performance hit can be acceptable. Since you are matching on nonce first. I think matching process of a tuple stops on first non matching element. So you wouldn't reach the record check for cases where Nonce is different from the value you've provided.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743859770
##########
src/chttpd/src/chttpd_misc.erl:
##########
@@ -224,6 +225,110 @@ handle_task_status_req(#httpd{method = 'GET'} = Req) ->
handle_task_status_req(Req) ->
send_method_not_allowed(Req, "GET,HEAD").
+handle_resource_status_req(#httpd{method = 'POST'} = Req) ->
+ ok = chttpd:verify_is_server_admin(Req),
+ chttpd:validate_ctype(Req, "application/json"),
+ {Props} = chttpd:json_body_obj(Req),
+ Action = proplists:get_value(<<"action">>, Props),
+ Key = proplists:get_value(<<"key">>, Props),
+ Val = proplists:get_value(<<"val">>, Props),
+
+ CountBy = fun couch_stats_resource_tracker:count_by/1,
+ GroupBy = fun couch_stats_resource_tracker:group_by/2,
+ SortedBy1 = fun couch_stats_resource_tracker:sorted_by/1,
+ SortedBy2 = fun couch_stats_resource_tracker:sorted_by/2,
+ ConvertEle = fun(K) -> list_to_existing_atom(binary_to_list(K)) end,
+ ConvertList = fun(L) -> [ConvertEle(E) || E <- L] end,
+ ToJson = fun couch_stats_resource_tracker:term_to_flat_json/1,
+ JsonKeys = fun(PL) -> [[ToJson(K), V] || {K, V} <- PL] end,
+
+ Fun = case {Action, Key, Val} of
+ {<<"count_by">>, Keys, undefined} when is_list(Keys) ->
+ Keys1 = [ConvertEle(K) || K <- Keys],
+ fun() -> CountBy(Keys1) end;
+ {<<"count_by">>, Key, undefined} ->
+ Key1 = ConvertEle(Key),
+ fun() -> CountBy(Key1) end;
+ {<<"group_by">>, Keys, Vals} when is_list(Keys) andalso is_list(Vals) ->
+ Keys1 = ConvertList(Keys),
+ Vals1 = ConvertList(Vals),
+ fun() -> GroupBy(Keys1, Vals1) end;
+ {<<"group_by">>, Key, Vals} when is_list(Vals) ->
+ Key1 = ConvertEle(Key),
+ Vals1 = ConvertList(Vals),
+ fun() -> GroupBy(Key1, Vals1) end;
+ {<<"group_by">>, Keys, Val} when is_list(Keys) ->
+ Keys1 = ConvertList(Keys),
+ Val1 = ConvertEle(Val),
+ fun() -> GroupBy(Keys1, Val1) end;
+ {<<"group_by">>, Key, Val} ->
+ Key1 = ConvertEle(Key),
+ Val1 = ConvertList(Val),
+ fun() -> GroupBy(Key1, Val1) end;
+
+ {<<"sorted_by">>, Key, undefined} ->
+ Key1 = ConvertEle(Key),
+ fun() -> JsonKeys(SortedBy1(Key1)) end;
+ {<<"sorted_by">>, Keys, undefined} when is_list(Keys) ->
+ Keys1 = [ConvertEle(K) || K <- Keys],
+ fun() -> JsonKeys(SortedBy1(Keys1)) end;
+ {<<"sorted_by">>, Keys, Vals} when is_list(Keys) andalso is_list(Vals) ->
+ Keys1 = ConvertList(Keys),
+ Vals1 = ConvertList(Vals),
+ fun() -> JsonKeys(SortedBy2(Keys1, Vals1)) end;
+ {<<"sorted_by">>, Key, Vals} when is_list(Vals) ->
+ Key1 = ConvertEle(Key),
+ Vals1 = ConvertList(Vals),
+ fun() -> JsonKeys(SortedBy2(Key1, Vals1)) end;
+ {<<"sorted_by">>, Keys, Val} when is_list(Keys) ->
+ Keys1 = ConvertList(Keys),
+ Val1 = ConvertEle(Val),
+ fun() -> JsonKeys(SortedBy2(Keys1, Val1)) end;
+ {<<"sorted_by">>, Key, Val} ->
+ Key1 = ConvertEle(Key),
+ Val1 = ConvertList(Val),
+ fun() -> JsonKeys(SortedBy2(Key1, Val1)) end;
+ _ ->
+ throw({badrequest, invalid_resource_request})
+ end,
+
+ Fun1 = fun() ->
+ case Fun() of
+ Map when is_map(Map) ->
+ {maps:fold(
+ fun
+ (_K,0,A) -> A; %% TODO: Skip 0 value entries?
+ (K,V,A) -> [{ToJson(K), V} | A]
+ end,
+ [], Map)};
+ List when is_list(List) ->
+ List
+ end
+ end,
+
+ {Resp, _Bad} = rpc:multicall(erlang, apply, [
+ fun() ->
+ {node(), Fun1()}
+ end,
+ []
+ ]),
+ %%io:format("{CSRT}***** GOT RESP: ~p~n", [Resp]),
+ send_json(Req, {Resp});
+handle_resource_status_req(#httpd{method = 'GET'} = Req) ->
+ ok = chttpd:verify_is_server_admin(Req),
+ {Resp, Bad} = rpc:multicall(erlang, apply, [
+ fun() ->
+ {node(), couch_stats_resource_tracker:active()}
+ end,
+ []
+ ]),
+ %% TODO: incorporate Bad responses
Review Comment:
consider using erpc, the handling of bad responses is a bit cleaner there, I think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743873513
##########
src/couch/priv/stats_descriptions.cfg:
##########
@@ -418,10 +422,47 @@
{type, counter},
{desc, <<"number of other requests">>}
]}.
+{[couchdb, query_server, js_filter], [
+ {type, counter},
+ {desc, <<"number of JS filter invocations">>}
+]}.
Review Comment:
There is already a `.query_server.calls.ddoc_filter` counter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743913513
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
Review Comment:
Minor nit: consider adding units to the function name. `now_msec` or something like that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743966480
##########
src/fabric/src/fabric_util.erl:
##########
@@ -136,24 +136,36 @@ get_shard([#shard{node = Node, name = Name} | Rest], Opts, Timeout, Factor) ->
MFA = {fabric_rpc, open_shard, [Name, [{timeout, Timeout} | Opts]]},
Ref = rexi:cast(Node, self(), MFA, [sync]),
try
- receive
- {Ref, {ok, Db}} ->
- {ok, Db};
- {Ref, {'rexi_EXIT', {{unauthorized, _} = Error, _}}} ->
- throw(Error);
- {Ref, {'rexi_EXIT', {{forbidden, _} = Error, _}}} ->
- throw(Error);
- {Ref, Reason} ->
- couch_log:debug("Failed to open shard ~p because: ~p", [Name, Reason]),
- get_shard(Rest, Opts, Timeout, Factor)
- after Timeout ->
- couch_log:debug("Failed to open shard ~p after: ~p", [Name, Timeout]),
- get_shard(Rest, Opts, Factor * Timeout, Factor)
- end
+ await_shard_response(Ref, Name, Rest, Opts, Factor, Timeout)
after
rexi_monitor:stop(Mon)
end.
+await_shard_response(Ref, Name, Rest, Opts, Factor, Timeout) ->
+ receive
+ Msg0 ->
+ {Msg, Delta} = rexi_utils:extract_delta(Msg0),
+ couch_stats_resource_tracker:accumulate_delta(Delta),
+ case Msg of
+ {Ref, {ok, Db}} ->
+ {ok, Db};
+ {Ref, {'rexi_EXIT', {{unauthorized, _} = Error, _}}} ->
+ throw(Error);
+ {Ref, {'rexi_EXIT', {{forbidden, _} = Error, _}}} ->
+ throw(Error);
+ {Ref, Reason} ->
+ couch_log:debug("Failed to open shard ~p because: ~p", [Name, Reason]),
+ get_shard(Rest, Opts, Timeout, Factor);
+ %% {OldRef, {ok, Db}} -> <-- stale db resp that got here late, should we do something?
+ _ ->
+ %% Got a message from an old Ref that timed out, try again
+ await_shard_response(Ref, Name, Rest, Opts, Factor, Timeout)
Review Comment:
This changes the logic a bit to drop any messages. This doesn't seem to be related to the stats tracking, but it's a good change and might be better do it as a separate PR to discuss there.
This goes back to the effort some years back to use {rexi, ...} prefix for rexi messages to avoid dropping or consuming any 2 items tuple message. Wonder if we should finish that work first and incorporate message cleanup with it. But overall it does seems like a problem for a different PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743939890
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
+ case conf_get("log_pid_usage_report", "coordinator") of
+ "coordinator" ->
+ coordinator;
+ "true" ->
+ true;
+ _ ->
+ false
+ end.
+
+should_log(undefined) ->
+ false;
+should_log(#rctx{}=Rctx) ->
+ should_log(Rctx, logging_enabled()).
+
+should_log(undefined, _) ->
+ false;
+should_log(#rctx{}, true) ->
+ true;
+should_log(#rctx{}, false) ->
+ false;
+should_log(#rctx{type = {coordinator, _, _}}, coordinator) ->
+ true;
+should_log(#rctx{type = {worker, fabric_rpc, FName}}, _) ->
+ case conf_get("log_fabric_rpc") of
+ "true" ->
+ true;
+ undefined ->
+ false;
+ Name ->
+ Name =:= atom_to_list(FName)
+ end;
+should_log(#rctx{}, _) ->
+ false.
+
+%%
+%% gen_server callbacks
+%%
+
+start_link() ->
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+
+init([]) ->
+ ets:new(?MODULE, [
+ named_table,
+ public,
+ {decentralized_counters, true},
+ {write_concurrency, true},
+ {read_concurrency, true},
+ {keypos, #rctx.pid_ref}
+ ]),
+ {ok, #st{}}.
+
+handle_call(fetch, _from, #st{} = St) ->
+ {reply, {ok, St}, St};
+handle_call({call_search, _}, _From, St) ->
+ %% TODO: provide isolated search queries here
+ {reply, ok, St};
+handle_call(Msg, _From, St) ->
+ {stop, {unknown_call, Msg}, St}.
+
+handle_cast(Msg, St) ->
+ {stop, {unknown_cast, Msg}, St}.
+
+handle_info(Msg, St) ->
+ {stop, {unknown_info, Msg}, St}.
+
+terminate(_Reason, _St) ->
+ ok.
Review Comment:
This is optional
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1807067405
##########
src/couch/priv/stats_descriptions.cfg:
##########
@@ -418,10 +422,47 @@
{type, counter},
{desc, <<"number of other requests">>}
]}.
+{[couchdb, query_server, js_filter], [
+ {type, counter},
+ {desc, <<"number of JS filter invocations">>}
+]}.
+{[couchdb, query_server, js_filtered_docs], [
+ {type, counter},
+ {desc, <<"number of docs filtered through JS invocations">>}
+]}.
+{[couchdb, query_server, js_filter_error], [
+ {type, counter},
+ {desc, <<"number of JS filter invocation errors">>}
+]}.
{[couchdb, legacy_checksums], [
{type, counter},
{desc, <<"number of legacy checksums found in couch_file instances">>}
]}.
+{[couchdb, btree, folds], [
+ {type, counter},
+ {desc, <<"number of couch btree kv fold callback invocations">>}
+]}.
+{[couchdb, btree, get_node, kp_node], [
+ {type, counter},
+ {desc, <<"number of couch btree kp_nodes read">>}
+]}.
+{[couchdb, btree, get_node, kv_node], [
+ {type, counter},
+ {desc, <<"number of couch btree kv_nodes read">>}
+]}.
Review Comment:
> It's not clear which btrees the metrics refer to.
So that's the thing, we don't have current visibility into the volume of `couch_btree:get_node` calls _or_ what type of operations are inducing them. These could come from normal db doc reads, all_docs folds, compaction folds, index build folds, all these different systems induce heavy couch_btree operations and we don't know where/when/why.
The idea here is to start tracking the amount of core btree operations by the workers, allowing us to track it at a process level like we're doing with the rest of the stats in this PR, and then we'll at least be able to see the total volume of couch_btree operations induced, and then use the reports to quantify the requests/dbs/etc that performed those operations.
A key aspect of the tracking here is that `couch_btree:*` are still inside the _caller_ processes, eg the workers themselves, so we can do the process local tracking without having to extend beyond process boundaries. This CSRT engine does not yet handle nested work tracking, so for instance we can't funnel back information from the couch_file pid back through IOQ. Tracking the data here is a solid boundary of per request work being induced that we can track, while also providing much needed insight into overall database operations.
As for tracking of `kv_node` separately from `kp_node`, I think this is important because we're in a btree and performance is proportional to the log of the size of the btree, and we currently don't have any metrics illuminating the impact of btree size on read throughput. In my experience it's a fairly common experience for users to forget that while `logN` in a B+tree is _really_ good, it's _not_ a constant value and once you get into databases with billions of documents it takes longer to get through the btree, on every request. My hope here is that by tracking `kp_node` reads separately from `kv_nodes` we can more easily understand the workloads induced by larger databases, and more easily quantify that, something that I personally find challenging to do today.
I thought about trying to funnel down into `couch_btree` an indicator variable of the datatype, so for instance we could track changes btree operations independently of compaction folds, but that'd either require passing through a more informative context into `couch_btree:*` or some ugly pdict hacks. I think this approach of getting as close to the caller boundary in as we can before we funnel off to another process gives us low level insight into `couch_btree` operations while also allowing us to track it at a request/worker level so that retroactively we'll be able to go look at the report logs and quantify what requests were responsible for the induced btree operations. It's important to note that this approach will also extend naturally to background worker CSRT tracking, eg we can easily extend compaction, replication, and indexing to create a CSRT context and then we'll be able to quantify the work induced by those background jobs too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743939047
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
+ case conf_get("log_pid_usage_report", "coordinator") of
+ "coordinator" ->
+ coordinator;
+ "true" ->
+ true;
+ _ ->
+ false
+ end.
+
+should_log(undefined) ->
+ false;
+should_log(#rctx{}=Rctx) ->
+ should_log(Rctx, logging_enabled()).
+
+should_log(undefined, _) ->
+ false;
+should_log(#rctx{}, true) ->
+ true;
+should_log(#rctx{}, false) ->
+ false;
+should_log(#rctx{type = {coordinator, _, _}}, coordinator) ->
+ true;
+should_log(#rctx{type = {worker, fabric_rpc, FName}}, _) ->
+ case conf_get("log_fabric_rpc") of
+ "true" ->
+ true;
+ undefined ->
+ false;
+ Name ->
+ Name =:= atom_to_list(FName)
+ end;
+should_log(#rctx{}, _) ->
+ false.
+
+%%
+%% gen_server callbacks
+%%
+
+start_link() ->
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+
+init([]) ->
+ ets:new(?MODULE, [
+ named_table,
+ public,
+ {decentralized_counters, true},
+ {write_concurrency, true},
+ {read_concurrency, true},
+ {keypos, #rctx.pid_ref}
+ ]),
+ {ok, #st{}}.
+
+handle_call(fetch, _from, #st{} = St) ->
+ {reply, {ok, St}, St};
+handle_call({call_search, _}, _From, St) ->
+ %% TODO: provide isolated search queries here
+ {reply, ok, St};
+handle_call(Msg, _From, St) ->
+ {stop, {unknown_call, Msg}, St}.
+
+handle_cast(Msg, St) ->
+ {stop, {unknown_cast, Msg}, St}.
+
+handle_info(Msg, St) ->
+ {stop, {unknown_info, Msg}, St}.
+
+terminate(_Reason, _St) ->
+ ok.
+
+code_change(_OldVsn, St, _Extra) ->
+ {ok, St}.
Review Comment:
This is optional we don't need to define it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1750554301
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
Review Comment:
This might be hard to maintain. Because we might forget to update the tuple shape when we change it in other place. I suggest to define a record. The data representation would be the same, However record syntax is easier to read and maintain.
```
-record(worker, {
module :: atom() | '_',
function :: atom() | '_'
}).
-record(coordinator, {
module :: atom() | '_',
function :: atom() | '_'
}).
select_by_type(coordinators) ->
ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = #coordinator{}} = R) -> R end));
select_by_type(workers) ->
ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = #coordinator{}} = R) -> R end)).
```
OR
```
-record(worker, {
module :: atom() | '_',
function :: atom() | '_'
}).
-record(coordinator, {
module :: atom() | '_',
function :: atom() | '_'
}).
coordinators() ->
ets:fun2ms(fun(#rctx{type = #coordinator{}} = R) -> R end).
workers() ->
ets:fun2ms(fun(#rctx{type = #worker{}} = R) -> R end).
select_by_type(coordinators) ->
ets:select(?MODULE, coordinators());
select_by_type(workers) ->
ets:select(?MODULE, workers()).
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1750565364
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
Review Comment:
If `worker` record would be defined (as suggested in another comment) this can be written.
```
create_context(MFA, #worker{module = M, function = F}, null, From, Nonce);
```
This is more verbose. One option to make it less verbose is to define constructor.
```
worker({M, F, _A}) ->
#worker{module = M, function = F}.
create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
....
create_context(MFA, worker(MFA), null, From, Nonce);
....
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1750693303
##########
src/rexi/src/rexi.erl:
##########
@@ -222,7 +222,11 @@ stream_ack(Client) ->
%%
ping() ->
{Caller, _} = get(rexi_from),
- erlang:send(Caller, {rexi, '$rexi_ping'}).
+ %% It is essential ping/0 includes deltas as otherwise long running
Review Comment:
Thank you for the comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1743969490
##########
src/mem3/src/mem3_rpc.erl:
##########
@@ -378,20 +378,34 @@ rexi_call(Node, MFA, Timeout) ->
Mon = rexi_monitor:start([rexi_utils:server_pid(Node)]),
Ref = rexi:cast(Node, self(), MFA, [sync]),
try
- receive
- {Ref, {ok, Reply}} ->
- Reply;
- {Ref, Error} ->
- erlang:error(Error);
- {rexi_DOWN, Mon, _, Reason} ->
- erlang:error({rexi_DOWN, {Node, Reason}})
- after Timeout ->
- erlang:error(timeout)
- end
+ wait_message(Node, Ref, Mon, Timeout)
after
rexi_monitor:stop(Mon)
end.
+wait_message(Node, Ref, Mon, Timeout) ->
+ receive
+ Msg ->
+ process_raw_message(Msg, Node, Ref, Mon, Timeout)
+ after Timeout ->
+ erlang:error(timeout)
+ end.
+
+process_raw_message(Msg0, Node, Ref, Mon, Timeout) ->
+ {Msg, Delta} = rexi_utils:extract_delta(Msg0),
+ couch_stats_resource_tracker:accumulate_delta(Delta),
+ case Msg of
+ {Ref, {ok, Reply}} ->
+ Reply;
+ {Ref, Error} ->
+ erlang:error(Error);
+ {rexi_DOWN, Mon, _, Reason} ->
+ erlang:error({rexi_DOWN, {Node, Reason}});
+ Other ->
+ ?LOG_UNEXPECTED_MSG(Other),
+ wait_message(Node, Ref, Mon, Timeout)
Review Comment:
This changes the logic here by dropping unknown(stale?) message. See a similar comment for get_shard. It might be good to do that in a separate PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "iilyak (via GitHub)" <gi...@apache.org>.
iilyak commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1816982151
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
Review Comment:
I looked briefly into bif implementation of `persistent_term`. It is nice:
- hash map internally
- the term sharing is preserved for complex containers (tuples, lists)
- the literals like numbers and strings are copied
- only one allocation during `get` (to create reference for GC)
My conclusion is - it should be way faster than `ets` or `gen_server` based approach. When updates are rare. Updates causing locking of a data structure so all concurrent reads are on hold.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1817432277
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
Review Comment:
That's nice to know that `ets:fun2ms` can handle multiple function heads, I wasn't sure as it's a "pseudo function".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1807070042
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
+ destroy_context(PidRef),
+ %% TODO: should we pass reason to log_process_lifetime_report?
+ %% Reason = case Reason0 of
+ %% {shutdown, Shutdown0} ->
+ %% Shutdown = atom_to_binary(Shutdown0),
+ %% <<"shutdown: ", Shutdown/binary>>;
+ %% Reason0 ->
+ %% Reason0
+ %% end,
+ %% TODO: should we send the induced work delta to the coordinator?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef)
+ end.
+
+log_process_lifetime_report(PidRef) ->
+ case is_enabled() andalso is_logging_enabled() of
+ true ->
+ Rctx = get_resource(PidRef),
+ case should_log(Rctx) of
+ true ->
+ couch_log:report("csrt-pid-usage-lifetime", to_json(Rctx));
+ _ ->
+ ok
+ end;
+ false ->
+ ok
+ end.
+
+is_logging_enabled() ->
+ logging_enabled() =/= false.
+
+logging_enabled() ->
+ case conf_get("log_pid_usage_report", "coordinator") of
+ "coordinator" ->
+ coordinator;
+ "true" ->
+ true;
+ _ ->
+ false
+ end.
+
+should_log(undefined) ->
+ false;
+should_log(#rctx{}=Rctx) ->
+ should_log(Rctx, logging_enabled()).
+
+should_log(undefined, _) ->
+ false;
+should_log(#rctx{}, true) ->
+ true;
+should_log(#rctx{}, false) ->
+ false;
+should_log(#rctx{type = {coordinator, _, _}}, coordinator) ->
+ true;
+should_log(#rctx{type = {worker, fabric_rpc, FName}}, _) ->
+ case conf_get("log_fabric_rpc") of
+ "true" ->
+ true;
+ undefined ->
+ false;
+ Name ->
+ Name =:= atom_to_list(FName)
+ end;
+should_log(#rctx{}, _) ->
+ false.
+
+%%
+%% gen_server callbacks
+%%
+
+start_link() ->
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+
+init([]) ->
+ ets:new(?MODULE, [
+ named_table,
+ public,
+ {decentralized_counters, true},
+ {write_concurrency, true},
Review Comment:
CSRT is predicated on ETS `write_concurrency` and `decentralized_counters` being essential. I can't think of a scenario where we wouldn't ever want `write_concurrency` on the table, especially given Erlang no longer supports non SMP builds. I think it's worth being explicit with our intentions here, and probably worth it for me to add a comment here in the code around our expectations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Couch stats resource tracker v2 [couchdb]
Posted by "chewbranca (via GitHub)" <gi...@apache.org>.
chewbranca commented on code in PR #5213:
URL: https://github.com/apache/couchdb/pull/5213#discussion_r1803921001
##########
src/couch_stats/src/couch_stats_resource_tracker.erl:
##########
@@ -0,0 +1,912 @@
+% Licensed under the Apache License, Version 2.0 (the "License"); you may not
+% use this file except in compliance with the License. You may obtain a copy of
+% the License at
+%
+% http://www.apache.org/licenses/LICENSE-2.0
+%
+% Unless required by applicable law or agreed to in writing, software
+% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+% WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+% License for the specific language governing permissions and limitations under
+% the License.
+
+-module(couch_stats_resource_tracker).
+
+-behaviour(gen_server).
+
+-export([
+ start_link/0,
+ init/1,
+ handle_call/3,
+ handle_cast/2,
+ handle_info/2,
+ code_change/3,
+ terminate/2
+]).
+
+%% PidRef API
+-export([
+ get_pid_ref/0,
+ set_pid_ref/1,
+ create_pid_ref/0,
+ close_pid_ref/0, close_pid_ref/1
+]).
+
+%% Context API
+-export([
+ create_resource/1,
+ create_context/5,
+ create_coordinator_context/2,
+ create_worker_context/3,
+ destroy_context/0, destroy_context/1,
+
+ get_resource/0, get_resource/1,
+
+ set_context_dbname/1, set_context_dbname/2,
+ set_context_handler_fun/1, set_context_handler_fun/2,
+ set_context_username/1, set_context_username/2
+]).
+
+%% stats collection api
+-export([
+ is_enabled/0,
+
+ inc/1, inc/2,
+ maybe_inc/2,
+ accumulate_delta/1,
+ make_delta/0,
+
+ ioq_called/0,
+
+ should_track/1
+]).
+
+%% aggregate query api
+-export([
+ active/0, active/1,
+ active_coordinators/0, active_coordinators/1,
+ active_workers/0, active_workers/1,
+
+ count_by/1,
+ group_by/2, group_by/3,
+ sorted/1,
+ sorted_by/1, sorted_by/2, sorted_by/3,
+
+ find_by_pid/1,
+ find_by_pidref/1,
+ find_by_nonce/1,
+ find_workers_by_pidref/1
+]).
+
+%% Process lifetime reporting api
+-export([
+ log_process_lifetime_report/1,
+ is_logging_enabled/0,
+ logging_enabled/0,
+ should_log/1, should_log/2,
+ tracker/1
+]).
+
+-include_lib("couch/include/couch_db.hrl").
+-include_lib("stdlib/include/ms_transform.hrl").
+
+%% Module pdict markers
+-define(DELTA_TA, csrt_delta_ta).
+-define(DELTA_TZ, csrt_delta_tz). %% T Zed instead of T0
+-define(PID_REF, csrt_pid_ref). %% track local ID
+-define(TRACKER_PID, csrt_tracker). %% tracker pid
+
+-define(MANGO_EVAL_MATCH, mango_eval_match).
+-define(DB_OPEN_DOC, docs_read).
+-define(DB_OPEN, db_open).
+-define(COUCH_SERVER_OPEN, db_open).
+-define(COUCH_BT_GET_KP_NODE, get_kp_node).
+-define(COUCH_BT_GET_KV_NODE, get_kv_node).
+-define(COUCH_BT_WRITE_KP_NODE, write_kp_node).
+-define(COUCH_BT_WRITE_KV_NODE, write_kv_node).
+-define(COUCH_JS_FILTER, js_filter).
+-define(COUCH_JS_FILTER_ERROR, js_filter_error).
+-define(COUCH_JS_FILTERED_DOCS, js_filtered_docs).
+-define(IOQ_CALLS, ioq_calls).
+-define(ROWS_READ, rows_read).
+
+%% TODO: overlap between this and couch btree fold invocations
+%% TODO: need some way to distinguish fols on views vs find vs all_docs
+-define(FRPC_CHANGES_ROW, changes_processed).
+-define(FRPC_CHANGES_RETURNED, changes_returned).
+
+-record(st, {}).
+
+-record(rctx, {
+ %% Metadata
+ started_at = tnow(),
+ updated_at = tnow(),
+ pid_ref,
+ mfa,
+ nonce,
+ from,
+ type = unknown, %% unknown/background/system/rpc/coordinator/fabric_rpc/etc_rpc/etc
+ dbname,
+ username,
+ path,
+
+ %% Stats counters
+ db_open = 0,
+ docs_read = 0,
+ rows_read = 0,
+ changes_processed = 0,
+ changes_returned = 0,
+ ioq_calls = 0,
+ io_bytes_read = 0,
+ io_bytes_written = 0,
+ js_evals = 0,
+ js_filter = 0,
+ js_filter_error = 0,
+ js_filtered_docs = 0,
+ mango_eval_match = 0,
+ %% TODO: switch record definitions to be macro based, eg:
+ %% ?COUCH_BT_GET_KP_NODE = 0,
+ get_kv_node = 0,
+ get_kp_node = 0,
+ write_kv_node = 0,
+ write_kp_node = 0
+}).
+
+%%
+%% Public API
+%%
+
+%%
+%% PidRef operations
+%%
+
+get_pid_ref() ->
+ get(?PID_REF).
+
+set_pid_ref(PidRef) ->
+ erlang:put(?PID_REF, PidRef),
+ PidRef.
+
+create_pid_ref() ->
+ case get_pid_ref() of
+ undefined ->
+ ok;
+ PidRef0 ->
+ %% TODO: what to do when it already exists?
+ throw({epidexist, PidRef0}),
+ close_pid_ref(PidRef0)
+ end,
+ PidRef = {self(), make_ref()},
+ set_pid_ref(PidRef),
+ PidRef.
+
+close_pid_ref() ->
+ close_pid_ref(get_pid_ref()).
+
+%%close_pid_ref(undefined) ->
+%% undefined;
+close_pid_ref(_PidRef) ->
+ erase(?PID_REF).
+
+get_resource() ->
+ get_resource(get_pid_ref()).
+
+get_resource(undefined) ->
+ undefined;
+get_resource(PidRef) ->
+ catch case ets:lookup(?MODULE, PidRef) of
+ [#rctx{}=Rctx] ->
+ Rctx;
+ [] ->
+ undefined
+ end.
+
+%% monotonic time now in millisecionds
+tnow() ->
+ erlang:monotonic_time(millisecond).
+
+is_enabled() ->
+ config:get_boolean(?MODULE_STRING, "enabled", true).
+
+%%
+%% Aggregate query API
+%%
+
+active() -> active_int(all).
+active_coordinators() -> active_int(coordinators).
+active_workers() -> active_int(workers).
+
+%% active_json() or active(json)?
+active(json) -> to_json_list(active_int(all)).
+active_coordinators(json) -> to_json_list(active_int(coordinators)).
+active_workers(json) -> to_json_list(active_int(workers)).
+
+
+active_int(coordinators) ->
+ select_by_type(coordinators);
+active_int(workers) ->
+ select_by_type(workers);
+active_int(all) ->
+ select_by_type(all).
+
+
+select_by_type(coordinators) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {coordinator, _, _}} = R) -> R end));
+select_by_type(workers) ->
+ ets:select(?MODULE, ets:fun2ms(fun(#rctx{type = {worker, _, _}} = R) -> R end));
+select_by_type(all) ->
+ ets:tab2list(?MODULE).
+
+find_by_nonce(Nonce) ->
+ ets:match_object(?MODULE, ets:fun2ms(fun(#rctx{nonce = Nonce1} = R) when Nonce =:= Nonce1 -> R end)).
+
+find_by_pid(Pid) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref={Pid, '_'}, _ = '_'})].
+
+find_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{pid_ref=PidRef, _ = '_'})].
+
+find_workers_by_pidref(PidRef) ->
+ [R || #rctx{} = R <- ets:match_object(?MODULE, #rctx{from=PidRef, _ = '_'})].
+
+field(#rctx{pid_ref=Val}, pid_ref) -> Val;
+%% NOTE: Pros and cons to doing these convert functions here
+%% Ideally, this would be done later so as to prefer the core data structures
+%% as long as possible, but we currently need the output of this function to
+%% be jiffy:encode'able. The tricky bit is dynamically encoding the group_by
+%% structure provided by the caller of *_by aggregator functions below.
+%% For now, we just always return jiffy:encode'able data types.
+field(#rctx{mfa=Val}, mfa) -> convert_mfa(Val);
+field(#rctx{nonce=Val}, nonce) -> Val;
+field(#rctx{from=Val}, from) -> Val;
+field(#rctx{type=Val}, type) -> convert_type(Val);
+field(#rctx{dbname=Val}, dbname) -> Val;
+field(#rctx{username=Val}, username) -> Val;
+field(#rctx{path=Val}, path) -> Val;
+field(#rctx{db_open=Val}, db_open) -> Val;
+field(#rctx{docs_read=Val}, docs_read) -> Val;
+field(#rctx{rows_read=Val}, rows_read) -> Val;
+field(#rctx{changes_processed=Val}, changes_processed) -> Val;
+field(#rctx{changes_returned=Val}, changes_returned) -> Val;
+field(#rctx{ioq_calls=Val}, ioq_calls) -> Val;
+field(#rctx{io_bytes_read=Val}, io_bytes_read) -> Val;
+field(#rctx{io_bytes_written=Val}, io_bytes_written) -> Val;
+field(#rctx{js_evals=Val}, js_evals) -> Val;
+field(#rctx{js_filter=Val}, js_filter) -> Val;
+field(#rctx{js_filter_error=Val}, js_filter_error) -> Val;
+field(#rctx{js_filtered_docs=Val}, js_filtered_docs) -> Val;
+field(#rctx{mango_eval_match=Val}, mango_eval_match) -> Val;
+field(#rctx{get_kv_node=Val}, get_kv_node) -> Val;
+field(#rctx{get_kp_node=Val}, get_kp_node) -> Val.
+
+curry_field(Field) ->
+ fun(Ele) -> field(Ele, Field) end.
+
+count_by(KeyFun) ->
+ group_by(KeyFun, fun(_) -> 1 end).
+
+group_by(KeyFun, ValFun) ->
+ group_by(KeyFun, ValFun, fun erlang:'+'/2).
+
+%% eg: group_by(mfa, docs_read).
+%% eg: group_by(fun(#rctx{mfa=MFA,docs_read=DR}) -> {MFA, DR} end, ioq_calls).
+%% eg: ^^ or: group_by([mfa, docs_read], ioq_calls).
+%% eg: group_by([username, dbname, mfa], docs_read).
+%% eg: group_by([username, dbname, mfa], ioq_calls).
+%% eg: group_by([username, dbname, mfa], js_filters).
+group_by(KeyL, ValFun, AggFun) when is_list(KeyL) ->
+ KeyFun = fun(Ele) -> list_to_tuple([field(Ele, Key) || Key <- KeyL]) end,
+ group_by(KeyFun, ValFun, AggFun);
+group_by(Key, ValFun, AggFun) when is_atom(Key) ->
+ group_by(curry_field(Key), ValFun, AggFun);
+group_by(KeyFun, Val, AggFun) when is_atom(Val) ->
+ group_by(KeyFun, curry_field(Val), AggFun);
+group_by(KeyFun, ValFun, AggFun) ->
+ FoldFun = fun(Ele, Acc) ->
+ Key = KeyFun(Ele),
+ Val = ValFun(Ele),
+ CurrVal = maps:get(Key, Acc, 0),
+ NewVal = AggFun(CurrVal, Val),
+ %% TODO: should we skip here? how to make this optional?
+ case NewVal > 0 of
+ true ->
+ maps:put(Key, NewVal, Acc);
+ false ->
+ Acc
+ end
+ end,
+ ets:foldl(FoldFun, #{}, ?MODULE).
+
+%% Sorts largest first
+sorted(Map) when is_map(Map) ->
+ lists:sort(fun({_K1, A}, {_K2, B}) -> B < A end, maps:to_list(Map)).
+
+shortened(L) ->
+ lists:sublist(L, 10).
+
+%% eg: sorted_by([username, dbname, mfa], ioq_calls)
+%% eg: sorted_by([dbname, mfa], doc_reads)
+sorted_by(KeyFun) -> shortened(sorted(count_by(KeyFun))).
+sorted_by(KeyFun, ValFun) -> shortened(sorted(group_by(KeyFun, ValFun))).
+sorted_by(KeyFun, ValFun, AggFun) -> shortened(sorted(group_by(KeyFun, ValFun, AggFun))).
+
+%%
+%% Conversion API for outputting JSON
+%%
+
+convert_mfa(MFA) when is_list(MFA) ->
+ list_to_binary(MFA);
+convert_mfa({M0, F0, A0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ A = integer_to_binary(A0),
+ <<M/binary, ":", F/binary, "/", A/binary>>;
+convert_mfa(null) ->
+ null;
+convert_mfa(undefined) ->
+ null.
+
+convert_type(Atom) when is_atom(Atom) ->
+ atom_to_binary(Atom);
+convert_type({coordinator, Verb0, Atom0}) when is_atom(Atom0) ->
+ Verb = atom_to_binary(Verb0),
+ Atom = atom_to_binary(Atom0),
+ <<"coordinator:", Verb/binary, ":", Atom/binary>>;
+convert_type({coordinator, Verb0, Path0}) ->
+ Verb = atom_to_binary(Verb0),
+ Path = list_to_binary(Path0),
+ <<"coordinator:", Verb/binary, ":", Path/binary>>;
+convert_type({worker, M0, F0}) ->
+ M = atom_to_binary(M0),
+ F = atom_to_binary(F0),
+ <<"worker:", M/binary, ":", F/binary>>;
+convert_type(null) ->
+ null;
+convert_type(undefined) ->
+ null.
+
+convert_path(undefined) ->
+ null;
+convert_path(Path) when is_binary(Path) ->
+ Path.
+
+convert_pidref({Parent0, ParentRef0}) ->
+ Parent = convert_pid(Parent0),
+ ParentRef = convert_ref(ParentRef0),
+ <<Parent/binary, ":", ParentRef/binary>>;
+convert_pidref(null) ->
+ null;
+convert_pidref(undefined) ->
+ null.
+
+convert_pid(Pid) when is_pid(Pid) ->
+ ?l2b(pid_to_list(Pid)).
+
+convert_ref(Ref) when is_reference(Ref) ->
+ ?l2b(ref_to_list(Ref)).
+
+to_json(#rctx{}=Rctx) ->
+ #rctx{
+ updated_at = TP,
+ started_at = TInit,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ from = From,
+ dbname = DbName,
+ path = Path,
+ username = UserName,
+ db_open = DbOpens,
+ docs_read = DocsRead,
+ rows_read = RowsRead,
+ js_filter = JSFilters,
+ js_filter_error = JSFilterErrors,
+ js_filtered_docs = JSFilteredDocss,
+ type = Type,
+ get_kp_node = GetKpNodes,
+ get_kv_node = GetKvNodes,
+ %%write_kp_node = WriteKpNodes,
+ %%write_kv_node = WriteKvNodes,
+ changes_returned = ChangesReturned,
+ ioq_calls = IoqCalls
+ } = Rctx,
+
+ #{
+ updated_at => TP,
+ started_at => TInit,
+ pid_ref => convert_pidref(PidRef),
+ mfa => convert_mfa(MFA),
+ nonce => Nonce,
+ from => convert_pidref(From),
+ dbname => DbName,
+ path => convert_path(Path),
+ username => UserName,
+ db_open => DbOpens,
+ docs_read => DocsRead,
+ js_filter => JSFilters,
+ js_filter_error => JSFilterErrors,
+ js_filtered_docs => JSFilteredDocss,
+ rows_read => RowsRead,
+ type => convert_type(Type),
+ get_kp_nodes => GetKpNodes,
+ get_kv_nodes => GetKvNodes,
+ %%write_kp_nodes => WriteKpNodes,
+ %%write_kv_nodes => WriteKvNodes,
+ changes_returned => ChangesReturned,
+ ioq_calls => IoqCalls
+ }.
+
+%%
+%% Context lifecycle API
+%%
+
+create_resource(#rctx{} = Rctx) ->
+ catch ets:insert(?MODULE, Rctx).
+
+create_worker_context(From, {M,F,_A} = MFA, Nonce) ->
+ case is_enabled() of
+ true ->
+ create_context(MFA, {worker, M, F}, null, From, Nonce);
+ false ->
+ false
+ end.
+
+create_coordinator_context(#httpd{} = Req, Path0) ->
+ case is_enabled() of
+ true ->
+ #httpd{
+ method = Verb,
+ nonce = Nonce
+ %%path_parts = Parts
+ } = Req,
+ %%Path = list_to_binary([$/ | io_lib:format("~p", [Parts])]),
+ Path = list_to_binary([$/ | Path0]),
+ Type = {coordinator, Verb, init},
+ create_context(null, Type, Path, null, Nonce);
+ false ->
+ false
+ end.
+
+create_context(MFA, Type, Path, From, Nonce) ->
+ PidRef = create_pid_ref(),
+ Rctx = #rctx{
+ from = From,
+ pid_ref = PidRef,
+ mfa = MFA,
+ nonce = Nonce,
+ path = Path,
+ type = Type
+ },
+ erlang:put(?DELTA_TZ, Rctx),
+ create_resource(Rctx),
+ track(Rctx),
+ PidRef.
+
+set_context_dbname(DbName) ->
+ set_context_dbname(DbName, get_pid_ref()).
+
+set_context_dbname(_, undefined) ->
+ ok;
+set_context_dbname(DbName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.dbname, DbName}]).
+
+set_context_handler_fun(Fun) when is_function(Fun) ->
+ set_context_handler_fun(Fun, get_pid_ref()).
+set_context_handler_fun(_, undefined) ->
+ ok;
+set_context_handler_fun(Fun, PidRef) when is_function(Fun) ->
+ case is_enabled() of
+ false ->
+ ok;
+ true ->
+ FunName = erlang:fun_to_list(Fun),
+ #rctx{type={coordinator, Verb, _}} = get_resource(),
+ Update = [{#rctx.type, {coordinator, Verb, FunName}}],
+ update_element(PidRef, Update)
+ end.
+
+set_context_username(null) ->
+ ok;
+set_context_username(undefined) ->
+ ok;
+set_context_username(User) ->
+ set_context_username(User, get_pid_ref()).
+
+set_context_username(null, _) ->
+ ok;
+set_context_username(_, undefined) ->
+ ok;
+set_context_username(#httpd{user_ctx = Ctx}, PidRef) ->
+ set_context_username(Ctx, PidRef);
+set_context_username(#user_ctx{name = Name}, PidRef) ->
+ set_context_username(Name, PidRef);
+set_context_username(UserName, PidRef) ->
+ is_enabled() andalso update_element(PidRef, [{#rctx.username, UserName}]).
+
+destroy_context() ->
+ destroy_context(get_pid_ref()).
+
+destroy_context(undefined) ->
+ ok;
+destroy_context({_, _} = PidRef) ->
+ stop_tracker(get_tracker()),
+ close_pid_ref(PidRef),
+ ok.
+
+%% Stat collection API
+
+inc(Key) ->
+ inc(Key, 1).
+
+%% TODO: inc(io_bytes_read, N) ->
+%% TODO: inc(io_bytes_written, N) ->
+%% TODO: inc(js_evals, N) ->
+inc(?DB_OPEN, N) ->
+ update_counter(#rctx.?DB_OPEN, N);
+inc(?ROWS_READ, N) ->
+ update_counter(#rctx.?ROWS_READ, N);
+inc(?FRPC_CHANGES_RETURNED, N) ->
+ update_counter(#rctx.?FRPC_CHANGES_RETURNED, N);
+inc(?IOQ_CALLS, N) ->
+ update_counter(#rctx.?IOQ_CALLS, N);
+inc(?COUCH_JS_FILTER, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER, N);
+inc(?COUCH_JS_FILTER_ERROR, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTER_ERROR, N);
+inc(?COUCH_JS_FILTERED_DOCS, N) ->
+ update_counter(#rctx.?COUCH_JS_FILTERED_DOCS, N);
+inc(?MANGO_EVAL_MATCH, N) ->
+ update_counter(#rctx.?MANGO_EVAL_MATCH, N);
+inc(?DB_OPEN_DOC, N) ->
+ update_counter(#rctx.?DB_OPEN_DOC, N);
+inc(?FRPC_CHANGES_ROW, N) ->
+ update_counter(#rctx.?ROWS_READ, N); %% TODO: rework double use of rows_read
+inc(?COUCH_BT_GET_KP_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KP_NODE, N);
+inc(?COUCH_BT_GET_KV_NODE, N) ->
+ update_counter(#rctx.?COUCH_BT_GET_KV_NODE, N);
+inc(_, _) ->
+ %% inc needs to allow unknown types to pass for accumulate_update to handle
+ %% updates from nodes with newer data formats
+ 0.
+
+maybe_inc([mango, evaluate_selector], Val) ->
+ inc(?MANGO_EVAL_MATCH, Val);
+maybe_inc([couchdb, database_reads], Val) ->
+ inc(?DB_OPEN_DOC, Val);
+maybe_inc([fabric_rpc, changes, processed], Val) ->
+ inc(?FRPC_CHANGES_ROW, Val);
+maybe_inc([fabric_rpc, changes, returned], Val) ->
+ inc(?FRPC_CHANGES_RETURNED, Val);
+maybe_inc([fabric_rpc, view, rows_read], Val) ->
+ inc(?ROWS_READ, Val);
+maybe_inc([couchdb, couch_server, open], Val) ->
+ inc(?DB_OPEN, Val);
+maybe_inc([couchdb, btree, get_node, kp_node], Val) ->
+ inc(?COUCH_BT_GET_KP_NODE, Val);
+maybe_inc([couchdb, btree, get_node, kv_node], Val) ->
+ inc(?COUCH_BT_GET_KV_NODE, Val);
+%% The write_node logic won't pickup writes as none of the RPC
+%% processes actually perform the write operation
+%% TODO: bubble up induced work from other processes
+maybe_inc([couchdb, btree, write_node, kp_node], Val) ->
+ inc(?COUCH_BT_WRITE_KP_NODE, Val);
+maybe_inc([couchdb, btree, write_node, kv_node], Val) ->
+ inc(?COUCH_BT_WRITE_KV_NODE, Val);
+maybe_inc([couchdb, query_server, js_filter_error], Val) ->
+ inc(?COUCH_JS_FILTER_ERROR, Val);
+maybe_inc([couchdb, query_server, js_filter], Val) ->
+ inc(?COUCH_JS_FILTER, Val);
+maybe_inc([couchdb, query_server, js_filtered_docs], Val) ->
+ inc(?COUCH_JS_FILTERED_DOCS, Val);
+maybe_inc(_Metric, _Val) ->
+ %%io:format("SKIPPING MAYBE_INC METRIC[~p]: ~p~n", [Val, Metric]),
+ 0.
+
+%% TODO: update stats_descriptions.cfg for relevant apps
+should_track([fabric_rpc, all_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, processed]) ->
+ is_enabled();
+should_track([fabric_rpc, changes, returned]) ->
+ is_enabled();
+should_track([fabric_rpc, map_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, reduce_view, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, get_all_security, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_doc, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, update_docs, spawned]) ->
+ is_enabled();
+should_track([fabric_rpc, open_shard, spawned]) ->
+ is_enabled();
+should_track([mango_cursor, view, all_docs]) ->
+ is_enabled();
+should_track([mango_cursor, view, idx]) ->
+ is_enabled();
+should_track(_Metric) ->
+ %%io:format("SKIPPING METRIC: ~p~n", [Metric]),
+ false.
+
+ioq_called() ->
+ inc(ioq_calls).
+
+accumulate_delta(Delta) when is_map(Delta) ->
+ %% TODO: switch to creating a batch of updates to invoke a single
+ %% update_counter rather than sequentially invoking it for each field
+ is_enabled() andalso maps:foreach(fun inc/2, Delta);
+accumulate_delta(undefined) ->
+ ok.
+
+make_delta() ->
+ TA = case get(?DELTA_TA) of
+ undefined ->
+ %% Need to handle this better, can't just make a new T0 at T' as
+ %% the timestamps will be identical causing a divide by zero error.
+ %%
+ %% Realistically need to ensure that all invocations of database
+ %% operations sets T0 appropriately. Perhaps it's possible to do
+ %% this is the couch_db:open chain, and then similarly, in
+ %% couch_server, and uhhhh... couch_file, and...
+ %%
+ %% I think we need some type of approach for establishing a T0 that
+ %% doesn't result in outrageous deltas. For now zero out the
+ %% microseconds field, or subtract a second on the off chance that
+ %% microseconds is zero. I'm not uptodate on the latest Erlang time
+ %% libraries and don't remember how to easily get an
+ %% `os:timestamp()` out of now() - 100ms or some such.
+ %%
+ %% I think it's unavoidable that we'll have some codepaths that do
+ %% not properly instantiate the T0 at spawn resulting in needing to
+ %% do some time of "time warp" or ignoring the timing collection
+ %% entirely. Perhaps if we hoisted out the stats collection into
+ %% the primary flow of the database and funnel that through all the
+ %% function clauses we could then utilize Dialyzer to statically
+ %% analyze and assert all code paths that invoke database
+ %% operations have properly instantinated a T0 at the appropriate
+ %% start time such that we don't have to "fudge" deltas with a
+ %% missing start point, but we're a long ways from that happening
+ %% so I feel it necessary to address the NULL start time.
+
+ %% Track how often we fail to initiate T0 correctly
+ %% Perhaps somewhat naughty we're incrementing stats from within
+ %% couch_stats itself? Might need to handle this differently
+ %% TODO: determine appropriate course of action here
+ %% io:format("~n**********MISSING STARTING DELTA************~n~n", []),
+ couch_stats:increment_counter(
+ [couchdb, csrt, delta_missing_t0]),
+ %%[couch_stats_resource_tracker, delta_missing_t0]),
+
+ case erlang:get(?DELTA_TZ) of
+ undefined ->
+ TA0 = make_delta_base(),
+ %% TODO: handline missing deltas, otherwise divide by zero
+ set_delta_a(TA0),
+ TA0;
+ TA0 ->
+ TA0
+ end;
+ #rctx{} = TA0 ->
+ TA0
+ end,
+ TB = get_resource(),
+ Delta = make_delta(TA, TB),
+ set_delta_a(TB),
+ Delta.
+
+make_delta(#rctx{}=TA, #rctx{}=TB) ->
+ Delta = #{
+ docs_read => TB#rctx.docs_read - TA#rctx.docs_read,
+ js_filter => TB#rctx.js_filter - TA#rctx.js_filter,
+ js_filter_error => TB#rctx.js_filter_error - TA#rctx.js_filter_error,
+ js_filtered_docs => TB#rctx.js_filtered_docs - TA#rctx.js_filtered_docs,
+ rows_read => TB#rctx.rows_read - TA#rctx.rows_read,
+ changes_returned => TB#rctx.changes_returned - TA#rctx.changes_returned,
+ get_kp_node => TB#rctx.get_kp_node - TA#rctx.get_kp_node,
+ get_kv_node => TB#rctx.get_kv_node - TA#rctx.get_kv_node,
+ db_open => TB#rctx.db_open - TA#rctx.db_open,
+ ioq_calls => TB#rctx.ioq_calls - TA#rctx.ioq_calls,
+ dt => TB#rctx.updated_at - TA#rctx.updated_at
+ },
+ %% TODO: reevaluate this decision
+ %% Only return non zero (and also positive) delta fields
+ maps:filter(fun(_K,V) -> V > 0 end, Delta);
+make_delta(_, #rctx{}) ->
+ #{error => missing_beg_rctx};
+make_delta(#rctx{}, _) ->
+ #{error => missing_fin_rctx}.
+
+%% TODO: what to do when PidRef=undefined?
+make_delta_base(PidRef) ->
+ %% TODO: extract user_ctx and db/shard from request
+ Now = tnow(),
+ #rctx{
+ pid_ref = PidRef,
+ %% TODO: confirm this subtraction works
+ started_at = Now - 100, %% give us 100ms rewind time for missing T0
+ updated_at = Now
+ }.
+
+make_delta_base() ->
+ make_delta_base(get_pid_ref()).
+
+set_delta_a(TA) ->
+ erlang:put(?DELTA_TA, TA).
+
+update_counter(Field, Count) ->
+ is_enabled() andalso update_counter(get_pid_ref(), Field, Count).
+
+update_counter(undefined, _Field, _Count) ->
+ ok;
+update_counter({_Pid,_Ref}=PidRef, Field, Count) ->
+ %% TODO: mem3 crashes without catch, why do we lose the stats table?
+ is_enabled() andalso catch ets:update_counter(?MODULE, PidRef, {Field, Count}, #rctx{pid_ref=PidRef}).
+
+update_element(undefined, _Update) ->
+ ok;
+update_element({_Pid,_Ref}=PidRef, Update) ->
+ %% TODO: should we take any action when the update fails?
+ is_enabled() andalso catch ets:update_element(?MODULE, PidRef, Update).
+
+%% Process lifetime logging api
+
+track(#rctx{pid_ref=PidRef}) ->
+ case get_tracker() of
+ undefined ->
+ Pid = spawn(?MODULE, tracker, [PidRef]),
+ put_tracker(Pid),
+ Pid;
+ Pid when is_pid(Pid) ->
+ Pid
+ end.
+
+tracker({Pid, _Ref}=PidRef) ->
+ MonRef = erlang:monitor(process, Pid),
+ receive
+ stop ->
+ %% TODO: do we need cleanup here?
+ log_process_lifetime_report(PidRef),
+ catch evict(PidRef),
+ demonitor(MonRef),
+ ok;
+ {'DOWN', MonRef, _Type, _0DPid, _Reason0} ->
Review Comment:
Yeah that was a bit of old code, but if you look below I left a TODO to get some feedback on and I could use some thoughts:
```
%% TODO: should we pass reason to log_process_lifetime_report?
%% Reason = case Reason0 of
%% {shutdown, Shutdown0} ->
%% Shutdown = atom_to_binary(Shutdown0),
%% <<"shutdown: ", Shutdown/binary>>;
%% Reason0 ->
%% Reason0
%% end,
```
I want to record the shutdown reason when the process dies, but there's so many different formats to the potential `Reason` tuple that I wasn't sure of a good way to address that. I had a few approaches to translate some standard error formats, but it felt clunky and incompletely. Anyone have any good ideas on this front?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org