You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@couchdb.apache.org by ja...@apache.org on 2021/05/30 01:13:18 UTC
[couchdb] branch sm60-darwin-invalid-offset created (now a3f6313)
This is an automated email from the ASF dual-hosted git repository.
jaydoane pushed a change to branch sm60-darwin-invalid-offset
in repository https://gitbox.apache.org/repos/asf/couchdb.git.
at a3f6313 Implement aegis:key_manager/0
This branch includes the following new commits:
new bc3ae96 Do not warn invalid offset for spidermonkey 60 on mac
new e3d6911 Handle missing db encryption key from lookup
new 2f617e8 Fix aegis get_db_info server test for operational key manager
new a3f6313 Implement aegis:key_manager/0
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[couchdb] 04/04: Implement aegis:key_manager/0
Posted by ja...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jaydoane pushed a commit to branch sm60-darwin-invalid-offset
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit a3f6313b5799338c4322aef2fbe2bbb819651b55
Author: Jay Doane <ja...@apache.org>
AuthorDate: Sat May 29 18:09:35 2021 -0700
Implement aegis:key_manager/0
A convenience function for configuration debugging.
---
src/aegis/src/aegis.erl | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/src/aegis/src/aegis.erl b/src/aegis/src/aegis.erl
index d3f7f96..f2da69f 100644
--- a/src/aegis/src/aegis.erl
+++ b/src/aegis/src/aegis.erl
@@ -18,6 +18,7 @@
-export([
+ key_manager/0,
init_db/2,
open_db/1,
get_db_info/1,
@@ -28,6 +29,10 @@
wrap_fold_fun/2
]).
+key_manager() ->
+ ?AEGIS_KEY_MANAGER.
+
+
init_db(#{} = Db, Options) ->
Db#{
is_encrypted => aegis_server:init_db(Db, Options)
[couchdb] 02/04: Handle missing db encryption key from lookup
Posted by ja...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jaydoane pushed a commit to branch sm60-darwin-invalid-offset
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit e3d6911dd386141b934337af0c1ed270a7198580
Author: Jay Doane <ja...@apache.org>
AuthorDate: Thu May 27 22:17:15 2021 -0700
Handle missing db encryption key from lookup
While performance testing, stack traces were caused by a `badmatch` error
in aegis_server:
```
May 6 21:34:48 c-fdbcore-perf-api-5bc54ff569-jkjxx db error [error] 2021-05-06T21:34:41.337533Z dbcore@127.0.0.1 <0.18553.273> 6adfd54aa5 req_err(3074846293) {{badmatch,{error,not_found}},
[{aegis_server,handle_call,3,[{file,"src/aegis_server.erl"},{line,170}]},
{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,636}]},
{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,665}]},
{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]} : {gen_server,call,
[aegis_server,
{encrypt,
#{after_doc_read => undefined,before_doc_update => undefined,
check_current_ts => -576456460633,db_options => [],
db_prefix => <<21,22,21,15,1,23,5,202,10,0>>,
db_version => <<"f518950b94a759bb410f86d32126f421">>,
interactive => true,is_encrypted => true,
layer_prefix => <<21,22>>,
md_version => <<0,0,12,155,121,36,204,80,0,0>>,
name =>
<<"test-user-standard-0002/ccm_capacity_unit_1620334990_test-user-standard-1512">>,
revs_limit => 1000,
security_doc => {[]},
security_fun => undefined,
tx => {erlfdb_transaction,#Ref<0.4001737934.1270480897.28885>},
user_ctx =>
{user_ctx,<<"test-user-standard-0002">>,
[<<"_admin">>,<<"_reader">>,<<"_writer">>],
<<"cookie">>},
uuid => <<"97e681cfd7224680f72213b8bf59a25c">>,
validate_doc_update_funs => []},
[<<"gen_server:call/2 L206">>,<<"aegis_server:encrypt/3 L93">>,<<"fabric2_fdb:-write_doc_body/2-fun-0-/3 L1430">>,<<"lists:foreach/2 L1338">>,<<"fabric2_fdb:write_doc/6 L851">>,<<"fabric2_db:update_doc_interactive/4 L1944">>,<<"fabric2_db:update_docs_interactive/5 L1808">>,<<"fabric2_db:batch_update_interactive_tx/1 L1752">>]
```
This crash is occurrs in aegis_server [1] (now at a slightly different
line from the trace) because `lookup/2` returns `{error, not_found}`
[2] when unable to find the UUID in the cache.
This handles the error case, preventing a crash, and also DRYs out the
code by factoring a `handle_crypto_call/6` function for both
encryption operations.
[1] https://github.com/apache/couchdb/blob/main/src/aegis/src/aegis_server.erl#L173
[2] https://github.com/apache/couchdb/blob/main/src/aegis/src/aegis_server.erl#L339
---
src/aegis/src/aegis_server.erl | 64 ++++++++++++++++++------------------------
1 file changed, 28 insertions(+), 36 deletions(-)
diff --git a/src/aegis/src/aegis_server.erl b/src/aegis/src/aegis_server.erl
index 15fea4c..aebd528 100644
--- a/src/aegis/src/aegis_server.erl
+++ b/src/aegis/src/aegis_server.erl
@@ -165,43 +165,11 @@ handle_call({insert_key, UUID, DbKey}, _From, #{cache := Cache} = St) ->
NewSt = insert(St, UUID, DbKey),
{reply, ok, NewSt, ?TIMEOUT};
-handle_call({encrypt, #{uuid := UUID} = Db, Key, Value}, From, St) ->
-
- {ok, DbKey} = lookup(St, UUID),
-
- erlang:spawn(fun() ->
- process_flag(sensitive, true),
- try
- do_encrypt(DbKey, Db, Key, Value)
- of
- Resp ->
- gen_server:reply(From, Resp)
- catch
- _:Error ->
- gen_server:reply(From, {error, Error})
- end
- end),
-
- {noreply, St, ?TIMEOUT};
-
-handle_call({decrypt, #{uuid := UUID} = Db, Key, Value}, From, St) ->
-
- {ok, DbKey} = lookup(St, UUID),
-
- erlang:spawn(fun() ->
- process_flag(sensitive, true),
- try
- do_decrypt(DbKey, Db, Key, Value)
- of
- Resp ->
- gen_server:reply(From, Resp)
- catch
- _:Error ->
- gen_server:reply(From, {error, Error})
- end
- end),
+handle_call({encrypt, Db, Key, Value}, From, St) ->
+ handle_crypto_call(fun do_encrypt/4, Db, Key, Value, From, St);
- {noreply, St, ?TIMEOUT};
+handle_call({decrypt, Db, Key, Value}, From, St) ->
+ handle_crypto_call(fun do_decrypt/4, Db, Key, Value, From, St);
handle_call(_Msg, _From, St) ->
{noreply, St}.
@@ -233,6 +201,30 @@ code_change(_OldVsn, St, _Extra) ->
%% private functions
+
+handle_crypto_call(DoCryptoOp, Db, Key, Value, From, St) ->
+ #{uuid := UUID} = Db,
+ couch_log:info("handle_crypto_call ~p ~p ~p", [DoCryptoOp, Db, Key]),
+ case lookup(St, UUID) of
+ {error, not_found} ->
+ gen_server:reply(From, {error, {db_key_not_found_for, UUID}});
+ {ok, DbKey} ->
+ erlang:spawn(fun() ->
+ process_flag(sensitive, true),
+ try
+ DoCryptoOp(DbKey, Db, Key, Value)
+ of
+ Resp ->
+ gen_server:reply(From, Resp)
+ catch
+ _:Error ->
+ gen_server:reply(From, {error, Error})
+ end
+ end),
+ {noreply, St, ?TIMEOUT}
+ end.
+
+
do_open_db(#{uuid := UUID} = Db) ->
case ?AEGIS_KEY_MANAGER:open_db(Db) of
{ok, DbKey} ->
[couchdb] 01/04: Do not warn invalid offset for spidermonkey 60 on
mac
Posted by ja...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jaydoane pushed a commit to branch sm60-darwin-invalid-offset
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit bc3ae96fd24f937b31627bb21986093f78a42bb1
Author: Jay Doane <ja...@apache.org>
AuthorDate: Wed May 26 22:31:54 2021 -0700
Do not warn invalid offset for spidermonkey 60 on mac
---
src/couch/rebar.config.script | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/couch/rebar.config.script b/src/couch/rebar.config.script
index 721c813..96a0489 100644
--- a/src/couch/rebar.config.script
+++ b/src/couch/rebar.config.script
@@ -119,7 +119,7 @@ end.
};
{unix, darwin} when SMVsn == "60" ->
{
- "-DXP_UNIX -I/usr/include/mozjs-60 -I/usr/local/include/mozjs-60 -std=c++14",
+ "-DXP_UNIX -I/usr/include/mozjs-60 -I/usr/local/include/mozjs-60 -std=c++14 -Wno-invalid-offsetof",
"-L/usr/local/lib -lmozjs-60 -lm -std=c++14 -lc++"
};
{unix, _} when SMVsn == "60" ->
[couchdb] 03/04: Fix aegis get_db_info server test for operational
key manager
Posted by ja...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jaydoane pushed a commit to branch sm60-darwin-invalid-offset
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 2f617e8e6b58b76e4e97a669936e4313d70280c6
Author: Jay Doane <ja...@apache.org>
AuthorDate: Sat May 29 16:46:53 2021 -0700
Fix aegis get_db_info server test for operational key manager
When aegis is configured to use an operational key manager, the
get_db_info test can fail since the `passthrough` option is used, but
`get_db_info/1` doesn't get mocked, resulting in the operational key
manager's implementation unintentially being used in the test.
This replaces the `passthrough` option with `non_strict`, and mocks
`get_db_info/1` so that the same mocked functions are used in tests
regardless of whether `AEGIS_KEY_MANAGER` is `aegis_noop_key_manager`,
or an operational key manager.
---
src/aegis/rebar.config.script | 1 +
src/aegis/test/aegis_server_test.erl | 4 +++-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/src/aegis/rebar.config.script b/src/aegis/rebar.config.script
index ef148bf..a72f27a 100644
--- a/src/aegis/rebar.config.script
+++ b/src/aegis/rebar.config.script
@@ -25,6 +25,7 @@ AegisKeyManager = case lists:keyfind(aegis_key_manager, 1, CouchConfig) of
_ ->
aegis_noop_key_manager
end,
+io:format("AegisKeyManager = ~p~n", [AegisKeyManager]),
CurrentOpts = case lists:keyfind(erl_opts, 1, CONFIG) of
{erl_opts, Opts} -> Opts;
diff --git a/src/aegis/test/aegis_server_test.erl b/src/aegis/test/aegis_server_test.erl
index ff16475..8dcf01c 100644
--- a/src/aegis/test/aegis_server_test.erl
+++ b/src/aegis/test/aegis_server_test.erl
@@ -50,10 +50,12 @@ basic_test_() ->
setup() ->
+ ?debugVal(?AEGIS_KEY_MANAGER),
Ctx = test_util:start_couch([fabric]),
- meck:new([?AEGIS_KEY_MANAGER], [passthrough]),
+ meck:new([?AEGIS_KEY_MANAGER], [non_strict]),
ok = meck:expect(?AEGIS_KEY_MANAGER, init_db, 2, {ok, <<0:256>>}),
ok = meck:expect(?AEGIS_KEY_MANAGER, open_db, 1, {ok, <<0:256>>}),
+ ok = meck:expect(?AEGIS_KEY_MANAGER, get_db_info, 1, []),
Ctx.