You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@couchdb.apache.org by wo...@apache.org on 2017/10/10 21:24:07 UTC
[couchdb] branch 749-fix-couch_peruser-app-structure updated
(4336dd4 -> bf2a2c1)
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a change to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git.
omit 4336dd4 80 cols
omit 87b0df9 whitespace and more state fixes
omit 372fb35 fix state
omit cb64096 fix style
omit 94d0ffd fix state call
omit 564b901 unlink changes listeners before exiting them so we survive
omit 3420b87 document ini entries
omit 697ab2d update README
omit 3e32c76 whitespace
omit c9d4ac8 move function declaration around for internal consistency
omit 3a426db s,init/0,init_state/0,
omit 57459a4 s/clusterState/state/ && s/state/changes_state/
omit 2ac9d55 remove leftover code from olde notification system
omit b8334ee add registered modules
omit 5d3b902 simplify couch_persuer.app definition
omit 8b9cc5e fix tests
omit 6203e08 add type specs
omit 736c269 make sure peruser listeners are only initialised once per node
omit 0400ca5 remove reliance on couch_replicator_clustering, handle cluster state internally
omit a35c53b move couch_replication_clustering:owner/3 to mem3.erl
omit b10249b track cluster state in gen_server state and get notfied from mem3 directly
omit a29e164 Ensure a user creation is handlined on one node only
omit 18f558c feat: mango test runner: do not rely on timeout for CouchDB start alone
omit b975e24 Start and stop couch_peruser in the test suite
omit b66e2ef Make couch_peruser a proper Erlang app
add b030a86 Return reduce overflow errors to the client
add a0e0885 Merge pull request #797 from apache/improve-reduce-limit-errors
new 2f49de8 Make couch_peruser a proper Erlang app
new b12683c Start and stop couch_peruser in the test suite
new 7da9d8b feat: mango test runner: do not rely on timeout for CouchDB start alone
new 599dcb3 Ensure a user creation is handlined on one node only
new 1af7ed4 track cluster state in gen_server state and get notfied from mem3 directly
new 93fb6ee move couch_replication_clustering:owner/3 to mem3.erl
new e34d048 remove reliance on couch_replicator_clustering, handle cluster state internally
new f0c2883 make sure peruser listeners are only initialised once per node
new ee9c872 add type specs
new 22656c8 fix tests
new 64ee2b7 simplify couch_persuer.app definition
new 90d4934 add registered modules
new 6487e38 remove leftover code from olde notification system
new 557ab00 s/clusterState/state/ && s/state/changes_state/
new e22f784 s,init/0,init_state/0,
new 3ed4838 move function declaration around for internal consistency
new e10a04b whitespace
new 3771f35 update README
new cc275cd document ini entries
new 4847c02 unlink changes listeners before exiting them so we survive
new 9ae8851 fix state call
new 28b400d fix style
new 96efbff fix state
new 570d6ee whitespace and more state fixes
new bf2a2c1 80 cols
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (4336dd4)
\
N -- N -- N refs/heads/749-fix-couch_peruser-app-structure (bf2a2c1)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 25 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
src/couch/src/couch_query_servers.erl | 29 ++++++++++++++++++++++++-----
src/couch_mrview/src/couch_mrview_http.erl | 6 +++++-
src/fabric/src/fabric_util.erl | 2 --
src/fabric/src/fabric_view.erl | 2 ++
test/javascript/tests/view_errors.js | 4 ++--
5 files changed, 33 insertions(+), 10 deletions(-)
--
To stop receiving notification emails like this one, please contact
['"commits@couchdb.apache.org" <co...@couchdb.apache.org>'].
[couchdb] 03/25: feat: mango test runner: do not rely on timeout
for CouchDB start alone
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 7da9d8bb7e0b54aad9ddc17681f71c4cf7ec66d7
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Sun Oct 1 13:14:43 2017 +0200
feat: mango test runner: do not rely on timeout for CouchDB start alone
On slow build nodes, 10 seconds might not be enough of a wait.
---
test/build/test-run-couch-for-mango.sh | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/test/build/test-run-couch-for-mango.sh b/test/build/test-run-couch-for-mango.sh
index 6034a79..0597a8f 100755
--- a/test/build/test-run-couch-for-mango.sh
+++ b/test/build/test-run-couch-for-mango.sh
@@ -13,8 +13,17 @@
./dev/run -n 1 --admin=testuser:testpass &
export SERVER_PID=$!
-sleep 10
-curl http://dev:15984
+
+COUCH_STARTED=-1
+while ( [ $COUCH_STARTED -ne 0 ] ); do
+ curl -s http://127.0.0.1:15984
+ COUCH_STARTED=$?
+ if [ $COUCH_STARTED -ne 0 ]; then
+ # do not wait another 5 seconds if couch started now
+ sleep 5
+ fi
+done
+
cd src/mango/
nosetests
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 17/25: whitespace
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit e10a04bff527dbdf9ea6cf61f7f4bd1f8c554b32
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Mon Oct 9 08:12:31 2017 +0200
whitespace
---
src/couch_peruser/src/couch_peruser.erl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index 7943a9c..c5110da 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -133,7 +133,7 @@ changes_handler({change, {Doc}, _Prepend}, _ResType, ChangesState=#changes_state
% couch_log:debug("peruser: changes_handler() on DbName/Doc ~p/~p", [DbName, Doc]),
case couch_util:get_value(<<"id">>, Doc) of
- <<"org.couchdb.user:",User/binary>>=DocId ->
+ <<"org.couchdb.user:",User/binary>> = DocId ->
case should_handle_doc(DbName, DocId) of
true ->
case couch_util:get_value(<<"deleted">>, Doc, false) of
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 09/25: add type specs
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit ee9c8725901696034c2603d6509ef626711dc8b7
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Sun Oct 8 16:28:38 2017 +0200
add type specs
---
src/couch_peruser/src/couch_peruser.erl | 53 +++++++++++++++++++++------------
1 file changed, 34 insertions(+), 19 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index e722b7e..16305c6 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -33,20 +33,20 @@
]).
-record(state, {
- parent,
- db_name,
- delete_dbs,
- changes_pid,
- changes_ref
+ parent :: pid(),
+ db_name :: binary(),
+ delete_dbs :: boolean(),
+ changes_pid :: pid(),
+ changes_ref :: reference()
}).
-record(clusterState, {
- parent,
- db_name,
- delete_dbs,
- states,
- mem3_cluster_pid,
- cluster_stable
+ parent :: pid(),
+ db_name :: binary(),
+ delete_dbs :: boolean(),
+ states :: list(),
+ mem3_cluster_pid :: pid(),
+ cluster_stable :: boolean()
}).
-define(USERDB_PREFIX, "userdb-").
@@ -57,10 +57,11 @@
%%
%% Please leave in the commented-out couch_log:debug calls, thanks! — Jan
%%
-
+-spec start_link() -> {ok, pid()} | ignore | {error, term()}.
start_link() ->
gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+-spec init() -> #clusterState{}.
init() ->
couch_log:debug("peruser: starting on node ~p in pid ~p", [node(), self()]),
case config:get_boolean("couch_peruser", "enable", false) of
@@ -92,11 +93,12 @@ init() ->
end.
% Cluster membership change notification callback
--spec notify_cluster_event(pid(), {cluster, any()}) -> ok.
+-spec notify_cluster_event(Server :: pid(), Event :: {cluster, any()}) -> ok.
notify_cluster_event(Server, {cluster, _} = Event) ->
% couch_log:debug("peruser: received cluster event ~p on node ~p", [Event, node()]),
gen_server:cast(Server, Event).
+-spec start_listening(ClusterState :: #clusterState{}) -> #clusterState{} | ok.
start_listening(#clusterState{states=States}=ClusterState) when length(States) > 0 ->
% couch_log:debug("peruser: start_listening() already run on node ~p in pid ~p", [node(), self()]),
ClusterState;
@@ -119,6 +121,7 @@ start_listening(#clusterState{db_name=DbName, delete_dbs=DeleteDbs} = ClusterSta
config:set("couch_peruser", "enable", "false", lists:concat([binary_to_list(DbName), " is missing"]))
end.
+-spec init_changes_handler(State :: #state{}) -> ok.
init_changes_handler(#state{db_name=DbName} = State) ->
% couch_log:debug("peruser: init_changes_handler() on DbName ~p", [DbName]),
try
@@ -132,7 +135,8 @@ init_changes_handler(#state{db_name=DbName} = State) ->
ok
end.
-
+-type db_change() :: {atom(), tuple(), binary()}.
+-spec changes_handler(Change :: db_change(), ResultType :: any(), State :: #state{}) -> #state{}.
changes_handler({change, {Doc}, _Prepend}, _ResType, State=#state{db_name=DbName}) ->
% couch_log:debug("peruser: changes_handler() on DbName/Doc ~p/~p", [DbName, Doc]),
@@ -165,7 +169,7 @@ changes_handler({change, {Doc}, _Prepend}, _ResType, State=#state{db_name=DbName
changes_handler(_Event, _ResType, State) ->
State.
-
+-spec should_handle_doc(ShardName :: binary(), DocId::binary()) -> boolean().
should_handle_doc(ShardName, DocId) ->
case is_stable() of
false ->
@@ -178,6 +182,7 @@ should_handle_doc(ShardName, DocId) ->
should_handle_doc_int(ShardName, DocId)
end.
+-spec should_handle_doc_int(ShardName :: binary(), DocId :: binary()) -> boolean().
should_handle_doc_int(ShardName, DocId) ->
DbName = mem3:dbname(ShardName),
Live = [erlang:node() | erlang:nodes()],
@@ -192,7 +197,7 @@ should_handle_doc_int(ShardName, DocId) ->
false
end.
-
+-spec delete_user_db(User :: binary()) -> binary().
delete_user_db(User) ->
UserDb = user_db_name(User),
try
@@ -205,6 +210,7 @@ delete_user_db(User) ->
end,
UserDb.
+-spec ensure_user_db(User :: binary()) -> binary().
ensure_user_db(User) ->
UserDb = user_db_name(User),
try
@@ -218,6 +224,7 @@ ensure_user_db(User) ->
end,
UserDb.
+-spec add_user(User :: binary(), Properties :: tuple(), Acc :: tuple()) -> tuple().
add_user(User, Prop, {Modified, SecProps}) ->
{PropValue} = couch_util:get_value(Prop, SecProps, {[]}),
Names = couch_util:get_value(<<"names">>, PropValue, []),
@@ -234,6 +241,7 @@ add_user(User, Prop, {Modified, SecProps}) ->
{<<"names">>, [User | Names]})}})}
end.
+-spec remove_user(User :: binary(), Properties :: tuple(), Acc :: tuple()) -> tuple().
remove_user(User, Prop, {Modified, SecProps}) ->
{PropValue} = couch_util:get_value(Prop, SecProps, {[]}),
Names = couch_util:get_value(<<"names">>, PropValue, []),
@@ -250,10 +258,11 @@ remove_user(User, Prop, {Modified, SecProps}) ->
{<<"names">>, lists:delete(User, Names)})}})}
end.
+-spec ensure_security(User :: binary(), UserDb :: binary(), TransformFun :: fun()) -> ok.
ensure_security(User, UserDb, TransformFun) ->
case fabric:get_all_security(UserDb, [?ADMIN_CTX]) of
{error, no_majority} ->
- % single node, ignore
+ % TODO: make sure this is still true: single node, ignore
ok;
{ok, Shards} ->
{_ShardInfo, {SecProps}} = hd(Shards),
@@ -272,11 +281,13 @@ ensure_security(User, UserDb, TransformFun) ->
end
end.
+-spec user_db_name(User :: binary()) -> binary().
user_db_name(User) ->
HexUser = list_to_binary(
[string:to_lower(integer_to_list(X, 16)) || <<X>> <= User]),
<<?USERDB_PREFIX,HexUser/binary>>.
+-spec exit_changes(ClusterState :: #clusterState{}) -> ok.
exit_changes(ClusterState) ->
lists:foreach(fun (State) ->
demonitor(State#state.changes_ref, [flush]),
@@ -289,16 +300,20 @@ is_stable() ->
% Mem3 cluster callbacks
+% TODO: find out what type Server is
+-spec cluster_unstable(Server :: any()) -> any().
cluster_unstable(Server) ->
gen_server:cast(Server, cluster_unstable),
Server.
+% TODO: find out what type Server is
+-spec cluster_stable(Server :: any()) -> any().
cluster_stable(Server) ->
gen_server:cast(Server, cluster_stable),
Server.
%% gen_server callbacks
-
+-spec init(Options :: list()) -> {ok, #clusterState{}}.
init([]) ->
ok = subscribe_for_changes(),
{ok, init()}.
@@ -344,13 +359,13 @@ handle_info(restart_config_listener, State) ->
handle_info(_Msg, State) ->
{noreply, State}.
+-spec subscribe_for_changes() -> ok.
subscribe_for_changes() ->
config:subscribe_for_changes([
{"couch_httpd_auth", "authentication_db"},
"couch_peruser"
]).
-
terminate(_Reason, _State) ->
%% Everything should be linked or monitored, let nature
%% take its course.
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 04/25: Ensure a user creation is handlined on one node
only
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 599dcb32a7609359aa7e170d7ef8e95dd539b10d
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Sat Oct 7 17:04:54 2017 +0200
Ensure a user creation is handlined on one node only
This patch makes use of the mechanism that ensures that replications
are only run on one node.
When the cluster has nodes added/removed all changes listeners are
restarted.
---
src/couch_peruser/src/couch_peruser.app.src | 2 +-
src/couch_peruser/src/couch_peruser.erl | 158 +++++++++++++++++++---------
2 files changed, 110 insertions(+), 50 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.app.src b/src/couch_peruser/src/couch_peruser.app.src
index 777446d..42b7b25 100644
--- a/src/couch_peruser/src/couch_peruser.app.src
+++ b/src/couch_peruser/src/couch_peruser.app.src
@@ -14,7 +14,7 @@
{description, "couch_peruser - maintains per-user databases in CouchDB"},
{vsn, git},
{registered, []},
- {applications, [kernel, stdlib, config, couch, fabric]},
+ {applications, [kernel, stdlib, config, couch, fabric, couch_replicator, mem3]},
{mod, {couch_peruser_app, []}},
{env, []},
{modules, [couch_peruser, couch_peruser_app, couch_peruser_sup]}
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index 63ef084..9161f56 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -22,6 +22,9 @@
-export([start_link/0, init/1, handle_call/3, handle_cast/2, handle_info/2,
terminate/2, code_change/3]).
+% cluster state notification callback
+-export([notify_cluster_event/2]).
+
-export([init_changes_handler/1, changes_handler/3]).
-record(state, {parent, db_name, delete_dbs, changes_pid, changes_ref}).
@@ -34,10 +37,13 @@ start_link() ->
gen_server:start_link(?MODULE, [], []).
init() ->
+ couch_log:debug("peruser: starting on node ~p", [node()]),
case config:get_boolean("couch_peruser", "enable", false) of
false ->
+ couch_log:debug("peruser: disabled on node ~p", [node()]),
#clusterState{};
true ->
+ couch_log:debug("peruser: enabled on node ~p", [node()]),
DbName = ?l2b(config:get(
"couch_httpd_auth", "authentication_db", "_users")),
DeleteDbs = config:get_boolean("couch_peruser", "delete_dbs", false),
@@ -47,21 +53,37 @@ init() ->
db_name = DbName,
delete_dbs = DeleteDbs
},
- try
- States = lists:map(fun (A) ->
- S = #state{parent = ClusterState#clusterState.parent,
- db_name = A#shard.name,
- delete_dbs = DeleteDbs},
- {Pid, Ref} = spawn_opt(
- ?MODULE, init_changes_handler, [S], [link, monitor]),
- S#state{changes_pid=Pid, changes_ref=Ref}
- end, mem3:local_shards(DbName)),
-
- ClusterState#clusterState{states = States}
- catch error:database_does_not_exist ->
- couch_log:warning("couch_peruser can't proceed as underlying database (~s) is missing, disables itself.", [DbName]),
- config:set("couch_peruser", "enable", "false", lists:concat([binary_to_list(DbName), " is missing"]))
- end
+
+ % set up cluster-stable listener
+ couch_replicator_clustering:link_cluster_event_listener(?MODULE,
+ notify_cluster_event, [self()]),
+
+ couch_log:debug("peruser: registered for cluster event on node ~p", [node()]),
+ ClusterState
+ end.
+
+% Cluster membership change notification callback
+-spec notify_cluster_event(pid(), {cluster, any()}) -> ok.
+notify_cluster_event(Server, {cluster, _} = Event) ->
+ couch_log:debug("peruser: received cluster event ~p on node ~p", [Event, node()]),
+ gen_server:cast(Server, Event).
+
+start_listening(#clusterState{db_name=DbName, delete_dbs=DeleteDbs} = ClusterState) ->
+ couch_log:debug("peruser: start_listening() on node ~p", [node()]),
+ try
+ States = lists:map(fun (A) ->
+ S = #state{parent = ClusterState#clusterState.parent,
+ db_name = A#shard.name,
+ delete_dbs = DeleteDbs},
+ {Pid, Ref} = spawn_opt(
+ ?MODULE, init_changes_handler, [S], [link, monitor]),
+ S#state{changes_pid=Pid, changes_ref=Ref}
+ end, mem3:local_shards(DbName)),
+
+ ClusterState#clusterState{states = States}
+ catch error:database_does_not_exist ->
+ couch_log:warning("couch_peruser can't proceed as underlying database (~s) is missing, disables itself.", [DbName]),
+ config:set("couch_peruser", "enable", "false", lists:concat([binary_to_list(DbName), " is missing"]))
end.
init_changes_handler(#state{db_name=DbName} = State) ->
@@ -76,24 +98,30 @@ init_changes_handler(#state{db_name=DbName} = State) ->
ok
end.
-changes_handler({change, {Doc}, _Prepend}, _ResType, State=#state{}) ->
+
+changes_handler({change, {Doc}, _Prepend}, _ResType, State=#state{db_name=DbName}) ->
case couch_util:get_value(<<"id">>, Doc) of
- <<"org.couchdb.user:",User/binary>> ->
- case couch_util:get_value(<<"deleted">>, Doc, false) of
- false ->
- UserDb = ensure_user_db(User),
- ok = ensure_security(User, UserDb, fun add_user/3),
- State;
+ <<"org.couchdb.user:",User/binary>>=DocId ->
+ case should_handle_doc(DbName, DocId) of
true ->
- case State#state.delete_dbs of
- true ->
- _UserDb = delete_user_db(User),
- State;
+ case couch_util:get_value(<<"deleted">>, Doc, false) of
false ->
- UserDb = user_db_name(User),
- ok = ensure_security(User, UserDb, fun remove_user/3),
- State
- end
+ UserDb = ensure_user_db(User),
+ ok = ensure_security(User, UserDb, fun add_user/3),
+ State;
+ true ->
+ case State#state.delete_dbs of
+ true ->
+ _UserDb = delete_user_db(User),
+ State;
+ false ->
+ UserDb = user_db_name(User),
+ ok = ensure_security(User, UserDb, fun remove_user/3),
+ State
+ end
+ end;
+ false ->
+ State
end;
_ ->
State
@@ -101,6 +129,25 @@ changes_handler({change, {Doc}, _Prepend}, _ResType, State=#state{}) ->
changes_handler(_Event, _ResType, State) ->
State.
+should_handle_doc(DbName, DocId) ->
+ case couch_replicator_clustering:owner(DbName, DocId) of
+ unstable ->
+ % todo: when we do proper resume[1], we can return false here
+ % and rely on a module restart when the cluster is stable again
+ % in the meantime, we risk conflicts when the cluster gets unstable
+ % and users are being created.
+ % [1] https://github.com/apache/couchdb/issues/872
+ true;
+ ThisNode when ThisNode =:= node() ->
+ couch_log:debug("peruser: handling ~s/~s", [DbName, DocId]),
+ % do the deed
+ true;
+ _OtherNode ->
+ couch_log:debug("peruser: skipping ~s/~s", [DbName, DocId]),
+ false
+ end.
+
+
delete_user_db(User) ->
UserDb = user_db_name(User),
try
@@ -158,20 +205,25 @@ remove_user(User, Prop, {Modified, SecProps}) ->
end.
ensure_security(User, UserDb, TransformFun) ->
- {ok, Shards} = fabric:get_all_security(UserDb, [?ADMIN_CTX]),
- {_ShardInfo, {SecProps}} = hd(Shards),
- % assert that shards have the same security object
- true = lists:all(fun ({_, {SecProps1}}) ->
- SecProps =:= SecProps1
- end, Shards),
- case lists:foldl(
- fun (Prop, SAcc) -> TransformFun(User, Prop, SAcc) end,
- {false, SecProps},
- [<<"admins">>, <<"members">>]) of
- {false, _} ->
- ok;
- {true, SecProps1} ->
- ok = fabric:set_security(UserDb, {SecProps1}, [?ADMIN_CTX])
+ case fabric:get_all_security(UserDb, [?ADMIN_CTX]) of
+ {error, no_majority} ->
+ % single node, ignore
+ ok;
+ {ok, Shards} ->
+ {_ShardInfo, {SecProps}} = hd(Shards),
+ % assert that shards have the same security object
+ true = lists:all(fun ({_, {SecProps1}}) ->
+ SecProps =:= SecProps1
+ end, Shards),
+ case lists:foldl(
+ fun (Prop, SAcc) -> TransformFun(User, Prop, SAcc) end,
+ {false, SecProps},
+ [<<"admins">>, <<"members">>]) of
+ {false, _} ->
+ ok;
+ {true, SecProps1} ->
+ ok = fabric:set_security(UserDb, {SecProps1}, [?ADMIN_CTX])
+ end
end.
user_db_name(User) ->
@@ -179,6 +231,11 @@ user_db_name(User) ->
[string:to_lower(integer_to_list(X, 16)) || <<X>> <= User]),
<<?USERDB_PREFIX,HexUser/binary>>.
+exit_changes(ClusterState) ->
+ lists:foreach(fun (State) ->
+ demonitor(State#state.changes_ref, [flush]),
+ exit(State#state.changes_pid, kill)
+ end, ClusterState#clusterState.states).
%% gen_server callbacks
@@ -191,16 +248,19 @@ handle_call(_Msg, _From, State) ->
handle_cast(update_config, ClusterState) when ClusterState#clusterState.states =/= undefined ->
- lists:foreach(fun (State) ->
- demonitor(State#state.changes_ref, [flush]),
- exit(State#state.changes_pid, kill)
- end, ClusterState#clusterState.states),
-
+ exit_changes(ClusterState),
{noreply, init()};
handle_cast(update_config, _) ->
{noreply, init()};
handle_cast(stop, State) ->
{stop, normal, State};
+handle_cast({cluster, unstable}, ClusterState) when ClusterState#clusterState.states =/= undefined ->
+ exit_changes(ClusterState),
+ {noreply, init()};
+handle_cast({cluster, unstable}, _) ->
+ {noreply, init()};
+handle_cast({cluster, stable}, State) ->
+ {noreply, start_listening(State)};
handle_cast(_Msg, State) ->
{noreply, State}.
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 05/25: track cluster state in gen_server state and get
notfied from mem3 directly
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 1af7ed4b54875263c9fe62da210bba93f6168ad1
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Sat Oct 7 23:17:30 2017 +0200
track cluster state in gen_server state and get notfied from mem3 directly
---
src/couch_peruser/src/couch_peruser.erl | 65 ++++++++++++++++++++++++---------
1 file changed, 48 insertions(+), 17 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index 9161f56..a31ff60 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -12,12 +12,11 @@
-module(couch_peruser).
-behaviour(gen_server).
+-behaviour(mem3_cluster).
-include_lib("couch/include/couch_db.hrl").
-include_lib("mem3/include/mem3.hrl").
--define(USERDB_PREFIX, "userdb-").
-
% gen_server callbacks
-export([start_link/0, init/1, handle_call/3, handle_cast/2, handle_info/2,
terminate/2, code_change/3]).
@@ -27,10 +26,25 @@
-export([init_changes_handler/1, changes_handler/3]).
+% mem3_cluster callbacks
+-export([
+ cluster_stable/1,
+ cluster_unstable/1
+]).
+
-record(state, {parent, db_name, delete_dbs, changes_pid, changes_ref}).
--record(clusterState, {parent, db_name, delete_dbs, states}).
+-record(clusterState, {parent,
+ db_name,
+ delete_dbs,
+ states,
+ mem3_cluster_pid,
+ cluster_stable
+}).
+-define(USERDB_PREFIX, "userdb-").
-define(RELISTEN_DELAY, 5000).
+-define(DEFAULT_QUIET_PERIOD, 60). % seconds
+-define(DEFAULT_START_PERIOD, 5). % seconds
start_link() ->
@@ -48,18 +62,24 @@ init() ->
"couch_httpd_auth", "authentication_db", "_users")),
DeleteDbs = config:get_boolean("couch_peruser", "delete_dbs", false),
- ClusterState = #clusterState{
- parent = self(),
- db_name = DbName,
- delete_dbs = DeleteDbs
- },
-
% set up cluster-stable listener
- couch_replicator_clustering:link_cluster_event_listener(?MODULE,
- notify_cluster_event, [self()]),
+ Period = abs(config:get_integer("couch_peruser", "cluster_quiet_period",
+ ?DEFAULT_QUIET_PERIOD)),
+ StartPeriod = abs(config:get_integer("couch_peruser", "cluster_start_period",
+ ?DEFAULT_START_PERIOD)),
+
+ {ok, Mem3Cluster} = mem3_cluster:start_link(?MODULE, self(), StartPeriod,
+ Period),
couch_log:debug("peruser: registered for cluster event on node ~p", [node()]),
- ClusterState
+
+ #clusterState{
+ parent = self(),
+ db_name = DbName,
+ delete_dbs = DeleteDbs,
+ mem3_cluster_pid = Mem3Cluster,
+ cluster_stable = false
+ }
end.
% Cluster membership change notification callback
@@ -80,7 +100,7 @@ start_listening(#clusterState{db_name=DbName, delete_dbs=DeleteDbs} = ClusterSta
S#state{changes_pid=Pid, changes_ref=Ref}
end, mem3:local_shards(DbName)),
- ClusterState#clusterState{states = States}
+ ClusterState#clusterState{states = States, cluster_stable = true}
catch error:database_does_not_exist ->
couch_log:warning("couch_peruser can't proceed as underlying database (~s) is missing, disables itself.", [DbName]),
config:set("couch_peruser", "enable", "false", lists:concat([binary_to_list(DbName), " is missing"]))
@@ -166,6 +186,7 @@ ensure_user_db(User) ->
{ok, _DbInfo} = fabric:get_db_info(UserDb)
catch error:database_does_not_exist ->
case fabric:create_db(UserDb, [?ADMIN_CTX]) of
+ {error, file_exists} -> ok;
ok -> ok;
accepted -> ok
end
@@ -207,7 +228,7 @@ remove_user(User, Prop, {Modified, SecProps}) ->
ensure_security(User, UserDb, TransformFun) ->
case fabric:get_all_security(UserDb, [?ADMIN_CTX]) of
{error, no_majority} ->
- % single node, ignore
+ % single node, ignore
ok;
{ok, Shards} ->
{_ShardInfo, {SecProps}} = hd(Shards),
@@ -237,6 +258,16 @@ exit_changes(ClusterState) ->
exit(State#state.changes_pid, kill)
end, ClusterState#clusterState.states).
+% Mem3 cluster callbacks
+
+cluster_unstable(Server) ->
+ gen_server:cast(Server, cluster_unstable),
+ Server.
+
+cluster_stable(Server) ->
+ gen_server:cast(Server, cluster_stable),
+ Server.
+
%% gen_server callbacks
init([]) ->
@@ -254,12 +285,12 @@ handle_cast(update_config, _) ->
{noreply, init()};
handle_cast(stop, State) ->
{stop, normal, State};
-handle_cast({cluster, unstable}, ClusterState) when ClusterState#clusterState.states =/= undefined ->
+handle_cast(cluster_unstable, ClusterState) when ClusterState#clusterState.states =/= undefined ->
exit_changes(ClusterState),
{noreply, init()};
-handle_cast({cluster, unstable}, _) ->
+handle_cast(cluster_unstable, _) ->
{noreply, init()};
-handle_cast({cluster, stable}, State) ->
+handle_cast(cluster_stable, State) ->
{noreply, start_listening(State)};
handle_cast(_Msg, State) ->
{noreply, State}.
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 18/25: update README
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 3771f35e1678379406cbd0ee6366b0bf3aa8d190
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Mon Oct 9 08:45:44 2017 +0200
update README
---
src/couch_peruser/README.md | 19 ++++++++++++++++++-
1 file changed, 18 insertions(+), 1 deletion(-)
diff --git a/src/couch_peruser/README.md b/src/couch_peruser/README.md
index 70f8348..64a0518 100644
--- a/src/couch_peruser/README.md
+++ b/src/couch_peruser/README.md
@@ -1,6 +1,6 @@
# couch_peruser [](https://travis-ci.org/apache/couchdb-peruser)
-couch_peruser is a CouchDB daemon that ensures that a private per-user
+couch_peruser is a CouchDB application that ensures that a private per-user
database exists for each document in _users. These databases are
writable only by the corresponding user. Databases are in the form:
@@ -15,3 +15,20 @@ correctly implement in just about any language, especially JavaScript
and Erlang. Other encodings would be possible, but would require
additional client and server-side code to support that encoding. This
is the simplest scheme that is obviously correct.
+
+## Implementation Notes
+
+The module itself is a `gen_server` and it implements the `mem3_cluster`
+behaviour.
+
+In a CouchDB cluster, the module runs on each node in the cluster. On startup,
+it launches a changes listener for each shard of the `authentication_db`
+(`_users`).
+
+In a cluster, when a change notification comes in (after a user doc has been
+created/updated/deleted), each node independently calculates if it should
+handle the notification based on the current list of active nodes in the
+cluster. This ensures that we avoid trying to update the internal `_dbs`
+concurrently and causing conflicts. It also ensures that at least one node
+does handle a notification. The mechanism that handles this does survive
+cluster reconfigurations transparently.
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 07/25: remove reliance on couch_replicator_clustering,
handle cluster state internally
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit e34d0484c561600162f714d251decbea73abd423
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Sun Oct 8 11:14:28 2017 +0200
remove reliance on couch_replicator_clustering, handle cluster state internally
---
src/couch_peruser/src/couch_peruser.erl | 56 ++++++++++++++++++++++-----------
1 file changed, 37 insertions(+), 19 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index a31ff60..791431c 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -33,7 +33,8 @@
]).
-record(state, {parent, db_name, delete_dbs, changes_pid, changes_ref}).
--record(clusterState, {parent,
+-record(clusterState, {
+ parent,
db_name,
delete_dbs,
states,
@@ -48,10 +49,10 @@
start_link() ->
- gen_server:start_link(?MODULE, [], []).
+ gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
init() ->
- couch_log:debug("peruser: starting on node ~p", [node()]),
+ couch_log:debug("peruser: starting on node ~p in pid ~p", [node(), self()]),
case config:get_boolean("couch_peruser", "enable", false) of
false ->
couch_log:debug("peruser: disabled on node ~p", [node()]),
@@ -107,6 +108,8 @@ start_listening(#clusterState{db_name=DbName, delete_dbs=DeleteDbs} = ClusterSta
end.
init_changes_handler(#state{db_name=DbName} = State) ->
+ % leave for debugging
+ % couch_log:debug("peruser: init_changes_handler() on DbName ~p", [DbName]),
try
{ok, Db} = couch_db:open_int(DbName, [?ADMIN_CTX, sys_db]),
FunAcc = {fun ?MODULE:changes_handler/3, State},
@@ -120,6 +123,9 @@ init_changes_handler(#state{db_name=DbName} = State) ->
changes_handler({change, {Doc}, _Prepend}, _ResType, State=#state{db_name=DbName}) ->
+ % leave for debugging
+ % couch_log:debug("peruser: changes_handler() on DbName/Doc ~p/~p", [DbName, Doc]),
+
case couch_util:get_value(<<"id">>, Doc) of
<<"org.couchdb.user:",User/binary>>=DocId ->
case should_handle_doc(DbName, DocId) of
@@ -149,22 +155,28 @@ changes_handler({change, {Doc}, _Prepend}, _ResType, State=#state{db_name=DbName
changes_handler(_Event, _ResType, State) ->
State.
-should_handle_doc(DbName, DocId) ->
- case couch_replicator_clustering:owner(DbName, DocId) of
- unstable ->
- % todo: when we do proper resume[1], we can return false here
- % and rely on a module restart when the cluster is stable again
- % in the meantime, we risk conflicts when the cluster gets unstable
- % and users are being created.
- % [1] https://github.com/apache/couchdb/issues/872
- true;
- ThisNode when ThisNode =:= node() ->
- couch_log:debug("peruser: handling ~s/~s", [DbName, DocId]),
- % do the deed
- true;
- _OtherNode ->
- couch_log:debug("peruser: skipping ~s/~s", [DbName, DocId]),
- false
+should_handle_doc(ShardName, DocId) ->
+ should_handle_doc_int(ShardName, DocId, is_stable()).
+
+should_handle_doc_int(ShardName, DocId, false) ->
+ % when the cluster is unstable, we have already stopped all Listeners
+ % the next stable event will restart all listeners and pick up this
+ % doc change
+ couch_log:debug("peruser: skipping, cluster unstable ~s/~s", [ShardName, DocId]),
+ false;
+should_handle_doc_int(ShardName, DocId, true) ->
+ DbName = mem3:dbname(ShardName),
+ Live = [erlang:node() | erlang:nodes()],
+ Shards = mem3:shards(DbName, DocId),
+ Nodes = [N || #shard{node=N} <- Shards, lists:member(N, Live)],
+ case mem3:owner(DbName, DocId, Nodes) of
+ ThisNode when ThisNode =:= node() ->
+ couch_log:debug("peruser: handling ~s/~s", [DbName, DocId]),
+ % do the deed
+ true;
+ _OtherNode ->
+ couch_log:debug("peruser: skipping ~s/~s", [DbName, DocId]),
+ false
end.
@@ -258,6 +270,10 @@ exit_changes(ClusterState) ->
exit(State#state.changes_pid, kill)
end, ClusterState#clusterState.states).
+-spec is_stable() -> true | false.
+is_stable() ->
+ gen_server:call(?MODULE, is_stable).
+
% Mem3 cluster callbacks
cluster_unstable(Server) ->
@@ -274,6 +290,8 @@ init([]) ->
ok = subscribe_for_changes(),
{ok, init()}.
+handle_call(is_stable, _From, #clusterState{cluster_stable = IsStable} = State) ->
+ {reply, IsStable, State};
handle_call(_Msg, _From, State) ->
{reply, error, State}.
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 21/25: fix state call
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 9ae88514637d59a4e66bf9de906876739542a251
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Tue Oct 10 10:47:39 2017 +0200
fix state call
---
src/couch_peruser/src/couch_peruser.erl | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index 2493387..f130e13 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -279,10 +279,10 @@ user_db_name(User) ->
[string:to_lower(integer_to_list(X, 16)) || <<X>> <= User]),
<<?USERDB_PREFIX,HexUser/binary>>.
--spec exit_changes(State :: #state{}) -> ok.
-exit_changes(State) ->
+-spec exit_changes(ChangesState :: #changes_state{}) -> ok.
+exit_changes(ChangesState) ->
lists:foreach(fun (ChangesState) ->
- demonitor(State#changes_state.changes_ref, [flush]),
+ demonitor(ChangesState#changes_state.changes_ref, [flush]),
unlink(ChangesState#changes_state.changes_pid),
exit(ChangesState#changes_state.changes_pid, kill)
end, State#state.states).
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 19/25: document ini entries
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit cc275cdb494291c09bc1508fdb00340f0c176318
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Mon Oct 9 08:49:52 2017 +0200
document ini entries
---
rel/overlay/etc/default.ini | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/rel/overlay/etc/default.ini b/rel/overlay/etc/default.ini
index 56f9147..4e61deb 100644
--- a/rel/overlay/etc/default.ini
+++ b/rel/overlay/etc/default.ini
@@ -88,6 +88,10 @@ enable = false
; If set to true and a user is deleted, the respective database gets
; deleted as well.
delete_dbs = false
+; Wait this many seconds after startup before attaching changes listeners
+; cluster_start_period = 5
+; Re-check cluster state at least every cluster_quiet_period seconds
+; cluster_quiet_period = 60
[httpd]
port = {{backend_port}}
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 11/25: simplify couch_persuer.app definition
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 64ee2b7b90e4b5da804301a527a5388c98c98877
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Mon Oct 9 07:54:39 2017 +0200
simplify couch_persuer.app definition
---
src/couch_peruser/src/couch_peruser.app.src | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.app.src b/src/couch_peruser/src/couch_peruser.app.src
index 9859e03..f7c5134 100644
--- a/src/couch_peruser/src/couch_peruser.app.src
+++ b/src/couch_peruser/src/couch_peruser.app.src
@@ -16,6 +16,5 @@
{registered, []},
{applications, [kernel, stdlib, config, couch, fabric, mem3]},
{mod, {couch_peruser_app, []}},
- {env, []},
- {modules, [couch_peruser, couch_peruser_app, couch_peruser_sup]}
+ {env, []}
]}.
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 25/25: 80 cols
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit bf2a2c139efecbba184cdeba6b4d14c19e5e84a0
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Tue Oct 10 20:59:40 2017 +0200
80 cols
---
src/couch_peruser/src/couch_peruser.erl | 37 ++++++++++++++++++++++++++-------
1 file changed, 29 insertions(+), 8 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index a394659..0c76932 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -131,8 +131,14 @@ init_changes_handler(#changes_state{db_name=DbName} = ChangesState) ->
end.
-type db_change() :: {atom(), tuple(), binary()}.
--spec changes_handler(Change :: db_change(), ResultType :: any(), ChangesState :: #changes_state{}) -> #changes_state{}.
-changes_handler({change, {Doc}, _Prepend}, _ResType, ChangesState=#changes_state{db_name=DbName}) ->
+-spec changes_handler(
+ Change :: db_change(),
+ ResultType :: any(),
+ ChangesState :: #changes_state{}) -> #changes_state{}.
+changes_handler(
+ {change, {Doc}, _Prepend},
+ _ResType,
+ ChangesState=#changes_state{db_name=DbName}) ->
% couch_log:debug("peruser: changes_handler() on DbName/Doc ~p/~p", [DbName, Doc]),
case couch_util:get_value(<<"id">>, Doc) of
@@ -171,13 +177,16 @@ should_handle_doc(ShardName, DocId) ->
% when the cluster is unstable, we have already stopped all Listeners
% the next stable event will restart all listeners and pick up this
% doc change
- couch_log:debug("peruser: skipping, cluster unstable ~s/~s", [ShardName, DocId]),
+ couch_log:debug("peruser: skipping, cluster unstable ~s/~s",
+ [ShardName, DocId]),
false;
true ->
should_handle_doc_int(ShardName, DocId)
end.
--spec should_handle_doc_int(ShardName :: binary(), DocId :: binary()) -> boolean().
+-spec should_handle_doc_int(
+ ShardName :: binary(),
+ DocId :: binary()) -> boolean().
should_handle_doc_int(ShardName, DocId) ->
DbName = mem3:dbname(ShardName),
Live = [erlang:node() | erlang:nodes()],
@@ -219,7 +228,10 @@ ensure_user_db(User) ->
end,
UserDb.
--spec add_user(User :: binary(), Properties :: tuple(), Acc :: tuple()) -> tuple().
+-spec add_user(
+ User :: binary(),
+ Properties :: tuple(),
+ Acc :: tuple()) -> tuple().
add_user(User, Prop, {Modified, SecProps}) ->
{PropValue} = couch_util:get_value(Prop, SecProps, {[]}),
Names = couch_util:get_value(<<"names">>, PropValue, []),
@@ -236,7 +248,10 @@ add_user(User, Prop, {Modified, SecProps}) ->
{<<"names">>, [User | Names]})}})}
end.
--spec remove_user(User :: binary(), Properties :: tuple(), Acc :: tuple()) -> tuple().
+-spec remove_user(
+ User :: binary(),
+ Properties :: tuple(),
+ Acc :: tuple()) -> tuple().
remove_user(User, Prop, {Modified, SecProps}) ->
{PropValue} = couch_util:get_value(Prop, SecProps, {[]}),
Names = couch_util:get_value(<<"names">>, PropValue, []),
@@ -253,7 +268,10 @@ remove_user(User, Prop, {Modified, SecProps}) ->
{<<"names">>, lists:delete(User, Names)})}})}
end.
--spec ensure_security(User :: binary(), UserDb :: binary(), TransformFun :: fun()) -> ok.
+-spec ensure_security(
+ User :: binary(),
+ UserDb :: binary(),
+ TransformFun :: fun()) -> ok.
ensure_security(User, UserDb, TransformFun) ->
case fabric:get_all_security(UserDb, [?ADMIN_CTX]) of
{error, no_majority} ->
@@ -348,7 +366,10 @@ handle_info({'DOWN', _Ref, _, _, _Reason}, State) ->
{stop, normal, State};
handle_info({config_change, "couch_peruser", _, _, _}, State) ->
handle_cast(update_config, State);
-handle_info({config_change, "couch_httpd_auth", "authentication_db", _, _}, State) ->
+handle_info({
+ config_change,
+ "couch_httpd_auth",
+ "authentication_db", _, _}, State) ->
handle_cast(update_config, State);
handle_info({gen_event_EXIT, _Handler, _Reason}, State) ->
erlang:send_after(?RELISTEN_DELAY, self(), restart_config_listener),
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 15/25: s,init/0,init_state/0,
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit e22f7842b503c76a5d773e4589bb3034fa57a583
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Mon Oct 9 08:10:52 2017 +0200
s,init/0,init_state/0,
---
src/couch_peruser/src/couch_peruser.erl | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index bda74ec..f820fe6 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -58,8 +58,8 @@
start_link() ->
gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
--spec init() -> #state{}.
-init() ->
+-spec init_state() -> #state{}.
+init_state() ->
couch_log:debug("peruser: starting on node ~p in pid ~p", [node(), self()]),
case config:get_boolean("couch_peruser", "enable", false) of
false ->
@@ -308,7 +308,7 @@ cluster_stable(Server) ->
-spec init(Options :: list()) -> {ok, #state{}}.
init([]) ->
ok = subscribe_for_changes(),
- {ok, init()}.
+ {ok, init_state()}.
handle_call(is_stable, _From, #state{cluster_stable = IsStable} = State) ->
{reply, IsStable, State};
@@ -318,16 +318,16 @@ handle_call(_Msg, _From, State) ->
handle_cast(update_config, State) when State#state.states =/= undefined ->
exit_changes(State),
- {noreply, init()};
+ {noreply, init_state()};
handle_cast(update_config, _) ->
- {noreply, init()};
+ {noreply, init_state()};
handle_cast(stop, State) ->
{stop, normal, State};
handle_cast(cluster_unstable, State) when State#state.states =/= undefined ->
exit_changes(State),
- {noreply, init()};
+ {noreply, init_state()};
handle_cast(cluster_unstable, _) ->
- {noreply, init()};
+ {noreply, init_state()};
handle_cast(cluster_stable, State) ->
{noreply, start_listening(State)};
handle_cast(_Msg, State) ->
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 10/25: fix tests
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 22656c86b3d5f3ad977bb72c4375874403940158
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Sun Oct 8 16:54:17 2017 +0200
fix tests
---
src/couch_peruser/src/couch_peruser.app.src | 2 +-
src/couch_peruser/test/couch_peruser_test.erl | 17 +++++++++++++++++
2 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/src/couch_peruser/src/couch_peruser.app.src b/src/couch_peruser/src/couch_peruser.app.src
index 42b7b25..9859e03 100644
--- a/src/couch_peruser/src/couch_peruser.app.src
+++ b/src/couch_peruser/src/couch_peruser.app.src
@@ -14,7 +14,7 @@
{description, "couch_peruser - maintains per-user databases in CouchDB"},
{vsn, git},
{registered, []},
- {applications, [kernel, stdlib, config, couch, fabric, couch_replicator, mem3]},
+ {applications, [kernel, stdlib, config, couch, fabric, mem3]},
{mod, {couch_peruser_app, []}},
{env, []},
{modules, [couch_peruser, couch_peruser_app, couch_peruser_sup]}
diff --git a/src/couch_peruser/test/couch_peruser_test.erl b/src/couch_peruser/test/couch_peruser_test.erl
index f7ef8cd..726b2db 100644
--- a/src/couch_peruser/test/couch_peruser_test.erl
+++ b/src/couch_peruser/test/couch_peruser_test.erl
@@ -35,13 +35,22 @@ setup() ->
do_request(put, get_base_url() ++ "/" ++ ?b2l(TestAuthDb)),
do_request(put, get_cluster_base_url() ++ "/" ++ ?b2l(TestAuthDb)),
set_config("couch_httpd_auth", "authentication_db", ?b2l(TestAuthDb)),
+ set_config("couch_peruser", "cluster_quiet_period", "1"),
+ set_config("couch_peruser", "cluster_start_period", "1"),
set_config("couch_peruser", "enable", "true"),
+ set_config("cluster", "n", "1"),
+ set_config("log", "level", "debug"),
+ timer:sleep(6000),
TestAuthDb.
teardown(TestAuthDb) ->
set_config("couch_peruser", "enable", "false"),
set_config("couch_peruser", "delete_dbs", "false"),
set_config("couch_httpd_auth", "authentication_db", "_users"),
+ set_config("couch_peruser", "cluster_quiet_period", "60"),
+ set_config("couch_peruser", "cluster_start_period", "5"),
+ set_config("cluster", "n", "3"),
+ set_config("log", "level", "info"),
do_request(delete, get_cluster_base_url() ++ "/" ++ ?b2l(TestAuthDb)),
do_request(delete, get_base_url() ++ "/" ++ ?b2l(TestAuthDb)),
lists:foreach(fun (DbName) ->
@@ -153,8 +162,10 @@ should_delete_user_db(TestAuthDb) ->
UserDbName = <<"userdb-626172">>,
set_config("couch_peruser", "delete_dbs", "true"),
create_user(TestAuthDb, User),
+ timer:sleep(2000),
?assert(lists:member(UserDbName, all_dbs())),
delete_user(TestAuthDb, User),
+ timer:sleep(2000),
?_assert(not lists:member(UserDbName, all_dbs())).
should_reflect_config_changes(TestAuthDb) ->
@@ -162,20 +173,26 @@ should_reflect_config_changes(TestAuthDb) ->
UserDbName = <<"userdb-62617a">>,
set_config("couch_peruser", "delete_dbs", "true"),
create_user(TestAuthDb, User),
+ timer:sleep(2000),
?assert(lists:member(UserDbName, all_dbs())),
delete_user(TestAuthDb, User),
+ timer:sleep(2000),
?assert(not lists:member(UserDbName, all_dbs())),
create_user(TestAuthDb, User),
+ timer:sleep(2000),
?assert(lists:member(UserDbName, all_dbs())),
set_config("couch_peruser", "delete_dbs", "false"),
delete_user(TestAuthDb, User),
+ timer:sleep(2000),
?assert(lists:member(UserDbName, all_dbs())),
create_user(TestAuthDb, User),
set_config("couch_peruser", "delete_dbs", "true"),
delete_user(TestAuthDb, User),
+ timer:sleep(2000),
?assert(not lists:member(UserDbName, all_dbs())),
set_config("couch_peruser", "enable", "false"),
create_user(TestAuthDb, User),
+ timer:sleep(2000),
?_assert(not lists:member(UserDbName, all_dbs())).
should_add_user_to_db_admins(TestAuthDb) ->
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 13/25: remove leftover code from olde notification system
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 6487e38ce9b389aae8ca0e8f99cd945506380778
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Mon Oct 9 07:56:12 2017 +0200
remove leftover code from olde notification system
---
src/couch_peruser/src/couch_peruser.erl | 8 --------
1 file changed, 8 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index 16305c6..c2ac6ab 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -21,9 +21,6 @@
-export([start_link/0, init/1, handle_call/3, handle_cast/2, handle_info/2,
terminate/2, code_change/3]).
-% cluster state notification callback
--export([notify_cluster_event/2]).
-
-export([init_changes_handler/1, changes_handler/3]).
% mem3_cluster callbacks
@@ -92,11 +89,6 @@ init() ->
}
end.
-% Cluster membership change notification callback
--spec notify_cluster_event(Server :: pid(), Event :: {cluster, any()}) -> ok.
-notify_cluster_event(Server, {cluster, _} = Event) ->
- % couch_log:debug("peruser: received cluster event ~p on node ~p", [Event, node()]),
- gen_server:cast(Server, Event).
-spec start_listening(ClusterState :: #clusterState{}) -> #clusterState{} | ok.
start_listening(#clusterState{states=States}=ClusterState) when length(States) > 0 ->
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 08/25: make sure peruser listeners are only initialised
once per node
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit f0c28835a6a8d4b08de4eaf94b7d07d2d4865e8c
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Sun Oct 8 15:36:31 2017 +0200
make sure peruser listeners are only initialised once per node
---
src/couch_peruser/src/couch_peruser.erl | 59 ++++++++++++++++++++-------------
1 file changed, 36 insertions(+), 23 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index 791431c..e722b7e 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -32,7 +32,14 @@
cluster_unstable/1
]).
--record(state, {parent, db_name, delete_dbs, changes_pid, changes_ref}).
+-record(state, {
+ parent,
+ db_name,
+ delete_dbs,
+ changes_pid,
+ changes_ref
+}).
+
-record(clusterState, {
parent,
db_name,
@@ -47,6 +54,9 @@
-define(DEFAULT_QUIET_PERIOD, 60). % seconds
-define(DEFAULT_START_PERIOD, 5). % seconds
+%%
+%% Please leave in the commented-out couch_log:debug calls, thanks! — Jan
+%%
start_link() ->
gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
@@ -72,8 +82,6 @@ init() ->
{ok, Mem3Cluster} = mem3_cluster:start_link(?MODULE, self(), StartPeriod,
Period),
- couch_log:debug("peruser: registered for cluster event on node ~p", [node()]),
-
#clusterState{
parent = self(),
db_name = DbName,
@@ -86,11 +94,14 @@ init() ->
% Cluster membership change notification callback
-spec notify_cluster_event(pid(), {cluster, any()}) -> ok.
notify_cluster_event(Server, {cluster, _} = Event) ->
- couch_log:debug("peruser: received cluster event ~p on node ~p", [Event, node()]),
+ % couch_log:debug("peruser: received cluster event ~p on node ~p", [Event, node()]),
gen_server:cast(Server, Event).
+start_listening(#clusterState{states=States}=ClusterState) when length(States) > 0 ->
+ % couch_log:debug("peruser: start_listening() already run on node ~p in pid ~p", [node(), self()]),
+ ClusterState;
start_listening(#clusterState{db_name=DbName, delete_dbs=DeleteDbs} = ClusterState) ->
- couch_log:debug("peruser: start_listening() on node ~p", [node()]),
+ % couch_log:debug("peruser: start_listening() on node ~p", [node()]),
try
States = lists:map(fun (A) ->
S = #state{parent = ClusterState#clusterState.parent,
@@ -100,6 +111,7 @@ start_listening(#clusterState{db_name=DbName, delete_dbs=DeleteDbs} = ClusterSta
?MODULE, init_changes_handler, [S], [link, monitor]),
S#state{changes_pid=Pid, changes_ref=Ref}
end, mem3:local_shards(DbName)),
+ % couch_log:debug("peruser: start_listening() States ~p", [States]),
ClusterState#clusterState{states = States, cluster_stable = true}
catch error:database_does_not_exist ->
@@ -108,7 +120,6 @@ start_listening(#clusterState{db_name=DbName, delete_dbs=DeleteDbs} = ClusterSta
end.
init_changes_handler(#state{db_name=DbName} = State) ->
- % leave for debugging
% couch_log:debug("peruser: init_changes_handler() on DbName ~p", [DbName]),
try
{ok, Db} = couch_db:open_int(DbName, [?ADMIN_CTX, sys_db]),
@@ -123,7 +134,6 @@ init_changes_handler(#state{db_name=DbName} = State) ->
changes_handler({change, {Doc}, _Prepend}, _ResType, State=#state{db_name=DbName}) ->
- % leave for debugging
% couch_log:debug("peruser: changes_handler() on DbName/Doc ~p/~p", [DbName, Doc]),
case couch_util:get_value(<<"id">>, Doc) of
@@ -155,28 +165,31 @@ changes_handler({change, {Doc}, _Prepend}, _ResType, State=#state{db_name=DbName
changes_handler(_Event, _ResType, State) ->
State.
+
should_handle_doc(ShardName, DocId) ->
- should_handle_doc_int(ShardName, DocId, is_stable()).
-
-should_handle_doc_int(ShardName, DocId, false) ->
- % when the cluster is unstable, we have already stopped all Listeners
- % the next stable event will restart all listeners and pick up this
- % doc change
- couch_log:debug("peruser: skipping, cluster unstable ~s/~s", [ShardName, DocId]),
- false;
-should_handle_doc_int(ShardName, DocId, true) ->
+ case is_stable() of
+ false ->
+ % when the cluster is unstable, we have already stopped all Listeners
+ % the next stable event will restart all listeners and pick up this
+ % doc change
+ couch_log:debug("peruser: skipping, cluster unstable ~s/~s", [ShardName, DocId]),
+ false;
+ true ->
+ should_handle_doc_int(ShardName, DocId)
+ end.
+
+should_handle_doc_int(ShardName, DocId) ->
DbName = mem3:dbname(ShardName),
Live = [erlang:node() | erlang:nodes()],
Shards = mem3:shards(DbName, DocId),
Nodes = [N || #shard{node=N} <- Shards, lists:member(N, Live)],
case mem3:owner(DbName, DocId, Nodes) of
- ThisNode when ThisNode =:= node() ->
- couch_log:debug("peruser: handling ~s/~s", [DbName, DocId]),
- % do the deed
- true;
- _OtherNode ->
- couch_log:debug("peruser: skipping ~s/~s", [DbName, DocId]),
- false
+ ThisNode when ThisNode =:= node() ->
+ couch_log:debug("peruser: handling ~s/~s", [DbName, DocId]),
+ true; % do the database action
+ _OtherNode ->
+ couch_log:debug("peruser: skipping ~s/~s", [DbName, DocId]),
+ false
end.
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 02/25: Start and stop couch_peruser in the test suite
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit b12683ce7a85d28b868128b74eb06f184047c5d2
Author: Russell Branca <ch...@apache.org>
AuthorDate: Thu Aug 17 19:40:38 2017 +0000
Start and stop couch_peruser in the test suite
---
src/couch_peruser/test/couch_peruser_test.erl | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/couch_peruser/test/couch_peruser_test.erl b/src/couch_peruser/test/couch_peruser_test.erl
index c6fde03..f7ef8cd 100644
--- a/src/couch_peruser/test/couch_peruser_test.erl
+++ b/src/couch_peruser/test/couch_peruser_test.erl
@@ -20,12 +20,14 @@
setup_all() ->
TestCtx = test_util:start_couch([chttpd]),
+ ok = application:start(couch_peruser),
Hashed = couch_passwords:hash_admin_password(?ADMIN_PASSWORD),
ok = config:set("admins", ?ADMIN_USERNAME, ?b2l(Hashed), _Persist=false),
TestCtx.
teardown_all(TestCtx) ->
config:delete("admins", ?ADMIN_USERNAME),
+ ok = application:stop(couch_peruser),
test_util:stop_couch(TestCtx).
setup() ->
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 06/25: move couch_replication_clustering:owner/3 to
mem3.erl
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 93fb6ee9da907c53f44ad6887c6c9a1309f118aa
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Sat Oct 7 23:24:14 2017 +0200
move couch_replication_clustering:owner/3 to mem3.erl
---
src/couch_replicator/src/couch_replicator.erl | 2 +-
src/couch_replicator/src/couch_replicator_clustering.erl | 10 +---------
src/mem3/src/mem3.erl | 8 +++++++-
3 files changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/couch_replicator/src/couch_replicator.erl b/src/couch_replicator/src/couch_replicator.erl
index c67b37d..8b7cd5c 100644
--- a/src/couch_replicator/src/couch_replicator.erl
+++ b/src/couch_replicator/src/couch_replicator.erl
@@ -184,7 +184,7 @@ active_doc(DbName, DocId) ->
Live = [node() | nodes()],
Nodes = lists:usort([N || #shard{node=N} <- Shards,
lists:member(N, Live)]),
- Owner = couch_replicator_clustering:owner(DbName, DocId, Nodes),
+ Owner = mem3:owner(DbName, DocId, Nodes),
case active_doc_rpc(DbName, DocId, [Owner]) of
{ok, DocInfo} ->
{ok, DocInfo};
diff --git a/src/couch_replicator/src/couch_replicator_clustering.erl b/src/couch_replicator/src/couch_replicator_clustering.erl
index ed01465..3d5229b 100644
--- a/src/couch_replicator/src/couch_replicator_clustering.erl
+++ b/src/couch_replicator/src/couch_replicator_clustering.erl
@@ -45,7 +45,6 @@
-export([
owner/2,
- owner/3,
is_stable/0,
link_cluster_event_listener/3
]).
@@ -96,13 +95,6 @@ owner(_DbName, _DocId) ->
node().
-% Direct calculation of node membership. This is the algorithm part. It
-% doesn't read the shard map, just picks owner based on a hash.
--spec owner(binary(), binary(), [node()]) -> node().
-owner(DbName, DocId, Nodes) ->
- hd(mem3_util:rotate_list({DbName, DocId}, lists:usort(Nodes))).
-
-
-spec is_stable() -> true | false.
is_stable() ->
gen_server:call(?MODULE, is_stable).
@@ -200,4 +192,4 @@ owner_int(ShardName, DocId) ->
Live = [node() | nodes()],
Shards = mem3:shards(DbName, DocId),
Nodes = [N || #shard{node=N} <- Shards, lists:member(N, Live)],
- owner(DbName, DocId, Nodes).
+ mem3:owner(DbName, DocId, Nodes).
diff --git a/src/mem3/src/mem3.erl b/src/mem3/src/mem3.erl
index e2cbb2e..047154a 100644
--- a/src/mem3/src/mem3.erl
+++ b/src/mem3/src/mem3.erl
@@ -19,7 +19,7 @@
-export([compare_nodelists/0, compare_shards/1]).
-export([quorum/1, group_by_proximity/1]).
-export([live_shards/2]).
--export([belongs/2]).
+-export([belongs/2, owner/3]).
-export([get_placement/1]).
%% For mem3 use only.
@@ -311,6 +311,12 @@ name(#shard{name=Name}) ->
name(#ordered_shard{name=Name}) ->
Name.
+% Direct calculation of node membership. This is the algorithm part. It
+% doesn't read the shard map, just picks owner based on a hash.
+-spec owner(binary(), binary(), [node()]) -> node().
+owner(DbName, DocId, Nodes) ->
+ hd(mem3_util:rotate_list({DbName, DocId}, lists:usort(Nodes))).
+
-ifdef(TEST).
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 14/25: s/clusterState/state/ && s/state/changes_state/
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 557ab003149972355c2be019b4557a1694a42cc3
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Mon Oct 9 08:08:24 2017 +0200
s/clusterState/state/ && s/state/changes_state/
---
src/couch_peruser/src/couch_peruser.erl | 76 ++++++++++++++++-----------------
1 file changed, 38 insertions(+), 38 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index c2ac6ab..bda74ec 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -29,7 +29,7 @@
cluster_unstable/1
]).
--record(state, {
+-record(changes_state, {
parent :: pid(),
db_name :: binary(),
delete_dbs :: boolean(),
@@ -37,7 +37,7 @@
changes_ref :: reference()
}).
--record(clusterState, {
+-record(state, {
parent :: pid(),
db_name :: binary(),
delete_dbs :: boolean(),
@@ -58,13 +58,13 @@
start_link() ->
gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
--spec init() -> #clusterState{}.
+-spec init() -> #state{}.
init() ->
couch_log:debug("peruser: starting on node ~p in pid ~p", [node(), self()]),
case config:get_boolean("couch_peruser", "enable", false) of
false ->
couch_log:debug("peruser: disabled on node ~p", [node()]),
- #clusterState{};
+ #state{};
true ->
couch_log:debug("peruser: enabled on node ~p", [node()]),
DbName = ?l2b(config:get(
@@ -80,7 +80,7 @@ init() ->
{ok, Mem3Cluster} = mem3_cluster:start_link(?MODULE, self(), StartPeriod,
Period),
- #clusterState{
+ #state{
parent = self(),
db_name = DbName,
delete_dbs = DeleteDbs,
@@ -90,35 +90,35 @@ init() ->
end.
--spec start_listening(ClusterState :: #clusterState{}) -> #clusterState{} | ok.
-start_listening(#clusterState{states=States}=ClusterState) when length(States) > 0 ->
+-spec start_listening(State :: #state{}) -> #state{} | ok.
+start_listening(#state{states=ChangesStates}=State) when length(ChangesStates) > 0 ->
% couch_log:debug("peruser: start_listening() already run on node ~p in pid ~p", [node(), self()]),
- ClusterState;
-start_listening(#clusterState{db_name=DbName, delete_dbs=DeleteDbs} = ClusterState) ->
+ State;
+start_listening(#state{db_name=DbName, delete_dbs=DeleteDbs} = State) ->
% couch_log:debug("peruser: start_listening() on node ~p", [node()]),
try
States = lists:map(fun (A) ->
- S = #state{parent = ClusterState#clusterState.parent,
+ S = #changes_state{parent = State#state.parent,
db_name = A#shard.name,
delete_dbs = DeleteDbs},
{Pid, Ref} = spawn_opt(
?MODULE, init_changes_handler, [S], [link, monitor]),
- S#state{changes_pid=Pid, changes_ref=Ref}
+ S#changes_state{changes_pid=Pid, changes_ref=Ref}
end, mem3:local_shards(DbName)),
% couch_log:debug("peruser: start_listening() States ~p", [States]),
- ClusterState#clusterState{states = States, cluster_stable = true}
+ State#state{states = States, cluster_stable = true}
catch error:database_does_not_exist ->
couch_log:warning("couch_peruser can't proceed as underlying database (~s) is missing, disables itself.", [DbName]),
config:set("couch_peruser", "enable", "false", lists:concat([binary_to_list(DbName), " is missing"]))
end.
--spec init_changes_handler(State :: #state{}) -> ok.
-init_changes_handler(#state{db_name=DbName} = State) ->
+-spec init_changes_handler(ChangesState :: #changes_state{}) -> ok.
+init_changes_handler(#changes_state{db_name=DbName} = ChangesState) ->
% couch_log:debug("peruser: init_changes_handler() on DbName ~p", [DbName]),
try
{ok, Db} = couch_db:open_int(DbName, [?ADMIN_CTX, sys_db]),
- FunAcc = {fun ?MODULE:changes_handler/3, State},
+ FunAcc = {fun ?MODULE:changes_handler/3, ChangesState},
(couch_changes:handle_db_changes(
#changes_args{feed="continuous", timeout=infinity},
{json_req, null},
@@ -128,8 +128,8 @@ init_changes_handler(#state{db_name=DbName} = State) ->
end.
-type db_change() :: {atom(), tuple(), binary()}.
--spec changes_handler(Change :: db_change(), ResultType :: any(), State :: #state{}) -> #state{}.
-changes_handler({change, {Doc}, _Prepend}, _ResType, State=#state{db_name=DbName}) ->
+-spec changes_handler(Change :: db_change(), ResultType :: any(), ChangesState :: #changes_state{}) -> #changes_state{}.
+changes_handler({change, {Doc}, _Prepend}, _ResType, ChangesState=#changes_state{db_name=DbName}) ->
% couch_log:debug("peruser: changes_handler() on DbName/Doc ~p/~p", [DbName, Doc]),
case couch_util:get_value(<<"id">>, Doc) of
@@ -140,26 +140,26 @@ changes_handler({change, {Doc}, _Prepend}, _ResType, State=#state{db_name=DbName
false ->
UserDb = ensure_user_db(User),
ok = ensure_security(User, UserDb, fun add_user/3),
- State;
+ ChangesState;
true ->
- case State#state.delete_dbs of
+ case ChangesState#changes_state.delete_dbs of
true ->
_UserDb = delete_user_db(User),
- State;
+ ChangesState;
false ->
UserDb = user_db_name(User),
ok = ensure_security(User, UserDb, fun remove_user/3),
- State
+ ChangesState
end
end;
false ->
- State
+ ChangesState
end;
_ ->
- State
+ ChangesState
end;
-changes_handler(_Event, _ResType, State) ->
- State.
+changes_handler(_Event, _ResType, ChangesState) ->
+ ChangesState.
-spec should_handle_doc(ShardName :: binary(), DocId::binary()) -> boolean().
should_handle_doc(ShardName, DocId) ->
@@ -279,12 +279,12 @@ user_db_name(User) ->
[string:to_lower(integer_to_list(X, 16)) || <<X>> <= User]),
<<?USERDB_PREFIX,HexUser/binary>>.
--spec exit_changes(ClusterState :: #clusterState{}) -> ok.
-exit_changes(ClusterState) ->
- lists:foreach(fun (State) ->
- demonitor(State#state.changes_ref, [flush]),
- exit(State#state.changes_pid, kill)
- end, ClusterState#clusterState.states).
+-spec exit_changes(State :: #state{}) -> ok.
+exit_changes(State) ->
+ lists:foreach(fun (ChangesState) ->
+ demonitor(State#changes_state.changes_ref, [flush]),
+ exit(ChangesState#changes_state.changes_pid, kill)
+ end, State#state.states).
-spec is_stable() -> true | false.
is_stable() ->
@@ -305,26 +305,26 @@ cluster_stable(Server) ->
Server.
%% gen_server callbacks
--spec init(Options :: list()) -> {ok, #clusterState{}}.
+-spec init(Options :: list()) -> {ok, #state{}}.
init([]) ->
ok = subscribe_for_changes(),
{ok, init()}.
-handle_call(is_stable, _From, #clusterState{cluster_stable = IsStable} = State) ->
+handle_call(is_stable, _From, #state{cluster_stable = IsStable} = State) ->
{reply, IsStable, State};
handle_call(_Msg, _From, State) ->
{reply, error, State}.
-handle_cast(update_config, ClusterState) when ClusterState#clusterState.states =/= undefined ->
- exit_changes(ClusterState),
+handle_cast(update_config, State) when State#state.states =/= undefined ->
+ exit_changes(State),
{noreply, init()};
handle_cast(update_config, _) ->
{noreply, init()};
handle_cast(stop, State) ->
{stop, normal, State};
-handle_cast(cluster_unstable, ClusterState) when ClusterState#clusterState.states =/= undefined ->
- exit_changes(ClusterState),
+handle_cast(cluster_unstable, State) when State#state.states =/= undefined ->
+ exit_changes(State),
{noreply, init()};
handle_cast(cluster_unstable, _) ->
{noreply, init()};
@@ -333,7 +333,7 @@ handle_cast(cluster_stable, State) ->
handle_cast(_Msg, State) ->
{noreply, State}.
-handle_info({'DOWN', Ref, _, _, _Reason}, #state{changes_ref=Ref} = State) ->
+handle_info({'DOWN', Ref, _, _, _Reason}, #changes_state{changes_ref=Ref} = State) ->
{stop, normal, State};
handle_info({config_change, "couch_peruser", _, _, _}, State) ->
handle_cast(update_config, State);
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 20/25: unlink changes listeners before exiting them so we
survive
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 4847c02a743478b39e37f1f6182ab7476860ec31
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Mon Oct 9 21:26:19 2017 +0200
unlink changes listeners before exiting them so we survive
---
src/couch_peruser/src/couch_peruser.erl | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index c5110da..2493387 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -283,6 +283,7 @@ user_db_name(User) ->
exit_changes(State) ->
lists:foreach(fun (ChangesState) ->
demonitor(State#changes_state.changes_ref, [flush]),
+ unlink(ChangesState#changes_state.changes_pid),
exit(ChangesState#changes_state.changes_pid, kill)
end, State#state.states).
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 23/25: fix state
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 96efbfffd4414fb7776542338daf9c92453aca5f
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Tue Oct 10 10:52:09 2017 +0200
fix state
---
src/couch_peruser/src/couch_peruser.erl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index 23def43..8394db5 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -343,7 +343,7 @@ handle_cast(cluster_stable, State) ->
handle_cast(_Msg, State) ->
{noreply, State}.
-handle_info({'DOWN', Ref, _, _, _Reason}, #changes_state{changes_ref=Ref} = State) ->
+handle_info({'DOWN', _Ref, _, _, _Reason}, State) ->
{stop, normal, State};
handle_info({config_change, "couch_peruser", _, _, _}, State) ->
handle_cast(update_config, State);
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 16/25: move function declaration around for internal
consistency
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 3ed4838f3065164cac73d9b95b561dc32a4ea529
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Mon Oct 9 08:12:01 2017 +0200
move function declaration around for internal consistency
---
src/couch_peruser/src/couch_peruser.erl | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index f820fe6..7943a9c 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -290,6 +290,13 @@ exit_changes(State) ->
is_stable() ->
gen_server:call(?MODULE, is_stable).
+-spec subscribe_for_changes() -> ok.
+subscribe_for_changes() ->
+ config:subscribe_for_changes([
+ {"couch_httpd_auth", "authentication_db"},
+ "couch_peruser"
+ ]).
+
% Mem3 cluster callbacks
% TODO: find out what type Server is
@@ -351,13 +358,6 @@ handle_info(restart_config_listener, State) ->
handle_info(_Msg, State) ->
{noreply, State}.
--spec subscribe_for_changes() -> ok.
-subscribe_for_changes() ->
- config:subscribe_for_changes([
- {"couch_httpd_auth", "authentication_db"},
- "couch_peruser"
- ]).
-
terminate(_Reason, _State) ->
%% Everything should be linked or monitored, let nature
%% take its course.
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 12/25: add registered modules
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 90d49349c3f794010a1da06751d2db9c0bf25964
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Mon Oct 9 07:55:27 2017 +0200
add registered modules
---
src/couch_peruser/src/couch_peruser.app.src | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/couch_peruser/src/couch_peruser.app.src b/src/couch_peruser/src/couch_peruser.app.src
index f7c5134..6cfaf44 100644
--- a/src/couch_peruser/src/couch_peruser.app.src
+++ b/src/couch_peruser/src/couch_peruser.app.src
@@ -13,7 +13,7 @@
{application, couch_peruser, [
{description, "couch_peruser - maintains per-user databases in CouchDB"},
{vsn, git},
- {registered, []},
+ {registered, [couch_peruser, couch_peruser_sup]},
{applications, [kernel, stdlib, config, couch, fabric, mem3]},
{mod, {couch_peruser_app, []}},
{env, []}
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 22/25: fix style
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 28b400dcfa664b48f090ed455c7e678e562bc1d2
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Tue Oct 10 10:49:24 2017 +0200
fix style
---
src/couch_peruser/src/couch_peruser.erl | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index f130e13..23def43 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -98,9 +98,11 @@ start_listening(#state{db_name=DbName, delete_dbs=DeleteDbs} = State) ->
% couch_log:debug("peruser: start_listening() on node ~p", [node()]),
try
States = lists:map(fun (A) ->
- S = #changes_state{parent = State#state.parent,
- db_name = A#shard.name,
- delete_dbs = DeleteDbs},
+ S = #changes_state{
+ parent = State#state.parent,
+ db_name = A#shard.name,
+ delete_dbs = DeleteDbs
+ },
{Pid, Ref} = spawn_opt(
?MODULE, init_changes_handler, [S], [link, monitor]),
S#changes_state{changes_pid=Pid, changes_ref=Ref}
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 01/25: Make couch_peruser a proper Erlang app
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 2f49de81c8f85d8ab34483c524e71e092de542c5
Author: Russell Branca <ch...@apache.org>
AuthorDate: Thu Aug 17 17:21:35 2017 +0000
Make couch_peruser a proper Erlang app
---
rel/overlay/etc/default.ini | 1 -
src/couch_peruser/src/couch_peruser.app.src | 5 ++++-
...couch_peruser.app.src => couch_peruser_app.erl} | 22 +++++++++++++------
...couch_peruser.app.src => couch_peruser_sup.erl} | 25 ++++++++++++++++------
4 files changed, 37 insertions(+), 16 deletions(-)
diff --git a/rel/overlay/etc/default.ini b/rel/overlay/etc/default.ini
index 1228535..56f9147 100644
--- a/rel/overlay/etc/default.ini
+++ b/rel/overlay/etc/default.ini
@@ -254,7 +254,6 @@ uuids={couch_uuids, start, []}
auth_cache={couch_auth_cache, start_link, []}
os_daemons={couch_os_daemons, start_link, []}
compaction_daemon={couch_compaction_daemon, start_link, []}
-couch_peruser={couch_peruser, start_link, []}
[mango]
; Set to true to disable the "index all fields" text index, which can lead
diff --git a/src/couch_peruser/src/couch_peruser.app.src b/src/couch_peruser/src/couch_peruser.app.src
index fb6d45b..777446d 100644
--- a/src/couch_peruser/src/couch_peruser.app.src
+++ b/src/couch_peruser/src/couch_peruser.app.src
@@ -14,5 +14,8 @@
{description, "couch_peruser - maintains per-user databases in CouchDB"},
{vsn, git},
{registered, []},
- {applications, [kernel, stdlib, config, couch, fabric]}
+ {applications, [kernel, stdlib, config, couch, fabric]},
+ {mod, {couch_peruser_app, []}},
+ {env, []},
+ {modules, [couch_peruser, couch_peruser_app, couch_peruser_sup]}
]}.
diff --git a/src/couch_peruser/src/couch_peruser.app.src b/src/couch_peruser/src/couch_peruser_app.erl
similarity index 65%
copy from src/couch_peruser/src/couch_peruser.app.src
copy to src/couch_peruser/src/couch_peruser_app.erl
index fb6d45b..770c082 100644
--- a/src/couch_peruser/src/couch_peruser.app.src
+++ b/src/couch_peruser/src/couch_peruser_app.erl
@@ -2,7 +2,7 @@
% use this file except in compliance with the License. You may obtain a copy of
% the License at
%
-% http://www.apache.org/licenses/LICENSE-2.0
+% http://www.apache.org/licenses/LICENSE-2.0
%
% Unless required by applicable law or agreed to in writing, software
% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -10,9 +10,17 @@
% License for the specific language governing permissions and limitations under
% the License.
-{application, couch_peruser, [
- {description, "couch_peruser - maintains per-user databases in CouchDB"},
- {vsn, git},
- {registered, []},
- {applications, [kernel, stdlib, config, couch, fabric]}
-]}.
+-module(couch_peruser_app).
+
+-behaviour(application).
+
+-export([start/2, stop/1]).
+
+
+start(_Type, _StartArgs) ->
+ couch_peruser_sup:start_link().
+
+
+stop(_State) ->
+ ok.
+
diff --git a/src/couch_peruser/src/couch_peruser.app.src b/src/couch_peruser/src/couch_peruser_sup.erl
similarity index 53%
copy from src/couch_peruser/src/couch_peruser.app.src
copy to src/couch_peruser/src/couch_peruser_sup.erl
index fb6d45b..b89a363 100644
--- a/src/couch_peruser/src/couch_peruser.app.src
+++ b/src/couch_peruser/src/couch_peruser_sup.erl
@@ -2,7 +2,7 @@
% use this file except in compliance with the License. You may obtain a copy of
% the License at
%
-% http://www.apache.org/licenses/LICENSE-2.0
+% http://www.apache.org/licenses/LICENSE-2.0
%
% Unless required by applicable law or agreed to in writing, software
% distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -10,9 +10,20 @@
% License for the specific language governing permissions and limitations under
% the License.
-{application, couch_peruser, [
- {description, "couch_peruser - maintains per-user databases in CouchDB"},
- {vsn, git},
- {registered, []},
- {applications, [kernel, stdlib, config, couch, fabric]}
-]}.
+-module(couch_peruser_sup).
+
+-behaviour(supervisor).
+
+-export([start_link/0, init/1]).
+
+%% Helper macro for declaring children of supervisor
+-define(CHILD(I, Type), {I, {I, start_link, []}, permanent, 5000, Type, [I]}).
+
+
+start_link() ->
+ supervisor:start_link({local, ?MODULE}, ?MODULE, []).
+
+
+init([]) ->
+ {ok, { {one_for_one, 5, 10}, [?CHILD(couch_peruser, worker)]}}.
+
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 24/25: whitespace and more state fixes
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch 749-fix-couch_peruser-app-structure
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 570d6eeb0beaac3d5e86b1c70b97fa128eb1dc06
Author: Jan Lehnardt <ja...@apache.org>
AuthorDate: Tue Oct 10 20:57:01 2017 +0200
whitespace and more state fixes
---
src/couch_peruser/src/couch_peruser.erl | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/src/couch_peruser/src/couch_peruser.erl b/src/couch_peruser/src/couch_peruser.erl
index 8394db5..a394659 100644
--- a/src/couch_peruser/src/couch_peruser.erl
+++ b/src/couch_peruser/src/couch_peruser.erl
@@ -74,8 +74,8 @@ init_state() ->
% set up cluster-stable listener
Period = abs(config:get_integer("couch_peruser", "cluster_quiet_period",
?DEFAULT_QUIET_PERIOD)),
- StartPeriod = abs(config:get_integer("couch_peruser", "cluster_start_period",
- ?DEFAULT_START_PERIOD)),
+ StartPeriod = abs(config:get_integer("couch_peruser",
+ "cluster_start_period", ?DEFAULT_START_PERIOD)),
{ok, Mem3Cluster} = mem3_cluster:start_link(?MODULE, self(), StartPeriod,
Period),
@@ -91,7 +91,8 @@ init_state() ->
-spec start_listening(State :: #state{}) -> #state{} | ok.
-start_listening(#state{states=ChangesStates}=State) when length(ChangesStates) > 0 ->
+start_listening(#state{states=ChangesStates}=State)
+ when length(ChangesStates) > 0 ->
% couch_log:debug("peruser: start_listening() already run on node ~p in pid ~p", [node(), self()]),
State;
start_listening(#state{db_name=DbName, delete_dbs=DeleteDbs} = State) ->
@@ -281,8 +282,8 @@ user_db_name(User) ->
[string:to_lower(integer_to_list(X, 16)) || <<X>> <= User]),
<<?USERDB_PREFIX,HexUser/binary>>.
--spec exit_changes(ChangesState :: #changes_state{}) -> ok.
-exit_changes(ChangesState) ->
+-spec exit_changes(State :: #state{}) -> ok.
+exit_changes(State) ->
lists:foreach(fun (ChangesState) ->
demonitor(ChangesState#changes_state.changes_ref, [flush]),
unlink(ChangesState#changes_state.changes_pid),
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.