You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Jan Lehnardt <ja...@apache.org> on 2014/10/26 20:04:06 UTC

Re: Setting up CouchDB 2.0

Hey all,

thanks all for your input! I have a revised proposal that I’d like to
share with you for vetting.

It is a bit simpler than the last one and it includes a more detailed
specification for the _setup endpoint that drives all this.

This proposal retains admin party mode. *If* we decide to remove admin
party, not much has to be changed for this proposal, so ignore that
particular bit.

Best to be read in monospace, here’s a copy: https://gist.github.com/janl/b1b15abec31c41169dae

N. End User Action
 - What happens behind the scenes.


1. Launch CouchDB with `$ couchdb`, or init.d, or any other way, exactly
like it is done in 1.x.x.
 - CouchDB launches and listens on 127.0.0.1:5984

From here on, there are two paths, one is via Fauxton (a) the other is
using a HTTP endpoint (b). Fauxton just uses the HTTP endpoint in (b).
(b) can be used to set up a cluster programmatically.


2.a. Go to Fauxton. There is a “Cluster Setup” tab in the sidebar. Go
to the tab and get presented with a form that asks you to enter an admin
username, admin password and optionally a bind_address and port to bind
to publicly. Submit the form with the [Enable Cluster] button.

 - POST to /_setup with
   {
     "action": "enable_cluster",
     "admin": {
       "user": "username",
       "pass": "password"
     },
     ["bind_address": "xxxx",]
     ["port": yyyy]
   }

 This sets up the admin user on the current node and binds to 0.0.0.0:5984
 or the specified ip:port. Logs admin user into Fauxton automatically.

2.b. POST to /_setup as shown above.

Repeat on all nodes.
 - keep the same username/password everywhere.


3. Pick any one node, for simplicity use the first one, to be the
“setup coordination node”.
 - this is a “master” node that manages the setup and requires all
   other nodes to be able to see it and vice versa. Setup won’t work
   with unavailable nodes (duh). The notion of “master” will be gone
   once the setup is finished. At that point, the system has no
   master node. Ignore I ever said “master”.

a. Go to Fauxton / Cluster Setup, once we have enabled the cluster, the
UI shows an “Add Node” interface with the fields admin, and node:
 - POST to /_setup with
   {
     "action": "add_node",
     "admin": { // should be auto-filled from Fauxton
       "user": "username",
       "pass": "password"
     },
     "node": {
       "host": "hostname",
       ["port": 5984]
     }
   }

b. as in a, but without the Fauxton bits, just POST to /_setup
 - this request will do this:
  - on the “setup coordination node”:
   - check if we have an Erlang Cookie Secret. If not, generate
     a UUID and set the erlang cookie to to that UUID.
     // TBD: persist the cookie, so it survives restarts
   - make a POST request to the node specified in the body above
     using the admin credentials in the body above:
     POST to http://username:password@node_b:5984/_setup with:
     {
       "action": "receive_cookie",
       "cookie": "<secretcookie>",
     }
     // TBD: persist the cookie on node B, so it survives restarts

   - when the request to node B returns, we know the Erlang-level
     inter-cluster communication is enabled and we can start adding
     the node on the CouchDB level. To do that, the “setup
     coordination node” does this to it’s own HTTP endpoint:
     PUT /nodes/node_b:5984 or the same thing with internal APIs.

- Repeat for all nodes.
- Fauxton keeps a list of all set up nodes for users to see.


4.a. When all nodes are added, click the [Finish Cluster Setup] button
in Fauxton.
 - this does POST /_setup
   {
     "action": "finish_setup"
   }

b. Same as in a.

 - this manages the final setup bits, like creating the _users,
   _replicator and _db_updates endpoints and whatever else is needed.
   // TBD: collect what else is needed.


## The Setup Endpoint

This is not a REST-y endpoint, it is a simple state machine operated
by HTTP POST with JSON bodies that have an `action` field.

### State 1: No Cluster Enabled

This is right after starting a node for the first time, and any time
before the cluster is enabled as outlined above.

GET /_setup
{"state": "cluster_disabled"}

POST /_setup {"action":"enable_cluster"...} -> Transition to State 2
POST /_setup {"action":"enable_cluster"...} with empty admin user/pass or invalid host/post or host/port not available -> Error
POST /_setup {"action":"anything_but_enable_cluster"...} -> Error


### State 2: Cluster enabled, admin user set, waiting for nodes to be added.

GET /_setup
{"state":"cluster_enabled","nodes":[]}

POST /_setup {"action":"enable_cluster"...} -> Error
POST /_setup {"action":"add_node"...} -> Stay in State 2, but return "nodes":["node B"}] on GET
POST /_setup {"action":"add_node"...} -> if target node not available, Error
POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
POST /_setup {"action":"finish_cluster"} -> Transition to State 3


### State 3: Cluster set up, all nodes operational

GET /_setup
{"state":"cluster_finished","nodes":["node a", "node b", ...]}

POST /_setup {"action":"enable_cluster"...} -> Error
POST /_setup {"action":"finish_cluster"...} -> Stay in State 3, do nothing
POST /_setup {"action":"add_node"...} -> Error
POST /_setup?i_know_what_i_am_doing=true {"action":"add_node"...} -> Add node, stay in State 3.

// TBD: we need to persist the setup state somewhere.

* * *

So far.

I think this is simpler than the first try. It retains the simplicity
of getting started of 1.x.x. It is fully scriptable via HTTP. It has
the same setup-security properties as 1.x.x. It requires an admin account
for cluster setup, but only on the last possible occasion in the
process. It still fully hides the Erlang cluster / secure cookie setup
from the end-user.

Does this sound sensible to you? Am I missing anything aside from the
TBD bits, for which, if you have ideas, I’d love your input! :)

Are there any deal-breaker flaws in this? What can be made simpler or
more clear?

Note that all the HTTP endpoints and field names are just placeholders,
no need to bikeshed on these just yet :)

I’m looking forward to your feedback!

Best
Jan
--





> On 25 Sep 2014, at 20:06 , Alexander Shorin <kx...@gmail.com> wrote:
> 
> On Thu, Sep 25, 2014 at 7:56 PM, Nick Pavlica <li...@gmail.com> wrote:
>> -The short version:
>> 
>> I would like to propose that CouchDB be developed and maintained separately
>> from a management GUI.”
> 
> That's still would be possible. The main idea of Jan's proposal is to
> simplify the cluster setup process using Fauxton UI and magic /_setup
> helper. But that doesn't means that you won't be able to do the same
> manually from command line interface and using plain old text configs.
> Just a little bit more work to do in this case for you.
> 
>> -The longer rambling version:
>> 
>> I would like to see CouchDB 2.x+ adopt a model that resembles that of Riak,
>> Cassandra, and others, where there is a core server and everything else is
>> optional.  It’s so easy to setup a Cassandra and Riak server or cluster
>> from the command line with just a bit of good documentation.  I really like
>> the fact that they are decoupled from an administrative tool like Futon or
>> Faxuton.  By decoupling the admin GUI’s from the database, it paves the way
>> for others to create new GUI tools, and reduces the effort to release new
>> database versions.  While taking a current build of the Master branch for a
>> spin, I was trying to use Futon only to discover that it wasn’t ready for
>> the changes made in 2.0.  After some discussion on IRC, I learned that it
>> would be replaced with Faxuton.  Once I left Futon behind, and used the
>> command line I was up and running.  In the end, it was much easier to work
>> from the command line than trying to work with an outdated tool that was
>> hurting more than helping.  This is not to say that Futon, and Faxuton
>> aren’t great tools, but they add additional complexity, and development
>> effort outside the core objective of a database.  Having a minimal database
>> core also allows administrators to have a reduced burden when it comes to
>> system administration because there are fewer system dependencies to
>> update, manage, and distribute.  On small systems it isn’t as big deal, but
>> the larger systems become, the harder they are to manage.  Additionally,
>> having to interact with a UI makes it harder to setup a cluster with
>> deployment tools like Chef, etc.  CoreOS, really highlights the need to
>> reduce the administrative burden when managing large systems.  Here is a
>> video that illistrates why/how CoreOS has striped down everthing but what
>> is needed (https://www.youtube.com/watch?v=JeICd9XyXfY).  While not exactly
>> relevant, it does convey the general idea.  It could be called CoreCouch,
>> CouchCore, or … :)
> 
> CouchDB 2.0 has new project layout where each component lives in his
> own repository and roughly plug-able: you can easily turn off
> component you don't need (sure if they're not the core) and Fauxton or
> docs are the such.
> 
> --
> ,,,^..^,,,


Re: Setting up CouchDB 2.0

Posted by Alexander Shorin <kx...@gmail.com>.
Forgot few more:

- How could /_setup helps with admin password change among the all
cluster nodes?
- If I add a new node after "finish_cluster" setup, will it have all
system databases (global_changes, cassim, _users...whatever else)
created?
--
,,,^..^,,,


On Sun, Oct 26, 2014 at 10:31 PM, Alexander Shorin <kx...@gmail.com> wrote:
> Hi Jan,
>
> Suddenly, gists doesn't supports per-line comments. Replied inline
> with questions.
>
> On Sun, Oct 26, 2014 at 10:04 PM, Jan Lehnardt <ja...@apache.org> wrote:
>> N. End User Action
>>  - What happens behind the scenes.
>>
>>
>> 1. Launch CouchDB with `$ couchdb`, or init.d, or any other way, exactly
>> like it is done in 1.x.x.
>>  - CouchDB launches and listens on 127.0.0.1:5984
>>
>> From here on, there are two paths, one is via Fauxton (a) the other is
>> using a HTTP endpoint (b). Fauxton just uses the HTTP endpoint in (b).
>> (b) can be used to set up a cluster programmatically.
>>
>>
>> 2.a. Go to Fauxton. There is a “Cluster Setup” tab in the sidebar. Go
>> to the tab and get presented with a form that asks you to enter an admin
>> username, admin password and optionally a bind_address and port to bind
>> to publicly. Submit the form with the [Enable Cluster] button.
>>
>>  - POST to /_setup with
>>    {
>>      "action": "enable_cluster",
>>      "admin": {
>>        "user": "username",
>>        "pass": "password"
>>      },
>>      ["bind_address": "xxxx",]
>>      ["port": yyyy]
>>    }
>>
>>  This sets up the admin user on the current node and binds to 0.0.0.0:5984
>>  or the specified ip:port. Logs admin user into Fauxton automatically.
>>
>> 2.b. POST to /_setup as shown above.
>>
>> Repeat on all nodes.
>>  - keep the same username/password everywhere.
>>
>
> Question:
> - Is /_setup admin-only resource?
> - If node already has admin-party fixed should it accepts new admin credentials?
> - Any reasons to replace 1-3 PUT requests to /_config with single POST
> one in this case?
>
>> 3. Pick any one node, for simplicity use the first one, to be the
>> “setup coordination node”.
>>  - this is a “master” node that manages the setup and requires all
>>    other nodes to be able to see it and vice versa. Setup won’t work
>>    with unavailable nodes (duh). The notion of “master” will be gone
>>    once the setup is finished. At that point, the system has no
>>    master node. Ignore I ever said “master”.
>>
>> a. Go to Fauxton / Cluster Setup, once we have enabled the cluster, the
>> UI shows an “Add Node” interface with the fields admin, and node:
>>  - POST to /_setup with
>>    {
>>      "action": "add_node",
>>      "admin": { // should be auto-filled from Fauxton
>>        "user": "username",
>>        "pass": "password"
>>      },
>>      "node": {
>>        "host": "hostname",
>>        ["port": 5984]
>>      }
>>    }
>>
>> b. as in a, but without the Fauxton bits, just POST to /_setup
>>  - this request will do this:
>>   - on the “setup coordination node”:
>>    - check if we have an Erlang Cookie Secret. If not, generate
>>      a UUID and set the erlang cookie to to that UUID.
>>      // TBD: persist the cookie, so it survives restarts
>>    - make a POST request to the node specified in the body above
>>      using the admin credentials in the body above:
>>      POST to http://username:password@node_b:5984/_setup with:
>>      {
>>        "action": "receive_cookie",
>>        "cookie": "<secretcookie>",
>>      }
>>      // TBD: persist the cookie on node B, so it survives restarts
>>
>>    - when the request to node B returns, we know the Erlang-level
>>      inter-cluster communication is enabled and we can start adding
>>      the node on the CouchDB level. To do that, the “setup
>>      coordination node” does this to it’s own HTTP endpoint:
>>      PUT /nodes/node_b:5984 or the same thing with internal APIs.
>>
>> - Repeat for all nodes.
>> - Fauxton keeps a list of all set up nodes for users to see.
>
> Question:
> - Since Fauxton already known all the nodes admin credentials and all
> the nodes are bounded to 0.0.0.0 iface (from previous step), will
> Fauxton automate nodes join into the cluster? This is about to skip
> "Repeat on all nodes" step
> - If some of my nodes have different admin credentials, is this the
> blocker error case or should Fauxton ask me for these credentials?
> - Any reasons for replacing regular request to /_nodes with custom
> /_setup? Point about cookie counts.
>
>> 4.a. When all nodes are added, click the [Finish Cluster Setup] button
>> in Fauxton.
>>  - this does POST /_setup
>>    {
>>      "action": "finish_setup"
>>    }
>>
>> b. Same as in a.
>>
>>  - this manages the final setup bits, like creating the _users,
>>    _replicator and _db_updates endpoints and whatever else is needed.
>>    // TBD: collect what else is needed.
>
> This is the only useful thing that /_setup does from my current point
> of view - everything else was just masking standard requests to
> existed API.
>
>> ## The Setup Endpoint
>>
>> This is not a REST-y endpoint, it is a simple state machine operated
>> by HTTP POST with JSON bodies that have an `action` field.
>>
>> ### State 1: No Cluster Enabled
>>
>> This is right after starting a node for the first time, and any time
>> before the cluster is enabled as outlined above.
>>
>> GET /_setup
>> {"state": "cluster_disabled"}
>>
>> POST /_setup {"action":"enable_cluster"...} -> Transition to State 2
>> POST /_setup {"action":"enable_cluster"...} with empty admin user/pass or invalid host/post or host/port not available -> Error
>> POST /_setup {"action":"anything_but_enable_cluster"...} -> Error
>>
>
> If "enable_cluster" only creates/setups admin and bind address, could
> this step be skipped? Because the same actions are possible to do via
> regular config setup.
>
>
>> ### State 2: Cluster enabled, admin user set, waiting for nodes to be added.
>>
>> GET /_setup
>> {"state":"cluster_enabled","nodes":[]}
>>
>> POST /_setup {"action":"enable_cluster"...} -> Error
>> POST /_setup {"action":"add_node"...} -> Stay in State 2, but return "nodes":["node B"}] on GET
>> POST /_setup {"action":"add_node"...} -> if target node not available, Error
>> POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
>> POST /_setup {"action":"finish_cluster"} -> Transition to State 3
>>
>
> Questions:
> - How much nodes required to be added? 1? 2? 3?...
> - How to remove accidentally added node from cluster?
>
>> ### State 3: Cluster set up, all nodes operational
>>
>> GET /_setup
>> {"state":"cluster_finished","nodes":["node a", "node b", ...]}
>>
>> POST /_setup {"action":"enable_cluster"...} -> Error
>> POST /_setup {"action":"finish_cluster"...} -> Stay in State 3, do nothing
>> POST /_setup {"action":"add_node"...} -> Error
>> POST /_setup?i_know_what_i_am_doing=true {"action":"add_node"...} -> Add node, stay in State 3.
>>
>> // TBD: we need to persist the setup state somewhere.
>>
>
> Questions:
> - Why adding a new node after finish_cluster is some specific case to
> mark it with "i_know_what_i_am_doing" parameter?
> - How to enlarge / reduce cluster after his setup or even disband it?
> Or this isn't what /_setup should cares about?
> - What happens with /_setup resource after finish_cluster? Any case
> for it to be useful?
>
>> * * *
>>
>> So far.
>>
>> I think this is simpler than the first try. It retains the simplicity
>> of getting started of 1.x.x. It is fully scriptable via HTTP. It has
>> the same setup-security properties as 1.x.x. It requires an admin account
>> for cluster setup, but only on the last possible occasion in the
>> process. It still fully hides the Erlang cluster / secure cookie setup
>> from the end-user.
>>
>> Does this sound sensible to you? Am I missing anything aside from the
>> TBD bits, for which, if you have ideas, I’d love your input! :)
>>
>> Are there any deal-breaker flaws in this? What can be made simpler or
>> more clear?
>>
>> Note that all the HTTP endpoints and field names are just placeholders,
>> no need to bikeshed on these just yet :)
>>
>> I’m looking forward to your feedback!
>
>
>
> --
> ,,,^..^,,,

Re: Setting up CouchDB 2.0

Posted by Jan Lehnardt <ja...@apache.org>.
> On 26 Oct 2014, at 20:31 , Alexander Shorin <kx...@gmail.com> wrote:
> 
> Hi Jan,
> 
> Suddenly, gists doesn't supports per-line comments. Replied inline
> with questions.

Ah yes, please reply here :)

Best
Jan
--

> 
> On Sun, Oct 26, 2014 at 10:04 PM, Jan Lehnardt <ja...@apache.org> wrote:
>> N. End User Action
>> - What happens behind the scenes.
>> 
>> 
>> 1. Launch CouchDB with `$ couchdb`, or init.d, or any other way, exactly
>> like it is done in 1.x.x.
>> - CouchDB launches and listens on 127.0.0.1:5984
>> 
>> From here on, there are two paths, one is via Fauxton (a) the other is
>> using a HTTP endpoint (b). Fauxton just uses the HTTP endpoint in (b).
>> (b) can be used to set up a cluster programmatically.
>> 
>> 
>> 2.a. Go to Fauxton. There is a “Cluster Setup” tab in the sidebar. Go
>> to the tab and get presented with a form that asks you to enter an admin
>> username, admin password and optionally a bind_address and port to bind
>> to publicly. Submit the form with the [Enable Cluster] button.
>> 
>> - POST to /_setup with
>>   {
>>     "action": "enable_cluster",
>>     "admin": {
>>       "user": "username",
>>       "pass": "password"
>>     },
>>     ["bind_address": "xxxx",]
>>     ["port": yyyy]
>>   }
>> 
>> This sets up the admin user on the current node and binds to 0.0.0.0:5984
>> or the specified ip:port. Logs admin user into Fauxton automatically.
>> 
>> 2.b. POST to /_setup as shown above.
>> 
>> Repeat on all nodes.
>> - keep the same username/password everywhere.
>> 
> 
> Question:
> - Is /_setup admin-only resource?
> - If node already has admin-party fixed should it accepts new admin credentials?
> - Any reasons to replace 1-3 PUT requests to /_config with single POST
> one in this case?
> 
>> 3. Pick any one node, for simplicity use the first one, to be the
>> “setup coordination node”.
>> - this is a “master” node that manages the setup and requires all
>>   other nodes to be able to see it and vice versa. Setup won’t work
>>   with unavailable nodes (duh). The notion of “master” will be gone
>>   once the setup is finished. At that point, the system has no
>>   master node. Ignore I ever said “master”.
>> 
>> a. Go to Fauxton / Cluster Setup, once we have enabled the cluster, the
>> UI shows an “Add Node” interface with the fields admin, and node:
>> - POST to /_setup with
>>   {
>>     "action": "add_node",
>>     "admin": { // should be auto-filled from Fauxton
>>       "user": "username",
>>       "pass": "password"
>>     },
>>     "node": {
>>       "host": "hostname",
>>       ["port": 5984]
>>     }
>>   }
>> 
>> b. as in a, but without the Fauxton bits, just POST to /_setup
>> - this request will do this:
>>  - on the “setup coordination node”:
>>   - check if we have an Erlang Cookie Secret. If not, generate
>>     a UUID and set the erlang cookie to to that UUID.
>>     // TBD: persist the cookie, so it survives restarts
>>   - make a POST request to the node specified in the body above
>>     using the admin credentials in the body above:
>>     POST to http://username:password@node_b:5984/_setup with:
>>     {
>>       "action": "receive_cookie",
>>       "cookie": "<secretcookie>",
>>     }
>>     // TBD: persist the cookie on node B, so it survives restarts
>> 
>>   - when the request to node B returns, we know the Erlang-level
>>     inter-cluster communication is enabled and we can start adding
>>     the node on the CouchDB level. To do that, the “setup
>>     coordination node” does this to it’s own HTTP endpoint:
>>     PUT /nodes/node_b:5984 or the same thing with internal APIs.
>> 
>> - Repeat for all nodes.
>> - Fauxton keeps a list of all set up nodes for users to see.
> 
> Question:
> - Since Fauxton already known all the nodes admin credentials and all
> the nodes are bounded to 0.0.0.0 iface (from previous step), will
> Fauxton automate nodes join into the cluster? This is about to skip
> "Repeat on all nodes" step
> - If some of my nodes have different admin credentials, is this the
> blocker error case or should Fauxton ask me for these credentials?
> - Any reasons for replacing regular request to /_nodes with custom
> /_setup? Point about cookie counts.
> 
>> 4.a. When all nodes are added, click the [Finish Cluster Setup] button
>> in Fauxton.
>> - this does POST /_setup
>>   {
>>     "action": "finish_setup"
>>   }
>> 
>> b. Same as in a.
>> 
>> - this manages the final setup bits, like creating the _users,
>>   _replicator and _db_updates endpoints and whatever else is needed.
>>   // TBD: collect what else is needed.
> 
> This is the only useful thing that /_setup does from my current point
> of view - everything else was just masking standard requests to
> existed API.
> 
>> ## The Setup Endpoint
>> 
>> This is not a REST-y endpoint, it is a simple state machine operated
>> by HTTP POST with JSON bodies that have an `action` field.
>> 
>> ### State 1: No Cluster Enabled
>> 
>> This is right after starting a node for the first time, and any time
>> before the cluster is enabled as outlined above.
>> 
>> GET /_setup
>> {"state": "cluster_disabled"}
>> 
>> POST /_setup {"action":"enable_cluster"...} -> Transition to State 2
>> POST /_setup {"action":"enable_cluster"...} with empty admin user/pass or invalid host/post or host/port not available -> Error
>> POST /_setup {"action":"anything_but_enable_cluster"...} -> Error
>> 
> 
> If "enable_cluster" only creates/setups admin and bind address, could
> this step be skipped? Because the same actions are possible to do via
> regular config setup.
> 
> 
>> ### State 2: Cluster enabled, admin user set, waiting for nodes to be added.
>> 
>> GET /_setup
>> {"state":"cluster_enabled","nodes":[]}
>> 
>> POST /_setup {"action":"enable_cluster"...} -> Error
>> POST /_setup {"action":"add_node"...} -> Stay in State 2, but return "nodes":["node B"}] on GET
>> POST /_setup {"action":"add_node"...} -> if target node not available, Error
>> POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
>> POST /_setup {"action":"finish_cluster"} -> Transition to State 3
>> 
> 
> Questions:
> - How much nodes required to be added? 1? 2? 3?...
> - How to remove accidentally added node from cluster?
> 
>> ### State 3: Cluster set up, all nodes operational
>> 
>> GET /_setup
>> {"state":"cluster_finished","nodes":["node a", "node b", ...]}
>> 
>> POST /_setup {"action":"enable_cluster"...} -> Error
>> POST /_setup {"action":"finish_cluster"...} -> Stay in State 3, do nothing
>> POST /_setup {"action":"add_node"...} -> Error
>> POST /_setup?i_know_what_i_am_doing=true {"action":"add_node"...} -> Add node, stay in State 3.
>> 
>> // TBD: we need to persist the setup state somewhere.
>> 
> 
> Questions:
> - Why adding a new node after finish_cluster is some specific case to
> mark it with "i_know_what_i_am_doing" parameter?
> - How to enlarge / reduce cluster after his setup or even disband it?
> Or this isn't what /_setup should cares about?
> - What happens with /_setup resource after finish_cluster? Any case
> for it to be useful?
> 
>> * * *
>> 
>> So far.
>> 
>> I think this is simpler than the first try. It retains the simplicity
>> of getting started of 1.x.x. It is fully scriptable via HTTP. It has
>> the same setup-security properties as 1.x.x. It requires an admin account
>> for cluster setup, but only on the last possible occasion in the
>> process. It still fully hides the Erlang cluster / secure cookie setup
>> from the end-user.
>> 
>> Does this sound sensible to you? Am I missing anything aside from the
>> TBD bits, for which, if you have ideas, I’d love your input! :)
>> 
>> Are there any deal-breaker flaws in this? What can be made simpler or
>> more clear?
>> 
>> Note that all the HTTP endpoints and field names are just placeholders,
>> no need to bikeshed on these just yet :)
>> 
>> I’m looking forward to your feedback!
> 
> 
> 
> --
> ,,,^..^,,,


Re: Setting up CouchDB 2.0

Posted by Alexander Shorin <kx...@gmail.com>.
Great news, Jan!
--
,,,^..^,,,


On Fri, Nov 7, 2014 at 5:49 PM, Jan Lehnardt <ja...@apache.org> wrote:
> Hey all,
>
> I’ve spent some time coding up the /_cluster_setup endpoint and the basic happy case already works, yay :)
>
> You can follow along here https://git-wip-us.apache.org/repos/asf?p=couchdb-setup.git;a=summary or here https://github.com/janl/couchdb-setup
>
> Any feedback welcome.
>
> Next steps are:
> - collecting feedback
> - test all error conditions
> - solicit help from the Fauxton team to build the frontend bits of this *puppyeyes* <3
>
> Woop.
>
> Best
> Jan
> --
>
>
>
>
>
>
>> On 31 Oct 2014, at 17:31 , Jan Lehnardt <ja...@apache.org> wrote:
>>
>>>
>>> On 31 Oct 2014, at 14:33 , Jan Lehnardt <ja...@apache.org> wrote:
>>>
>>>>
>>>> On 26 Oct 2014, at 22:11 , Alexander Shorin <kx...@gmail.com> wrote:
>>>>
>>>> On Sun, Oct 26, 2014 at 11:25 PM, Jan Lehnardt <ja...@apache.org> wrote:
>>>>> Definitely, sorry for missing that bit.
>>>>>
>>>>
>>>> No worries. Let's clear this up (:
>>>>
>>>>>> - If node already has admin-party fixed should it accepts new admin credentials?
>>>>>
>>>>> Good question, I’d say if an admin already exists, no new admin credentials are needed/
>>>>>
>>>>>
>>>>>> - Any reasons to replace 1-3 PUT requests to /_config with single POST
>>>>>> one in this case?
>>>>>
>>>>> I’m not sure what the 1-3 PUT requests are?
>>>>
>>>> These ones:
>>>> curl -XPUT http://localhost:5984/_config/admin/root -d '"password"' -H
>>>> 'Content-Type: application/json'
>>>> curl -XPUT http://localhost:5984/_config/httpd/bind_address -d
>>>> '"0.0.0.0"' -H 'Content-Type: application/json'
>>>> curl -XPUT http://localhost:5984/_config/httpd/port -d '"5984"' -H
>>>> 'Content-Type: application/json'
>>>>
>>>> The last two are optional as like as the related fields are optional
>>>> for /_setup call.
>>>
>>> Thank, got it. If it is only these settings, then the one /_setup call
>>> with action enable_cluster could be replaced by this. I think having
>>> a deliberate /_setup that duplicates some /_config stuff is actually
>>> helpful.
>>>
>>> Incidentally, how does /_config behave in a cluster? Does it write
>>> back to all nodes’s local.ini file?
>>>
>>>
>>>>
>>>>>> 3. Pick any one node, for simplicity use the first one, to be the
>>>>>>> “setup coordination node”.
>>>>>>> - this is a “master” node that manages the setup and requires all
>>>>>>> other nodes to be able to see it and vice versa. Setup won’t work
>>>>>>> with unavailable nodes (duh). The notion of “master” will be gone
>>>>>>> once the setup is finished. At that point, the system has no
>>>>>>> master node. Ignore I ever said “master”.
>>>>>>>
>>>>>>> a. Go to Fauxton / Cluster Setup, once we have enabled the cluster, the
>>>>>>> UI shows an “Add Node” interface with the fields admin, and node:
>>>>>>> - POST to /_setup with
>>>>>>> {
>>>>>>>  "action": "add_node",
>>>>>>>  "admin": { // should be auto-filled from Fauxton
>>>>>>>    "user": "username",
>>>>>>>    "pass": "password"
>>>>>>>  },
>>>>>>>  "node": {
>>>>>>>    "host": "hostname",
>>>>>>>    ["port": 5984]
>>>>>>>  }
>>>>>>> }
>>>>>>>
>>>>>>> b. as in a, but without the Fauxton bits, just POST to /_setup
>>>>>>> - this request will do this:
>>>>>>> - on the “setup coordination node”:
>>>>>>> - check if we have an Erlang Cookie Secret. If not, generate
>>>>>>>  a UUID and set the erlang cookie to to that UUID.
>>>>>>>  // TBD: persist the cookie, so it survives restarts
>>>>>>> - make a POST request to the node specified in the body above
>>>>>>>  using the admin credentials in the body above:
>>>>>>>  POST to http://username:password@node_b:5984/_setup with:
>>>>>>>  {
>>>>>>>    "action": "receive_cookie",
>>>>>>>    "cookie": "<secretcookie>",
>>>>>>>  }
>>>>>>>  // TBD: persist the cookie on node B, so it survives restarts
>>>>>>>
>>>>>>> - when the request to node B returns, we know the Erlang-level
>>>>>>>  inter-cluster communication is enabled and we can start adding
>>>>>>>  the node on the CouchDB level. To do that, the “setup
>>>>>>>  coordination node” does this to it’s own HTTP endpoint:
>>>>>>>  PUT /nodes/node_b:5984 or the same thing with internal APIs.
>>>>>>>
>>>>>>> - Repeat for all nodes.
>>>>>>> - Fauxton keeps a list of all set up nodes for users to see.
>>>>>>
>>>>>> Question:
>>>>>> - Since Fauxton already known all the nodes admin credentials and all
>>>>>> the nodes are bounded to 0.0.0.0 iface (from previous step), will
>>>>>> Fauxton automate nodes join into the cluster? This is about to skip
>>>>>> "Repeat on all nodes" step
>>>>>
>>>>> How does Fauxton know about the other nodes at this point?
>>>>> (I guess since the Erlang cluster is already set up, it could expose that
>>>>> info to Fauxton in a zeroconf kind of fashion and auto-populate the Fauxton
>>>>> UI with nodes that then can be joined with just a click of a button.)
>>>>
>>>> Oh, right. The "should be auto-filled from Fauxton" comment confused
>>>> me, so I thought about that Fauxton is already aware about nodes list.
>>>> However, zeroconf is desirable, but this is another feature to add. So
>>>> everything is ok here.
>>>
>>> I don’t mean zeroconf specifically, but your comment got me a new idea,
>>> about using Erlang cluster functions to auto-detect nodes, but now that
>>> I think about it, I don’t think it works like I want, so let’s shelve
>>> that part. We can opt into proper zeroconf anytime later.
>>>
>>>>
>>>>>> - If some of my nodes have different admin credentials, is this the
>>>>>> blocker error case or should Fauxton ask me for these credentials?
>>>>>
>>>>> That’s why `add_node` takes a username and password as options, you
>>>>> can set that up if you want. / This could also be made an error case.
>>>>> It should certainly not be recommended.
>>>>>
>>>>
>>>> Right, same confusion by "auto-filled" commentary (:
>>>>
>>>>>> - Any reasons for replacing regular request to /_nodes with custom
>>>>>> /_setup?
>>>>>
>>>>> I don’t know what /_nodes is. Do you mean /nodes? — The reason this isn’t
>>>>> using /nodes at this point is that /nodes already has a special meaning
>>>>> and I didn’t want to complicate the existing logic. In addition, /nodes
>>>>> might have to be adjusted to carry the username and password of the target
>>>>> CouchDB to do the setup (if we otherwise keep the proposed model, happy
>>>>> to see alternatives, though!).
>>>>>
>>>>> If we can reduce all of what I outlined to `PUT /nodes/node_b|c|d`, that
>>>>> would be nice. Fauxton could then offer the setup UI based on whether /nodes
>>>>> has any entries. But I don’t know enough about the semantics and other
>>>>> uses of /nodes, so I haven’t thought about this option too much.
>>>>>
>>>>
>>>> *Bikeshedding alert*: shouldn't system database names be started with
>>>> leading underscore?(:
>>>> Yes, /nodes. Btw, nice idea about storing there node credentials -
>>>> this should help with cluster management in case when admin
>>>> credentials are different everywhere. I only worry that this would
>>>> cause a conflict with cassim logic.
>>>
>>> /nodes is what BigCouch uses and I quickly conferred with rnewson on
>>> IRC. /nodes is only available on :5986, e.g the per-node administration
>>> port. It is not part of the public API on :5984, so I think we can skip
>>> the underscoring there for now. (Incidentally, that is why we can’t do
>>> the setup just from Fauxton, because we need to write to :5986 from
>>> :5984.
>>>
>>>
>>>>>> Point about cookie counts.
>>>>>
>>>>> Not sure I follow.
>>>>
>>>> I'd tried to overcome your reply. I could be wrong, but /nodes doesn't
>>>> knows anything about Erlang cookies and how to work with them while
>>>> your /_setup provides such functionality.  I eventually trying to find
>>>> reasons to avoid having special HTTP resource which will used only
>>>> once for whole cluster lifespan while there exists other which are
>>>> able to made the same job. Setting up cookies makes a sense to have it
>>>> instead.
>>>
>>> Yeah, that’s exactly my point, I think if we can make /nodes (that I now
>>> learn is node-only (see above)) understand Erlang cookie business, we
>>> could avoid /_setup, but since /nodes is only available on :5986 and we
>>> are trying new users to never have to see anything but :5984, we need
>>> /_setup as a “proxy”. In addition, /nodes only “works” after the Erlang
>>> cookie is set up in all nodes, so we are in a chicken and egg situation
>>> here, and I think that’s the final thing that requires us to use /_setup.
>>>
>>>
>>>> 4.a. When all nodes are added, click the [Finish Cluster Setup] button
>>>>>>> in Fauxton.
>>>>>>> - this does POST /_setup
>>>>>>> {
>>>>>>>  "action": "finish_setup"
>>>>>>> }
>>>>>>>
>>>>>>> b. Same as in a.
>>>>>>>
>>>>>>> - this manages the final setup bits, like creating the _users,
>>>>>>> _replicator and _db_updates endpoints and whatever else is needed.
>>>>>>> // TBD: collect what else is needed.
>>>>>>
>>>>>> This is the only useful thing that /_setup does from my current point
>>>>>> of view - everything else was just masking standard requests to
>>>>>> existed API.
>>>>>
>>>>> Which existing API in particular?
>>>>>
>>>>> If you mean that this all can be done over /_config and /nodes, yes totally,
>>>>> but Fauxton on node_a can’t access /_config on node_b. That’s one of the
>>>>> reasons of why I suggest using /_setup, so it can do all this from a single
>>>>> node via Fauxton. The other reason is that it is a dedicated API end-point
>>>>> that hides a lot of complexity instead of having end-users hit a bunch of
>>>>> seemingly random endpoints (although this *could* be hidden in Fauxton maybe,
>>>>> except for the cross domain issue).
>>>>
>>>> Yes, I'm about /_config and /nodes. But why Fauxton cannot access to
>>>> config on node_b? Especially, if it knows the credentials and node_b
>>>> bounded to 0.0.0.0 iface.
>>>
>>> Same origin policy in browsers, see above :)
>>>
>>>
>>>> About API usage complexity: for followers of Fauxton-driven way they
>>>> really don't care about what HTTP requests will be made behind while
>>>> nice spinner loops in their browser. As for fellows of console way
>>>> this isn't an issue too: some small cluster installations are easily
>>>> to made via "seemingly random endpoints" following our guidelines; for
>>>> bigger clusters these processes tends to be automated by provisioning
>>>> tools.
>>>
>>> Yeah, I can get behind the reasoning that complexity can be hidden
>>> behind Fauxton and cli setup can be a bit more complex. I just like
>>> the idea of making this a first-class setup citizen :)
>>>
>>>
>>>>>>> ## The Setup Endpoint
>>>>>>>
>>>>>>> This is not a REST-y endpoint, it is a simple state machine operated
>>>>>>> by HTTP POST with JSON bodies that have an `action` field.
>>>>>>>
>>>>>>> ### State 1: No Cluster Enabled
>>>>>>>
>>>>>>> This is right after starting a node for the first time, and any time
>>>>>>> before the cluster is enabled as outlined above.
>>>>>>>
>>>>>>> GET /_setup
>>>>>>> {"state": "cluster_disabled"}
>>>>>>>
>>>>>>> POST /_setup {"action":"enable_cluster"...} -> Transition to State 2
>>>>>>> POST /_setup {"action":"enable_cluster"...} with empty admin user/pass or invalid host/post or host/port not available -> Error
>>>>>>> POST /_setup {"action":"anything_but_enable_cluster"...} -> Error
>>>>>>>
>>>>>>
>>>>>> If "enable_cluster" only creates/setups admin and bind address, could
>>>>>> this step be skipped? Because the same actions are possible to do via
>>>>>> regular config setup.
>>>>>
>>>>> Yes! It just needs to ensure these things are done. If Fauxton detects
>>>>> they *are* done, it can skip the enable step and show the add_node interface
>>>>> right away.
>>>>
>>>> Good!
>>>>
>>>>>>
>>>>>>
>>>>>>> ### State 2: Cluster enabled, admin user set, waiting for nodes to be added.
>>>>>>>
>>>>>>> GET /_setup
>>>>>>> {"state":"cluster_enabled","nodes":[]}
>>>>>>>
>>>>>>> POST /_setup {"action":"enable_cluster"...} -> Error
>>>>>>> POST /_setup {"action":"add_node"...} -> Stay in State 2, but return "nodes":["node B"}] on GET
>>>>>>> POST /_setup {"action":"add_node"...} -> if target node not available, Error
>>>>>>> POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
>>>>>>> POST /_setup {"action":"finish_cluster"} -> Transition to State 3
>>>>>>>
>>>>>>
>>>>>> Questions:
>>>>>> - How much nodes required to be added? 1? 2? 3?...
>>>>>
>>>>> Doesn’t matter.
>>>>
>>>> Then a case:
>>>> POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
>>>>
>>>> will never happens since there will be always at least one done in
>>>> cluster - those one who runs setup (:
>>>
>>> From Fauxton, yes, but an API user could just call enable_cluster and
>>> then finish_cluster, and they should get an appropriate error :)
>>>
>>>
>>>>>> - How to remove accidentally added node from cluster?
>>>>>
>>>>> Delete from /nodes database. Could be added as a UI element in Fauxton.
>>>>
>>>> That's what I worried about: we adding nodes via /_setup, but have to
>>>> remove them via /nodes. Consistency have to be preserved (:
>>>
>>> Now that I know about /nodes being on :5986 only, /_setup needs a
>>> remove_node action as well. Thanks for flagging that :)
>>>
>>>
>>>
>>>>
>>>>
>>>>>>
>>>>>>> ### State 3: Cluster set up, all nodes operational
>>>>>>>
>>>>>>> GET /_setup
>>>>>>> {"state":"cluster_finished","nodes":["node a", "node b", ...]}
>>>>>>>
>>>>>>> POST /_setup {"action":"enable_cluster"...} -> Error
>>>>>>> POST /_setup {"action":"finish_cluster"...} -> Stay in State 3, do nothing
>>>>>>> POST /_setup {"action":"add_node"...} -> Error
>>>>>>> POST /_setup?i_know_what_i_am_doing=true {"action":"add_node"...} -> Add node, stay in State 3.
>>>>>>>
>>>>>>> // TBD: we need to persist the setup state somewhere.
>>>>>>>
>>>>>>
>>>>>> Questions:
>>>>>> - Why adding a new node after finish_cluster is some specific case to
>>>>>> mark it with "i_know_what_i_am_doing" parameter?
>>>>>
>>>>> Because I think it is not advisable to do this regularly, but someone might
>>>>> want to do this regardless (see next).
>>>>>
>>>>>
>>>>>> - How to enlarge / reduce cluster after his setup or even disband it?
>>>>>
>>>>> Enlarge: see above.
>>>>> Reduce: delete from /nodes
>>>>> Disband: shut down all CouchDB processes :)
>>>>>
>>>>> I don’t know the BigCouch/Cloudant best practices for this. I’ll chalk
>>>>> this down as a “needs input from Cloudant people” :)
>>>>>
>>>>
>>>> You run me into recursion with (see next) and (see above) notes! Nice
>>>> trick, but still unclear how to let your cluster grow - this isn't
>>>> some exceptional case. Reducing (not eventual during network issues)
>>>> is what more rarely could happens. +1 for having more info from
>>>> Cloudant people (:
>>>>
>>>>
>>>>>> Or this isn't what /_setup should cares about?
>>>>>
>>>>> In general, this isn’t really covered by the setup proposal here. I’d like
>>>>> to keep this out of scope for now, but we should have good answers to that
>>>>> going forward.
>>>>
>>>> Agreed.
>>>>
>>>>>> - What happens with /_setup resource after finish_cluster? Any case
>>>>>> for it to be useful?
>>>>>
>>>>> Only for Fauxton to show the correct setup state.
>>>>>
>>>>
>>>> If so then I just figured out some better name for it: /_cluster
>>>> - it setups the cluster as you planned
>>>> - it shows cluster state as you planned
>>>> - it allows to manage cluster nodes in the way which isn't suitable
>>>> for /nodes API (like setting cookies)
>>>> - it becomes useful after cluster setup
>>>> and it could handle other cluster-wide tasks.
>>>>
>>>> What do you think about?
>>>
>>> I’m not attached to any particular name /_cluster or /_cluster_setup
>>> work for me :)
>>>
>>>
>>>>>> - How could /_setup helps with admin password change among the all
>>>>>> cluster nodes?
>>>>>
>>>>> At least on the first run setup, Fauxton can just keep the new password in
>>>>> memory and pre-fill the add_node screens with the same username and password.
>>>>> /_setup then transports it over.
>>>>>
>>>>> For later setups, I don’t know, as we would have the admin to enter the password
>>>>> in plaintext so we can send it. Alternatively, we could use the /_config API to
>>>>> read and send the PBKDF2 hash *waves hands*.
>>>>>
>>>>
>>>> "Fauxton can just keep the new password in memory" opens a door to the
>>>> issue when you accidentally refreshes page / closes tab / loses page
>>>> memory in other way. Not an flaw, just a case to remember about.
>>>>
>>>> As about sending (or replicating) PBKDF2 hash looks good for me.
>>>
>>> Yeah, that sounds like a more robust solution. We’d need something
>>> simliar for the HTTP auth secret.
>>
>> Meh, catch-22 again. Since /_setup is admin-only and we have already set up
>> the target node, we will need the cluster password in plain text added there.
>> We could mitigate some of the problems you outline by storing the password
>> in localStorage until we finsh_cluster (or a timeout, whichever occurs first).
>>
>>>
>>>
>>>>>> - If I add a new node after "finish_cluster" setup, will it have all
>>>>>> system databases (global_changes, cassim, _users...whatever else)
>>>>>> created?
>>>>>
>>>>> That is unspecified at this point. I’d need more input from the Cloudant people
>>>>> on this one. I’m happy to go either way, or make it an option for later joined
>>>>> nodes.
>>>>
>>>> Ok. Let's wait what Cloudant people say. I'm pretty sure they already
>>>> know the solution for all these problems or at least knows their
>>>> specifics.
>>>>
>>>> Thanks a lot, Jan! (:
>>>
>>> Thank you Alex, this helps a lot! :)
>>>
>>> * * *
>>>
>>> I’ll start putting this proposal into the wiki, so we can see it evolve from
>>> now on.
>>>
>>> * * *
>>>
>>> Best
>>> Jan
>>> --
>

Re: Setting up CouchDB 2.0

Posted by Jan Lehnardt <ja...@apache.org>.
Hey all,

I’ve spent some time coding up the /_cluster_setup endpoint and the basic happy case already works, yay :)

You can follow along here https://git-wip-us.apache.org/repos/asf?p=couchdb-setup.git;a=summary or here https://github.com/janl/couchdb-setup

Any feedback welcome.

Next steps are:
- collecting feedback
- test all error conditions
- solicit help from the Fauxton team to build the frontend bits of this *puppyeyes* <3

Woop.

Best
Jan
--






> On 31 Oct 2014, at 17:31 , Jan Lehnardt <ja...@apache.org> wrote:
> 
>> 
>> On 31 Oct 2014, at 14:33 , Jan Lehnardt <ja...@apache.org> wrote:
>> 
>>> 
>>> On 26 Oct 2014, at 22:11 , Alexander Shorin <kx...@gmail.com> wrote:
>>> 
>>> On Sun, Oct 26, 2014 at 11:25 PM, Jan Lehnardt <ja...@apache.org> wrote:
>>>> Definitely, sorry for missing that bit.
>>>> 
>>> 
>>> No worries. Let's clear this up (:
>>> 
>>>>> - If node already has admin-party fixed should it accepts new admin credentials?
>>>> 
>>>> Good question, I’d say if an admin already exists, no new admin credentials are needed/
>>>> 
>>>> 
>>>>> - Any reasons to replace 1-3 PUT requests to /_config with single POST
>>>>> one in this case?
>>>> 
>>>> I’m not sure what the 1-3 PUT requests are?
>>> 
>>> These ones:
>>> curl -XPUT http://localhost:5984/_config/admin/root -d '"password"' -H
>>> 'Content-Type: application/json'
>>> curl -XPUT http://localhost:5984/_config/httpd/bind_address -d
>>> '"0.0.0.0"' -H 'Content-Type: application/json'
>>> curl -XPUT http://localhost:5984/_config/httpd/port -d '"5984"' -H
>>> 'Content-Type: application/json'
>>> 
>>> The last two are optional as like as the related fields are optional
>>> for /_setup call.
>> 
>> Thank, got it. If it is only these settings, then the one /_setup call
>> with action enable_cluster could be replaced by this. I think having
>> a deliberate /_setup that duplicates some /_config stuff is actually
>> helpful.
>> 
>> Incidentally, how does /_config behave in a cluster? Does it write
>> back to all nodes’s local.ini file?
>> 
>> 
>>> 
>>>>> 3. Pick any one node, for simplicity use the first one, to be the
>>>>>> “setup coordination node”.
>>>>>> - this is a “master” node that manages the setup and requires all
>>>>>> other nodes to be able to see it and vice versa. Setup won’t work
>>>>>> with unavailable nodes (duh). The notion of “master” will be gone
>>>>>> once the setup is finished. At that point, the system has no
>>>>>> master node. Ignore I ever said “master”.
>>>>>> 
>>>>>> a. Go to Fauxton / Cluster Setup, once we have enabled the cluster, the
>>>>>> UI shows an “Add Node” interface with the fields admin, and node:
>>>>>> - POST to /_setup with
>>>>>> {
>>>>>>  "action": "add_node",
>>>>>>  "admin": { // should be auto-filled from Fauxton
>>>>>>    "user": "username",
>>>>>>    "pass": "password"
>>>>>>  },
>>>>>>  "node": {
>>>>>>    "host": "hostname",
>>>>>>    ["port": 5984]
>>>>>>  }
>>>>>> }
>>>>>> 
>>>>>> b. as in a, but without the Fauxton bits, just POST to /_setup
>>>>>> - this request will do this:
>>>>>> - on the “setup coordination node”:
>>>>>> - check if we have an Erlang Cookie Secret. If not, generate
>>>>>>  a UUID and set the erlang cookie to to that UUID.
>>>>>>  // TBD: persist the cookie, so it survives restarts
>>>>>> - make a POST request to the node specified in the body above
>>>>>>  using the admin credentials in the body above:
>>>>>>  POST to http://username:password@node_b:5984/_setup with:
>>>>>>  {
>>>>>>    "action": "receive_cookie",
>>>>>>    "cookie": "<secretcookie>",
>>>>>>  }
>>>>>>  // TBD: persist the cookie on node B, so it survives restarts
>>>>>> 
>>>>>> - when the request to node B returns, we know the Erlang-level
>>>>>>  inter-cluster communication is enabled and we can start adding
>>>>>>  the node on the CouchDB level. To do that, the “setup
>>>>>>  coordination node” does this to it’s own HTTP endpoint:
>>>>>>  PUT /nodes/node_b:5984 or the same thing with internal APIs.
>>>>>> 
>>>>>> - Repeat for all nodes.
>>>>>> - Fauxton keeps a list of all set up nodes for users to see.
>>>>> 
>>>>> Question:
>>>>> - Since Fauxton already known all the nodes admin credentials and all
>>>>> the nodes are bounded to 0.0.0.0 iface (from previous step), will
>>>>> Fauxton automate nodes join into the cluster? This is about to skip
>>>>> "Repeat on all nodes" step
>>>> 
>>>> How does Fauxton know about the other nodes at this point?
>>>> (I guess since the Erlang cluster is already set up, it could expose that
>>>> info to Fauxton in a zeroconf kind of fashion and auto-populate the Fauxton
>>>> UI with nodes that then can be joined with just a click of a button.)
>>> 
>>> Oh, right. The "should be auto-filled from Fauxton" comment confused
>>> me, so I thought about that Fauxton is already aware about nodes list.
>>> However, zeroconf is desirable, but this is another feature to add. So
>>> everything is ok here.
>> 
>> I don’t mean zeroconf specifically, but your comment got me a new idea,
>> about using Erlang cluster functions to auto-detect nodes, but now that
>> I think about it, I don’t think it works like I want, so let’s shelve
>> that part. We can opt into proper zeroconf anytime later.
>> 
>>> 
>>>>> - If some of my nodes have different admin credentials, is this the
>>>>> blocker error case or should Fauxton ask me for these credentials?
>>>> 
>>>> That’s why `add_node` takes a username and password as options, you
>>>> can set that up if you want. / This could also be made an error case.
>>>> It should certainly not be recommended.
>>>> 
>>> 
>>> Right, same confusion by "auto-filled" commentary (:
>>> 
>>>>> - Any reasons for replacing regular request to /_nodes with custom
>>>>> /_setup?
>>>> 
>>>> I don’t know what /_nodes is. Do you mean /nodes? — The reason this isn’t
>>>> using /nodes at this point is that /nodes already has a special meaning
>>>> and I didn’t want to complicate the existing logic. In addition, /nodes
>>>> might have to be adjusted to carry the username and password of the target
>>>> CouchDB to do the setup (if we otherwise keep the proposed model, happy
>>>> to see alternatives, though!).
>>>> 
>>>> If we can reduce all of what I outlined to `PUT /nodes/node_b|c|d`, that
>>>> would be nice. Fauxton could then offer the setup UI based on whether /nodes
>>>> has any entries. But I don’t know enough about the semantics and other
>>>> uses of /nodes, so I haven’t thought about this option too much.
>>>> 
>>> 
>>> *Bikeshedding alert*: shouldn't system database names be started with
>>> leading underscore?(:
>>> Yes, /nodes. Btw, nice idea about storing there node credentials -
>>> this should help with cluster management in case when admin
>>> credentials are different everywhere. I only worry that this would
>>> cause a conflict with cassim logic.
>> 
>> /nodes is what BigCouch uses and I quickly conferred with rnewson on
>> IRC. /nodes is only available on :5986, e.g the per-node administration
>> port. It is not part of the public API on :5984, so I think we can skip
>> the underscoring there for now. (Incidentally, that is why we can’t do
>> the setup just from Fauxton, because we need to write to :5986 from
>> :5984.
>> 
>> 
>>>>> Point about cookie counts.
>>>> 
>>>> Not sure I follow.
>>> 
>>> I'd tried to overcome your reply. I could be wrong, but /nodes doesn't
>>> knows anything about Erlang cookies and how to work with them while
>>> your /_setup provides such functionality.  I eventually trying to find
>>> reasons to avoid having special HTTP resource which will used only
>>> once for whole cluster lifespan while there exists other which are
>>> able to made the same job. Setting up cookies makes a sense to have it
>>> instead.
>> 
>> Yeah, that’s exactly my point, I think if we can make /nodes (that I now
>> learn is node-only (see above)) understand Erlang cookie business, we
>> could avoid /_setup, but since /nodes is only available on :5986 and we
>> are trying new users to never have to see anything but :5984, we need
>> /_setup as a “proxy”. In addition, /nodes only “works” after the Erlang
>> cookie is set up in all nodes, so we are in a chicken and egg situation
>> here, and I think that’s the final thing that requires us to use /_setup.
>> 
>> 
>>> 4.a. When all nodes are added, click the [Finish Cluster Setup] button
>>>>>> in Fauxton.
>>>>>> - this does POST /_setup
>>>>>> {
>>>>>>  "action": "finish_setup"
>>>>>> }
>>>>>> 
>>>>>> b. Same as in a.
>>>>>> 
>>>>>> - this manages the final setup bits, like creating the _users,
>>>>>> _replicator and _db_updates endpoints and whatever else is needed.
>>>>>> // TBD: collect what else is needed.
>>>>> 
>>>>> This is the only useful thing that /_setup does from my current point
>>>>> of view - everything else was just masking standard requests to
>>>>> existed API.
>>>> 
>>>> Which existing API in particular?
>>>> 
>>>> If you mean that this all can be done over /_config and /nodes, yes totally,
>>>> but Fauxton on node_a can’t access /_config on node_b. That’s one of the
>>>> reasons of why I suggest using /_setup, so it can do all this from a single
>>>> node via Fauxton. The other reason is that it is a dedicated API end-point
>>>> that hides a lot of complexity instead of having end-users hit a bunch of
>>>> seemingly random endpoints (although this *could* be hidden in Fauxton maybe,
>>>> except for the cross domain issue).
>>> 
>>> Yes, I'm about /_config and /nodes. But why Fauxton cannot access to
>>> config on node_b? Especially, if it knows the credentials and node_b
>>> bounded to 0.0.0.0 iface.
>> 
>> Same origin policy in browsers, see above :)
>> 
>> 
>>> About API usage complexity: for followers of Fauxton-driven way they
>>> really don't care about what HTTP requests will be made behind while
>>> nice spinner loops in their browser. As for fellows of console way
>>> this isn't an issue too: some small cluster installations are easily
>>> to made via "seemingly random endpoints" following our guidelines; for
>>> bigger clusters these processes tends to be automated by provisioning
>>> tools.
>> 
>> Yeah, I can get behind the reasoning that complexity can be hidden
>> behind Fauxton and cli setup can be a bit more complex. I just like
>> the idea of making this a first-class setup citizen :)
>> 
>> 
>>>>>> ## The Setup Endpoint
>>>>>> 
>>>>>> This is not a REST-y endpoint, it is a simple state machine operated
>>>>>> by HTTP POST with JSON bodies that have an `action` field.
>>>>>> 
>>>>>> ### State 1: No Cluster Enabled
>>>>>> 
>>>>>> This is right after starting a node for the first time, and any time
>>>>>> before the cluster is enabled as outlined above.
>>>>>> 
>>>>>> GET /_setup
>>>>>> {"state": "cluster_disabled"}
>>>>>> 
>>>>>> POST /_setup {"action":"enable_cluster"...} -> Transition to State 2
>>>>>> POST /_setup {"action":"enable_cluster"...} with empty admin user/pass or invalid host/post or host/port not available -> Error
>>>>>> POST /_setup {"action":"anything_but_enable_cluster"...} -> Error
>>>>>> 
>>>>> 
>>>>> If "enable_cluster" only creates/setups admin and bind address, could
>>>>> this step be skipped? Because the same actions are possible to do via
>>>>> regular config setup.
>>>> 
>>>> Yes! It just needs to ensure these things are done. If Fauxton detects
>>>> they *are* done, it can skip the enable step and show the add_node interface
>>>> right away.
>>> 
>>> Good!
>>> 
>>>>> 
>>>>> 
>>>>>> ### State 2: Cluster enabled, admin user set, waiting for nodes to be added.
>>>>>> 
>>>>>> GET /_setup
>>>>>> {"state":"cluster_enabled","nodes":[]}
>>>>>> 
>>>>>> POST /_setup {"action":"enable_cluster"...} -> Error
>>>>>> POST /_setup {"action":"add_node"...} -> Stay in State 2, but return "nodes":["node B"}] on GET
>>>>>> POST /_setup {"action":"add_node"...} -> if target node not available, Error
>>>>>> POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
>>>>>> POST /_setup {"action":"finish_cluster"} -> Transition to State 3
>>>>>> 
>>>>> 
>>>>> Questions:
>>>>> - How much nodes required to be added? 1? 2? 3?...
>>>> 
>>>> Doesn’t matter.
>>> 
>>> Then a case:
>>> POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
>>> 
>>> will never happens since there will be always at least one done in
>>> cluster - those one who runs setup (:
>> 
>> From Fauxton, yes, but an API user could just call enable_cluster and
>> then finish_cluster, and they should get an appropriate error :)
>> 
>> 
>>>>> - How to remove accidentally added node from cluster?
>>>> 
>>>> Delete from /nodes database. Could be added as a UI element in Fauxton.
>>> 
>>> That's what I worried about: we adding nodes via /_setup, but have to
>>> remove them via /nodes. Consistency have to be preserved (:
>> 
>> Now that I know about /nodes being on :5986 only, /_setup needs a
>> remove_node action as well. Thanks for flagging that :)
>> 
>> 
>> 
>>> 
>>> 
>>>>> 
>>>>>> ### State 3: Cluster set up, all nodes operational
>>>>>> 
>>>>>> GET /_setup
>>>>>> {"state":"cluster_finished","nodes":["node a", "node b", ...]}
>>>>>> 
>>>>>> POST /_setup {"action":"enable_cluster"...} -> Error
>>>>>> POST /_setup {"action":"finish_cluster"...} -> Stay in State 3, do nothing
>>>>>> POST /_setup {"action":"add_node"...} -> Error
>>>>>> POST /_setup?i_know_what_i_am_doing=true {"action":"add_node"...} -> Add node, stay in State 3.
>>>>>> 
>>>>>> // TBD: we need to persist the setup state somewhere.
>>>>>> 
>>>>> 
>>>>> Questions:
>>>>> - Why adding a new node after finish_cluster is some specific case to
>>>>> mark it with "i_know_what_i_am_doing" parameter?
>>>> 
>>>> Because I think it is not advisable to do this regularly, but someone might
>>>> want to do this regardless (see next).
>>>> 
>>>> 
>>>>> - How to enlarge / reduce cluster after his setup or even disband it?
>>>> 
>>>> Enlarge: see above.
>>>> Reduce: delete from /nodes
>>>> Disband: shut down all CouchDB processes :)
>>>> 
>>>> I don’t know the BigCouch/Cloudant best practices for this. I’ll chalk
>>>> this down as a “needs input from Cloudant people” :)
>>>> 
>>> 
>>> You run me into recursion with (see next) and (see above) notes! Nice
>>> trick, but still unclear how to let your cluster grow - this isn't
>>> some exceptional case. Reducing (not eventual during network issues)
>>> is what more rarely could happens. +1 for having more info from
>>> Cloudant people (:
>>> 
>>> 
>>>>> Or this isn't what /_setup should cares about?
>>>> 
>>>> In general, this isn’t really covered by the setup proposal here. I’d like
>>>> to keep this out of scope for now, but we should have good answers to that
>>>> going forward.
>>> 
>>> Agreed.
>>> 
>>>>> - What happens with /_setup resource after finish_cluster? Any case
>>>>> for it to be useful?
>>>> 
>>>> Only for Fauxton to show the correct setup state.
>>>> 
>>> 
>>> If so then I just figured out some better name for it: /_cluster
>>> - it setups the cluster as you planned
>>> - it shows cluster state as you planned
>>> - it allows to manage cluster nodes in the way which isn't suitable
>>> for /nodes API (like setting cookies)
>>> - it becomes useful after cluster setup
>>> and it could handle other cluster-wide tasks.
>>> 
>>> What do you think about?
>> 
>> I’m not attached to any particular name /_cluster or /_cluster_setup
>> work for me :)
>> 
>> 
>>>>> - How could /_setup helps with admin password change among the all
>>>>> cluster nodes?
>>>> 
>>>> At least on the first run setup, Fauxton can just keep the new password in
>>>> memory and pre-fill the add_node screens with the same username and password.
>>>> /_setup then transports it over.
>>>> 
>>>> For later setups, I don’t know, as we would have the admin to enter the password
>>>> in plaintext so we can send it. Alternatively, we could use the /_config API to
>>>> read and send the PBKDF2 hash *waves hands*.
>>>> 
>>> 
>>> "Fauxton can just keep the new password in memory" opens a door to the
>>> issue when you accidentally refreshes page / closes tab / loses page
>>> memory in other way. Not an flaw, just a case to remember about.
>>> 
>>> As about sending (or replicating) PBKDF2 hash looks good for me.
>> 
>> Yeah, that sounds like a more robust solution. We’d need something
>> simliar for the HTTP auth secret.
> 
> Meh, catch-22 again. Since /_setup is admin-only and we have already set up
> the target node, we will need the cluster password in plain text added there.
> We could mitigate some of the problems you outline by storing the password
> in localStorage until we finsh_cluster (or a timeout, whichever occurs first).
> 
>> 
>> 
>>>>> - If I add a new node after "finish_cluster" setup, will it have all
>>>>> system databases (global_changes, cassim, _users...whatever else)
>>>>> created?
>>>> 
>>>> That is unspecified at this point. I’d need more input from the Cloudant people
>>>> on this one. I’m happy to go either way, or make it an option for later joined
>>>> nodes.
>>> 
>>> Ok. Let's wait what Cloudant people say. I'm pretty sure they already
>>> know the solution for all these problems or at least knows their
>>> specifics.
>>> 
>>> Thanks a lot, Jan! (:
>> 
>> Thank you Alex, this helps a lot! :)
>> 
>> * * *
>> 
>> I’ll start putting this proposal into the wiki, so we can see it evolve from
>> now on.
>> 
>> * * *
>> 
>> Best
>> Jan
>> --


Re: Setting up CouchDB 2.0

Posted by Jan Lehnardt <ja...@apache.org>.
> On 31 Oct 2014, at 14:33 , Jan Lehnardt <ja...@apache.org> wrote:
> 
>> 
>> On 26 Oct 2014, at 22:11 , Alexander Shorin <kx...@gmail.com> wrote:
>> 
>> On Sun, Oct 26, 2014 at 11:25 PM, Jan Lehnardt <ja...@apache.org> wrote:
>>> Definitely, sorry for missing that bit.
>>> 
>> 
>> No worries. Let's clear this up (:
>> 
>>>> - If node already has admin-party fixed should it accepts new admin credentials?
>>> 
>>> Good question, I’d say if an admin already exists, no new admin credentials are needed/
>>> 
>>> 
>>>> - Any reasons to replace 1-3 PUT requests to /_config with single POST
>>>> one in this case?
>>> 
>>> I’m not sure what the 1-3 PUT requests are?
>> 
>> These ones:
>> curl -XPUT http://localhost:5984/_config/admin/root -d '"password"' -H
>> 'Content-Type: application/json'
>> curl -XPUT http://localhost:5984/_config/httpd/bind_address -d
>> '"0.0.0.0"' -H 'Content-Type: application/json'
>> curl -XPUT http://localhost:5984/_config/httpd/port -d '"5984"' -H
>> 'Content-Type: application/json'
>> 
>> The last two are optional as like as the related fields are optional
>> for /_setup call.
> 
> Thank, got it. If it is only these settings, then the one /_setup call
> with action enable_cluster could be replaced by this. I think having
> a deliberate /_setup that duplicates some /_config stuff is actually
> helpful.
> 
> Incidentally, how does /_config behave in a cluster? Does it write
> back to all nodes’s local.ini file?
> 
> 
>> 
>>>> 3. Pick any one node, for simplicity use the first one, to be the
>>>>> “setup coordination node”.
>>>>> - this is a “master” node that manages the setup and requires all
>>>>> other nodes to be able to see it and vice versa. Setup won’t work
>>>>> with unavailable nodes (duh). The notion of “master” will be gone
>>>>> once the setup is finished. At that point, the system has no
>>>>> master node. Ignore I ever said “master”.
>>>>> 
>>>>> a. Go to Fauxton / Cluster Setup, once we have enabled the cluster, the
>>>>> UI shows an “Add Node” interface with the fields admin, and node:
>>>>> - POST to /_setup with
>>>>> {
>>>>>   "action": "add_node",
>>>>>   "admin": { // should be auto-filled from Fauxton
>>>>>     "user": "username",
>>>>>     "pass": "password"
>>>>>   },
>>>>>   "node": {
>>>>>     "host": "hostname",
>>>>>     ["port": 5984]
>>>>>   }
>>>>> }
>>>>> 
>>>>> b. as in a, but without the Fauxton bits, just POST to /_setup
>>>>> - this request will do this:
>>>>> - on the “setup coordination node”:
>>>>> - check if we have an Erlang Cookie Secret. If not, generate
>>>>>   a UUID and set the erlang cookie to to that UUID.
>>>>>   // TBD: persist the cookie, so it survives restarts
>>>>> - make a POST request to the node specified in the body above
>>>>>   using the admin credentials in the body above:
>>>>>   POST to http://username:password@node_b:5984/_setup with:
>>>>>   {
>>>>>     "action": "receive_cookie",
>>>>>     "cookie": "<secretcookie>",
>>>>>   }
>>>>>   // TBD: persist the cookie on node B, so it survives restarts
>>>>> 
>>>>> - when the request to node B returns, we know the Erlang-level
>>>>>   inter-cluster communication is enabled and we can start adding
>>>>>   the node on the CouchDB level. To do that, the “setup
>>>>>   coordination node” does this to it’s own HTTP endpoint:
>>>>>   PUT /nodes/node_b:5984 or the same thing with internal APIs.
>>>>> 
>>>>> - Repeat for all nodes.
>>>>> - Fauxton keeps a list of all set up nodes for users to see.
>>>> 
>>>> Question:
>>>> - Since Fauxton already known all the nodes admin credentials and all
>>>> the nodes are bounded to 0.0.0.0 iface (from previous step), will
>>>> Fauxton automate nodes join into the cluster? This is about to skip
>>>> "Repeat on all nodes" step
>>> 
>>> How does Fauxton know about the other nodes at this point?
>>> (I guess since the Erlang cluster is already set up, it could expose that
>>> info to Fauxton in a zeroconf kind of fashion and auto-populate the Fauxton
>>> UI with nodes that then can be joined with just a click of a button.)
>> 
>> Oh, right. The "should be auto-filled from Fauxton" comment confused
>> me, so I thought about that Fauxton is already aware about nodes list.
>> However, zeroconf is desirable, but this is another feature to add. So
>> everything is ok here.
> 
> I don’t mean zeroconf specifically, but your comment got me a new idea,
> about using Erlang cluster functions to auto-detect nodes, but now that
> I think about it, I don’t think it works like I want, so let’s shelve
> that part. We can opt into proper zeroconf anytime later.
> 
>> 
>>>> - If some of my nodes have different admin credentials, is this the
>>>> blocker error case or should Fauxton ask me for these credentials?
>>> 
>>> That’s why `add_node` takes a username and password as options, you
>>> can set that up if you want. / This could also be made an error case.
>>> It should certainly not be recommended.
>>> 
>> 
>> Right, same confusion by "auto-filled" commentary (:
>> 
>>>> - Any reasons for replacing regular request to /_nodes with custom
>>>> /_setup?
>>> 
>>> I don’t know what /_nodes is. Do you mean /nodes? — The reason this isn’t
>>> using /nodes at this point is that /nodes already has a special meaning
>>> and I didn’t want to complicate the existing logic. In addition, /nodes
>>> might have to be adjusted to carry the username and password of the target
>>> CouchDB to do the setup (if we otherwise keep the proposed model, happy
>>> to see alternatives, though!).
>>> 
>>> If we can reduce all of what I outlined to `PUT /nodes/node_b|c|d`, that
>>> would be nice. Fauxton could then offer the setup UI based on whether /nodes
>>> has any entries. But I don’t know enough about the semantics and other
>>> uses of /nodes, so I haven’t thought about this option too much.
>>> 
>> 
>> *Bikeshedding alert*: shouldn't system database names be started with
>> leading underscore?(:
>> Yes, /nodes. Btw, nice idea about storing there node credentials -
>> this should help with cluster management in case when admin
>> credentials are different everywhere. I only worry that this would
>> cause a conflict with cassim logic.
> 
> /nodes is what BigCouch uses and I quickly conferred with rnewson on
> IRC. /nodes is only available on :5986, e.g the per-node administration
> port. It is not part of the public API on :5984, so I think we can skip
> the underscoring there for now. (Incidentally, that is why we can’t do
> the setup just from Fauxton, because we need to write to :5986 from
> :5984.
> 
> 
>>>> Point about cookie counts.
>>> 
>>> Not sure I follow.
>> 
>> I'd tried to overcome your reply. I could be wrong, but /nodes doesn't
>> knows anything about Erlang cookies and how to work with them while
>> your /_setup provides such functionality.  I eventually trying to find
>> reasons to avoid having special HTTP resource which will used only
>> once for whole cluster lifespan while there exists other which are
>> able to made the same job. Setting up cookies makes a sense to have it
>> instead.
> 
> Yeah, that’s exactly my point, I think if we can make /nodes (that I now
> learn is node-only (see above)) understand Erlang cookie business, we
> could avoid /_setup, but since /nodes is only available on :5986 and we
> are trying new users to never have to see anything but :5984, we need
> /_setup as a “proxy”. In addition, /nodes only “works” after the Erlang
> cookie is set up in all nodes, so we are in a chicken and egg situation
> here, and I think that’s the final thing that requires us to use /_setup.
> 
> 
>> 4.a. When all nodes are added, click the [Finish Cluster Setup] button
>>>>> in Fauxton.
>>>>> - this does POST /_setup
>>>>> {
>>>>>   "action": "finish_setup"
>>>>> }
>>>>> 
>>>>> b. Same as in a.
>>>>> 
>>>>> - this manages the final setup bits, like creating the _users,
>>>>> _replicator and _db_updates endpoints and whatever else is needed.
>>>>> // TBD: collect what else is needed.
>>>> 
>>>> This is the only useful thing that /_setup does from my current point
>>>> of view - everything else was just masking standard requests to
>>>> existed API.
>>> 
>>> Which existing API in particular?
>>> 
>>> If you mean that this all can be done over /_config and /nodes, yes totally,
>>> but Fauxton on node_a can’t access /_config on node_b. That’s one of the
>>> reasons of why I suggest using /_setup, so it can do all this from a single
>>> node via Fauxton. The other reason is that it is a dedicated API end-point
>>> that hides a lot of complexity instead of having end-users hit a bunch of
>>> seemingly random endpoints (although this *could* be hidden in Fauxton maybe,
>>> except for the cross domain issue).
>> 
>> Yes, I'm about /_config and /nodes. But why Fauxton cannot access to
>> config on node_b? Especially, if it knows the credentials and node_b
>> bounded to 0.0.0.0 iface.
> 
> Same origin policy in browsers, see above :)
> 
> 
>> About API usage complexity: for followers of Fauxton-driven way they
>> really don't care about what HTTP requests will be made behind while
>> nice spinner loops in their browser. As for fellows of console way
>> this isn't an issue too: some small cluster installations are easily
>> to made via "seemingly random endpoints" following our guidelines; for
>> bigger clusters these processes tends to be automated by provisioning
>> tools.
> 
> Yeah, I can get behind the reasoning that complexity can be hidden
> behind Fauxton and cli setup can be a bit more complex. I just like
> the idea of making this a first-class setup citizen :)
> 
> 
>>>>> ## The Setup Endpoint
>>>>> 
>>>>> This is not a REST-y endpoint, it is a simple state machine operated
>>>>> by HTTP POST with JSON bodies that have an `action` field.
>>>>> 
>>>>> ### State 1: No Cluster Enabled
>>>>> 
>>>>> This is right after starting a node for the first time, and any time
>>>>> before the cluster is enabled as outlined above.
>>>>> 
>>>>> GET /_setup
>>>>> {"state": "cluster_disabled"}
>>>>> 
>>>>> POST /_setup {"action":"enable_cluster"...} -> Transition to State 2
>>>>> POST /_setup {"action":"enable_cluster"...} with empty admin user/pass or invalid host/post or host/port not available -> Error
>>>>> POST /_setup {"action":"anything_but_enable_cluster"...} -> Error
>>>>> 
>>>> 
>>>> If "enable_cluster" only creates/setups admin and bind address, could
>>>> this step be skipped? Because the same actions are possible to do via
>>>> regular config setup.
>>> 
>>> Yes! It just needs to ensure these things are done. If Fauxton detects
>>> they *are* done, it can skip the enable step and show the add_node interface
>>> right away.
>> 
>> Good!
>> 
>>>> 
>>>> 
>>>>> ### State 2: Cluster enabled, admin user set, waiting for nodes to be added.
>>>>> 
>>>>> GET /_setup
>>>>> {"state":"cluster_enabled","nodes":[]}
>>>>> 
>>>>> POST /_setup {"action":"enable_cluster"...} -> Error
>>>>> POST /_setup {"action":"add_node"...} -> Stay in State 2, but return "nodes":["node B"}] on GET
>>>>> POST /_setup {"action":"add_node"...} -> if target node not available, Error
>>>>> POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
>>>>> POST /_setup {"action":"finish_cluster"} -> Transition to State 3
>>>>> 
>>>> 
>>>> Questions:
>>>> - How much nodes required to be added? 1? 2? 3?...
>>> 
>>> Doesn’t matter.
>> 
>> Then a case:
>> POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
>> 
>> will never happens since there will be always at least one done in
>> cluster - those one who runs setup (:
> 
> From Fauxton, yes, but an API user could just call enable_cluster and
> then finish_cluster, and they should get an appropriate error :)
> 
> 
>>>> - How to remove accidentally added node from cluster?
>>> 
>>> Delete from /nodes database. Could be added as a UI element in Fauxton.
>> 
>> That's what I worried about: we adding nodes via /_setup, but have to
>> remove them via /nodes. Consistency have to be preserved (:
> 
> Now that I know about /nodes being on :5986 only, /_setup needs a
> remove_node action as well. Thanks for flagging that :)
> 
> 
> 
>> 
>> 
>>>> 
>>>>> ### State 3: Cluster set up, all nodes operational
>>>>> 
>>>>> GET /_setup
>>>>> {"state":"cluster_finished","nodes":["node a", "node b", ...]}
>>>>> 
>>>>> POST /_setup {"action":"enable_cluster"...} -> Error
>>>>> POST /_setup {"action":"finish_cluster"...} -> Stay in State 3, do nothing
>>>>> POST /_setup {"action":"add_node"...} -> Error
>>>>> POST /_setup?i_know_what_i_am_doing=true {"action":"add_node"...} -> Add node, stay in State 3.
>>>>> 
>>>>> // TBD: we need to persist the setup state somewhere.
>>>>> 
>>>> 
>>>> Questions:
>>>> - Why adding a new node after finish_cluster is some specific case to
>>>> mark it with "i_know_what_i_am_doing" parameter?
>>> 
>>> Because I think it is not advisable to do this regularly, but someone might
>>> want to do this regardless (see next).
>>> 
>>> 
>>>> - How to enlarge / reduce cluster after his setup or even disband it?
>>> 
>>> Enlarge: see above.
>>> Reduce: delete from /nodes
>>> Disband: shut down all CouchDB processes :)
>>> 
>>> I don’t know the BigCouch/Cloudant best practices for this. I’ll chalk
>>> this down as a “needs input from Cloudant people” :)
>>> 
>> 
>> You run me into recursion with (see next) and (see above) notes! Nice
>> trick, but still unclear how to let your cluster grow - this isn't
>> some exceptional case. Reducing (not eventual during network issues)
>> is what more rarely could happens. +1 for having more info from
>> Cloudant people (:
>> 
>> 
>>>> Or this isn't what /_setup should cares about?
>>> 
>>> In general, this isn’t really covered by the setup proposal here. I’d like
>>> to keep this out of scope for now, but we should have good answers to that
>>> going forward.
>> 
>> Agreed.
>> 
>>>> - What happens with /_setup resource after finish_cluster? Any case
>>>> for it to be useful?
>>> 
>>> Only for Fauxton to show the correct setup state.
>>> 
>> 
>> If so then I just figured out some better name for it: /_cluster
>> - it setups the cluster as you planned
>> - it shows cluster state as you planned
>> - it allows to manage cluster nodes in the way which isn't suitable
>> for /nodes API (like setting cookies)
>> - it becomes useful after cluster setup
>> and it could handle other cluster-wide tasks.
>> 
>> What do you think about?
> 
> I’m not attached to any particular name /_cluster or /_cluster_setup
> work for me :)
> 
> 
>>>> - How could /_setup helps with admin password change among the all
>>>> cluster nodes?
>>> 
>>> At least on the first run setup, Fauxton can just keep the new password in
>>> memory and pre-fill the add_node screens with the same username and password.
>>> /_setup then transports it over.
>>> 
>>> For later setups, I don’t know, as we would have the admin to enter the password
>>> in plaintext so we can send it. Alternatively, we could use the /_config API to
>>> read and send the PBKDF2 hash *waves hands*.
>>> 
>> 
>> "Fauxton can just keep the new password in memory" opens a door to the
>> issue when you accidentally refreshes page / closes tab / loses page
>> memory in other way. Not an flaw, just a case to remember about.
>> 
>> As about sending (or replicating) PBKDF2 hash looks good for me.
> 
> Yeah, that sounds like a more robust solution. We’d need something
> simliar for the HTTP auth secret.

Meh, catch-22 again. Since /_setup is admin-only and we have already set up
the target node, we will need the cluster password in plain text added there.
We could mitigate some of the problems you outline by storing the password
in localStorage until we finsh_cluster (or a timeout, whichever occurs first).

> 
> 
>>>> - If I add a new node after "finish_cluster" setup, will it have all
>>>> system databases (global_changes, cassim, _users...whatever else)
>>>> created?
>>> 
>>> That is unspecified at this point. I’d need more input from the Cloudant people
>>> on this one. I’m happy to go either way, or make it an option for later joined
>>> nodes.
>> 
>> Ok. Let's wait what Cloudant people say. I'm pretty sure they already
>> know the solution for all these problems or at least knows their
>> specifics.
>> 
>> Thanks a lot, Jan! (:
> 
> Thank you Alex, this helps a lot! :)
> 
> * * *
> 
> I’ll start putting this proposal into the wiki, so we can see it evolve from
> now on.
> 
> * * *
> 
> Best
> Jan
> --


Re: Setting up CouchDB 2.0

Posted by Jan Lehnardt <ja...@apache.org>.
> On 26 Oct 2014, at 22:11 , Alexander Shorin <kx...@gmail.com> wrote:
> 
> On Sun, Oct 26, 2014 at 11:25 PM, Jan Lehnardt <ja...@apache.org> wrote:
>> Definitely, sorry for missing that bit.
>> 
> 
> No worries. Let's clear this up (:
> 
>>> - If node already has admin-party fixed should it accepts new admin credentials?
>> 
>> Good question, I’d say if an admin already exists, no new admin credentials are needed/
>> 
>> 
>>> - Any reasons to replace 1-3 PUT requests to /_config with single POST
>>> one in this case?
>> 
>> I’m not sure what the 1-3 PUT requests are?
> 
> These ones:
> curl -XPUT http://localhost:5984/_config/admin/root -d '"password"' -H
> 'Content-Type: application/json'
> curl -XPUT http://localhost:5984/_config/httpd/bind_address -d
> '"0.0.0.0"' -H 'Content-Type: application/json'
> curl -XPUT http://localhost:5984/_config/httpd/port -d '"5984"' -H
> 'Content-Type: application/json'
> 
> The last two are optional as like as the related fields are optional
> for /_setup call.

Thank, got it. If it is only these settings, then the one /_setup call
with action enable_cluster could be replaced by this. I think having
a deliberate /_setup that duplicates some /_config stuff is actually
helpful.

Incidentally, how does /_config behave in a cluster? Does it write
back to all nodes’s local.ini file?


> 
>>> 3. Pick any one node, for simplicity use the first one, to be the
>>>> “setup coordination node”.
>>>> - this is a “master” node that manages the setup and requires all
>>>>  other nodes to be able to see it and vice versa. Setup won’t work
>>>>  with unavailable nodes (duh). The notion of “master” will be gone
>>>>  once the setup is finished. At that point, the system has no
>>>>  master node. Ignore I ever said “master”.
>>>> 
>>>> a. Go to Fauxton / Cluster Setup, once we have enabled the cluster, the
>>>> UI shows an “Add Node” interface with the fields admin, and node:
>>>> - POST to /_setup with
>>>>  {
>>>>    "action": "add_node",
>>>>    "admin": { // should be auto-filled from Fauxton
>>>>      "user": "username",
>>>>      "pass": "password"
>>>>    },
>>>>    "node": {
>>>>      "host": "hostname",
>>>>      ["port": 5984]
>>>>    }
>>>>  }
>>>> 
>>>> b. as in a, but without the Fauxton bits, just POST to /_setup
>>>> - this request will do this:
>>>> - on the “setup coordination node”:
>>>>  - check if we have an Erlang Cookie Secret. If not, generate
>>>>    a UUID and set the erlang cookie to to that UUID.
>>>>    // TBD: persist the cookie, so it survives restarts
>>>>  - make a POST request to the node specified in the body above
>>>>    using the admin credentials in the body above:
>>>>    POST to http://username:password@node_b:5984/_setup with:
>>>>    {
>>>>      "action": "receive_cookie",
>>>>      "cookie": "<secretcookie>",
>>>>    }
>>>>    // TBD: persist the cookie on node B, so it survives restarts
>>>> 
>>>>  - when the request to node B returns, we know the Erlang-level
>>>>    inter-cluster communication is enabled and we can start adding
>>>>    the node on the CouchDB level. To do that, the “setup
>>>>    coordination node” does this to it’s own HTTP endpoint:
>>>>    PUT /nodes/node_b:5984 or the same thing with internal APIs.
>>>> 
>>>> - Repeat for all nodes.
>>>> - Fauxton keeps a list of all set up nodes for users to see.
>>> 
>>> Question:
>>> - Since Fauxton already known all the nodes admin credentials and all
>>> the nodes are bounded to 0.0.0.0 iface (from previous step), will
>>> Fauxton automate nodes join into the cluster? This is about to skip
>>> "Repeat on all nodes" step
>> 
>> How does Fauxton know about the other nodes at this point?
>> (I guess since the Erlang cluster is already set up, it could expose that
>> info to Fauxton in a zeroconf kind of fashion and auto-populate the Fauxton
>> UI with nodes that then can be joined with just a click of a button.)
> 
> Oh, right. The "should be auto-filled from Fauxton" comment confused
> me, so I thought about that Fauxton is already aware about nodes list.
> However, zeroconf is desirable, but this is another feature to add. So
> everything is ok here.

I don’t mean zeroconf specifically, but your comment got me a new idea,
about using Erlang cluster functions to auto-detect nodes, but now that
I think about it, I don’t think it works like I want, so let’s shelve
that part. We can opt into proper zeroconf anytime later.

> 
>>> - If some of my nodes have different admin credentials, is this the
>>> blocker error case or should Fauxton ask me for these credentials?
>> 
>> That’s why `add_node` takes a username and password as options, you
>> can set that up if you want. / This could also be made an error case.
>> It should certainly not be recommended.
>> 
> 
> Right, same confusion by "auto-filled" commentary (:
> 
>>> - Any reasons for replacing regular request to /_nodes with custom
>>> /_setup?
>> 
>> I don’t know what /_nodes is. Do you mean /nodes? — The reason this isn’t
>> using /nodes at this point is that /nodes already has a special meaning
>> and I didn’t want to complicate the existing logic. In addition, /nodes
>> might have to be adjusted to carry the username and password of the target
>> CouchDB to do the setup (if we otherwise keep the proposed model, happy
>> to see alternatives, though!).
>> 
>> If we can reduce all of what I outlined to `PUT /nodes/node_b|c|d`, that
>> would be nice. Fauxton could then offer the setup UI based on whether /nodes
>> has any entries. But I don’t know enough about the semantics and other
>> uses of /nodes, so I haven’t thought about this option too much.
>> 
> 
> *Bikeshedding alert*: shouldn't system database names be started with
> leading underscore?(:
> Yes, /nodes. Btw, nice idea about storing there node credentials -
> this should help with cluster management in case when admin
> credentials are different everywhere. I only worry that this would
> cause a conflict with cassim logic.

/nodes is what BigCouch uses and I quickly conferred with rnewson on
IRC. /nodes is only available on :5986, e.g the per-node administration
port. It is not part of the public API on :5984, so I think we can skip
the underscoring there for now. (Incidentally, that is why we can’t do
the setup just from Fauxton, because we need to write to :5986 from
:5984.


>>> Point about cookie counts.
>> 
>> Not sure I follow.
> 
> I'd tried to overcome your reply. I could be wrong, but /nodes doesn't
> knows anything about Erlang cookies and how to work with them while
> your /_setup provides such functionality.  I eventually trying to find
> reasons to avoid having special HTTP resource which will used only
> once for whole cluster lifespan while there exists other which are
> able to made the same job. Setting up cookies makes a sense to have it
> instead.

Yeah, that’s exactly my point, I think if we can make /nodes (that I now
learn is node-only (see above)) understand Erlang cookie business, we
could avoid /_setup, but since /nodes is only available on :5986 and we
are trying new users to never have to see anything but :5984, we need
/_setup as a “proxy”. In addition, /nodes only “works” after the Erlang
cookie is set up in all nodes, so we are in a chicken and egg situation
here, and I think that’s the final thing that requires us to use /_setup.


> 4.a. When all nodes are added, click the [Finish Cluster Setup] button
>>>> in Fauxton.
>>>> - this does POST /_setup
>>>>  {
>>>>    "action": "finish_setup"
>>>>  }
>>>> 
>>>> b. Same as in a.
>>>> 
>>>> - this manages the final setup bits, like creating the _users,
>>>>  _replicator and _db_updates endpoints and whatever else is needed.
>>>>  // TBD: collect what else is needed.
>>> 
>>> This is the only useful thing that /_setup does from my current point
>>> of view - everything else was just masking standard requests to
>>> existed API.
>> 
>> Which existing API in particular?
>> 
>> If you mean that this all can be done over /_config and /nodes, yes totally,
>> but Fauxton on node_a can’t access /_config on node_b. That’s one of the
>> reasons of why I suggest using /_setup, so it can do all this from a single
>> node via Fauxton. The other reason is that it is a dedicated API end-point
>> that hides a lot of complexity instead of having end-users hit a bunch of
>> seemingly random endpoints (although this *could* be hidden in Fauxton maybe,
>> except for the cross domain issue).
> 
> Yes, I'm about /_config and /nodes. But why Fauxton cannot access to
> config on node_b? Especially, if it knows the credentials and node_b
> bounded to 0.0.0.0 iface.

Same origin policy in browsers, see above :)


> About API usage complexity: for followers of Fauxton-driven way they
> really don't care about what HTTP requests will be made behind while
> nice spinner loops in their browser. As for fellows of console way
> this isn't an issue too: some small cluster installations are easily
> to made via "seemingly random endpoints" following our guidelines; for
> bigger clusters these processes tends to be automated by provisioning
> tools.

Yeah, I can get behind the reasoning that complexity can be hidden
behind Fauxton and cli setup can be a bit more complex. I just like
the idea of making this a first-class setup citizen :)


>>>> ## The Setup Endpoint
>>>> 
>>>> This is not a REST-y endpoint, it is a simple state machine operated
>>>> by HTTP POST with JSON bodies that have an `action` field.
>>>> 
>>>> ### State 1: No Cluster Enabled
>>>> 
>>>> This is right after starting a node for the first time, and any time
>>>> before the cluster is enabled as outlined above.
>>>> 
>>>> GET /_setup
>>>> {"state": "cluster_disabled"}
>>>> 
>>>> POST /_setup {"action":"enable_cluster"...} -> Transition to State 2
>>>> POST /_setup {"action":"enable_cluster"...} with empty admin user/pass or invalid host/post or host/port not available -> Error
>>>> POST /_setup {"action":"anything_but_enable_cluster"...} -> Error
>>>> 
>>> 
>>> If "enable_cluster" only creates/setups admin and bind address, could
>>> this step be skipped? Because the same actions are possible to do via
>>> regular config setup.
>> 
>> Yes! It just needs to ensure these things are done. If Fauxton detects
>> they *are* done, it can skip the enable step and show the add_node interface
>> right away.
> 
> Good!
> 
>>> 
>>> 
>>>> ### State 2: Cluster enabled, admin user set, waiting for nodes to be added.
>>>> 
>>>> GET /_setup
>>>> {"state":"cluster_enabled","nodes":[]}
>>>> 
>>>> POST /_setup {"action":"enable_cluster"...} -> Error
>>>> POST /_setup {"action":"add_node"...} -> Stay in State 2, but return "nodes":["node B"}] on GET
>>>> POST /_setup {"action":"add_node"...} -> if target node not available, Error
>>>> POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
>>>> POST /_setup {"action":"finish_cluster"} -> Transition to State 3
>>>> 
>>> 
>>> Questions:
>>> - How much nodes required to be added? 1? 2? 3?...
>> 
>> Doesn’t matter.
> 
> Then a case:
> POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
> 
> will never happens since there will be always at least one done in
> cluster - those one who runs setup (:

From Fauxton, yes, but an API user could just call enable_cluster and
then finish_cluster, and they should get an appropriate error :)


>>> - How to remove accidentally added node from cluster?
>> 
>> Delete from /nodes database. Could be added as a UI element in Fauxton.
> 
> That's what I worried about: we adding nodes via /_setup, but have to
> remove them via /nodes. Consistency have to be preserved (:

Now that I know about /nodes being on :5986 only, /_setup needs a
remove_node action as well. Thanks for flagging that :)



> 
> 
>>> 
>>>> ### State 3: Cluster set up, all nodes operational
>>>> 
>>>> GET /_setup
>>>> {"state":"cluster_finished","nodes":["node a", "node b", ...]}
>>>> 
>>>> POST /_setup {"action":"enable_cluster"...} -> Error
>>>> POST /_setup {"action":"finish_cluster"...} -> Stay in State 3, do nothing
>>>> POST /_setup {"action":"add_node"...} -> Error
>>>> POST /_setup?i_know_what_i_am_doing=true {"action":"add_node"...} -> Add node, stay in State 3.
>>>> 
>>>> // TBD: we need to persist the setup state somewhere.
>>>> 
>>> 
>>> Questions:
>>> - Why adding a new node after finish_cluster is some specific case to
>>> mark it with "i_know_what_i_am_doing" parameter?
>> 
>> Because I think it is not advisable to do this regularly, but someone might
>> want to do this regardless (see next).
>> 
>> 
>>> - How to enlarge / reduce cluster after his setup or even disband it?
>> 
>> Enlarge: see above.
>> Reduce: delete from /nodes
>> Disband: shut down all CouchDB processes :)
>> 
>> I don’t know the BigCouch/Cloudant best practices for this. I’ll chalk
>> this down as a “needs input from Cloudant people” :)
>> 
> 
> You run me into recursion with (see next) and (see above) notes! Nice
> trick, but still unclear how to let your cluster grow - this isn't
> some exceptional case. Reducing (not eventual during network issues)
> is what more rarely could happens. +1 for having more info from
> Cloudant people (:
> 
> 
>>> Or this isn't what /_setup should cares about?
>> 
>> In general, this isn’t really covered by the setup proposal here. I’d like
>> to keep this out of scope for now, but we should have good answers to that
>> going forward.
> 
> Agreed.
> 
>>> - What happens with /_setup resource after finish_cluster? Any case
>>> for it to be useful?
>> 
>> Only for Fauxton to show the correct setup state.
>> 
> 
> If so then I just figured out some better name for it: /_cluster
> - it setups the cluster as you planned
> - it shows cluster state as you planned
> - it allows to manage cluster nodes in the way which isn't suitable
> for /nodes API (like setting cookies)
> - it becomes useful after cluster setup
> and it could handle other cluster-wide tasks.
> 
> What do you think about?

I’m not attached to any particular name /_cluster or /_cluster_setup
work for me :)


>>> - How could /_setup helps with admin password change among the all
>>> cluster nodes?
>> 
>> At least on the first run setup, Fauxton can just keep the new password in
>> memory and pre-fill the add_node screens with the same username and password.
>> /_setup then transports it over.
>> 
>> For later setups, I don’t know, as we would have the admin to enter the password
>> in plaintext so we can send it. Alternatively, we could use the /_config API to
>> read and send the PBKDF2 hash *waves hands*.
>> 
> 
> "Fauxton can just keep the new password in memory" opens a door to the
> issue when you accidentally refreshes page / closes tab / loses page
> memory in other way. Not an flaw, just a case to remember about.
> 
> As about sending (or replicating) PBKDF2 hash looks good for me.

Yeah, that sounds like a more robust solution. We’d need something
simliar for the HTTP auth secret.


>>> - If I add a new node after "finish_cluster" setup, will it have all
>>> system databases (global_changes, cassim, _users...whatever else)
>>> created?
>> 
>> That is unspecified at this point. I’d need more input from the Cloudant people
>> on this one. I’m happy to go either way, or make it an option for later joined
>> nodes.
> 
> Ok. Let's wait what Cloudant people say. I'm pretty sure they already
> know the solution for all these problems or at least knows their
> specifics.
> 
> Thanks a lot, Jan! (:

Thank you Alex, this helps a lot! :)

* * *

I’ll start putting this proposal into the wiki, so we can see it evolve from
now on.

* * *

Best
Jan
--


Re: Setting up CouchDB 2.0

Posted by Alexander Shorin <kx...@gmail.com>.
On Sun, Oct 26, 2014 at 11:25 PM, Jan Lehnardt <ja...@apache.org> wrote:
> Definitely, sorry for missing that bit.
>

No worries. Let's clear this up (:

>> - If node already has admin-party fixed should it accepts new admin credentials?
>
> Good question, I’d say if an admin already exists, no new admin credentials are needed/
>
>
>> - Any reasons to replace 1-3 PUT requests to /_config with single POST
>> one in this case?
>
> I’m not sure what the 1-3 PUT requests are?

These ones:
curl -XPUT http://localhost:5984/_config/admin/root -d '"password"' -H
'Content-Type: application/json'
curl -XPUT http://localhost:5984/_config/httpd/bind_address -d
'"0.0.0.0"' -H 'Content-Type: application/json'
curl -XPUT http://localhost:5984/_config/httpd/port -d '"5984"' -H
'Content-Type: application/json'

The last two are optional as like as the related fields are optional
for /_setup call.

>> 3. Pick any one node, for simplicity use the first one, to be the
>>> “setup coordination node”.
>>> - this is a “master” node that manages the setup and requires all
>>>   other nodes to be able to see it and vice versa. Setup won’t work
>>>   with unavailable nodes (duh). The notion of “master” will be gone
>>>   once the setup is finished. At that point, the system has no
>>>   master node. Ignore I ever said “master”.
>>>
>>> a. Go to Fauxton / Cluster Setup, once we have enabled the cluster, the
>>> UI shows an “Add Node” interface with the fields admin, and node:
>>> - POST to /_setup with
>>>   {
>>>     "action": "add_node",
>>>     "admin": { // should be auto-filled from Fauxton
>>>       "user": "username",
>>>       "pass": "password"
>>>     },
>>>     "node": {
>>>       "host": "hostname",
>>>       ["port": 5984]
>>>     }
>>>   }
>>>
>>> b. as in a, but without the Fauxton bits, just POST to /_setup
>>> - this request will do this:
>>>  - on the “setup coordination node”:
>>>   - check if we have an Erlang Cookie Secret. If not, generate
>>>     a UUID and set the erlang cookie to to that UUID.
>>>     // TBD: persist the cookie, so it survives restarts
>>>   - make a POST request to the node specified in the body above
>>>     using the admin credentials in the body above:
>>>     POST to http://username:password@node_b:5984/_setup with:
>>>     {
>>>       "action": "receive_cookie",
>>>       "cookie": "<secretcookie>",
>>>     }
>>>     // TBD: persist the cookie on node B, so it survives restarts
>>>
>>>   - when the request to node B returns, we know the Erlang-level
>>>     inter-cluster communication is enabled and we can start adding
>>>     the node on the CouchDB level. To do that, the “setup
>>>     coordination node” does this to it’s own HTTP endpoint:
>>>     PUT /nodes/node_b:5984 or the same thing with internal APIs.
>>>
>>> - Repeat for all nodes.
>>> - Fauxton keeps a list of all set up nodes for users to see.
>>
>> Question:
>> - Since Fauxton already known all the nodes admin credentials and all
>> the nodes are bounded to 0.0.0.0 iface (from previous step), will
>> Fauxton automate nodes join into the cluster? This is about to skip
>> "Repeat on all nodes" step
>
> How does Fauxton know about the other nodes at this point?
> (I guess since the Erlang cluster is already set up, it could expose that
> info to Fauxton in a zeroconf kind of fashion and auto-populate the Fauxton
> UI with nodes that then can be joined with just a click of a button.)

Oh, right. The "should be auto-filled from Fauxton" comment confused
me, so I thought about that Fauxton is already aware about nodes list.
However, zeroconf is desirable, but this is another feature to add. So
everything is ok here.

>> - If some of my nodes have different admin credentials, is this the
>> blocker error case or should Fauxton ask me for these credentials?
>
> That’s why `add_node` takes a username and password as options, you
> can set that up if you want. / This could also be made an error case.
> It should certainly not be recommended.
>

Right, same confusion by "auto-filled" commentary (:

>> - Any reasons for replacing regular request to /_nodes with custom
>> /_setup?
>
> I don’t know what /_nodes is. Do you mean /nodes? — The reason this isn’t
> using /nodes at this point is that /nodes already has a special meaning
> and I didn’t want to complicate the existing logic. In addition, /nodes
> might have to be adjusted to carry the username and password of the target
> CouchDB to do the setup (if we otherwise keep the proposed model, happy
> to see alternatives, though!).
>
> If we can reduce all of what I outlined to `PUT /nodes/node_b|c|d`, that
> would be nice. Fauxton could then offer the setup UI based on whether /nodes
> has any entries. But I don’t know enough about the semantics and other
> uses of /nodes, so I haven’t thought about this option too much.
>

*Bikeshedding alert*: shouldn't system database names be started with
leading underscore?(:
Yes, /nodes. Btw, nice idea about storing there node credentials -
this should help with cluster management in case when admin
credentials are different everywhere. I only worry that this would
cause a conflict with cassim logic.

>> Point about cookie counts.
>
> Not sure I follow.

I'd tried to overcome your reply. I could be wrong, but /nodes doesn't
knows anything about Erlang cookies and how to work with them while
your /_setup provides such functionality.  I eventually trying to find
reasons to avoid having special HTTP resource which will used only
once for whole cluster lifespan while there exists other which are
able to made the same job. Setting up cookies makes a sense to have it
instead.

>> 4.a. When all nodes are added, click the [Finish Cluster Setup] button
>>> in Fauxton.
>>> - this does POST /_setup
>>>   {
>>>     "action": "finish_setup"
>>>   }
>>>
>>> b. Same as in a.
>>>
>>> - this manages the final setup bits, like creating the _users,
>>>   _replicator and _db_updates endpoints and whatever else is needed.
>>>   // TBD: collect what else is needed.
>>
>> This is the only useful thing that /_setup does from my current point
>> of view - everything else was just masking standard requests to
>> existed API.
>
> Which existing API in particular?
>
> If you mean that this all can be done over /_config and /nodes, yes totally,
> but Fauxton on node_a can’t access /_config on node_b. That’s one of the
> reasons of why I suggest using /_setup, so it can do all this from a single
> node via Fauxton. The other reason is that it is a dedicated API end-point
> that hides a lot of complexity instead of having end-users hit a bunch of
> seemingly random endpoints (although this *could* be hidden in Fauxton maybe,
> except for the cross domain issue).

Yes, I'm about /_config and /nodes. But why Fauxton cannot access to
config on node_b? Especially, if it knows the credentials and node_b
bounded to 0.0.0.0 iface.

About API usage complexity: for followers of Fauxton-driven way they
really don't care about what HTTP requests will be made behind while
nice spinner loops in their browser. As for fellows of console way
this isn't an issue too: some small cluster installations are easily
to made via "seemingly random endpoints" following our guidelines; for
bigger clusters these processes tends to be automated by provisioning
tools.



>>
>>> ## The Setup Endpoint
>>>
>>> This is not a REST-y endpoint, it is a simple state machine operated
>>> by HTTP POST with JSON bodies that have an `action` field.
>>>
>>> ### State 1: No Cluster Enabled
>>>
>>> This is right after starting a node for the first time, and any time
>>> before the cluster is enabled as outlined above.
>>>
>>> GET /_setup
>>> {"state": "cluster_disabled"}
>>>
>>> POST /_setup {"action":"enable_cluster"...} -> Transition to State 2
>>> POST /_setup {"action":"enable_cluster"...} with empty admin user/pass or invalid host/post or host/port not available -> Error
>>> POST /_setup {"action":"anything_but_enable_cluster"...} -> Error
>>>
>>
>> If "enable_cluster" only creates/setups admin and bind address, could
>> this step be skipped? Because the same actions are possible to do via
>> regular config setup.
>
> Yes! It just needs to ensure these things are done. If Fauxton detects
> they *are* done, it can skip the enable step and show the add_node interface
> right away.

Good!

>>
>>
>>> ### State 2: Cluster enabled, admin user set, waiting for nodes to be added.
>>>
>>> GET /_setup
>>> {"state":"cluster_enabled","nodes":[]}
>>>
>>> POST /_setup {"action":"enable_cluster"...} -> Error
>>> POST /_setup {"action":"add_node"...} -> Stay in State 2, but return "nodes":["node B"}] on GET
>>> POST /_setup {"action":"add_node"...} -> if target node not available, Error
>>> POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
>>> POST /_setup {"action":"finish_cluster"} -> Transition to State 3
>>>
>>
>> Questions:
>> - How much nodes required to be added? 1? 2? 3?...
>
> Doesn’t matter.

Then a case:
POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error

will never happens since there will be always at least one done in
cluster - those one who runs setup (:

>
>> - How to remove accidentally added node from cluster?
>
> Delete from /nodes database. Could be added as a UI element in Fauxton.

That's what I worried about: we adding nodes via /_setup, but have to
remove them via /nodes. Consistency have to be preserved (:


>>
>>> ### State 3: Cluster set up, all nodes operational
>>>
>>> GET /_setup
>>> {"state":"cluster_finished","nodes":["node a", "node b", ...]}
>>>
>>> POST /_setup {"action":"enable_cluster"...} -> Error
>>> POST /_setup {"action":"finish_cluster"...} -> Stay in State 3, do nothing
>>> POST /_setup {"action":"add_node"...} -> Error
>>> POST /_setup?i_know_what_i_am_doing=true {"action":"add_node"...} -> Add node, stay in State 3.
>>>
>>> // TBD: we need to persist the setup state somewhere.
>>>
>>
>> Questions:
>> - Why adding a new node after finish_cluster is some specific case to
>> mark it with "i_know_what_i_am_doing" parameter?
>
> Because I think it is not advisable to do this regularly, but someone might
> want to do this regardless (see next).
>
>
>> - How to enlarge / reduce cluster after his setup or even disband it?
>
> Enlarge: see above.
> Reduce: delete from /nodes
> Disband: shut down all CouchDB processes :)
>
> I don’t know the BigCouch/Cloudant best practices for this. I’ll chalk
> this down as a “needs input from Cloudant people” :)
>

You run me into recursion with (see next) and (see above) notes! Nice
trick, but still unclear how to let your cluster grow - this isn't
some exceptional case. Reducing (not eventual during network issues)
is what more rarely could happens. +1 for having more info from
Cloudant people (:


>> Or this isn't what /_setup should cares about?
>
> In general, this isn’t really covered by the setup proposal here. I’d like
> to keep this out of scope for now, but we should have good answers to that
> going forward.

Agreed.

>> - What happens with /_setup resource after finish_cluster? Any case
>> for it to be useful?
>
> Only for Fauxton to show the correct setup state.
>

If so then I just figured out some better name for it: /_cluster
- it setups the cluster as you planned
- it shows cluster state as you planned
- it allows to manage cluster nodes in the way which isn't suitable
for /nodes API (like setting cookies)
- it becomes useful after cluster setup
and it could handle other cluster-wide tasks.

What do you think about?

>> - How could /_setup helps with admin password change among the all
>> cluster nodes?
>
> At least on the first run setup, Fauxton can just keep the new password in
> memory and pre-fill the add_node screens with the same username and password.
> /_setup then transports it over.
>
> For later setups, I don’t know, as we would have the admin to enter the password
> in plaintext so we can send it. Alternatively, we could use the /_config API to
> read and send the PBKDF2 hash *waves hands*.
>

"Fauxton can just keep the new password in memory" opens a door to the
issue when you accidentally refreshes page / closes tab / loses page
memory in other way. Not an flaw, just a case to remember about.

As about sending (or replicating) PBKDF2 hash looks good for me.

>> - If I add a new node after "finish_cluster" setup, will it have all
>> system databases (global_changes, cassim, _users...whatever else)
>> created?
>
> That is unspecified at this point. I’d need more input from the Cloudant people
> on this one. I’m happy to go either way, or make it an option for later joined
> nodes.

Ok. Let's wait what Cloudant people say. I'm pretty sure they already
know the solution for all these problems or at least knows their
specifics.

Thanks a lot, Jan! (:

--
,,,^..^,,,

Re: Setting up CouchDB 2.0

Posted by Jan Lehnardt <ja...@apache.org>.
> On 26 Oct 2014, at 20:31 , Alexander Shorin <kx...@gmail.com> wrote:
> 
> Hi Jan,
> 
> Suddenly, gists doesn't supports per-line comments. Replied inline
> with questions.

Good questions, thanks! Answers inline :)


> 
> On Sun, Oct 26, 2014 at 10:04 PM, Jan Lehnardt <ja...@apache.org> wrote:
>> N. End User Action
>> - What happens behind the scenes.
>> 
>> 
>> 1. Launch CouchDB with `$ couchdb`, or init.d, or any other way, exactly
>> like it is done in 1.x.x.
>> - CouchDB launches and listens on 127.0.0.1:5984
>> 
>> From here on, there are two paths, one is via Fauxton (a) the other is
>> using a HTTP endpoint (b). Fauxton just uses the HTTP endpoint in (b).
>> (b) can be used to set up a cluster programmatically.
>> 
>> 
>> 2.a. Go to Fauxton. There is a “Cluster Setup” tab in the sidebar. Go
>> to the tab and get presented with a form that asks you to enter an admin
>> username, admin password and optionally a bind_address and port to bind
>> to publicly. Submit the form with the [Enable Cluster] button.
>> 
>> - POST to /_setup with
>>   {
>>     "action": "enable_cluster",
>>     "admin": {
>>       "user": "username",
>>       "pass": "password"
>>     },
>>     ["bind_address": "xxxx",]
>>     ["port": yyyy]
>>   }
>> 
>> This sets up the admin user on the current node and binds to 0.0.0.0:5984
>> or the specified ip:port. Logs admin user into Fauxton automatically.
>> 
>> 2.b. POST to /_setup as shown above.
>> 
>> Repeat on all nodes.
>> - keep the same username/password everywhere.
>> 
> 
> Question:
> - Is /_setup admin-only resource?

Definitely, sorry for missing that bit.


> - If node already has admin-party fixed should it accepts new admin credentials?

Good question, I’d say if an admin already exists, no new admin credentials are needed/


> - Any reasons to replace 1-3 PUT requests to /_config with single POST
> one in this case?

I’m not sure what the 1-3 PUT requests are?


> 3. Pick any one node, for simplicity use the first one, to be the
>> “setup coordination node”.
>> - this is a “master” node that manages the setup and requires all
>>   other nodes to be able to see it and vice versa. Setup won’t work
>>   with unavailable nodes (duh). The notion of “master” will be gone
>>   once the setup is finished. At that point, the system has no
>>   master node. Ignore I ever said “master”.
>> 
>> a. Go to Fauxton / Cluster Setup, once we have enabled the cluster, the
>> UI shows an “Add Node” interface with the fields admin, and node:
>> - POST to /_setup with
>>   {
>>     "action": "add_node",
>>     "admin": { // should be auto-filled from Fauxton
>>       "user": "username",
>>       "pass": "password"
>>     },
>>     "node": {
>>       "host": "hostname",
>>       ["port": 5984]
>>     }
>>   }
>> 
>> b. as in a, but without the Fauxton bits, just POST to /_setup
>> - this request will do this:
>>  - on the “setup coordination node”:
>>   - check if we have an Erlang Cookie Secret. If not, generate
>>     a UUID and set the erlang cookie to to that UUID.
>>     // TBD: persist the cookie, so it survives restarts
>>   - make a POST request to the node specified in the body above
>>     using the admin credentials in the body above:
>>     POST to http://username:password@node_b:5984/_setup with:
>>     {
>>       "action": "receive_cookie",
>>       "cookie": "<secretcookie>",
>>     }
>>     // TBD: persist the cookie on node B, so it survives restarts
>> 
>>   - when the request to node B returns, we know the Erlang-level
>>     inter-cluster communication is enabled and we can start adding
>>     the node on the CouchDB level. To do that, the “setup
>>     coordination node” does this to it’s own HTTP endpoint:
>>     PUT /nodes/node_b:5984 or the same thing with internal APIs.
>> 
>> - Repeat for all nodes.
>> - Fauxton keeps a list of all set up nodes for users to see.
> 
> Question:
> - Since Fauxton already known all the nodes admin credentials and all
> the nodes are bounded to 0.0.0.0 iface (from previous step), will
> Fauxton automate nodes join into the cluster? This is about to skip
> "Repeat on all nodes" step

How does Fauxton know about the other nodes at this point?
(I guess since the Erlang cluster is already set up, it could expose that
info to Fauxton in a zeroconf kind of fashion and auto-populate the Fauxton
UI with nodes that then can be joined with just a click of a button.)


> - If some of my nodes have different admin credentials, is this the
> blocker error case or should Fauxton ask me for these credentials?

That’s why `add_node` takes a username and password as options, you
can set that up if you want. / This could also be made an error case.
It should certainly not be recommended.


> - Any reasons for replacing regular request to /_nodes with custom
> /_setup?

I don’t know what /_nodes is. Do you mean /nodes? — The reason this isn’t
using /nodes at this point is that /nodes already has a special meaning
and I didn’t want to complicate the existing logic. In addition, /nodes
might have to be adjusted to carry the username and password of the target
CouchDB to do the setup (if we otherwise keep the proposed model, happy
to see alternatives, though!).

If we can reduce all of what I outlined to `PUT /nodes/node_b|c|d`, that
would be nice. Fauxton could then offer the setup UI based on whether /nodes
has any entries. But I don’t know enough about the semantics and other
uses of /nodes, so I haven’t thought about this option too much.


> Point about cookie counts.

Not sure I follow.


> 4.a. When all nodes are added, click the [Finish Cluster Setup] button
>> in Fauxton.
>> - this does POST /_setup
>>   {
>>     "action": "finish_setup"
>>   }
>> 
>> b. Same as in a.
>> 
>> - this manages the final setup bits, like creating the _users,
>>   _replicator and _db_updates endpoints and whatever else is needed.
>>   // TBD: collect what else is needed.
> 
> This is the only useful thing that /_setup does from my current point
> of view - everything else was just masking standard requests to
> existed API.

Which existing API in particular?

If you mean that this all can be done over /_config and /nodes, yes totally,
but Fauxton on node_a can’t access /_config on node_b. That’s one of the
reasons of why I suggest using /_setup, so it can do all this from a single
node via Fauxton. The other reason is that it is a dedicated API end-point
that hides a lot of complexity instead of having end-users hit a bunch of
seemingly random endpoints (although this *could* be hidden in Fauxton maybe,
except for the cross domain issue).


> 
>> ## The Setup Endpoint
>> 
>> This is not a REST-y endpoint, it is a simple state machine operated
>> by HTTP POST with JSON bodies that have an `action` field.
>> 
>> ### State 1: No Cluster Enabled
>> 
>> This is right after starting a node for the first time, and any time
>> before the cluster is enabled as outlined above.
>> 
>> GET /_setup
>> {"state": "cluster_disabled"}
>> 
>> POST /_setup {"action":"enable_cluster"...} -> Transition to State 2
>> POST /_setup {"action":"enable_cluster"...} with empty admin user/pass or invalid host/post or host/port not available -> Error
>> POST /_setup {"action":"anything_but_enable_cluster"...} -> Error
>> 
> 
> If "enable_cluster" only creates/setups admin and bind address, could
> this step be skipped? Because the same actions are possible to do via
> regular config setup.

Yes! It just needs to ensure these things are done. If Fauxton detects
they *are* done, it can skip the enable step and show the add_node interface
right away.

> 
> 
>> ### State 2: Cluster enabled, admin user set, waiting for nodes to be added.
>> 
>> GET /_setup
>> {"state":"cluster_enabled","nodes":[]}
>> 
>> POST /_setup {"action":"enable_cluster"...} -> Error
>> POST /_setup {"action":"add_node"...} -> Stay in State 2, but return "nodes":["node B"}] on GET
>> POST /_setup {"action":"add_node"...} -> if target node not available, Error
>> POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
>> POST /_setup {"action":"finish_cluster"} -> Transition to State 3
>> 
> 
> Questions:
> - How much nodes required to be added? 1? 2? 3?...

Doesn’t matter.


> - How to remove accidentally added node from cluster?

Delete from /nodes database. Could be added as a UI element in Fauxton.


> 
>> ### State 3: Cluster set up, all nodes operational
>> 
>> GET /_setup
>> {"state":"cluster_finished","nodes":["node a", "node b", ...]}
>> 
>> POST /_setup {"action":"enable_cluster"...} -> Error
>> POST /_setup {"action":"finish_cluster"...} -> Stay in State 3, do nothing
>> POST /_setup {"action":"add_node"...} -> Error
>> POST /_setup?i_know_what_i_am_doing=true {"action":"add_node"...} -> Add node, stay in State 3.
>> 
>> // TBD: we need to persist the setup state somewhere.
>> 
> 
> Questions:
> - Why adding a new node after finish_cluster is some specific case to
> mark it with "i_know_what_i_am_doing" parameter?

Because I think it is not advisable to do this regularly, but someone might
want to do this regardless (see next).


> - How to enlarge / reduce cluster after his setup or even disband it?

Enlarge: see above.
Reduce: delete from /nodes
Disband: shut down all CouchDB processes :)

I don’t know the BigCouch/Cloudant best practices for this. I’ll chalk
this down as a “needs input from Cloudant people” :)


> Or this isn't what /_setup should cares about?

In general, this isn’t really covered by the setup proposal here. I’d like
to keep this out of scope for now, but we should have good answers to that
going forward.


> - What happens with /_setup resource after finish_cluster? Any case
> for it to be useful?

Only for Fauxton to show the correct setup state.


> - How could /_setup helps with admin password change among the all
> cluster nodes?

At least on the first run setup, Fauxton can just keep the new password in
memory and pre-fill the add_node screens with the same username and password.
/_setup then transports it over.

For later setups, I don’t know, as we would have the admin to enter the password
in plaintext so we can send it. Alternatively, we could use the /_config API to
read and send the PBKDF2 hash *waves hands*.


> - If I add a new node after "finish_cluster" setup, will it have all
> system databases (global_changes, cassim, _users...whatever else)
> created?

That is unspecified at this point. I’d need more input from the Cloudant people
on this one. I’m happy to go either way, or make it an option for later joined
nodes.

Best
Jan
--




> 
>> * * *
>> 
>> So far.
>> 
>> I think this is simpler than the first try. It retains the simplicity
>> of getting started of 1.x.x. It is fully scriptable via HTTP. It has
>> the same setup-security properties as 1.x.x. It requires an admin account
>> for cluster setup, but only on the last possible occasion in the
>> process. It still fully hides the Erlang cluster / secure cookie setup
>> from the end-user.
>> 
>> Does this sound sensible to you? Am I missing anything aside from the
>> TBD bits, for which, if you have ideas, I’d love your input! :)
>> 
>> Are there any deal-breaker flaws in this? What can be made simpler or
>> more clear?
>> 
>> Note that all the HTTP endpoints and field names are just placeholders,
>> no need to bikeshed on these just yet :)
>> 
>> I’m looking forward to your feedback!
> 
> 
> 
> --
> ,,,^..^,,,


Re: Setting up CouchDB 2.0

Posted by Alexander Shorin <kx...@gmail.com>.
Hi Jan,

Suddenly, gists doesn't supports per-line comments. Replied inline
with questions.

On Sun, Oct 26, 2014 at 10:04 PM, Jan Lehnardt <ja...@apache.org> wrote:
> N. End User Action
>  - What happens behind the scenes.
>
>
> 1. Launch CouchDB with `$ couchdb`, or init.d, or any other way, exactly
> like it is done in 1.x.x.
>  - CouchDB launches and listens on 127.0.0.1:5984
>
> From here on, there are two paths, one is via Fauxton (a) the other is
> using a HTTP endpoint (b). Fauxton just uses the HTTP endpoint in (b).
> (b) can be used to set up a cluster programmatically.
>
>
> 2.a. Go to Fauxton. There is a “Cluster Setup” tab in the sidebar. Go
> to the tab and get presented with a form that asks you to enter an admin
> username, admin password and optionally a bind_address and port to bind
> to publicly. Submit the form with the [Enable Cluster] button.
>
>  - POST to /_setup with
>    {
>      "action": "enable_cluster",
>      "admin": {
>        "user": "username",
>        "pass": "password"
>      },
>      ["bind_address": "xxxx",]
>      ["port": yyyy]
>    }
>
>  This sets up the admin user on the current node and binds to 0.0.0.0:5984
>  or the specified ip:port. Logs admin user into Fauxton automatically.
>
> 2.b. POST to /_setup as shown above.
>
> Repeat on all nodes.
>  - keep the same username/password everywhere.
>

Question:
- Is /_setup admin-only resource?
- If node already has admin-party fixed should it accepts new admin credentials?
- Any reasons to replace 1-3 PUT requests to /_config with single POST
one in this case?

> 3. Pick any one node, for simplicity use the first one, to be the
> “setup coordination node”.
>  - this is a “master” node that manages the setup and requires all
>    other nodes to be able to see it and vice versa. Setup won’t work
>    with unavailable nodes (duh). The notion of “master” will be gone
>    once the setup is finished. At that point, the system has no
>    master node. Ignore I ever said “master”.
>
> a. Go to Fauxton / Cluster Setup, once we have enabled the cluster, the
> UI shows an “Add Node” interface with the fields admin, and node:
>  - POST to /_setup with
>    {
>      "action": "add_node",
>      "admin": { // should be auto-filled from Fauxton
>        "user": "username",
>        "pass": "password"
>      },
>      "node": {
>        "host": "hostname",
>        ["port": 5984]
>      }
>    }
>
> b. as in a, but without the Fauxton bits, just POST to /_setup
>  - this request will do this:
>   - on the “setup coordination node”:
>    - check if we have an Erlang Cookie Secret. If not, generate
>      a UUID and set the erlang cookie to to that UUID.
>      // TBD: persist the cookie, so it survives restarts
>    - make a POST request to the node specified in the body above
>      using the admin credentials in the body above:
>      POST to http://username:password@node_b:5984/_setup with:
>      {
>        "action": "receive_cookie",
>        "cookie": "<secretcookie>",
>      }
>      // TBD: persist the cookie on node B, so it survives restarts
>
>    - when the request to node B returns, we know the Erlang-level
>      inter-cluster communication is enabled and we can start adding
>      the node on the CouchDB level. To do that, the “setup
>      coordination node” does this to it’s own HTTP endpoint:
>      PUT /nodes/node_b:5984 or the same thing with internal APIs.
>
> - Repeat for all nodes.
> - Fauxton keeps a list of all set up nodes for users to see.

Question:
- Since Fauxton already known all the nodes admin credentials and all
the nodes are bounded to 0.0.0.0 iface (from previous step), will
Fauxton automate nodes join into the cluster? This is about to skip
"Repeat on all nodes" step
- If some of my nodes have different admin credentials, is this the
blocker error case or should Fauxton ask me for these credentials?
- Any reasons for replacing regular request to /_nodes with custom
/_setup? Point about cookie counts.

> 4.a. When all nodes are added, click the [Finish Cluster Setup] button
> in Fauxton.
>  - this does POST /_setup
>    {
>      "action": "finish_setup"
>    }
>
> b. Same as in a.
>
>  - this manages the final setup bits, like creating the _users,
>    _replicator and _db_updates endpoints and whatever else is needed.
>    // TBD: collect what else is needed.

This is the only useful thing that /_setup does from my current point
of view - everything else was just masking standard requests to
existed API.

> ## The Setup Endpoint
>
> This is not a REST-y endpoint, it is a simple state machine operated
> by HTTP POST with JSON bodies that have an `action` field.
>
> ### State 1: No Cluster Enabled
>
> This is right after starting a node for the first time, and any time
> before the cluster is enabled as outlined above.
>
> GET /_setup
> {"state": "cluster_disabled"}
>
> POST /_setup {"action":"enable_cluster"...} -> Transition to State 2
> POST /_setup {"action":"enable_cluster"...} with empty admin user/pass or invalid host/post or host/port not available -> Error
> POST /_setup {"action":"anything_but_enable_cluster"...} -> Error
>

If "enable_cluster" only creates/setups admin and bind address, could
this step be skipped? Because the same actions are possible to do via
regular config setup.


> ### State 2: Cluster enabled, admin user set, waiting for nodes to be added.
>
> GET /_setup
> {"state":"cluster_enabled","nodes":[]}
>
> POST /_setup {"action":"enable_cluster"...} -> Error
> POST /_setup {"action":"add_node"...} -> Stay in State 2, but return "nodes":["node B"}] on GET
> POST /_setup {"action":"add_node"...} -> if target node not available, Error
> POST /_setup {"action":"finish_cluster"} with no nodes set up -> Error
> POST /_setup {"action":"finish_cluster"} -> Transition to State 3
>

Questions:
- How much nodes required to be added? 1? 2? 3?...
- How to remove accidentally added node from cluster?

> ### State 3: Cluster set up, all nodes operational
>
> GET /_setup
> {"state":"cluster_finished","nodes":["node a", "node b", ...]}
>
> POST /_setup {"action":"enable_cluster"...} -> Error
> POST /_setup {"action":"finish_cluster"...} -> Stay in State 3, do nothing
> POST /_setup {"action":"add_node"...} -> Error
> POST /_setup?i_know_what_i_am_doing=true {"action":"add_node"...} -> Add node, stay in State 3.
>
> // TBD: we need to persist the setup state somewhere.
>

Questions:
- Why adding a new node after finish_cluster is some specific case to
mark it with "i_know_what_i_am_doing" parameter?
- How to enlarge / reduce cluster after his setup or even disband it?
Or this isn't what /_setup should cares about?
- What happens with /_setup resource after finish_cluster? Any case
for it to be useful?

> * * *
>
> So far.
>
> I think this is simpler than the first try. It retains the simplicity
> of getting started of 1.x.x. It is fully scriptable via HTTP. It has
> the same setup-security properties as 1.x.x. It requires an admin account
> for cluster setup, but only on the last possible occasion in the
> process. It still fully hides the Erlang cluster / secure cookie setup
> from the end-user.
>
> Does this sound sensible to you? Am I missing anything aside from the
> TBD bits, for which, if you have ideas, I’d love your input! :)
>
> Are there any deal-breaker flaws in this? What can be made simpler or
> more clear?
>
> Note that all the HTTP endpoints and field names are just placeholders,
> no need to bikeshed on these just yet :)
>
> I’m looking forward to your feedback!



--
,,,^..^,,,