[GitHub] jiangpengcheng commented on issue #1421: when one member of a cluster is restarted, it won't rejoin to the cluster automatically

[GitBox <gi...@apache.org>]

jiangpengcheng commented on issue #1421: when one member of a cluster is restarted, it won't rejoin to the cluster automatically
URL: https://github.com/apache/couchdb/issues/1421#issuecomment-402982983
 
 
   ## Steps to Reproduce (for bugs)
   1. ~I'm using CouchDB for this project [OpenWhisk](https://github.com/apache/incubator-openwhisk), to deploy a CouchDB cluster~ Now I use docker only, create couchdb containers on 3 nodes:
   
       ```shell
       # node1
       docker run -d --name couchdb -e COUCHDB_USER=couch_user -e COUCHDB_PASSWORD=couch_password -e NODENAME=192.168.253.137 -p 5984:5984 -p 4369:4369 -p 9100:9100 apache/couchdb:2.1
   
       # node2
       docker run -d --name couchdb -e COUCHDB_USER=couch_user -e COUCHDB_PASSWORD=couch_password -e NODENAME=192.168.253.138 -p 5984:5984 -p 4369:4369 -p 9100:9100 apache/couchdb:2.1
   
       # node3
       docker run -d --name couchdb -e COUCHDB_USER=couch_user -e COUCHDB_PASSWORD=couch_password -e NODENAME=192.168.253.139 -p 5984:5984 -p 4369:4369 -p 9100:9100 apache/couchdb:2.1
       ```
   
   2. setup the cluster
   
       I follow this [guide](http://docs.couchdb.org/en/2.1.1/cluster/setup.html#the-cluster-setup-api), and use node1 as the “setup coordination node”
   
       Below command returned an error, so I skipped this command:
       ```shell
       curl -X POST -H "Content-Type: application/json" http://couch_user:couch_password@127.0.0.1:5984/_cluster_setup -d '{"action": "enable_cluster", "bind_address":"0.0.0.0", "username": "couch_user", "password":"couch_password", "node_count":"3"}'
       {"error":"bad_request","reason":"Cluster is already enabled"}
       ```
   
       ```shell
       # add node2 to cluster
       curl -X POST -H "Content-Type: application/json" http://couch_user:couch_password@127.0.0.1:5984/_cluster_setup -d '{"action": "enable_cluster", "bind_address":"0.0.0.0", "username": "couch_user", "password":"couch_password", "port": 5984, "node_count": "3", "remote_node": "192.168.253.138", "remote_current_user": "couch_user", "remote_current_password": "couch_password" }'
       {"ok":true}
   
       curl -X POST -H "Content-Type: application/json" http://couch_user:couch_password@127.0.0.1:5984/_cluster_setup -d '{"action": "add_node", "host":"192.168.253.138", "port": "5984", "username": "couch_user", "password":"couch_password"}'
       {"ok":true}
   
       # add node3 to cluster
       curl -X POST -H "Content-Type: application/json" 
       http://couch_user:couch_password@127.0.0.1:5984/_cluster_setup -d '{"action": "enable_cluster", "bind_address":"0.0.0.0", "username": "couch_user", "password":"couch_password", "port": 5984, "node_count": "3", "remote_node": "192.168.253.139", "remote_current_user": "couch_user", "remote_current_password": "couch_password" }'
       {"ok":true}
   
       curl -X POST -H "Content-Type: application/json" 
       http://couch_user:couch_password@127.0.0.1:5984/_cluster_setup -d '{"action": "add_node", "host":"192.168.253.139", "port": "5984", "username": "couch_user", "password":"couch_password"}'
       {"ok":true}
   
       # finish cluster
       curl -X POST -H "Content-Type: application/json" http://couch_user:couch_password@127.0.0.1:5984/_cluster_setup -d '{"action": "finish_cluster"}'
       {"ok":true}
       ```
   
       the cluster was formed correctly:
       ```shell
       curl -X GET http://couch_user:couch_password@127.0.0.1:5984/_membership
       {"all_nodes":["couchdb@192.168.253.137","couchdb@192.168.253.138","couchdb@192.168.253.139"],"cluster_nodes":["couchdb@192.168.253.137","couchdb@192.168.253.138","couchdb@192.168.253.139"]}
       ```
   
   3. restart the 3rd node
   
   4. call the `_membership` API againt node1/node2, the result was:
       ```
       {"all_nodes":["couchdb@192.168.253.137","couchdb@192.168.253.138"],"cluster_nodes":["couchdb@192.168.253.137","couchdb@192.168.253.138","couchdb@192.168.253.139"]}
       ```
       and the node3 returned:
       ```
       {"all_nodes":["couchdb@192.168.253.139"],"cluster_nodes":["couchdb@192.168.253.137","couchdb@192.168.253.138","couchdb@192.168.253.139"]}
       ```
       we can see that the node3 has left the cluster
   
   
   ## Context
   
   A strange thing is that when I restarted the node1, it can rejoin the cluster, seems there is something special in the “setup coordination node”.
   
   So I cleaned the env, recreated 3 couchdb containers and use node2 as the “setup coordination node” this time, and find out that the node2 can rejoin the cluster automatically while node1 and node3 can not, this is confusing
   
   
   @wohali Could you please check this when you have some time? Many thanks

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services