[GitHub] epappas opened a new issue #963: CouchDB 2.1 Cluster doesn't replicate the data across

[GitBox <gi...@apache.org>]

epappas opened a new issue #963: CouchDB 2.1 Cluster doesn't replicate the data across
URL: https://github.com/apache/couchdb/issues/963
 
 
   I am trying to create a couchDB cluster, by trying to replicate the steps from 
   the docs [1] but it seems the actual result differs from my expectation. Whilst 
   by following these steps the Nodes discover each other, and they understand that
   they are in a cluster mode, they don't replicate the data across. So when I'm 
   creating a new Database on one of the nodes, the others never get an update. 
   Same if the database has a document.
   
   [1] http://docs.couchdb.org/en/latest/cluster/setup.html
   
   ## Expected Behavior
   
   I'm Expecting that in a cluster mode, CouchDB would replicate the data (in an 
   eventual consistency manner) across the participated nodes, if such update has 
   taken place to any node of the cluster.
   
   So if I'm creating a document in one node, I should be able to query another 
   node of the cluster to fetch the results that I have just created (time delay 
   is accepted).
   
   ## Current Behavior
   
   I am creating the cluster (apparently successfully) but the nodes don't get 
   updated for new documents that have been created into other nodes of the cluster.
   
   ## Possible Solution
   
   Somehow, the nodes need to be have consistent state. I'll need to dig deeper to 
   figure how.
   
   As a followup, I will try to trace the network between the nodes, to see if 
   there is any information actually been exchanged.
   
   The fact that the nodes are not listed under the `all_nodes` list, but only 
   under the `cluster_nodes` list of the `/_membership` endpoint seems to be giving
   away a clue.
   
   ## Steps to Reproduce (for bugs)
   
   Some ENV variables for example context
   
   ```
   SIZE="3"
   ADMIN_USER="admin"
   ADMIN_PASSWORD="secure_password"
   COUCHDB_COOKIE="yummy_cookie"
   COUCHDB_SECRET="spooky_secret"
   IP="10.100.20.xxx" # where xxx is the IP of the Host
   NODES="10.100.20.1 10.100.21.2 10.100.22.3" # the IPs of all three Nodes
   COORDINATION_NODE="10.100.20.1"
   ```
   
   I am running my Docker image as such; (FYI the `/mnt` directory points to an EBS)
   
   ```
   sudo docker run -d \
       --name=couchdb \
       --restart always \
       --net=host \
       -p 5984:5984 \
       -p 5986:5986 \
       -p 4369:4369 \
       -p 9100:9100 \
       -v /mnt/couchdb/data:/opt/couchdb/data \
       -e NODENAME="$IP" \
       -e COUCHDB_COOKIE="$COUCHDB_COOKIE" \
       -e COUCHDB_SECRET="$COUCHDB_SECRET" \
       apache/couchdb:2.1.0
   ```
   
   Then on Each Node (3 of them) I run;
   
   ```
   curl -X POST \
       -H "Content-Type: application/json" \
       http://127.0.0.1:5984/_cluster_setup \
       -d "{
           \"action\": \"enable_cluster\",
           \"username\": \"$ADMIN_USER\",
           \"password\": \"$ADMIN_PASSWORD\",
           \"bind_address\": \"0.0.0.0\",
           \"port\": 5984,
           \"node_count\": $SIZE,
           \"singlenode\": false
       }"
   
   # Execute only in the coordination node
   if [ $IP = $COORDINATION_NODE ]
   then
       for node in $NODES
       do
           curl -X POST -H "Content-Type: application/json" \
               "http://$ADMIN_USER:$ADMIN_PASSWORD@127.0.0.1:5984/_cluster_setup" \
               -d "{
                   \"action\": \"enable_cluster\", 
                   \"bind_address\": \"0.0.0.0\", 
                   \"username\": \"$ADMIN_USER\", 
                   \"password\": \"$ADMIN_PASSWORD\",
                   \"port\": 15984, 
                   \"node_count\": $SIZE,
                   \"remote_node\": \"$node\", 
                   \"remote_current_user\": \"$ADMIN_USER\", 
                   \"remote_current_password\": \"$ADMIN_PASSWORD\"
               }"
       done
   fi
   
   curl -X POST \
       -H "Content-Type: application/json" \
       "http://$ADMIN_USER:$ADMIN_PASSWORD@127.0.0.1:5984/_cluster_setup" \
       -d '{"action":"finish_cluster"}'
   
   for node in $NODES
   do
       curl -X PUT "http://$ADMIN_USER:$ADMIN_PASSWORD@$IP:5986/_nodes/couchdb@$node" -d {}
   done
   ```
   
   Now if you create a document in the node 10.100.20.1, the others two never get 
   the update.
   
   ## Context
   
   - I have tried to netcat the ports from within the running docker containers, and they seem to be succeeding
   - the `netstat` in the host machine seems to have the ports active and listening
   - The `/_membership` has such a result: (eg for node 10.100.20.1)
   
   ```
   {
       "all_nodes": [
           "couchdb@10.100.20.1"
       ],
       "cluster_nodes": [
           "couchdb@10.100.20.1",
           "couchdb@10.100.20.2",
           "couchdb@10.100.20.3"
       ]
   }
   ```
   
   ## Your Environment
   CouchDB 2.1.0, (docker image: apache/couchdb:2.1.0)
   Host machine: Ubuntu xenial
   Platform: AWS, private subnet
   
   Ideas? thanks!

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services