[jira] [Created] (CASSANDRA-13378) DS Cassandra3.0 Adding a datacenter to a cluster procedure not working for us

["Kevin Rivait (JIRA)" <ji...@apache.org>]

Kevin Rivait created CASSANDRA-13378:
----------------------------------------

             Summary: DS Cassandra3.0  Adding a datacenter to a cluster procedure not working for us
                 Key: CASSANDRA-13378
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13378
             Project: Cassandra
          Issue Type: Bug
          Components: Configuration
         Environment: 5 node cluster , 
•	Server Model: Dell R730xd
•	Processor: 2 * Intel Xeon E5-2630 – Total 16 cores 
•	Memory: 192 GB  
•	OS Drive: 2 * 120 GB 
•	Direct attached storage – 8 TB (4 * 2TB SSD) + 1 * 800GB SSD
•	10Gb Dual Port + 1Gb Dual Port Network Daughter Card 
OS - OEL 2.6.32-642.15.1.el6.x86_64
java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

6 nodes in other DC are identical

            Reporter: Kevin Rivait
            Priority: Blocker
             Fix For: 3.0.9
         Attachments: cassandra.yaml, DS-add-a-dc-C3.0.rtf

i have replicated the issue on my personal cluster using VMs.
we have many keyspaces and users developing on the Dev Cluster we are trying to stretch.
With my current VMs (10 total) i can create a 2 DC cluster  dc1 and dc2.
i rebuild all nodes - clear data dirs and restart dc1.
i also clear data dirs on dc2 nodes,  i donot restart yet,
now i have single dc1 5 nodes cluster.
snitch - GossipingPropertyFileSnitch
i create keyspaces on DC1
after i alter keyspaces with replication dc1 3   dc2 0    i can no longer query tables - not enough replicas available for query at consistency ONE.
same error with CQL using consistency local_one
continue with procedure, startup dc2 nodes,  
alter replication on keyspaces to  dc1 3  dc2 2
from dc2 nodes , nodetool rebuild -- dc1  fails

i am attaching detailed steps with errors  and cassandra.yaml




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)