multi-dc clusters with 'local' ips and no vpn

[Chris Burroughs <ch...@gmail.com>]

Cassandra makes the totally reasonable assumption that the entire
cluster is in one routable address space.  We unfortunately had a
situation where:
 * nodes can talk to each other in the same dc on an internal address,
but not talk to each other over their external 1:1 NAT address.
 * nodes can talk to nodes in the other dc over the external address,
but there is no usable shared internal address space they can talk over

In case anyone else finds themselves in the same situation we have what
we think is a working solution in pre-production.  CASSANDRA-5630
handles the "reconnect trick" to prefer the local ip when in the same
DC.  And some iptables rules allow the local nodes to do the initial
gossiping with each other before that switch.

for each node in same dc:
    'iptables -t nat -A OUTPUT -j DNAT -p tcp --dst %s --dport 7000 -o
eth0  --to-destination %s' % (ext_ip, local_ip)