You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Chris Marino <ch...@vsider.com> on 2011/07/29 23:11:14 UTC
Running across multiple EC2 regions...
Hello,
Me and some of my colleagues are about to start some experiments running
Cassandra across EC2 regions using virtual networks and have some questions
about how this is going to work.
I've read these threads about patching the .yaml file to bind to the Listen
address to the public IP...
http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.<http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.com%3E>
com%3E<http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.com%3E>
http://mail-archives.apache.org/mod_mbox/cassandra-user/201103.mbox/%3CAANLkTikWsuUOjEU228niWi0iDTAO5J=5wO=i=hG33AnM@mail.gmail.com%3E
And this EC2 Snitch patch that lets it work across regions...
https://issues.apache.org/jira/browse/CASSANDRA-2452
I'm pretty familiar with EC2 networking, but not very familiar how Cassandra
will use the RPC and Listen ports with this Snitch.
So, my question is: If we use the EC2 Snitch patch to set up across regions,
will the instance's private IP/interface *ever* be used? Or will all traffic
always go in and out of the public interface?? Using the public interface is
slower and more expensive that the private interface.
What I'm trying to do is set up a virtual network that lets all the nodes
use private IPs, but can still communicate across regions.
We're going to try this with a virtual network as well as with the EC2
Snitch to see how things compare. Being able to use the EC2 private
interface is going to make big difference.
Thanks in advance.
CM
Re: Running across multiple EC2 regions...
Posted by Vijay <vi...@gmail.com>.
Chris,
It is internal to cassandra.(IEndpointStateChangeSubscriber)... the snitch
basically starts gossiping via public ip and once it sees the Location of
the node to be local (it gets notified via IESCS) and then switches to
private IP (safely - flush and reconnect). hope this helps... I didnt intend
to make it complicated i thought the operations part is what you have to be
worried about... (Change the Security group in both the regions to allow the
Public IP's)....
Regards,
</VJ>
On Fri, Jul 29, 2011 at 4:37 PM, Chris Marino <ch...@vcider.com> wrote:
> Thanks Vijay, that helps a lot.
>
> FYI, I did read the comments but didn't understand what I was reading
> since I don't know what IECSC is?? Googled it but still came up empty. What
> is this??
>
> Sorry, if this is obvious, but I'm pretty new to all this....
>
> Thanks
> CM
>
> On Fri, Jul 29, 2011 at 3:18 PM, Vijay <vi...@gmail.com> wrote:
>
>> "So, my question is: If we use the EC2 Snitch patch to set up across
>> regions, will the instance's private IP/interface *ever* be used? Or will
>> all traffic always go in and out of the public interface?? Using the public
>> interface is slower and more expensive that the private interface. "
>>
>> If you use 2452, communication within a region is via private ip and
>> communication between the regions are public (Handshaking or intial
>> communication will still be via public ip). In EC2 they dont have 2
>> interface but they nat the public IP.... even then this patch will do the
>> right thing for you.
>>
>> There is comments in the patch....
>>
>>
>> + * 1) Snitch will automatically set the public IP by querying the AWS API
>> + *
>> + * 2) Snitch will set the private IP as a Gossip application state.
>> + *
>> + * 3) Snitch implements IESCS and will reset the connection if it is within the
>> + * same region to communicate via private IP.
>> + *
>> + * Implements Ec2Snitch to inherit its functionality and extend it for
>> + * Multi-Region.
>> + *
>> + * Operational: All the nodes in this cluster needs to be able to (modify the
>> + * Security group settings in AWS) communicate via Public IP's.
>>
>>
>> Regards,
>> </VJ>
>>
>>
>>
>>
>> On Fri, Jul 29, 2011 at 2:11 PM, Chris Marino <ch...@vsider.com> wrote:
>>
>>> Hello,
>>>
>>> Me and some of my colleagues are about to start some experiments running
>>> Cassandra across EC2 regions using virtual networks and have some questions
>>> about how this is going to work.
>>>
>>> I've read these threads about patching the .yaml file to bind to the
>>> Listen address to the public IP...
>>>
>>>
>>> http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.<http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.com%3E>
>>> com%3E<http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.com%3E>
>>>
>>>
>>> http://mail-archives.apache.org/mod_mbox/cassandra-user/201103.mbox/%3CAANLkTikWsuUOjEU228niWi0iDTAO5J=5wO=i=hG33AnM@mail.gmail.com%3E
>>>
>>> And this EC2 Snitch patch that lets it work across regions...
>>>
>>> https://issues.apache.org/jira/browse/CASSANDRA-2452
>>>
>>> I'm pretty familiar with EC2 networking, but not very familiar how
>>> Cassandra will use the RPC and Listen ports with this Snitch.
>>>
>>> So, my question is: If we use the EC2 Snitch patch to set up across
>>> regions, will the instance's private IP/interface *ever* be used? Or will
>>> all traffic always go in and out of the public interface?? Using the public
>>> interface is slower and more expensive that the private interface.
>>>
>>> What I'm trying to do is set up a virtual network that lets all the nodes
>>> use private IPs, but can still communicate across regions.
>>>
>>> We're going to try this with a virtual network as well as with the EC2
>>> Snitch to see how things compare. Being able to use the EC2 private
>>> interface is going to make big difference.
>>>
>>> Thanks in advance.
>>> CM
>>>
>>>
>>
>
Re: Running across multiple EC2 regions...
Posted by Chris Marino <ch...@vcider.com>.
Thanks Vijay, that helps a lot.
FYI, I did read the comments but didn't understand what I was reading since
I don't know what IECSC is?? Googled it but still came up empty. What is
this??
Sorry, if this is obvious, but I'm pretty new to all this....
Thanks
CM
On Fri, Jul 29, 2011 at 3:18 PM, Vijay <vi...@gmail.com> wrote:
> "So, my question is: If we use the EC2 Snitch patch to set up across
> regions, will the instance's private IP/interface *ever* be used? Or will
> all traffic always go in and out of the public interface?? Using the public
> interface is slower and more expensive that the private interface. "
>
> If you use 2452, communication within a region is via private ip and
> communication between the regions are public (Handshaking or intial
> communication will still be via public ip). In EC2 they dont have 2
> interface but they nat the public IP.... even then this patch will do the
> right thing for you.
>
> There is comments in the patch....
>
> + * 1) Snitch will automatically set the public IP by querying the AWS API
> + *
> + * 2) Snitch will set the private IP as a Gossip application state.
> + *
> + * 3) Snitch implements IESCS and will reset the connection if it is within the
> + * same region to communicate via private IP.
> + *
> + * Implements Ec2Snitch to inherit its functionality and extend it for
> + * Multi-Region.
> + *
> + * Operational: All the nodes in this cluster needs to be able to (modify the
> + * Security group settings in AWS) communicate via Public IP's.
>
>
> Regards,
> </VJ>
>
>
>
>
> On Fri, Jul 29, 2011 at 2:11 PM, Chris Marino <ch...@vsider.com> wrote:
>
>> Hello,
>>
>> Me and some of my colleagues are about to start some experiments running
>> Cassandra across EC2 regions using virtual networks and have some questions
>> about how this is going to work.
>>
>> I've read these threads about patching the .yaml file to bind to the
>> Listen address to the public IP...
>>
>>
>> http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.<http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.com%3E>
>> com%3E<http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.com%3E>
>>
>>
>> http://mail-archives.apache.org/mod_mbox/cassandra-user/201103.mbox/%3CAANLkTikWsuUOjEU228niWi0iDTAO5J=5wO=i=hG33AnM@mail.gmail.com%3E
>>
>> And this EC2 Snitch patch that lets it work across regions...
>>
>> https://issues.apache.org/jira/browse/CASSANDRA-2452
>>
>> I'm pretty familiar with EC2 networking, but not very familiar how
>> Cassandra will use the RPC and Listen ports with this Snitch.
>>
>> So, my question is: If we use the EC2 Snitch patch to set up across
>> regions, will the instance's private IP/interface *ever* be used? Or will
>> all traffic always go in and out of the public interface?? Using the public
>> interface is slower and more expensive that the private interface.
>>
>> What I'm trying to do is set up a virtual network that lets all the nodes
>> use private IPs, but can still communicate across regions.
>>
>> We're going to try this with a virtual network as well as with the EC2
>> Snitch to see how things compare. Being able to use the EC2 private
>> interface is going to make big difference.
>>
>> Thanks in advance.
>> CM
>>
>>
>
Re: Running across multiple EC2 regions...
Posted by Vijay <vi...@gmail.com>.
"So, my question is: If we use the EC2 Snitch patch to set up across
regions, will the instance's private IP/interface *ever* be used? Or will
all traffic always go in and out of the public interface?? Using the public
interface is slower and more expensive that the private interface. "
If you use 2452, communication within a region is via private ip and
communication between the regions are public (Handshaking or intial
communication will still be via public ip). In EC2 they dont have 2
interface but they nat the public IP.... even then this patch will do the
right thing for you.
There is comments in the patch....
+ * 1) Snitch will automatically set the public IP by querying the AWS API
+ *
+ * 2) Snitch will set the private IP as a Gossip application state.
+ *
+ * 3) Snitch implements IESCS and will reset the connection if it is within the
+ * same region to communicate via private IP.
+ *
+ * Implements Ec2Snitch to inherit its functionality and extend it for
+ * Multi-Region.
+ *
+ * Operational: All the nodes in this cluster needs to be able to (modify the
+ * Security group settings in AWS) communicate via Public IP's.
Regards,
</VJ>
On Fri, Jul 29, 2011 at 2:11 PM, Chris Marino <ch...@vsider.com> wrote:
> Hello,
>
> Me and some of my colleagues are about to start some experiments running
> Cassandra across EC2 regions using virtual networks and have some questions
> about how this is going to work.
>
> I've read these threads about patching the .yaml file to bind to the Listen
> address to the public IP...
>
>
> http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.<http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.com%3E>
> com%3E<http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.com%3E>
>
>
> http://mail-archives.apache.org/mod_mbox/cassandra-user/201103.mbox/%3CAANLkTikWsuUOjEU228niWi0iDTAO5J=5wO=i=hG33AnM@mail.gmail.com%3E
>
> And this EC2 Snitch patch that lets it work across regions...
>
> https://issues.apache.org/jira/browse/CASSANDRA-2452
>
> I'm pretty familiar with EC2 networking, but not very familiar how
> Cassandra will use the RPC and Listen ports with this Snitch.
>
> So, my question is: If we use the EC2 Snitch patch to set up across
> regions, will the instance's private IP/interface *ever* be used? Or will
> all traffic always go in and out of the public interface?? Using the public
> interface is slower and more expensive that the private interface.
>
> What I'm trying to do is set up a virtual network that lets all the nodes
> use private IPs, but can still communicate across regions.
>
> We're going to try this with a virtual network as well as with the EC2
> Snitch to see how things compare. Being able to use the EC2 private
> interface is going to make big difference.
>
> Thanks in advance.
> CM
>
>