You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2010/10/23 06:59:21 UTC
[jira] Created: (CASSANDRA-1654) EC2Snitch
EC2Snitch
---------
Key: CASSANDRA-1654
URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Jonathan Ellis
Assignee: Jon Hermes
Fix For: 0.7.1
This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1654) EC2Snitch
Posted by "Jon Hermes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924754#action_12924754 ]
Jon Hermes commented on CASSANDRA-1654:
---------------------------------------
A region is a DC, a zone is a Rack.
The metadata is per-node, so it'll have to go somewhere in Config.
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jon Hermes
> Fix For: 0.7.1
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-1654:
----------------------------------------
Attachment: (was: 1654-v2.txt)
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-incomplete.txt, 1654-v2.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-1654:
----------------------------------------
Attachment: 1654-v2.txt
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-incomplete.txt, 1654-v2.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1654) EC2Snitch
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925861#action_12925861 ]
Jonathan Ellis commented on CASSANDRA-1654:
-------------------------------------------
P.S. let's use Ec2Snitch (lowercase C) for consistency w/ other class names
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jon Hermes
> Fix For: 0.7.0
>
> Attachments: 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Jon Hermes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jon Hermes updated CASSANDRA-1654:
----------------------------------
Attachment: 1654.txt
svn st:
M conf/cassandra.yaml
A src/java/org/apache/cassandra/locator/EC2Snitch.java
M src/java/org/apache/cassandra/config/DatabaseDescriptor.java
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jon Hermes
> Fix For: 0.7.1
>
> Attachments: 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1654:
--------------------------------------
Fix Version/s: (was: 0.7.0)
0.7.1
Assignee: (was: Jon Hermes)
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Fix For: 0.7.1
>
> Attachments: 1654-incomplete.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1654) EC2Snitch
Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12976334#action_12976334 ]
Matthew F. Dennis commented on CASSANDRA-1654:
----------------------------------------------
doesn't appear to work on EC2
{noformat}
INFO 19:02:09,673 EC2Snitch using region: us-east, zone: 1d.
ERROR 19:02:09,762 Exception encountered during startup.
java.lang.ExceptionInInitializerError
at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:100)
at org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:54)
at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:240)
at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:133)
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:424)
... 4 more
Caused by: java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
at org.apache.cassandra.gms.Gossiper.addLocalApplicationState(Gossiper.java:877)
at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:300)
... 4 more
Exception encountered during startup.
Heap
par new generation total 38336K, used 14355K [0x00007fa51eb10000, 0x00007fa5214a0000, 0x00007fa5214a0000)
eden space 34112K, 42% used [0x00007fa51eb10000, 0x00007fa51f914d08, 0x00007fa520c60000)
from space 4224K, 0% used [0x00007fa520c60000, 0x00007fa520c60000, 0x00007fa521080000)
to space 4224K, 0% used [0x00007fa521080000, 0x00007fa521080000, 0x00007fa5214a0000)
concurrent mark-sweep generation total 4151744K, used 0K [0x00007fa5214a0000, 0x00007fa61eb10000, 0x00007fa61eb10000)
concurrent-mark-sweep perm gen total 21248K, used 10733K [0x00007fa61eb10000, 0x00007fa61ffd0000, 0x00007fa623f10000)
{noformat}
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-incomplete.txt, 1654-v2.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-1654:
----------------------------------------
Attachment: 1654-v2.txt
v2 which should actually gossip values around. Untested.
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-incomplete.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CASSANDRA-1654) EC2Snitch
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-1654:
-----------------------------------------
Assignee: Brandon Williams
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-incomplete.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Jon Hermes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jon Hermes updated CASSANDRA-1654:
----------------------------------
Attachment: 1654-incomplete.txt
Conserving remaining sanity.
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jon Hermes
> Fix For: 0.7.0
>
> Attachments: 1654-incomplete.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1654) EC2Snitch
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927469#action_12927469 ]
Jonathan Ellis commented on CASSANDRA-1654:
-------------------------------------------
(imo the most straightforward way to spread dc+rack via gossip is to add a ApplicationState.LOCATION state.)
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jon Hermes
> Fix For: 0.7.0
>
> Attachments: 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-1654:
----------------------------------------
Attachment: (was: 1654-v2.txt)
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-incomplete.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-1654:
----------------------------------------
Attachment: 1654-v2.txt
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-incomplete.txt, 1654-v2.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Jon Hermes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jon Hermes updated CASSANDRA-1654:
----------------------------------
Attachment: 1654.txt
$ bin/cassandra -f
...
INFO 19:42:51,530 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
*INFO 19:42:51,558 EC2Snitch using region: us-east, zone: 1c.*
INFO 19:42:51,623 Creating new commitlog segment /var/lib/cassandra/commitlog/CommitLog-1288208571623.log
...
\o/
If the response code isn't 200 OK, then it flubs. If for some reason it does return a response, the region/zone parsing logic will most likely fail. But if they run on ec2, It Just Works (tm).
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jon Hermes
> Fix For: 0.7.1
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1654) EC2Snitch
Posted by "Jon Hermes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925478#action_12925478 ]
Jon Hermes commented on CASSANDRA-1654:
---------------------------------------
Current working strategy is to poke 169.254.169.154/x/x/availability-zone, which will respond with the information I need ("us-east-1a", etc.).
During node bootup, if DD sees EC2Snitch as the endpoint_snitch, it will populate the region/zone off that information, and staticly return that when needed to the EC2Snitch.
The usage is commented, "Don't use EC2Snitch if you're not on EC2", as that url will most likely 404, but I'm not sure what else I can do to prevent/safeguard it's usage. Should I be letting the GET fail? Defaulting region/zone to "" (all nodes are in the same region/zone)? This is during boot-time, so it's highly visible in any event.
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jon Hermes
> Fix For: 0.7.1
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CASSANDRA-1654) EC2Snitch
Posted by "Jon Hermes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925525#action_12925525 ]
Jon Hermes edited comment on CASSANDRA-1654 at 10/27/10 4:26 PM:
-----------------------------------------------------------------
$ bin/cassandra -f
...
INFO 19:42:51,530 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
*INFO 19:42:51,558 EC2Snitch using region: us-east, zone: 1c.*
INFO 19:42:51,623 Creating new commitlog segment /var/lib/cassandra/commitlog/CommitLog-1288208571623.log
...
\o/
If 169.254.169.254 doesn't exist, then there will be a "No Route to Host" error. If the response code isn't 200 OK, then it flubs. If for some reason it does return a response, the region/zone parsing logic will most likely fail. But if they run on ec2, It Just Works (tm).
was (Author: jhermes):
$ bin/cassandra -f
...
INFO 19:42:51,530 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
*INFO 19:42:51,558 EC2Snitch using region: us-east, zone: 1c.*
INFO 19:42:51,623 Creating new commitlog segment /var/lib/cassandra/commitlog/CommitLog-1288208571623.log
...
\o/
If the response code isn't 200 OK, then it flubs. If for some reason it does return a response, the region/zone parsing logic will most likely fail. But if they run on ec2, It Just Works (tm).
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jon Hermes
> Fix For: 0.7.1
>
> Attachments: 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-1654:
----------------------------------------
Attachment: (was: 1654-v2.txt)
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-incomplete.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1654:
--------------------------------------
Remaining Estimate: 16h
Original Estimate: 16h
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-cassandra-0.7-v3.txt, 1654-incomplete.txt, 1654-v2.txt, 1654-v4.txt, 1654.txt
>
> Original Estimate: 16h
> Remaining Estimate: 16h
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew F. Dennis updated CASSANDRA-1654:
-----------------------------------------
Attachment: 1654-cassandra-0.7-v3.txt
1654-cassandra-0.7-v3.txt adds the EC2Snitch info to Gossiper after Gossiper has been initialized preventing the NPE above.
EC2Snitch is still not entirely working though.
{noformat}
ERROR [pool-1-thread-3] 2010-12-31 20:42:49,085 Cassandra.java (line 3044) Internal error processing batch_mutate
java.lang.NullPointerException
at org.apache.cassandra.locator.Ec2Snitch.getDatacenter(Ec2Snitch.java:76)
at org.apache.cassandra.locator.DynamicEndpointSnitch.getDatacenter(DynamicEndpointSnitch.java:115)
at org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:104)
at org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:412)
at org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:385)
at org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.process(Cassandra.java:3036)
at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2555)
at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-cassandra-0.7-v3.txt, 1654-incomplete.txt, 1654-v2.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew F. Dennis updated CASSANDRA-1654:
-----------------------------------------
Attachment: 1654-cassandra-0.7-v5.txt
v5 fixes NPE when used with DynamicSnitch
+1
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-cassandra-0.7-v3.txt, 1654-cassandra-0.7-v5.txt, 1654-incomplete.txt, 1654-v2.txt, 1654-v4.txt, 1654.txt
>
> Original Estimate: 16h
> Remaining Estimate: 16h
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1654) EC2Snitch
Posted by "Jon Hermes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924789#action_12924789 ]
Jon Hermes commented on CASSANDRA-1654:
---------------------------------------
Should I have the user enter it in as "region: us-east, zone: 1a" or enter "availability_zone = us-east-1a" and split it myself?
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jon Hermes
> Fix For: 0.7.1
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Jon Hermes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jon Hermes updated CASSANDRA-1654:
----------------------------------
Attachment: (was: 1654.txt)
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jon Hermes
> Fix For: 0.7.1
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CASSANDRA-1654) EC2Snitch
Posted by "Jon Hermes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925478#action_12925478 ]
Jon Hermes edited comment on CASSANDRA-1654 at 10/27/10 2:01 PM:
-----------------------------------------------------------------
Current working strategy is to poke 169.254.169.254(zeroconf)/x/x/availability-zone, which will respond with the information I need ("us-east-1a", etc.).
During node bootup, if DD sees EC2Snitch as the endpoint_snitch, it will populate the region/zone off that information, and staticly return that when needed to the EC2Snitch.
The usage is commented, "Don't use EC2Snitch if you're not on EC2", as that url will most likely 404, but I'm not sure what else I can do to prevent/safeguard it's usage. Should I be letting the GET fail? Defaulting region/zone to "" (all nodes are in the same region/zone)? This is during boot-time, so it's highly visible in any event.
was (Author: jhermes):
Current working strategy is to poke 169.254.169.154/x/x/availability-zone, which will respond with the information I need ("us-east-1a", etc.).
During node bootup, if DD sees EC2Snitch as the endpoint_snitch, it will populate the region/zone off that information, and staticly return that when needed to the EC2Snitch.
The usage is commented, "Don't use EC2Snitch if you're not on EC2", as that url will most likely 404, but I'm not sure what else I can do to prevent/safeguard it's usage. Should I be letting the GET fail? Defaulting region/zone to "" (all nodes are in the same region/zone)? This is during boot-time, so it's highly visible in any event.
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jon Hermes
> Fix For: 0.7.1
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1654) EC2Snitch
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980878#action_12980878 ]
Hudson commented on CASSANDRA-1654:
-----------------------------------
Integrated in Cassandra-0.7 #154 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/154/])
ec2 snitch. Patch by Jon Hermes, Matt Dennis, and brandonwilliams.
Reviewed by brandonwilliams for CASSANDRA-1654
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-cassandra-0.7-v3.txt, 1654-cassandra-0.7-v5.txt, 1654-incomplete.txt, 1654-v2.txt, 1654-v4.txt, 1654.txt
>
> Original Estimate: 16h
> Remaining Estimate: 16h
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew F. Dennis updated CASSANDRA-1654:
-----------------------------------------
Reviewer: mdennis
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-incomplete.txt, 1654-v2.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-1654:
----------------------------------------
Attachment: 1654-v2.txt
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-incomplete.txt, 1654-v2.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1654) EC2Snitch
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925860#action_12925860 ]
Jonathan Ellis commented on CASSANDRA-1654:
-------------------------------------------
does this actually work? how does node X find region/zone of node Y?
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jon Hermes
> Fix For: 0.7.0
>
> Attachments: 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1654) EC2Snitch
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-1654:
----------------------------------------
Attachment: 1654-v4.txt
v4 fixes the NPE caused by the node asking the snitch for its own dc/rack info, which of course is not in gossip.
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Attachments: 1654-cassandra-0.7-v3.txt, 1654-incomplete.txt, 1654-v2.txt, 1654-v4.txt, 1654.txt
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1654) EC2Snitch
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924824#action_12924824 ]
Jonathan Ellis commented on CASSANDRA-1654:
-------------------------------------------
No, you should introspect it from the instance metadata. http://docs.amazonwebservices.com/AmazonEC2/dg/2007-01-03/AESDG-chapter-instancedata.html
I think this will require the nodes to announce their DC/zone in gossip.
> EC2Snitch
> ---------
>
> Key: CASSANDRA-1654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1654
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Jon Hermes
> Fix For: 0.7.1
>
>
> This will be a snitch extending AbstractNetworkTopologySnitch that pulls DC from instance metadata placement_availability_zone.
> I don't think we can get rack information so the choice is between picking something unique-per-machine to be the rack or assigning everything in the DC the same rack. Second seems easier, so let's use placement_availability_zone for rack, too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.