You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Shanker Balan <sh...@shapeblue.com> on 2013/10/24 15:10:02 UTC
Metadata URL location
Hi Guys,
CloudStack metadata services are on the default gateway while on EC2,
its at 169.254.169.254. Am curious to know why CloudStack does not
use a link local address for meta data services.
A search of the Wiki (https://cwiki.apache.org/confluence/dosearchsite.action?where=CLOUDSTACK&tooltip=Type+ALL%3A+in+your+query+to+search+all+of+Confluence&spaceSearch=true&queryString=metadata) didn’t seem to list any doc related to the design of this service.
TIA.
--
@shankerbalan
M: +91 98860 60539 | O: +91 (80) 67935867
shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade Centre, Bangalore - 560 055
CloudStack Bootcamp Training on 27/28 November, Bangalore
http://www.shapeblue.com/cloudstack-training/
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: Metadata URL location
Posted by Shanker Balan <sh...@shapeblue.com>.
On 24-Oct-2013, at 9:21 pm, Darren Shepherd <da...@gmail.com> wrote:
> So additionally you need to do
>
> ip addr add dev eth0 169.254.169.254/0
Thanks Kris and Darren. Thats very useful information.
The reason I ask is currently there is a bit of heuristics involved
in obtaining the meta data server IP from the DHCP lease files.
Scripts are not portable across different OSes and distribution
as they use different DHCP clients.
Having a consistent well known meta data server IP 169.254.169.254 would be nice. :)
>
> On Thu, Oct 24, 2013 at 8:29 AM, Kris G. Lindgren <kl...@godaddy.com> wrote:
>> You would also need to supernet 169.254.169.254 on the virtual router
>> (assigning it as 169.254.169.254 netmask 0.0.0.0 on eth0) that way it will
>> still arp to servers that are calling it that have real ip addresses.
>>
>> Additionally we had some other iptables rules in there that would change
>> the the ip address of the incoming request to metadata based upon the mac
>> address that was hitting it. This was to prevent spoofing of another vm's
>> IP and getting someone else's metadata (at least in our metadata
>> implementation we keyed off of the VM IP calling into metadata). This
>> also allowed a user to set whatever ipaddress they wanted, but as long as
>> the mac address was the same and they still had a zeroconfig route on the
>> VM, they still got only their metadata.
>> ____________________________________________
>>
>> Kris Lindgren
>> Senior Linux Systems Engineer
>> GoDaddy, LLC.
>>
>>
>> This email message and any attachment(s) hereto are intended for use only
>> by its intended recipient(s) and may contain confidential information. If
>> you have received this email in error, please immediately notify the
>> sender and permanently delete the original and any copy of this message
>> and its attachments.
>>
>>
>>
>>
>>
>>
>>
>> On 10/24/13 9:12 AM, "Darren Shepherd" <da...@gmail.com> wrote:
>>
>>> My guess, I don't really know, would be because its hard. The VR uses
>>> link local for the control network so 169.254/16 is bound to the wrong
>>> nic. To fix this you just need some ip routing magic in linux (credit
>>> goes to Kris Lindgren who showed me how to do this). Add the below to
>>> a file, substitute eth0 for the guest network nic, run "ip -b <FILE>"
>>> The below effectively creates a routing table dedicated to the IP
>>> 169.254.169.254 that sets it default route to go out the guest network
>>> nic.
>>>
>>> rule add from 169.254.169.254 table 70
>>> rule add to 169.254.169.254 dev eth0 table 70
>>> route flush table 70
>>> route add default dev eth0 src 169.254.169.254 table 70
>>> route flush cache
>>>
>>> Darren
>>>
>>> On Thu, Oct 24, 2013 at 6:10 AM, Shanker Balan
>>> <sh...@shapeblue.com> wrote:
>>>> Hi Guys,
>>>>
>>>> CloudStack metadata services are on the default gateway while on EC2,
>>>> its at 169.254.169.254. Am curious to know why CloudStack does not
>>>> use a link local address for meta data services.
>>>>
>>>> A search of the Wiki
>>>> (https://cwiki.apache.org/confluence/dosearchsite.action?where=CLOUDSTACK
>>>> &tooltip=Type+ALL%3A+in+your+query+to+search+all+of+Confluence&spaceSearc
>>>> h=true&queryString=metadata) didn¹t seem to list any doc related to the
>>>> design of this service.
>>>>
>>>> TIA.
>>>>
>>>> --
>>>> @shankerbalan
>>>>
>>>> M: +91 98860 60539 | O: +91 (80) 67935867
>>>> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
>>>> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade
>>>> Centre, Bangalore - 560 055
>>>>
>>>> CloudStack Bootcamp Training on 27/28 November, Bangalore
>>>> http://www.shapeblue.com/cloudstack-training/
>>>>
>>>>
>>>>
>>>>
>>>> This email and any attachments to it may be confidential and are
>>>> intended solely for the use of the individual to whom it is addressed.
>>>> Any views or opinions expressed are solely those of the author and do
>>>> not necessarily represent those of Shape Blue Ltd or related companies.
>>>> If you are not the intended recipient of this email, you must neither
>>>> take any action based upon its contents, nor copy or show it to anyone.
>>>> Please contact the sender if you believe you have received this email in
>>>> error. Shape Blue Ltd is a company incorporated in England & Wales.
>>>> ShapeBlue Services India LLP is a company incorporated in India and is
>>>> operated under license from Shape Blue Ltd. Shape Blue Brasil
>>>> Consultoria Ltda is a company incorporated in Brasil and is operated
>>>> under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
--
@shankerbalan
M: +91 98860 60539 | O: +91 (80) 67935867
shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade Centre, Bangalore - 560 055
CloudStack Bootcamp Training on 27/28 November, Bangalore
http://www.shapeblue.com/cloudstack-training/
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: Metadata URL location
Posted by Darren Shepherd <da...@gmail.com>.
I think it should be fine. Conceptually, 169.254/16 is not supposed
to be routed. The standard protocol is that you can randomly assign
yourself an IP as long as nobody responds to ARP. So the definition
of the subnet is that it's your local subnet. EC2 is funky though.
If you launch an amazon linux AMI it will come up with the following
route.
169.254.169.254 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
Now it really don't matter if that route exists because ARP is useless
in EC2. If you look at your ARP cache (or do arping), you'll notice
everything is FE:FF:FF:FF:FF:FF. So all packets leaving your VM go to
the same place and then EC2 magic ensues....
Darren
On Fri, Oct 25, 2013 at 7:51 AM, Chiradeep Vittal
<Ch...@citrix.com> wrote:
> Hmm. Windows may not work then? Needs testing
>
> On 10/24/13 5:04 PM, "Darren Shepherd" <da...@gmail.com> wrote:
>
>>Chiradeep,
>>
>>Linux distros set 169.254/16 route on the primary interface. It's just
>>there now. I'm not sure if that's because of ec2 or if it's always been
>>that way, but all modern distros will assign it if you have a standard
>>base install.
>>
>>In the VPC case I think we would have to use network namespaces to bind
>>the same IP to multiple NICs. You could bind the IP to loopback, but
>>then it won't ARP. So since linux distros already have the route, they
>>won't send to the gateway.
>>
>>Yes systemvm will need to be allowed to talk over 169.254, so that's
>>needs to change.
>>
>>Again, I said the reason I thought this wasn't done is because it's hard.
>> But still doable. I'd like to see us do this change. At least in the
>>KVM case, it would be real nice to be able to pickup the standard ubuntu
>>(or fedora) cloud qcow and have it run in CloudStack.
>>
>>Darren
>>
>>> On Oct 24, 2013, at 3:57 PM, Chiradeep Vittal
>>><Ch...@citrix.com> wrote:
>>>
>>> For the VPC virtual router case this would this have to be done on all
>>> guest interfaces?
>>> Could we alias localhost on the VR to 169.254.169.254?
>>> For shared networks, basic zone and networks where the VR is not the
>>> default gateway, we would have to send another (169.254.0.0/16) route in
>>> the DHCP response to point to the VR.
>>>
>>> Additionally in basic zone the anti-spoof filters in dom0 would have to
>>>be
>>> adjusted to let 169.254.169.254 addresses
>>>
>>>> On 10/24/13 8:51 AM, "Darren Shepherd" <da...@gmail.com>
>>>>wrote:
>>>>
>>>> So additionally you need to do
>>>>
>>>> ip addr add dev eth0 169.254.169.254/0
>>>>
>>>> On Thu, Oct 24, 2013 at 8:29 AM, Kris G. Lindgren
>>>><kl...@godaddy.com>
>>>> wrote:
>>>>> You would also need to supernet 169.254.169.254 on the virtual router
>>>>> (assigning it as 169.254.169.254 netmask 0.0.0.0 on eth0) that way it
>>>>> will
>>>>> still arp to servers that are calling it that have real ip addresses.
>>>>>
>>>>> Additionally we had some other iptables rules in there that would
>>>>>change
>>>>> the the ip address of the incoming request to metadata based upon the
>>>>> mac
>>>>> address that was hitting it. This was to prevent spoofing of another
>>>>> vm's
>>>>> IP and getting someone else's metadata (at least in our metadata
>>>>> implementation we keyed off of the VM IP calling into metadata). This
>>>>> also allowed a user to set whatever ipaddress they wanted, but as long
>>>>> as
>>>>> the mac address was the same and they still had a zeroconfig route on
>>>>> the
>>>>> VM, they still got only their metadata.
>>>>> ____________________________________________
>>>>>
>>>>> Kris Lindgren
>>>>> Senior Linux Systems Engineer
>>>>> GoDaddy, LLC.
>>>>>
>>>>>
>>>>> This email message and any attachment(s) hereto are intended for use
>>>>> only
>>>>> by its intended recipient(s) and may contain confidential information.
>>>>> If
>>>>> you have received this email in error, please immediately notify the
>>>>> sender and permanently delete the original and any copy of this
>>>>>message
>>>>> and its attachments.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 10/24/13 9:12 AM, "Darren Shepherd" <da...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> My guess, I don't really know, would be because its hard. The VR
>>>>>>uses
>>>>>> link local for the control network so 169.254/16 is bound to the
>>>>>>wrong
>>>>>> nic. To fix this you just need some ip routing magic in linux
>>>>>>(credit
>>>>>> goes to Kris Lindgren who showed me how to do this). Add the below
>>>>>>to
>>>>>> a file, substitute eth0 for the guest network nic, run "ip -b <FILE>"
>>>>>> The below effectively creates a routing table dedicated to the IP
>>>>>> 169.254.169.254 that sets it default route to go out the guest
>>>>>>network
>>>>>> nic.
>>>>>>
>>>>>> rule add from 169.254.169.254 table 70
>>>>>> rule add to 169.254.169.254 dev eth0 table 70
>>>>>> route flush table 70
>>>>>> route add default dev eth0 src 169.254.169.254 table 70
>>>>>> route flush cache
>>>>>>
>>>>>> Darren
>>>>>>
>>>>>> On Thu, Oct 24, 2013 at 6:10 AM, Shanker Balan
>>>>>> <sh...@shapeblue.com> wrote:
>>>>>>> Hi Guys,
>>>>>>>
>>>>>>> CloudStack metadata services are on the default gateway while on
>>>>>>>EC2,
>>>>>>> its at 169.254.169.254. Am curious to know why CloudStack does not
>>>>>>> use a link local address for meta data services.
>>>>>>>
>>>>>>> A search of the Wiki
>>>>>>>
>>>>>>>(https://cwiki.apache.org/confluence/dosearchsite.action?where=CLOUDS
>>>>>>>TA
>>>>>>> CK
>>>>>>>
>>>>>>>&tooltip=Type+ALL%3A+in+your+query+to+search+all+of+Confluence&spaceS
>>>>>>>ea
>>>>>>> rc
>>>>>>> h=true&queryString=metadata) didn¹t seem to list any doc related to
>>>>>>>the
>>>>>>> design of this service.
>>>>>>>
>>>>>>> TIA.
>>>>>>>
>>>>>>> --
>>>>>>> @shankerbalan
>>>>>>>
>>>>>>> M: +91 98860 60539 | O: +91 (80) 67935867
>>>>>>> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
>>>>>>> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade
>>>>>>> Centre, Bangalore - 560 055
>>>>>>>
>>>>>>> CloudStack Bootcamp Training on 27/28 November, Bangalore
>>>>>>> http://www.shapeblue.com/cloudstack-training/
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> This email and any attachments to it may be confidential and are
>>>>>>> intended solely for the use of the individual to whom it is
>>>>>>>addressed.
>>>>>>> Any views or opinions expressed are solely those of the author and
>>>>>>>do
>>>>>>> not necessarily represent those of Shape Blue Ltd or related
>>>>>>>companies.
>>>>>>> If you are not the intended recipient of this email, you must
>>>>>>>neither
>>>>>>> take any action based upon its contents, nor copy or show it to
>>>>>>>anyone.
>>>>>>> Please contact the sender if you believe you have received this
>>>>>>>email
>>>>>>> in
>>>>>>> error. Shape Blue Ltd is a company incorporated in England & Wales.
>>>>>>> ShapeBlue Services India LLP is a company incorporated in India and
>>>>>>>is
>>>>>>> operated under license from Shape Blue Ltd. Shape Blue Brasil
>>>>>>> Consultoria Ltda is a company incorporated in Brasil and is operated
>>>>>>> under license from Shape Blue Ltd. ShapeBlue is a registered
>>>>>>>trademark.
>>>
>
Re: Metadata URL location
Posted by Chiradeep Vittal <Ch...@citrix.com>.
Hmm. Windows may not work then? Needs testing
On 10/24/13 5:04 PM, "Darren Shepherd" <da...@gmail.com> wrote:
>Chiradeep,
>
>Linux distros set 169.254/16 route on the primary interface. It's just
>there now. I'm not sure if that's because of ec2 or if it's always been
>that way, but all modern distros will assign it if you have a standard
>base install.
>
>In the VPC case I think we would have to use network namespaces to bind
>the same IP to multiple NICs. You could bind the IP to loopback, but
>then it won't ARP. So since linux distros already have the route, they
>won't send to the gateway.
>
>Yes systemvm will need to be allowed to talk over 169.254, so that's
>needs to change.
>
>Again, I said the reason I thought this wasn't done is because it's hard.
> But still doable. I'd like to see us do this change. At least in the
>KVM case, it would be real nice to be able to pickup the standard ubuntu
>(or fedora) cloud qcow and have it run in CloudStack.
>
>Darren
>
>> On Oct 24, 2013, at 3:57 PM, Chiradeep Vittal
>><Ch...@citrix.com> wrote:
>>
>> For the VPC virtual router case this would this have to be done on all
>> guest interfaces?
>> Could we alias localhost on the VR to 169.254.169.254?
>> For shared networks, basic zone and networks where the VR is not the
>> default gateway, we would have to send another (169.254.0.0/16) route in
>> the DHCP response to point to the VR.
>>
>> Additionally in basic zone the anti-spoof filters in dom0 would have to
>>be
>> adjusted to let 169.254.169.254 addresses
>>
>>> On 10/24/13 8:51 AM, "Darren Shepherd" <da...@gmail.com>
>>>wrote:
>>>
>>> So additionally you need to do
>>>
>>> ip addr add dev eth0 169.254.169.254/0
>>>
>>> On Thu, Oct 24, 2013 at 8:29 AM, Kris G. Lindgren
>>><kl...@godaddy.com>
>>> wrote:
>>>> You would also need to supernet 169.254.169.254 on the virtual router
>>>> (assigning it as 169.254.169.254 netmask 0.0.0.0 on eth0) that way it
>>>> will
>>>> still arp to servers that are calling it that have real ip addresses.
>>>>
>>>> Additionally we had some other iptables rules in there that would
>>>>change
>>>> the the ip address of the incoming request to metadata based upon the
>>>> mac
>>>> address that was hitting it. This was to prevent spoofing of another
>>>> vm's
>>>> IP and getting someone else's metadata (at least in our metadata
>>>> implementation we keyed off of the VM IP calling into metadata). This
>>>> also allowed a user to set whatever ipaddress they wanted, but as long
>>>> as
>>>> the mac address was the same and they still had a zeroconfig route on
>>>> the
>>>> VM, they still got only their metadata.
>>>> ____________________________________________
>>>>
>>>> Kris Lindgren
>>>> Senior Linux Systems Engineer
>>>> GoDaddy, LLC.
>>>>
>>>>
>>>> This email message and any attachment(s) hereto are intended for use
>>>> only
>>>> by its intended recipient(s) and may contain confidential information.
>>>> If
>>>> you have received this email in error, please immediately notify the
>>>> sender and permanently delete the original and any copy of this
>>>>message
>>>> and its attachments.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 10/24/13 9:12 AM, "Darren Shepherd" <da...@gmail.com>
>>>> wrote:
>>>>
>>>>> My guess, I don't really know, would be because its hard. The VR
>>>>>uses
>>>>> link local for the control network so 169.254/16 is bound to the
>>>>>wrong
>>>>> nic. To fix this you just need some ip routing magic in linux
>>>>>(credit
>>>>> goes to Kris Lindgren who showed me how to do this). Add the below
>>>>>to
>>>>> a file, substitute eth0 for the guest network nic, run "ip -b <FILE>"
>>>>> The below effectively creates a routing table dedicated to the IP
>>>>> 169.254.169.254 that sets it default route to go out the guest
>>>>>network
>>>>> nic.
>>>>>
>>>>> rule add from 169.254.169.254 table 70
>>>>> rule add to 169.254.169.254 dev eth0 table 70
>>>>> route flush table 70
>>>>> route add default dev eth0 src 169.254.169.254 table 70
>>>>> route flush cache
>>>>>
>>>>> Darren
>>>>>
>>>>> On Thu, Oct 24, 2013 at 6:10 AM, Shanker Balan
>>>>> <sh...@shapeblue.com> wrote:
>>>>>> Hi Guys,
>>>>>>
>>>>>> CloudStack metadata services are on the default gateway while on
>>>>>>EC2,
>>>>>> its at 169.254.169.254. Am curious to know why CloudStack does not
>>>>>> use a link local address for meta data services.
>>>>>>
>>>>>> A search of the Wiki
>>>>>>
>>>>>>(https://cwiki.apache.org/confluence/dosearchsite.action?where=CLOUDS
>>>>>>TA
>>>>>> CK
>>>>>>
>>>>>>&tooltip=Type+ALL%3A+in+your+query+to+search+all+of+Confluence&spaceS
>>>>>>ea
>>>>>> rc
>>>>>> h=true&queryString=metadata) didn¹t seem to list any doc related to
>>>>>>the
>>>>>> design of this service.
>>>>>>
>>>>>> TIA.
>>>>>>
>>>>>> --
>>>>>> @shankerbalan
>>>>>>
>>>>>> M: +91 98860 60539 | O: +91 (80) 67935867
>>>>>> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
>>>>>> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade
>>>>>> Centre, Bangalore - 560 055
>>>>>>
>>>>>> CloudStack Bootcamp Training on 27/28 November, Bangalore
>>>>>> http://www.shapeblue.com/cloudstack-training/
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> This email and any attachments to it may be confidential and are
>>>>>> intended solely for the use of the individual to whom it is
>>>>>>addressed.
>>>>>> Any views or opinions expressed are solely those of the author and
>>>>>>do
>>>>>> not necessarily represent those of Shape Blue Ltd or related
>>>>>>companies.
>>>>>> If you are not the intended recipient of this email, you must
>>>>>>neither
>>>>>> take any action based upon its contents, nor copy or show it to
>>>>>>anyone.
>>>>>> Please contact the sender if you believe you have received this
>>>>>>email
>>>>>> in
>>>>>> error. Shape Blue Ltd is a company incorporated in England & Wales.
>>>>>> ShapeBlue Services India LLP is a company incorporated in India and
>>>>>>is
>>>>>> operated under license from Shape Blue Ltd. Shape Blue Brasil
>>>>>> Consultoria Ltda is a company incorporated in Brasil and is operated
>>>>>> under license from Shape Blue Ltd. ShapeBlue is a registered
>>>>>>trademark.
>>
Re: Metadata URL location
Posted by Darren Shepherd <da...@gmail.com>.
Chiradeep,
Linux distros set 169.254/16 route on the primary interface. It's just there now. I'm not sure if that's because of ec2 or if it's always been that way, but all modern distros will assign it if you have a standard base install.
In the VPC case I think we would have to use network namespaces to bind the same IP to multiple NICs. You could bind the IP to loopback, but then it won't ARP. So since linux distros already have the route, they won't send to the gateway.
Yes systemvm will need to be allowed to talk over 169.254, so that's needs to change.
Again, I said the reason I thought this wasn't done is because it's hard. But still doable. I'd like to see us do this change. At least in the KVM case, it would be real nice to be able to pickup the standard ubuntu (or fedora) cloud qcow and have it run in CloudStack.
Darren
> On Oct 24, 2013, at 3:57 PM, Chiradeep Vittal <Ch...@citrix.com> wrote:
>
> For the VPC virtual router case this would this have to be done on all
> guest interfaces?
> Could we alias localhost on the VR to 169.254.169.254?
> For shared networks, basic zone and networks where the VR is not the
> default gateway, we would have to send another (169.254.0.0/16) route in
> the DHCP response to point to the VR.
>
> Additionally in basic zone the anti-spoof filters in dom0 would have to be
> adjusted to let 169.254.169.254 addresses
>
>> On 10/24/13 8:51 AM, "Darren Shepherd" <da...@gmail.com> wrote:
>>
>> So additionally you need to do
>>
>> ip addr add dev eth0 169.254.169.254/0
>>
>> On Thu, Oct 24, 2013 at 8:29 AM, Kris G. Lindgren <kl...@godaddy.com>
>> wrote:
>>> You would also need to supernet 169.254.169.254 on the virtual router
>>> (assigning it as 169.254.169.254 netmask 0.0.0.0 on eth0) that way it
>>> will
>>> still arp to servers that are calling it that have real ip addresses.
>>>
>>> Additionally we had some other iptables rules in there that would change
>>> the the ip address of the incoming request to metadata based upon the
>>> mac
>>> address that was hitting it. This was to prevent spoofing of another
>>> vm's
>>> IP and getting someone else's metadata (at least in our metadata
>>> implementation we keyed off of the VM IP calling into metadata). This
>>> also allowed a user to set whatever ipaddress they wanted, but as long
>>> as
>>> the mac address was the same and they still had a zeroconfig route on
>>> the
>>> VM, they still got only their metadata.
>>> ____________________________________________
>>>
>>> Kris Lindgren
>>> Senior Linux Systems Engineer
>>> GoDaddy, LLC.
>>>
>>>
>>> This email message and any attachment(s) hereto are intended for use
>>> only
>>> by its intended recipient(s) and may contain confidential information.
>>> If
>>> you have received this email in error, please immediately notify the
>>> sender and permanently delete the original and any copy of this message
>>> and its attachments.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 10/24/13 9:12 AM, "Darren Shepherd" <da...@gmail.com>
>>> wrote:
>>>
>>>> My guess, I don't really know, would be because its hard. The VR uses
>>>> link local for the control network so 169.254/16 is bound to the wrong
>>>> nic. To fix this you just need some ip routing magic in linux (credit
>>>> goes to Kris Lindgren who showed me how to do this). Add the below to
>>>> a file, substitute eth0 for the guest network nic, run "ip -b <FILE>"
>>>> The below effectively creates a routing table dedicated to the IP
>>>> 169.254.169.254 that sets it default route to go out the guest network
>>>> nic.
>>>>
>>>> rule add from 169.254.169.254 table 70
>>>> rule add to 169.254.169.254 dev eth0 table 70
>>>> route flush table 70
>>>> route add default dev eth0 src 169.254.169.254 table 70
>>>> route flush cache
>>>>
>>>> Darren
>>>>
>>>> On Thu, Oct 24, 2013 at 6:10 AM, Shanker Balan
>>>> <sh...@shapeblue.com> wrote:
>>>>> Hi Guys,
>>>>>
>>>>> CloudStack metadata services are on the default gateway while on EC2,
>>>>> its at 169.254.169.254. Am curious to know why CloudStack does not
>>>>> use a link local address for meta data services.
>>>>>
>>>>> A search of the Wiki
>>>>> (https://cwiki.apache.org/confluence/dosearchsite.action?where=CLOUDSTA
>>>>> CK
>>>>> &tooltip=Type+ALL%3A+in+your+query+to+search+all+of+Confluence&spaceSea
>>>>> rc
>>>>> h=true&queryString=metadata) didn¹t seem to list any doc related to the
>>>>> design of this service.
>>>>>
>>>>> TIA.
>>>>>
>>>>> --
>>>>> @shankerbalan
>>>>>
>>>>> M: +91 98860 60539 | O: +91 (80) 67935867
>>>>> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
>>>>> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade
>>>>> Centre, Bangalore - 560 055
>>>>>
>>>>> CloudStack Bootcamp Training on 27/28 November, Bangalore
>>>>> http://www.shapeblue.com/cloudstack-training/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> This email and any attachments to it may be confidential and are
>>>>> intended solely for the use of the individual to whom it is addressed.
>>>>> Any views or opinions expressed are solely those of the author and do
>>>>> not necessarily represent those of Shape Blue Ltd or related companies.
>>>>> If you are not the intended recipient of this email, you must neither
>>>>> take any action based upon its contents, nor copy or show it to anyone.
>>>>> Please contact the sender if you believe you have received this email
>>>>> in
>>>>> error. Shape Blue Ltd is a company incorporated in England & Wales.
>>>>> ShapeBlue Services India LLP is a company incorporated in India and is
>>>>> operated under license from Shape Blue Ltd. Shape Blue Brasil
>>>>> Consultoria Ltda is a company incorporated in Brasil and is operated
>>>>> under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>
Re: Metadata URL location
Posted by Chiradeep Vittal <Ch...@citrix.com>.
For the VPC virtual router case this would this have to be done on all
guest interfaces?
Could we alias localhost on the VR to 169.254.169.254?
For shared networks, basic zone and networks where the VR is not the
default gateway, we would have to send another (169.254.0.0/16) route in
the DHCP response to point to the VR.
Additionally in basic zone the anti-spoof filters in dom0 would have to be
adjusted to let 169.254.169.254 addresses
On 10/24/13 8:51 AM, "Darren Shepherd" <da...@gmail.com> wrote:
>So additionally you need to do
>
>ip addr add dev eth0 169.254.169.254/0
>
>On Thu, Oct 24, 2013 at 8:29 AM, Kris G. Lindgren <kl...@godaddy.com>
>wrote:
>> You would also need to supernet 169.254.169.254 on the virtual router
>> (assigning it as 169.254.169.254 netmask 0.0.0.0 on eth0) that way it
>>will
>> still arp to servers that are calling it that have real ip addresses.
>>
>> Additionally we had some other iptables rules in there that would change
>> the the ip address of the incoming request to metadata based upon the
>>mac
>> address that was hitting it. This was to prevent spoofing of another
>>vm's
>> IP and getting someone else's metadata (at least in our metadata
>> implementation we keyed off of the VM IP calling into metadata). This
>> also allowed a user to set whatever ipaddress they wanted, but as long
>>as
>> the mac address was the same and they still had a zeroconfig route on
>>the
>> VM, they still got only their metadata.
>> ____________________________________________
>>
>> Kris Lindgren
>> Senior Linux Systems Engineer
>> GoDaddy, LLC.
>>
>>
>> This email message and any attachment(s) hereto are intended for use
>>only
>> by its intended recipient(s) and may contain confidential information.
>>If
>> you have received this email in error, please immediately notify the
>> sender and permanently delete the original and any copy of this message
>> and its attachments.
>>
>>
>>
>>
>>
>>
>>
>> On 10/24/13 9:12 AM, "Darren Shepherd" <da...@gmail.com>
>>wrote:
>>
>>>My guess, I don't really know, would be because its hard. The VR uses
>>>link local for the control network so 169.254/16 is bound to the wrong
>>>nic. To fix this you just need some ip routing magic in linux (credit
>>>goes to Kris Lindgren who showed me how to do this). Add the below to
>>>a file, substitute eth0 for the guest network nic, run "ip -b <FILE>"
>>>The below effectively creates a routing table dedicated to the IP
>>>169.254.169.254 that sets it default route to go out the guest network
>>>nic.
>>>
>>>rule add from 169.254.169.254 table 70
>>>rule add to 169.254.169.254 dev eth0 table 70
>>>route flush table 70
>>>route add default dev eth0 src 169.254.169.254 table 70
>>>route flush cache
>>>
>>>Darren
>>>
>>>On Thu, Oct 24, 2013 at 6:10 AM, Shanker Balan
>>><sh...@shapeblue.com> wrote:
>>>> Hi Guys,
>>>>
>>>> CloudStack metadata services are on the default gateway while on EC2,
>>>> its at 169.254.169.254. Am curious to know why CloudStack does not
>>>> use a link local address for meta data services.
>>>>
>>>> A search of the Wiki
>>>>(https://cwiki.apache.org/confluence/dosearchsite.action?where=CLOUDSTA
>>>>CK
>>>>&tooltip=Type+ALL%3A+in+your+query+to+search+all+of+Confluence&spaceSea
>>>>rc
>>>>h=true&queryString=metadata) didn¹t seem to list any doc related to the
>>>>design of this service.
>>>>
>>>> TIA.
>>>>
>>>> --
>>>> @shankerbalan
>>>>
>>>> M: +91 98860 60539 | O: +91 (80) 67935867
>>>> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
>>>> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade
>>>>Centre, Bangalore - 560 055
>>>>
>>>> CloudStack Bootcamp Training on 27/28 November, Bangalore
>>>> http://www.shapeblue.com/cloudstack-training/
>>>>
>>>>
>>>>
>>>>
>>>> This email and any attachments to it may be confidential and are
>>>>intended solely for the use of the individual to whom it is addressed.
>>>>Any views or opinions expressed are solely those of the author and do
>>>>not necessarily represent those of Shape Blue Ltd or related companies.
>>>>If you are not the intended recipient of this email, you must neither
>>>>take any action based upon its contents, nor copy or show it to anyone.
>>>>Please contact the sender if you believe you have received this email
>>>>in
>>>>error. Shape Blue Ltd is a company incorporated in England & Wales.
>>>>ShapeBlue Services India LLP is a company incorporated in India and is
>>>>operated under license from Shape Blue Ltd. Shape Blue Brasil
>>>>Consultoria Ltda is a company incorporated in Brasil and is operated
>>>>under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>>
Re: Metadata URL location
Posted by Darren Shepherd <da...@gmail.com>.
So additionally you need to do
ip addr add dev eth0 169.254.169.254/0
On Thu, Oct 24, 2013 at 8:29 AM, Kris G. Lindgren <kl...@godaddy.com> wrote:
> You would also need to supernet 169.254.169.254 on the virtual router
> (assigning it as 169.254.169.254 netmask 0.0.0.0 on eth0) that way it will
> still arp to servers that are calling it that have real ip addresses.
>
> Additionally we had some other iptables rules in there that would change
> the the ip address of the incoming request to metadata based upon the mac
> address that was hitting it. This was to prevent spoofing of another vm's
> IP and getting someone else's metadata (at least in our metadata
> implementation we keyed off of the VM IP calling into metadata). This
> also allowed a user to set whatever ipaddress they wanted, but as long as
> the mac address was the same and they still had a zeroconfig route on the
> VM, they still got only their metadata.
> ____________________________________________
>
> Kris Lindgren
> Senior Linux Systems Engineer
> GoDaddy, LLC.
>
>
> This email message and any attachment(s) hereto are intended for use only
> by its intended recipient(s) and may contain confidential information. If
> you have received this email in error, please immediately notify the
> sender and permanently delete the original and any copy of this message
> and its attachments.
>
>
>
>
>
>
>
> On 10/24/13 9:12 AM, "Darren Shepherd" <da...@gmail.com> wrote:
>
>>My guess, I don't really know, would be because its hard. The VR uses
>>link local for the control network so 169.254/16 is bound to the wrong
>>nic. To fix this you just need some ip routing magic in linux (credit
>>goes to Kris Lindgren who showed me how to do this). Add the below to
>>a file, substitute eth0 for the guest network nic, run "ip -b <FILE>"
>>The below effectively creates a routing table dedicated to the IP
>>169.254.169.254 that sets it default route to go out the guest network
>>nic.
>>
>>rule add from 169.254.169.254 table 70
>>rule add to 169.254.169.254 dev eth0 table 70
>>route flush table 70
>>route add default dev eth0 src 169.254.169.254 table 70
>>route flush cache
>>
>>Darren
>>
>>On Thu, Oct 24, 2013 at 6:10 AM, Shanker Balan
>><sh...@shapeblue.com> wrote:
>>> Hi Guys,
>>>
>>> CloudStack metadata services are on the default gateway while on EC2,
>>> its at 169.254.169.254. Am curious to know why CloudStack does not
>>> use a link local address for meta data services.
>>>
>>> A search of the Wiki
>>>(https://cwiki.apache.org/confluence/dosearchsite.action?where=CLOUDSTACK
>>>&tooltip=Type+ALL%3A+in+your+query+to+search+all+of+Confluence&spaceSearc
>>>h=true&queryString=metadata) didn¹t seem to list any doc related to the
>>>design of this service.
>>>
>>> TIA.
>>>
>>> --
>>> @shankerbalan
>>>
>>> M: +91 98860 60539 | O: +91 (80) 67935867
>>> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
>>> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade
>>>Centre, Bangalore - 560 055
>>>
>>> CloudStack Bootcamp Training on 27/28 November, Bangalore
>>> http://www.shapeblue.com/cloudstack-training/
>>>
>>>
>>>
>>>
>>> This email and any attachments to it may be confidential and are
>>>intended solely for the use of the individual to whom it is addressed.
>>>Any views or opinions expressed are solely those of the author and do
>>>not necessarily represent those of Shape Blue Ltd or related companies.
>>>If you are not the intended recipient of this email, you must neither
>>>take any action based upon its contents, nor copy or show it to anyone.
>>>Please contact the sender if you believe you have received this email in
>>>error. Shape Blue Ltd is a company incorporated in England & Wales.
>>>ShapeBlue Services India LLP is a company incorporated in India and is
>>>operated under license from Shape Blue Ltd. Shape Blue Brasil
>>>Consultoria Ltda is a company incorporated in Brasil and is operated
>>>under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>
Re: Metadata URL location
Posted by Darren Shepherd <da...@gmail.com>.
Alex,
I don't think that's correct. The instructions that I gave in the
earlier email was changes to the VR to serve metadata from the VR.
Regardless of hypervisor, it should work.
Darren
On Thu, Oct 24, 2013 at 10:54 AM, Alex Huang <Al...@citrix.com> wrote:
> In order to use an link local address inside the end user vm, that metadata service must be setup on every hypervisor's dom0 or it has to be proxied out of the dom0. That's not doable for VmWare. Instead, CloudStack uses VR to serve the data, which works for all three hypervisors.
>
> --Alex
>
>> -----Original Message-----
>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>> Sent: Thursday, October 24, 2013 8:13 AM
>> To: dev@cloudstack.apache.org
>> Cc: klindgren@godaddy.com
>> Subject: Re: Metadata URL location
>>
>> My guess, I don't really know, would be because its hard. The VR uses link
>> local for the control network so 169.254/16 is bound to the wrong nic. To fix
>> this you just need some ip routing magic in linux (credit goes to Kris Lindgren
>> who showed me how to do this). Add the below to a file, substitute eth0 for
>> the guest network nic, run "ip -b <FILE>"
>> The below effectively creates a routing table dedicated to the IP
>> 169.254.169.254 that sets it default route to go out the guest network nic.
>>
>> rule add from 169.254.169.254 table 70
>> rule add to 169.254.169.254 dev eth0 table 70 route flush table 70 route add
>> default dev eth0 src 169.254.169.254 table 70 route flush cache
>>
>> Darren
>>
>> On Thu, Oct 24, 2013 at 6:10 AM, Shanker Balan
>> <sh...@shapeblue.com> wrote:
>> > Hi Guys,
>> >
>> > CloudStack metadata services are on the default gateway while on EC2,
>> > its at 169.254.169.254. Am curious to know why CloudStack does not use
>> > a link local address for meta data services.
>> >
>> > A search of the Wiki
>> (https://cwiki.apache.org/confluence/dosearchsite.action?where=CLOUDST
>> ACK&tooltip=Type+ALL%3A+in+your+query+to+search+all+of+Confluence&
>> spaceSearch=true&queryString=metadata) didn't seem to list any doc
>> related to the design of this service.
>> >
>> > TIA.
>> >
>> > --
>> > @shankerbalan
>> >
>> > M: +91 98860 60539 | O: +91 (80) 67935867 shanker.balan@shapeblue.com
>> > | www.shapeblue.com | Twitter:@shapeblue ShapeBlue Services India LLP,
>> > 22nd floor, Unit 2201A, World Trade Centre, Bangalore - 560 055
>> >
>> > CloudStack Bootcamp Training on 27/28 November, Bangalore
>> > http://www.shapeblue.com/cloudstack-training/
>> >
>> >
>> >
>> >
>> > This email and any attachments to it may be confidential and are intended
>> solely for the use of the individual to whom it is addressed. Any views or
>> opinions expressed are solely those of the author and do not necessarily
>> represent those of Shape Blue Ltd or related companies. If you are not the
>> intended recipient of this email, you must neither take any action based
>> upon its contents, nor copy or show it to anyone. Please contact the sender if
>> you believe you have received this email in error. Shape Blue Ltd is a
>> company incorporated in England & Wales. ShapeBlue Services India LLP is a
>> company incorporated in India and is operated under license from Shape
>> Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
>> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a
>> registered trademark.
RE: Metadata URL location
Posted by Alex Huang <Al...@citrix.com>.
In order to use an link local address inside the end user vm, that metadata service must be setup on every hypervisor's dom0 or it has to be proxied out of the dom0. That's not doable for VmWare. Instead, CloudStack uses VR to serve the data, which works for all three hypervisors.
--Alex
> -----Original Message-----
> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
> Sent: Thursday, October 24, 2013 8:13 AM
> To: dev@cloudstack.apache.org
> Cc: klindgren@godaddy.com
> Subject: Re: Metadata URL location
>
> My guess, I don't really know, would be because its hard. The VR uses link
> local for the control network so 169.254/16 is bound to the wrong nic. To fix
> this you just need some ip routing magic in linux (credit goes to Kris Lindgren
> who showed me how to do this). Add the below to a file, substitute eth0 for
> the guest network nic, run "ip -b <FILE>"
> The below effectively creates a routing table dedicated to the IP
> 169.254.169.254 that sets it default route to go out the guest network nic.
>
> rule add from 169.254.169.254 table 70
> rule add to 169.254.169.254 dev eth0 table 70 route flush table 70 route add
> default dev eth0 src 169.254.169.254 table 70 route flush cache
>
> Darren
>
> On Thu, Oct 24, 2013 at 6:10 AM, Shanker Balan
> <sh...@shapeblue.com> wrote:
> > Hi Guys,
> >
> > CloudStack metadata services are on the default gateway while on EC2,
> > its at 169.254.169.254. Am curious to know why CloudStack does not use
> > a link local address for meta data services.
> >
> > A search of the Wiki
> (https://cwiki.apache.org/confluence/dosearchsite.action?where=CLOUDST
> ACK&tooltip=Type+ALL%3A+in+your+query+to+search+all+of+Confluence&
> spaceSearch=true&queryString=metadata) didn't seem to list any doc
> related to the design of this service.
> >
> > TIA.
> >
> > --
> > @shankerbalan
> >
> > M: +91 98860 60539 | O: +91 (80) 67935867 shanker.balan@shapeblue.com
> > | www.shapeblue.com | Twitter:@shapeblue ShapeBlue Services India LLP,
> > 22nd floor, Unit 2201A, World Trade Centre, Bangalore - 560 055
> >
> > CloudStack Bootcamp Training on 27/28 November, Bangalore
> > http://www.shapeblue.com/cloudstack-training/
> >
> >
> >
> >
> > This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender if
> you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape
> Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a
> registered trademark.
Re: Metadata URL location
Posted by "Kris G. Lindgren" <kl...@godaddy.com>.
You would also need to supernet 169.254.169.254 on the virtual router
(assigning it as 169.254.169.254 netmask 0.0.0.0 on eth0) that way it will
still arp to servers that are calling it that have real ip addresses.
Additionally we had some other iptables rules in there that would change
the the ip address of the incoming request to metadata based upon the mac
address that was hitting it. This was to prevent spoofing of another vm's
IP and getting someone else's metadata (at least in our metadata
implementation we keyed off of the VM IP calling into metadata). This
also allowed a user to set whatever ipaddress they wanted, but as long as
the mac address was the same and they still had a zeroconfig route on the
VM, they still got only their metadata.
____________________________________________
Kris Lindgren
Senior Linux Systems Engineer
GoDaddy, LLC.
This email message and any attachment(s) hereto are intended for use only
by its intended recipient(s) and may contain confidential information. If
you have received this email in error, please immediately notify the
sender and permanently delete the original and any copy of this message
and its attachments.
On 10/24/13 9:12 AM, "Darren Shepherd" <da...@gmail.com> wrote:
>My guess, I don't really know, would be because its hard. The VR uses
>link local for the control network so 169.254/16 is bound to the wrong
>nic. To fix this you just need some ip routing magic in linux (credit
>goes to Kris Lindgren who showed me how to do this). Add the below to
>a file, substitute eth0 for the guest network nic, run "ip -b <FILE>"
>The below effectively creates a routing table dedicated to the IP
>169.254.169.254 that sets it default route to go out the guest network
>nic.
>
>rule add from 169.254.169.254 table 70
>rule add to 169.254.169.254 dev eth0 table 70
>route flush table 70
>route add default dev eth0 src 169.254.169.254 table 70
>route flush cache
>
>Darren
>
>On Thu, Oct 24, 2013 at 6:10 AM, Shanker Balan
><sh...@shapeblue.com> wrote:
>> Hi Guys,
>>
>> CloudStack metadata services are on the default gateway while on EC2,
>> its at 169.254.169.254. Am curious to know why CloudStack does not
>> use a link local address for meta data services.
>>
>> A search of the Wiki
>>(https://cwiki.apache.org/confluence/dosearchsite.action?where=CLOUDSTACK
>>&tooltip=Type+ALL%3A+in+your+query+to+search+all+of+Confluence&spaceSearc
>>h=true&queryString=metadata) didn¹t seem to list any doc related to the
>>design of this service.
>>
>> TIA.
>>
>> --
>> @shankerbalan
>>
>> M: +91 98860 60539 | O: +91 (80) 67935867
>> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
>> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade
>>Centre, Bangalore - 560 055
>>
>> CloudStack Bootcamp Training on 27/28 November, Bangalore
>> http://www.shapeblue.com/cloudstack-training/
>>
>>
>>
>>
>> This email and any attachments to it may be confidential and are
>>intended solely for the use of the individual to whom it is addressed.
>>Any views or opinions expressed are solely those of the author and do
>>not necessarily represent those of Shape Blue Ltd or related companies.
>>If you are not the intended recipient of this email, you must neither
>>take any action based upon its contents, nor copy or show it to anyone.
>>Please contact the sender if you believe you have received this email in
>>error. Shape Blue Ltd is a company incorporated in England & Wales.
>>ShapeBlue Services India LLP is a company incorporated in India and is
>>operated under license from Shape Blue Ltd. Shape Blue Brasil
>>Consultoria Ltda is a company incorporated in Brasil and is operated
>>under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: Metadata URL location
Posted by Darren Shepherd <da...@gmail.com>.
My guess, I don't really know, would be because its hard. The VR uses
link local for the control network so 169.254/16 is bound to the wrong
nic. To fix this you just need some ip routing magic in linux (credit
goes to Kris Lindgren who showed me how to do this). Add the below to
a file, substitute eth0 for the guest network nic, run "ip -b <FILE>"
The below effectively creates a routing table dedicated to the IP
169.254.169.254 that sets it default route to go out the guest network
nic.
rule add from 169.254.169.254 table 70
rule add to 169.254.169.254 dev eth0 table 70
route flush table 70
route add default dev eth0 src 169.254.169.254 table 70
route flush cache
Darren
On Thu, Oct 24, 2013 at 6:10 AM, Shanker Balan
<sh...@shapeblue.com> wrote:
> Hi Guys,
>
> CloudStack metadata services are on the default gateway while on EC2,
> its at 169.254.169.254. Am curious to know why CloudStack does not
> use a link local address for meta data services.
>
> A search of the Wiki (https://cwiki.apache.org/confluence/dosearchsite.action?where=CLOUDSTACK&tooltip=Type+ALL%3A+in+your+query+to+search+all+of+Confluence&spaceSearch=true&queryString=metadata) didn’t seem to list any doc related to the design of this service.
>
> TIA.
>
> --
> @shankerbalan
>
> M: +91 98860 60539 | O: +91 (80) 67935867
> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade Centre, Bangalore - 560 055
>
> CloudStack Bootcamp Training on 27/28 November, Bangalore
> http://www.shapeblue.com/cloudstack-training/
>
>
>
>
> This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.