You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by John Burwell <jb...@basho.com> on 2012/10/04 04:11:14 UTC
SSVM Network Configuration
All,
How do you configure networking to permit the SSVM to connect to the public Internet or another internal network? I have been trying to understand the network configuration from the documentation, but am missing something in my configuration attempt.
Thank you for your assistance,
-John
Re: SSVM Network Configuration
Posted by "Kelceydamage@bbits" <ke...@bbits.ca>.
I'll start with the first question about storage network not located locally.
When you set up the storage network, make sure the zone gateway has a route to the S3 or whichever storage system.
The SSVM will create a static route via the gateway to the storage network.
I might have this a little mixed up, does anyone care to comment?
Also NAT works via you router/gateway you can set up ip address translation to let a remote ip communicate through a router to reach your internal network.
Sent from my iPhone
On Oct 4, 2012, at 9:16 AM, John Burwell <jb...@basho.com> wrote:
> Kelcey,
>
> I am a bit confused about how secstorage.allowed.internal.sites is used which stems to lack of knowledge regarding the devcloud network configuration. Also, is there documentation available for setting up such a NAT?
>
> As a point of clarification to my original question, I am working in the devcloud environment (using the OVA downloaded from the wiki) where I need to get the SSVM to connect to S3 or to a local VirtualBox VM running an S3-compatible object store. Thus far, I have been unable to get devcloud to bring up a second NIC on a host-only network. I have attempted to setup an advanced network configuration as follows:
>
> Physical Network with VLAN isolation method
> Management Server: 10.0.2.15 -> Gateway: 10.0.2.2
> Storage Network: 10.0.2.50-10.0.2.59 -> Gateway 10.0.2.2 on VLAN0
> Management Network: 10.0.2.200-10.0.2.220 -> Gateway 10.0.2.2
> Public Network: 10.0.2.100-10.0.2.199 -> VLAN0
>
> Obviously, my network configuration is incorrect, but I have the reached the limits of my CloudStack and Xen knowledge to identify the problem(s).
>
> Given this information, what is the best way to give the SSVM access to the Internet and/or a VirtualBox host-only network?
>
> Thank you for your help,
> -John
>
> On Oct 3, 2012, at 10:39 PM, "Kelceydamage@bbits" <ke...@bbits.ca> wrote:
>
>> The the secondary storage VM can be NATed to from any network router, however the console proxy does not work over NAT.
>>
>> Sent from my iPhone
>>
>> On Oct 3, 2012, at 7:32 PM, Edison Su <Ed...@citrix.com> wrote:
>>
>>> System vm will have 4 nics, eth2 is on the public network, eth1 is the private(mgt) network.
>>> The IP address of eth2 is got from pod configuration: in one of IP address range ["startip", "endip"] in createPod API.
>>> The IP address of eth1 is got from guest network, if it's basic network mode, this IP range is configured by createVlanIpRanges API
>>> SSVM will connect to mgt server through eth1(mgt server's ip address is configured to route through eth1), and download template from eth2.
>>> What's your specific issue about network configuration?
>>>
>>>> -----Original Message-----
>>>> From: John Burwell [mailto:jburwell@basho.com]
>>>> Sent: Wednesday, October 03, 2012 7:11 PM
>>>> To: cloudstack-dev@incubator.apache.org
>>>> Subject: SSVM Network Configuration
>>>>
>>>> All,
>>>>
>>>> How do you configure networking to permit the SSVM to connect to the
>>>> public Internet or another internal network? I have been trying to
>>>> understand the network configuration from the documentation, but am
>>>> missing something in my configuration attempt.
>>>>
>>>> Thank you for your assistance,
>>>> -John
>
RE: SSVM Network Configuration
Posted by "Kelcey Damage (BBITS)" <ke...@bbits.ca>.
Awesome, I learnt something here too.
Glad it's working for you now.
KELCEY DAMAGE
Infrastructure Systems Architect
www.backbonetechnology.com
-------------------------------------------------------------------------
kelcey@bbits.ca
address: 55 East 7th Ave, Vancouver, BC, V5T 1M4
tel: +1 604 713 8560 ext:114
fax: +1 604 605 0964
skype: kelcey.damage
-----Original Message-----
From: John Burwell [mailto:jburwell@basho.com]
Sent: Thursday, October 04, 2012 2:39 PM
To: cloudstack-dev@incubator.apache.org
Subject: Re: SSVM Network Configuration
Ahmad,
You were correct on the rp_filter issue. Once disabled, the SSVM was able to connect outbound to S3, as well as, any host reachable from devcloud. I noticed that rp_filter is disabled in sysctl.conf yet it is somehow being enabled at runtime. Is this behavior intended?
Thanks,
-John
On Oct 4, 2012, at 1:07 PM, Ahmad Emneina <Ah...@citrix.com> wrote:
> On 10/4/12 9:16 AM, "John Burwell" <jb...@basho.com> wrote:
>
>> Kelcey,
>>
>> I am a bit confused about how secstorage.allowed.internal.sites is
>> used which stems to lack of knowledge regarding the devcloud network
>> configuration. Also, is there documentation available for setting up
>> such a NAT?
>>
>> As a point of clarification to my original question, I am working in
>> the devcloud environment (using the OVA downloaded from the wiki)
>> where I need to get the SSVM to connect to S3 or to a local
>> VirtualBox VM running an S3-compatible object store. Thus far, I
>> have been unable to get devcloud to bring up a second NIC on a
>> host-only network. I have attempted to setup an advanced network configuration as follows:
>>
>> Physical Network with VLAN isolation method Management Server:
>> 10.0.2.15 -> Gateway: 10.0.2.2 Storage Network: 10.0.2.50-10.0.2.59
>> -> Gateway 10.0.2.2 on VLAN0 Management Network:
>> 10.0.2.200-10.0.2.220 -> Gateway 10.0.2.2 Public Network:
>> 10.0.2.100-10.0.2.199 -> VLAN0
>
> The issue that gets created here is you get system vm's that are
> multi-homed. Your system vm's get a nic (leg) on each network... But
> that network is one and the same. Why this is an issue is rp_filter is
> enabled by default on the system vm's, message comes in on one of
> those nics, but it's default route out is another nic... Thus blocking the response.
>
> Ideally you'd use a basic zone for this kind of configuration, or else
> you'll end up having to log into the system vm's every time a new one
> is spawned and disabling rp_filter for the nics. You might want to
> test this, by logging in and disabling rp_filter on the nics and see
> if things start working as expected.
>
>
>>
>> Obviously, my network configuration is incorrect, but I have the
>> reached the limits of my CloudStack and Xen knowledge to identify the problem(s).
>>
>> Given this information, what is the best way to give the SSVM access
>> to the Internet and/or a VirtualBox host-only network?
>>
>> Thank you for your help,
>> -John
>>
>> On Oct 3, 2012, at 10:39 PM, "Kelceydamage@bbits" <ke...@bbits.ca> wrote:
>>
>>> The the secondary storage VM can be NATed to from any network
>>> router, however the console proxy does not work over NAT.
>>>
>>> Sent from my iPhone
>>>
>>> On Oct 3, 2012, at 7:32 PM, Edison Su <Ed...@citrix.com> wrote:
>>>
>>>> System vm will have 4 nics, eth2 is on the public network, eth1 is
>>>> the
>>>> private(mgt) network.
>>>> The IP address of eth2 is got from pod configuration: in one of IP
>>>> address range ["startip", "endip"] in createPod API.
>>>> The IP address of eth1 is got from guest network, if it's basic
>>>> network mode, this IP range is configured by createVlanIpRanges API
>>>> SSVM will connect to mgt server through eth1(mgt server's ip
>>>> address is configured to route through eth1), and download template from eth2.
>>>> What's your specific issue about network configuration?
>>>>
>>>>> -----Original Message-----
>>>>> From: John Burwell [mailto:jburwell@basho.com]
>>>>> Sent: Wednesday, October 03, 2012 7:11 PM
>>>>> To: cloudstack-dev@incubator.apache.org
>>>>> Subject: SSVM Network Configuration
>>>>>
>>>>> All,
>>>>>
>>>>> How do you configure networking to permit the SSVM to connect to
>>>>> the public Internet or another internal network? I have been
>>>>> trying to understand the network configuration from the
>>>>> documentation, but am missing something in my configuration attempt.
>>>>>
>>>>> Thank you for your assistance,
>>>>> -John
>>
>>
>
>
> --
> Æ
>
>
>
Re: SSVM Network Configuration
Posted by John Burwell <jb...@basho.com>.
Chiradeep,
I have made the change in my feature branch. I will submit it as part of larger patch, S3 secondary storage, that I plan submit later this week or early next.
Thanks,
-John
On Oct 29, 2012, at 4:31 PM, Chiradeep Vittal <Ch...@citrix.com> wrote:
> Yes, 0.
> I think that the discrepancy is because sysctl.conf is modified too late
> in the game.
> 0. SSVM boots for first time
> 1. cloud-early-config figures out that scripts need to be patched
> 2. Scripts are patched and reboot is initiated. Sysctl is not modified yet
> 3. SSVM boots, steps 1-2 do not take place, figures out it is an SSVM
> 4. cloud-early-config modifies /etc/sysctl.conf, but DOES NOT execute
> sysctl -w
>
> Hence the runtime value of rp_filter remains 1 while the config file says
> "0".
> --
> Chiradeep
>
> On 10/29/12 1:00 PM, "John Burwell" <jb...@basho.com> wrote:
>
>> Chiradeep,
>>
>> Currently, net.ipv4.conf.default.rp_filer is set to 1 in
>> systemvm/debian/config/etc/sysctl.conf. Should it be modified to be 0?
>>
>> Thanks,
>> -John
>>
>> On Oct 4, 2012, at 6:09 PM, Chiradeep Vittal
>> <Ch...@citrix.com> wrote:
>>
>>> It is disabled in sysctl.conf, not sure how it gets re-enabled. See
>>> patches/systemvm/debian/config/etc/init.d/cloud-early-config (function
>>> disable_rpfilter).
>>> Perhaps it is interface-specific rather than "all".
>>>
>>> On 10/4/12 2:39 PM, "John Burwell" <jb...@basho.com> wrote:
>>>
>>>> Ahmad,
>>>>
>>>> You were correct on the rp_filter issue. Once disabled, the SSVM was
>>>> able to connect outbound to S3, as well as, any host reachable from
>>>> devcloud. I noticed that rp_filter is disabled in sysctl.conf yet it
>>>> is
>>>> somehow being enabled at runtime. Is this behavior intended?
>>>>
>>>> Thanks,
>>>> -John
>>>>
>>>> On Oct 4, 2012, at 1:07 PM, Ahmad Emneina <Ah...@citrix.com>
>>>> wrote:
>>>>
>>>>> On 10/4/12 9:16 AM, "John Burwell" <jb...@basho.com> wrote:
>>>>>
>>>>>> Kelcey,
>>>>>>
>>>>>> I am a bit confused about how secstorage.allowed.internal.sites is
>>>>>> used
>>>>>> which stems to lack of knowledge regarding the devcloud network
>>>>>> configuration. Also, is there documentation available for setting up
>>>>>> such a NAT?
>>>>>>
>>>>>> As a point of clarification to my original question, I am working in
>>>>>> the
>>>>>> devcloud environment (using the OVA downloaded from the wiki) where I
>>>>>> need to get the SSVM to connect to S3 or to a local VirtualBox VM
>>>>>> running
>>>>>> an S3-compatible object store. Thus far, I have been unable to get
>>>>>> devcloud to bring up a second NIC on a host-only network. I have
>>>>>> attempted to setup an advanced network configuration as follows:
>>>>>>
>>>>>> Physical Network with VLAN isolation method
>>>>>> Management Server: 10.0.2.15 -> Gateway: 10.0.2.2
>>>>>> Storage Network: 10.0.2.50-10.0.2.59 -> Gateway 10.0.2.2 on VLAN0
>>>>>> Management Network: 10.0.2.200-10.0.2.220 -> Gateway 10.0.2.2
>>>>>> Public Network: 10.0.2.100-10.0.2.199 -> VLAN0
>>>>>
>>>>> The issue that gets created here is you get system vm's that are
>>>>> multi-homed. Your system vm's get a nic (leg) on each network... But
>>>>> that
>>>>> network is one and the same. Why this is an issue is rp_filter is
>>>>> enabled
>>>>> by default on the system vm's, message comes in on one of those nics,
>>>>> but
>>>>> it's default route out is another nic... Thus blocking the response.
>>>>>
>>>>> Ideally you'd use a basic zone for this kind of configuration, or else
>>>>> you'll end up having to log into the system vm's every time a new one
>>>>> is
>>>>> spawned and disabling rp_filter for the nics. You might want to test
>>>>> this,
>>>>> by logging in and disabling rp_filter on the nics and see if things
>>>>> start
>>>>> working as expected.
>>>>>
>>>>>
>>>>>>
>>>>>> Obviously, my network configuration is incorrect, but I have the
>>>>>> reached
>>>>>> the limits of my CloudStack and Xen knowledge to identify the
>>>>>> problem(s).
>>>>>>
>>>>>> Given this information, what is the best way to give the SSVM access
>>>>>> to
>>>>>> the Internet and/or a VirtualBox host-only network?
>>>>>>
>>>>>> Thank you for your help,
>>>>>> -John
>>>>>>
>>>>>> On Oct 3, 2012, at 10:39 PM, "Kelceydamage@bbits" <ke...@bbits.ca>
>>>>>> wrote:
>>>>>>
>>>>>>> The the secondary storage VM can be NATed to from any network
>>>>>>> router,
>>>>>>> however the console proxy does not work over NAT.
>>>>>>>
>>>>>>> Sent from my iPhone
>>>>>>>
>>>>>>> On Oct 3, 2012, at 7:32 PM, Edison Su <Ed...@citrix.com> wrote:
>>>>>>>
>>>>>>>> System vm will have 4 nics, eth2 is on the public network, eth1 is
>>>>>>>> the
>>>>>>>> private(mgt) network.
>>>>>>>> The IP address of eth2 is got from pod configuration: in one of IP
>>>>>>>> address range ["startip", "endip"] in createPod API.
>>>>>>>> The IP address of eth1 is got from guest network, if it's basic
>>>>>>>> network mode, this IP range is configured by createVlanIpRanges API
>>>>>>>> SSVM will connect to mgt server through eth1(mgt server's ip
>>>>>>>> address
>>>>>>>> is configured to route through eth1), and download template from
>>>>>>>> eth2.
>>>>>>>> What's your specific issue about network configuration?
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: John Burwell [mailto:jburwell@basho.com]
>>>>>>>>> Sent: Wednesday, October 03, 2012 7:11 PM
>>>>>>>>> To: cloudstack-dev@incubator.apache.org
>>>>>>>>> Subject: SSVM Network Configuration
>>>>>>>>>
>>>>>>>>> All,
>>>>>>>>>
>>>>>>>>> How do you configure networking to permit the SSVM to connect to
>>>>>>>>> the
>>>>>>>>> public Internet or another internal network? I have been trying
>>>>>>>>> to
>>>>>>>>> understand the network configuration from the documentation, but
>>>>>>>>> am
>>>>>>>>> missing something in my configuration attempt.
>>>>>>>>>
>>>>>>>>> Thank you for your assistance,
>>>>>>>>> -John
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Æ
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: SSVM Network Configuration
Posted by Chiradeep Vittal <Ch...@citrix.com>.
Yes, 0.
I think that the discrepancy is because sysctl.conf is modified too late
in the game.
0. SSVM boots for first time
1. cloud-early-config figures out that scripts need to be patched
2. Scripts are patched and reboot is initiated. Sysctl is not modified yet
3. SSVM boots, steps 1-2 do not take place, figures out it is an SSVM
4. cloud-early-config modifies /etc/sysctl.conf, but DOES NOT execute
sysctl -w
Hence the runtime value of rp_filter remains 1 while the config file says
"0".
--
Chiradeep
On 10/29/12 1:00 PM, "John Burwell" <jb...@basho.com> wrote:
>Chiradeep,
>
>Currently, net.ipv4.conf.default.rp_filer is set to 1 in
>systemvm/debian/config/etc/sysctl.conf. Should it be modified to be 0?
>
>Thanks,
>-John
>
>On Oct 4, 2012, at 6:09 PM, Chiradeep Vittal
><Ch...@citrix.com> wrote:
>
>> It is disabled in sysctl.conf, not sure how it gets re-enabled. See
>> patches/systemvm/debian/config/etc/init.d/cloud-early-config (function
>> disable_rpfilter).
>> Perhaps it is interface-specific rather than "all".
>>
>> On 10/4/12 2:39 PM, "John Burwell" <jb...@basho.com> wrote:
>>
>>> Ahmad,
>>>
>>> You were correct on the rp_filter issue. Once disabled, the SSVM was
>>> able to connect outbound to S3, as well as, any host reachable from
>>> devcloud. I noticed that rp_filter is disabled in sysctl.conf yet it
>>>is
>>> somehow being enabled at runtime. Is this behavior intended?
>>>
>>> Thanks,
>>> -John
>>>
>>> On Oct 4, 2012, at 1:07 PM, Ahmad Emneina <Ah...@citrix.com>
>>> wrote:
>>>
>>>> On 10/4/12 9:16 AM, "John Burwell" <jb...@basho.com> wrote:
>>>>
>>>>> Kelcey,
>>>>>
>>>>> I am a bit confused about how secstorage.allowed.internal.sites is
>>>>>used
>>>>> which stems to lack of knowledge regarding the devcloud network
>>>>> configuration. Also, is there documentation available for setting up
>>>>> such a NAT?
>>>>>
>>>>> As a point of clarification to my original question, I am working in
>>>>> the
>>>>> devcloud environment (using the OVA downloaded from the wiki) where I
>>>>> need to get the SSVM to connect to S3 or to a local VirtualBox VM
>>>>> running
>>>>> an S3-compatible object store. Thus far, I have been unable to get
>>>>> devcloud to bring up a second NIC on a host-only network. I have
>>>>> attempted to setup an advanced network configuration as follows:
>>>>>
>>>>> Physical Network with VLAN isolation method
>>>>> Management Server: 10.0.2.15 -> Gateway: 10.0.2.2
>>>>> Storage Network: 10.0.2.50-10.0.2.59 -> Gateway 10.0.2.2 on VLAN0
>>>>> Management Network: 10.0.2.200-10.0.2.220 -> Gateway 10.0.2.2
>>>>> Public Network: 10.0.2.100-10.0.2.199 -> VLAN0
>>>>
>>>> The issue that gets created here is you get system vm's that are
>>>> multi-homed. Your system vm's get a nic (leg) on each network... But
>>>> that
>>>> network is one and the same. Why this is an issue is rp_filter is
>>>> enabled
>>>> by default on the system vm's, message comes in on one of those nics,
>>>> but
>>>> it's default route out is another nic... Thus blocking the response.
>>>>
>>>> Ideally you'd use a basic zone for this kind of configuration, or else
>>>> you'll end up having to log into the system vm's every time a new one
>>>>is
>>>> spawned and disabling rp_filter for the nics. You might want to test
>>>> this,
>>>> by logging in and disabling rp_filter on the nics and see if things
>>>> start
>>>> working as expected.
>>>>
>>>>
>>>>>
>>>>> Obviously, my network configuration is incorrect, but I have the
>>>>> reached
>>>>> the limits of my CloudStack and Xen knowledge to identify the
>>>>> problem(s).
>>>>>
>>>>> Given this information, what is the best way to give the SSVM access
>>>>>to
>>>>> the Internet and/or a VirtualBox host-only network?
>>>>>
>>>>> Thank you for your help,
>>>>> -John
>>>>>
>>>>> On Oct 3, 2012, at 10:39 PM, "Kelceydamage@bbits" <ke...@bbits.ca>
>>>>> wrote:
>>>>>
>>>>>> The the secondary storage VM can be NATed to from any network
>>>>>>router,
>>>>>> however the console proxy does not work over NAT.
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>> On Oct 3, 2012, at 7:32 PM, Edison Su <Ed...@citrix.com> wrote:
>>>>>>
>>>>>>> System vm will have 4 nics, eth2 is on the public network, eth1 is
>>>>>>> the
>>>>>>> private(mgt) network.
>>>>>>> The IP address of eth2 is got from pod configuration: in one of IP
>>>>>>> address range ["startip", "endip"] in createPod API.
>>>>>>> The IP address of eth1 is got from guest network, if it's basic
>>>>>>> network mode, this IP range is configured by createVlanIpRanges API
>>>>>>> SSVM will connect to mgt server through eth1(mgt server's ip
>>>>>>>address
>>>>>>> is configured to route through eth1), and download template from
>>>>>>> eth2.
>>>>>>> What's your specific issue about network configuration?
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: John Burwell [mailto:jburwell@basho.com]
>>>>>>>> Sent: Wednesday, October 03, 2012 7:11 PM
>>>>>>>> To: cloudstack-dev@incubator.apache.org
>>>>>>>> Subject: SSVM Network Configuration
>>>>>>>>
>>>>>>>> All,
>>>>>>>>
>>>>>>>> How do you configure networking to permit the SSVM to connect to
>>>>>>>>the
>>>>>>>> public Internet or another internal network? I have been trying
>>>>>>>>to
>>>>>>>> understand the network configuration from the documentation, but
>>>>>>>>am
>>>>>>>> missing something in my configuration attempt.
>>>>>>>>
>>>>>>>> Thank you for your assistance,
>>>>>>>> -John
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Æ
>>>>
>>>>
>>>>
>>>
>>
>
Re: SSVM Network Configuration
Posted by John Burwell <jb...@basho.com>.
Chiradeep,
Currently, net.ipv4.conf.default.rp_filer is set to 1 in systemvm/debian/config/etc/sysctl.conf. Should it be modified to be 0?
Thanks,
-John
On Oct 4, 2012, at 6:09 PM, Chiradeep Vittal <Ch...@citrix.com> wrote:
> It is disabled in sysctl.conf, not sure how it gets re-enabled. See
> patches/systemvm/debian/config/etc/init.d/cloud-early-config (function
> disable_rpfilter).
> Perhaps it is interface-specific rather than "all".
>
> On 10/4/12 2:39 PM, "John Burwell" <jb...@basho.com> wrote:
>
>> Ahmad,
>>
>> You were correct on the rp_filter issue. Once disabled, the SSVM was
>> able to connect outbound to S3, as well as, any host reachable from
>> devcloud. I noticed that rp_filter is disabled in sysctl.conf yet it is
>> somehow being enabled at runtime. Is this behavior intended?
>>
>> Thanks,
>> -John
>>
>> On Oct 4, 2012, at 1:07 PM, Ahmad Emneina <Ah...@citrix.com>
>> wrote:
>>
>>> On 10/4/12 9:16 AM, "John Burwell" <jb...@basho.com> wrote:
>>>
>>>> Kelcey,
>>>>
>>>> I am a bit confused about how secstorage.allowed.internal.sites is used
>>>> which stems to lack of knowledge regarding the devcloud network
>>>> configuration. Also, is there documentation available for setting up
>>>> such a NAT?
>>>>
>>>> As a point of clarification to my original question, I am working in
>>>> the
>>>> devcloud environment (using the OVA downloaded from the wiki) where I
>>>> need to get the SSVM to connect to S3 or to a local VirtualBox VM
>>>> running
>>>> an S3-compatible object store. Thus far, I have been unable to get
>>>> devcloud to bring up a second NIC on a host-only network. I have
>>>> attempted to setup an advanced network configuration as follows:
>>>>
>>>> Physical Network with VLAN isolation method
>>>> Management Server: 10.0.2.15 -> Gateway: 10.0.2.2
>>>> Storage Network: 10.0.2.50-10.0.2.59 -> Gateway 10.0.2.2 on VLAN0
>>>> Management Network: 10.0.2.200-10.0.2.220 -> Gateway 10.0.2.2
>>>> Public Network: 10.0.2.100-10.0.2.199 -> VLAN0
>>>
>>> The issue that gets created here is you get system vm's that are
>>> multi-homed. Your system vm's get a nic (leg) on each network... But
>>> that
>>> network is one and the same. Why this is an issue is rp_filter is
>>> enabled
>>> by default on the system vm's, message comes in on one of those nics,
>>> but
>>> it's default route out is another nic... Thus blocking the response.
>>>
>>> Ideally you'd use a basic zone for this kind of configuration, or else
>>> you'll end up having to log into the system vm's every time a new one is
>>> spawned and disabling rp_filter for the nics. You might want to test
>>> this,
>>> by logging in and disabling rp_filter on the nics and see if things
>>> start
>>> working as expected.
>>>
>>>
>>>>
>>>> Obviously, my network configuration is incorrect, but I have the
>>>> reached
>>>> the limits of my CloudStack and Xen knowledge to identify the
>>>> problem(s).
>>>>
>>>> Given this information, what is the best way to give the SSVM access to
>>>> the Internet and/or a VirtualBox host-only network?
>>>>
>>>> Thank you for your help,
>>>> -John
>>>>
>>>> On Oct 3, 2012, at 10:39 PM, "Kelceydamage@bbits" <ke...@bbits.ca>
>>>> wrote:
>>>>
>>>>> The the secondary storage VM can be NATed to from any network router,
>>>>> however the console proxy does not work over NAT.
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On Oct 3, 2012, at 7:32 PM, Edison Su <Ed...@citrix.com> wrote:
>>>>>
>>>>>> System vm will have 4 nics, eth2 is on the public network, eth1 is
>>>>>> the
>>>>>> private(mgt) network.
>>>>>> The IP address of eth2 is got from pod configuration: in one of IP
>>>>>> address range ["startip", "endip"] in createPod API.
>>>>>> The IP address of eth1 is got from guest network, if it's basic
>>>>>> network mode, this IP range is configured by createVlanIpRanges API
>>>>>> SSVM will connect to mgt server through eth1(mgt server's ip address
>>>>>> is configured to route through eth1), and download template from
>>>>>> eth2.
>>>>>> What's your specific issue about network configuration?
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: John Burwell [mailto:jburwell@basho.com]
>>>>>>> Sent: Wednesday, October 03, 2012 7:11 PM
>>>>>>> To: cloudstack-dev@incubator.apache.org
>>>>>>> Subject: SSVM Network Configuration
>>>>>>>
>>>>>>> All,
>>>>>>>
>>>>>>> How do you configure networking to permit the SSVM to connect to the
>>>>>>> public Internet or another internal network? I have been trying to
>>>>>>> understand the network configuration from the documentation, but am
>>>>>>> missing something in my configuration attempt.
>>>>>>>
>>>>>>> Thank you for your assistance,
>>>>>>> -John
>>>>
>>>>
>>>
>>>
>>> --
>>> Æ
>>>
>>>
>>>
>>
>
Re: SSVM Network Configuration
Posted by Chiradeep Vittal <Ch...@citrix.com>.
It is disabled in sysctl.conf, not sure how it gets re-enabled. See
patches/systemvm/debian/config/etc/init.d/cloud-early-config (function
disable_rpfilter).
Perhaps it is interface-specific rather than "all".
On 10/4/12 2:39 PM, "John Burwell" <jb...@basho.com> wrote:
>Ahmad,
>
>You were correct on the rp_filter issue. Once disabled, the SSVM was
>able to connect outbound to S3, as well as, any host reachable from
>devcloud. I noticed that rp_filter is disabled in sysctl.conf yet it is
>somehow being enabled at runtime. Is this behavior intended?
>
>Thanks,
>-John
>
>On Oct 4, 2012, at 1:07 PM, Ahmad Emneina <Ah...@citrix.com>
>wrote:
>
>> On 10/4/12 9:16 AM, "John Burwell" <jb...@basho.com> wrote:
>>
>>> Kelcey,
>>>
>>> I am a bit confused about how secstorage.allowed.internal.sites is used
>>> which stems to lack of knowledge regarding the devcloud network
>>> configuration. Also, is there documentation available for setting up
>>> such a NAT?
>>>
>>> As a point of clarification to my original question, I am working in
>>>the
>>> devcloud environment (using the OVA downloaded from the wiki) where I
>>> need to get the SSVM to connect to S3 or to a local VirtualBox VM
>>>running
>>> an S3-compatible object store. Thus far, I have been unable to get
>>> devcloud to bring up a second NIC on a host-only network. I have
>>> attempted to setup an advanced network configuration as follows:
>>>
>>> Physical Network with VLAN isolation method
>>> Management Server: 10.0.2.15 -> Gateway: 10.0.2.2
>>> Storage Network: 10.0.2.50-10.0.2.59 -> Gateway 10.0.2.2 on VLAN0
>>> Management Network: 10.0.2.200-10.0.2.220 -> Gateway 10.0.2.2
>>> Public Network: 10.0.2.100-10.0.2.199 -> VLAN0
>>
>> The issue that gets created here is you get system vm's that are
>> multi-homed. Your system vm's get a nic (leg) on each network... But
>>that
>> network is one and the same. Why this is an issue is rp_filter is
>>enabled
>> by default on the system vm's, message comes in on one of those nics,
>>but
>> it's default route out is another nic... Thus blocking the response.
>>
>> Ideally you'd use a basic zone for this kind of configuration, or else
>> you'll end up having to log into the system vm's every time a new one is
>> spawned and disabling rp_filter for the nics. You might want to test
>>this,
>> by logging in and disabling rp_filter on the nics and see if things
>>start
>> working as expected.
>>
>>
>>>
>>> Obviously, my network configuration is incorrect, but I have the
>>>reached
>>> the limits of my CloudStack and Xen knowledge to identify the
>>>problem(s).
>>>
>>> Given this information, what is the best way to give the SSVM access to
>>> the Internet and/or a VirtualBox host-only network?
>>>
>>> Thank you for your help,
>>> -John
>>>
>>> On Oct 3, 2012, at 10:39 PM, "Kelceydamage@bbits" <ke...@bbits.ca>
>>>wrote:
>>>
>>>> The the secondary storage VM can be NATed to from any network router,
>>>> however the console proxy does not work over NAT.
>>>>
>>>> Sent from my iPhone
>>>>
>>>> On Oct 3, 2012, at 7:32 PM, Edison Su <Ed...@citrix.com> wrote:
>>>>
>>>>> System vm will have 4 nics, eth2 is on the public network, eth1 is
>>>>>the
>>>>> private(mgt) network.
>>>>> The IP address of eth2 is got from pod configuration: in one of IP
>>>>> address range ["startip", "endip"] in createPod API.
>>>>> The IP address of eth1 is got from guest network, if it's basic
>>>>> network mode, this IP range is configured by createVlanIpRanges API
>>>>> SSVM will connect to mgt server through eth1(mgt server's ip address
>>>>> is configured to route through eth1), and download template from
>>>>>eth2.
>>>>> What's your specific issue about network configuration?
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: John Burwell [mailto:jburwell@basho.com]
>>>>>> Sent: Wednesday, October 03, 2012 7:11 PM
>>>>>> To: cloudstack-dev@incubator.apache.org
>>>>>> Subject: SSVM Network Configuration
>>>>>>
>>>>>> All,
>>>>>>
>>>>>> How do you configure networking to permit the SSVM to connect to the
>>>>>> public Internet or another internal network? I have been trying to
>>>>>> understand the network configuration from the documentation, but am
>>>>>> missing something in my configuration attempt.
>>>>>>
>>>>>> Thank you for your assistance,
>>>>>> -John
>>>
>>>
>>
>>
>> --
>> Æ
>>
>>
>>
>
Re: SSVM Network Configuration
Posted by John Burwell <jb...@basho.com>.
Ahmad,
You were correct on the rp_filter issue. Once disabled, the SSVM was able to connect outbound to S3, as well as, any host reachable from devcloud. I noticed that rp_filter is disabled in sysctl.conf yet it is somehow being enabled at runtime. Is this behavior intended?
Thanks,
-John
On Oct 4, 2012, at 1:07 PM, Ahmad Emneina <Ah...@citrix.com> wrote:
> On 10/4/12 9:16 AM, "John Burwell" <jb...@basho.com> wrote:
>
>> Kelcey,
>>
>> I am a bit confused about how secstorage.allowed.internal.sites is used
>> which stems to lack of knowledge regarding the devcloud network
>> configuration. Also, is there documentation available for setting up
>> such a NAT?
>>
>> As a point of clarification to my original question, I am working in the
>> devcloud environment (using the OVA downloaded from the wiki) where I
>> need to get the SSVM to connect to S3 or to a local VirtualBox VM running
>> an S3-compatible object store. Thus far, I have been unable to get
>> devcloud to bring up a second NIC on a host-only network. I have
>> attempted to setup an advanced network configuration as follows:
>>
>> Physical Network with VLAN isolation method
>> Management Server: 10.0.2.15 -> Gateway: 10.0.2.2
>> Storage Network: 10.0.2.50-10.0.2.59 -> Gateway 10.0.2.2 on VLAN0
>> Management Network: 10.0.2.200-10.0.2.220 -> Gateway 10.0.2.2
>> Public Network: 10.0.2.100-10.0.2.199 -> VLAN0
>
> The issue that gets created here is you get system vm's that are
> multi-homed. Your system vm's get a nic (leg) on each network... But that
> network is one and the same. Why this is an issue is rp_filter is enabled
> by default on the system vm's, message comes in on one of those nics, but
> it's default route out is another nic... Thus blocking the response.
>
> Ideally you'd use a basic zone for this kind of configuration, or else
> you'll end up having to log into the system vm's every time a new one is
> spawned and disabling rp_filter for the nics. You might want to test this,
> by logging in and disabling rp_filter on the nics and see if things start
> working as expected.
>
>
>>
>> Obviously, my network configuration is incorrect, but I have the reached
>> the limits of my CloudStack and Xen knowledge to identify the problem(s).
>>
>> Given this information, what is the best way to give the SSVM access to
>> the Internet and/or a VirtualBox host-only network?
>>
>> Thank you for your help,
>> -John
>>
>> On Oct 3, 2012, at 10:39 PM, "Kelceydamage@bbits" <ke...@bbits.ca> wrote:
>>
>>> The the secondary storage VM can be NATed to from any network router,
>>> however the console proxy does not work over NAT.
>>>
>>> Sent from my iPhone
>>>
>>> On Oct 3, 2012, at 7:32 PM, Edison Su <Ed...@citrix.com> wrote:
>>>
>>>> System vm will have 4 nics, eth2 is on the public network, eth1 is the
>>>> private(mgt) network.
>>>> The IP address of eth2 is got from pod configuration: in one of IP
>>>> address range ["startip", "endip"] in createPod API.
>>>> The IP address of eth1 is got from guest network, if it's basic
>>>> network mode, this IP range is configured by createVlanIpRanges API
>>>> SSVM will connect to mgt server through eth1(mgt server's ip address
>>>> is configured to route through eth1), and download template from eth2.
>>>> What's your specific issue about network configuration?
>>>>
>>>>> -----Original Message-----
>>>>> From: John Burwell [mailto:jburwell@basho.com]
>>>>> Sent: Wednesday, October 03, 2012 7:11 PM
>>>>> To: cloudstack-dev@incubator.apache.org
>>>>> Subject: SSVM Network Configuration
>>>>>
>>>>> All,
>>>>>
>>>>> How do you configure networking to permit the SSVM to connect to the
>>>>> public Internet or another internal network? I have been trying to
>>>>> understand the network configuration from the documentation, but am
>>>>> missing something in my configuration attempt.
>>>>>
>>>>> Thank you for your assistance,
>>>>> -John
>>
>>
>
>
> --
> Æ
>
>
>
Re: SSVM Network Configuration
Posted by Ahmad Emneina <Ah...@citrix.com>.
On 10/4/12 9:16 AM, "John Burwell" <jb...@basho.com> wrote:
>Kelcey,
>
>I am a bit confused about how secstorage.allowed.internal.sites is used
>which stems to lack of knowledge regarding the devcloud network
>configuration. Also, is there documentation available for setting up
>such a NAT?
>
>As a point of clarification to my original question, I am working in the
>devcloud environment (using the OVA downloaded from the wiki) where I
>need to get the SSVM to connect to S3 or to a local VirtualBox VM running
>an S3-compatible object store. Thus far, I have been unable to get
>devcloud to bring up a second NIC on a host-only network. I have
>attempted to setup an advanced network configuration as follows:
>
>Physical Network with VLAN isolation method
>Management Server: 10.0.2.15 -> Gateway: 10.0.2.2
>Storage Network: 10.0.2.50-10.0.2.59 -> Gateway 10.0.2.2 on VLAN0
>Management Network: 10.0.2.200-10.0.2.220 -> Gateway 10.0.2.2
>Public Network: 10.0.2.100-10.0.2.199 -> VLAN0
The issue that gets created here is you get system vm's that are
multi-homed. Your system vm's get a nic (leg) on each network... But that
network is one and the same. Why this is an issue is rp_filter is enabled
by default on the system vm's, message comes in on one of those nics, but
it's default route out is another nic... Thus blocking the response.
Ideally you'd use a basic zone for this kind of configuration, or else
you'll end up having to log into the system vm's every time a new one is
spawned and disabling rp_filter for the nics. You might want to test this,
by logging in and disabling rp_filter on the nics and see if things start
working as expected.
>
>Obviously, my network configuration is incorrect, but I have the reached
>the limits of my CloudStack and Xen knowledge to identify the problem(s).
>
>Given this information, what is the best way to give the SSVM access to
>the Internet and/or a VirtualBox host-only network?
>
>Thank you for your help,
>-John
>
>On Oct 3, 2012, at 10:39 PM, "Kelceydamage@bbits" <ke...@bbits.ca> wrote:
>
>> The the secondary storage VM can be NATed to from any network router,
>>however the console proxy does not work over NAT.
>>
>> Sent from my iPhone
>>
>> On Oct 3, 2012, at 7:32 PM, Edison Su <Ed...@citrix.com> wrote:
>>
>>> System vm will have 4 nics, eth2 is on the public network, eth1 is the
>>>private(mgt) network.
>>> The IP address of eth2 is got from pod configuration: in one of IP
>>>address range ["startip", "endip"] in createPod API.
>>> The IP address of eth1 is got from guest network, if it's basic
>>>network mode, this IP range is configured by createVlanIpRanges API
>>> SSVM will connect to mgt server through eth1(mgt server's ip address
>>>is configured to route through eth1), and download template from eth2.
>>> What's your specific issue about network configuration?
>>>
>>>> -----Original Message-----
>>>> From: John Burwell [mailto:jburwell@basho.com]
>>>> Sent: Wednesday, October 03, 2012 7:11 PM
>>>> To: cloudstack-dev@incubator.apache.org
>>>> Subject: SSVM Network Configuration
>>>>
>>>> All,
>>>>
>>>> How do you configure networking to permit the SSVM to connect to the
>>>> public Internet or another internal network? I have been trying to
>>>> understand the network configuration from the documentation, but am
>>>> missing something in my configuration attempt.
>>>>
>>>> Thank you for your assistance,
>>>> -John
>
>
--
Æ
Re: SSVM Network Configuration
Posted by John Burwell <jb...@basho.com>.
Kelcey,
I am a bit confused about how secstorage.allowed.internal.sites is used which stems to lack of knowledge regarding the devcloud network configuration. Also, is there documentation available for setting up such a NAT?
As a point of clarification to my original question, I am working in the devcloud environment (using the OVA downloaded from the wiki) where I need to get the SSVM to connect to S3 or to a local VirtualBox VM running an S3-compatible object store. Thus far, I have been unable to get devcloud to bring up a second NIC on a host-only network. I have attempted to setup an advanced network configuration as follows:
Physical Network with VLAN isolation method
Management Server: 10.0.2.15 -> Gateway: 10.0.2.2
Storage Network: 10.0.2.50-10.0.2.59 -> Gateway 10.0.2.2 on VLAN0
Management Network: 10.0.2.200-10.0.2.220 -> Gateway 10.0.2.2
Public Network: 10.0.2.100-10.0.2.199 -> VLAN0
Obviously, my network configuration is incorrect, but I have the reached the limits of my CloudStack and Xen knowledge to identify the problem(s).
Given this information, what is the best way to give the SSVM access to the Internet and/or a VirtualBox host-only network?
Thank you for your help,
-John
On Oct 3, 2012, at 10:39 PM, "Kelceydamage@bbits" <ke...@bbits.ca> wrote:
> The the secondary storage VM can be NATed to from any network router, however the console proxy does not work over NAT.
>
> Sent from my iPhone
>
> On Oct 3, 2012, at 7:32 PM, Edison Su <Ed...@citrix.com> wrote:
>
>> System vm will have 4 nics, eth2 is on the public network, eth1 is the private(mgt) network.
>> The IP address of eth2 is got from pod configuration: in one of IP address range ["startip", "endip"] in createPod API.
>> The IP address of eth1 is got from guest network, if it's basic network mode, this IP range is configured by createVlanIpRanges API
>> SSVM will connect to mgt server through eth1(mgt server's ip address is configured to route through eth1), and download template from eth2.
>> What's your specific issue about network configuration?
>>
>>> -----Original Message-----
>>> From: John Burwell [mailto:jburwell@basho.com]
>>> Sent: Wednesday, October 03, 2012 7:11 PM
>>> To: cloudstack-dev@incubator.apache.org
>>> Subject: SSVM Network Configuration
>>>
>>> All,
>>>
>>> How do you configure networking to permit the SSVM to connect to the
>>> public Internet or another internal network? I have been trying to
>>> understand the network configuration from the documentation, but am
>>> missing something in my configuration attempt.
>>>
>>> Thank you for your assistance,
>>> -John
Re: SSVM Network Configuration
Posted by "Kelceydamage@bbits" <ke...@bbits.ca>.
The the secondary storage VM can be NATed to from any network router, however the console proxy does not work over NAT.
Sent from my iPhone
On Oct 3, 2012, at 7:32 PM, Edison Su <Ed...@citrix.com> wrote:
> System vm will have 4 nics, eth2 is on the public network, eth1 is the private(mgt) network.
> The IP address of eth2 is got from pod configuration: in one of IP address range ["startip", "endip"] in createPod API.
> The IP address of eth1 is got from guest network, if it's basic network mode, this IP range is configured by createVlanIpRanges API
> SSVM will connect to mgt server through eth1(mgt server's ip address is configured to route through eth1), and download template from eth2.
> What's your specific issue about network configuration?
>
>> -----Original Message-----
>> From: John Burwell [mailto:jburwell@basho.com]
>> Sent: Wednesday, October 03, 2012 7:11 PM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: SSVM Network Configuration
>>
>> All,
>>
>> How do you configure networking to permit the SSVM to connect to the
>> public Internet or another internal network? I have been trying to
>> understand the network configuration from the documentation, but am
>> missing something in my configuration attempt.
>>
>> Thank you for your assistance,
>> -John
Re: SSVM Network Configuration
Posted by David Nalley <da...@gnsa.us>.
On Wed, Oct 3, 2012 at 10:32 PM, Edison Su <Ed...@citrix.com> wrote:
> System vm will have 4 nics, eth2 is on the public network, eth1 is the private(mgt) network.
> The IP address of eth2 is got from pod configuration: in one of IP address range ["startip", "endip"] in createPod API.
> The IP address of eth1 is got from guest network, if it's basic network mode, this IP range is configured by createVlanIpRanges API
> SSVM will connect to mgt server through eth1(mgt server's ip address is configured to route through eth1), and download template from eth2.
> What's your specific issue about network configuration?
>
>> -----Original Message-----
>> From: John Burwell [mailto:jburwell@basho.com]
>> Sent: Wednesday, October 03, 2012 7:11 PM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: SSVM Network Configuration
>>
>> All,
>>
>> How do you configure networking to permit the SSVM to connect to the
>> public Internet or another internal network? I have been trying to
>> understand the network configuration from the documentation, but am
>> missing something in my configuration attempt.
>>
>> Thank you for your assistance,
>> -John
Do you have secstorage.allowed.internal.sites set for your internal
network? It should be deny all by default on internal network.
--David
RE: SSVM Network Configuration
Posted by Edison Su <Ed...@citrix.com>.
System vm will have 4 nics, eth2 is on the public network, eth1 is the private(mgt) network.
The IP address of eth2 is got from pod configuration: in one of IP address range ["startip", "endip"] in createPod API.
The IP address of eth1 is got from guest network, if it's basic network mode, this IP range is configured by createVlanIpRanges API
SSVM will connect to mgt server through eth1(mgt server's ip address is configured to route through eth1), and download template from eth2.
What's your specific issue about network configuration?
> -----Original Message-----
> From: John Burwell [mailto:jburwell@basho.com]
> Sent: Wednesday, October 03, 2012 7:11 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: SSVM Network Configuration
>
> All,
>
> How do you configure networking to permit the SSVM to connect to the
> public Internet or another internal network? I have been trying to
> understand the network configuration from the documentation, but am
> missing something in my configuration attempt.
>
> Thank you for your assistance,
> -John