You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Will Stevens <wi...@gmail.com> on 2016/09/12 20:20:11 UTC
[DISCUSS] Replacing the VR
*Disclaimer:* This is a thought experiment and should be treated as such.
Please weigh in with the good and bad of this idea...
A couple of us have been discussing the idea of potentially replacing the
ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a license
issue because I think it is licensed under GPL, but for the sake of
discussion, let's assume we can overcome any license issues.
I have spent some time recently with the VyOS and I have to admit, I was
pretty impressed. It is simple and intuitive and it gives you a lot more
options for auditing the configuration etc...
Items of potential interest:
- Clean up our current VR script spaghetti to a simpler more auditable
configuration workflow.
- Gives a cleaner path for IPv6 support.
- Handles VPN configuration via the same configuration interface.
- Support for OSPF & BGP.
- VPN support through OpenVPN & StrongSwan.
- Easily supports HA (redundant routers) through VRRP.
- VXLAN support.
- Transaction based changes to the VR with rollback on error.
Items that could be difficult to solve:
- Userdata password reset workflow and implementation.
- Upgrade process.
The VyOS is not the only option if we were to consider this approach.
Another option, which I don't know as well, would be CloudRouter (AGPL
license) [2] which is purely API driven.
Anyway, would love to hear your thoughts...
Will
[1] https://vyos.io/
[2] https://cloudrouter.org/
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
At this point, I think we are still throwing around ideas. If you have
links to more options you think we should have on the table, please add
them to the discussion. :)
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Mon, Sep 12, 2016 at 6:30 PM, Marty Godsey <ma...@gonsource.com> wrote:
> Have we thought of other SDx routers such as pfsense? Its licensed under
> the BSD license and is well maintained.
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: Monday, September 12, 2016 6:16 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> You have probably looked into this more than I have Rene.
>
> I am not sure there existed a time when the VR was ever "great". In my
> eyes, the core ACS dev team should not be building and managing its own
> VR. I feel like that is a liability because the subset of developers who
> are proficient in networking is quite small. That means we could be at
> risk of losing the majority of our "experts" with a few people changing
> their $dayjob. It feels safer to work with an existing technology which
> has its own development community focused on doing that piece well.
> Obviously this has its own drawbacks, but in general, we need the VR
> implementation to be built by dedicated network engineers and not jack of
> all trade developers. No offense to current company...
>
> I agree with your list of what you would like to see. Rock solid and not
> over complicated is key. That being said, if it ONLY handles what ACS
> needs today, then WE have to be the ones to develop any changes. For
> example; we need IPv6, VXLAN support, etc... My point is that if we only
> focus on what we need today, then we end up building everything we need for
> the future and I think we end up back where we are now down the road...
>
> I love everything you are saying, just not sure I want us building and
> maintaining it all...
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Mon, Sep 12, 2016 at 5:47 PM, Rene Moser <ma...@renemoser.net> wrote:
>
> > Hi
> >
> > On 09/12/2016 10:20 PM, Will Stevens wrote:
> > > *Disclaimer:* This is a thought experiment and should be treated as
> such.
> > > Please weigh in with the good and bad of this idea...
> > >
> > > A couple of us have been discussing the idea of potentially
> > > replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
> > > There may be a license issue because I think it is licensed under
> > > GPL, but for the sake of discussion, let's assume we can overcome any
> license issues.
> >
> > VyOS is Debian based, much like the current VR. As long as it is not
> > shipped with CloudStack, all fine.
> >
> > > I have spent some time recently with the VyOS and I have to admit, I
> > > was pretty impressed. It is simple and intuitive and it gives you a
> > > lot more options for auditing the configuration etc...
> >
> > I had the same "crazy" thoughts when I heard about VyOS the first time.
> >
> > When I looked at VyOS, the release cycle were not very frequent and
> > the current stable release is still based on Debian 6 (EOL [1] since
> > 02.16)
> >
> > However to me, it doesn't matter if it's VyOS or CloudLinux, or
> > another solution.
> >
> > The question is more like what is wrong with the current VR and how
> > can we make the VR great again. Things I would like to see:
> >
> > - VR must have a "clean", programmable, documented, API, supporting
> > batch processing.
> > - VR must be rock solid (minimal shell) state of the art, up to date,
> > but small (Only contain things CloudStack needs, not more)
> > - VR must be scale well (...) and support stateful HA
> > - VR must be easy to upgrade (security) without downtimes.
> >
> > Christmas is soon... ;)
> >
> > René
> >
> > [1] https://www.debian.org/News/2016/20160212
> >
> >
>
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
Have we thought of other SDx routers such as pfsense? Its licensed under the BSD license and is well maintained.
-----Original Message-----
From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
Sent: Monday, September 12, 2016 6:16 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
You have probably looked into this more than I have Rene.
I am not sure there existed a time when the VR was ever "great". In my eyes, the core ACS dev team should not be building and managing its own VR. I feel like that is a liability because the subset of developers who are proficient in networking is quite small. That means we could be at risk of losing the majority of our "experts" with a few people changing their $dayjob. It feels safer to work with an existing technology which has its own development community focused on doing that piece well.
Obviously this has its own drawbacks, but in general, we need the VR implementation to be built by dedicated network engineers and not jack of all trade developers. No offense to current company...
I agree with your list of what you would like to see. Rock solid and not over complicated is key. That being said, if it ONLY handles what ACS needs today, then WE have to be the ones to develop any changes. For example; we need IPv6, VXLAN support, etc... My point is that if we only focus on what we need today, then we end up building everything we need for the future and I think we end up back where we are now down the road...
I love everything you are saying, just not sure I want us building and maintaining it all...
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_
On Mon, Sep 12, 2016 at 5:47 PM, Rene Moser <ma...@renemoser.net> wrote:
> Hi
>
> On 09/12/2016 10:20 PM, Will Stevens wrote:
> > *Disclaimer:* This is a thought experiment and should be treated as such.
> > Please weigh in with the good and bad of this idea...
> >
> > A couple of us have been discussing the idea of potentially
> > replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
> > There may be a license issue because I think it is licensed under
> > GPL, but for the sake of discussion, let's assume we can overcome any license issues.
>
> VyOS is Debian based, much like the current VR. As long as it is not
> shipped with CloudStack, all fine.
>
> > I have spent some time recently with the VyOS and I have to admit, I
> > was pretty impressed. It is simple and intuitive and it gives you a
> > lot more options for auditing the configuration etc...
>
> I had the same "crazy" thoughts when I heard about VyOS the first time.
>
> When I looked at VyOS, the release cycle were not very frequent and
> the current stable release is still based on Debian 6 (EOL [1] since
> 02.16)
>
> However to me, it doesn't matter if it's VyOS or CloudLinux, or
> another solution.
>
> The question is more like what is wrong with the current VR and how
> can we make the VR great again. Things I would like to see:
>
> - VR must have a "clean", programmable, documented, API, supporting
> batch processing.
> - VR must be rock solid (minimal shell) state of the art, up to date,
> but small (Only contain things CloudStack needs, not more)
> - VR must be scale well (...) and support stateful HA
> - VR must be easy to upgrade (security) without downtimes.
>
> Christmas is soon... ;)
>
> René
>
> [1] https://www.debian.org/News/2016/20160212
>
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
You have probably looked into this more than I have Rene.
I am not sure there existed a time when the VR was ever "great". In my
eyes, the core ACS dev team should not be building and managing its own
VR. I feel like that is a liability because the subset of developers who
are proficient in networking is quite small. That means we could be at
risk of losing the majority of our "experts" with a few people changing
their $dayjob. It feels safer to work with an existing technology which
has its own development community focused on doing that piece well.
Obviously this has its own drawbacks, but in general, we need the VR
implementation to be built by dedicated network engineers and not jack of
all trade developers. No offense to current company...
I agree with your list of what you would like to see. Rock solid and not
over complicated is key. That being said, if it ONLY handles what ACS
needs today, then WE have to be the ones to develop any changes. For
example; we need IPv6, VXLAN support, etc... My point is that if we only
focus on what we need today, then we end up building everything we need for
the future and I think we end up back where we are now down the road...
I love everything you are saying, just not sure I want us building and
maintaining it all...
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Mon, Sep 12, 2016 at 5:47 PM, Rene Moser <ma...@renemoser.net> wrote:
> Hi
>
> On 09/12/2016 10:20 PM, Will Stevens wrote:
> > *Disclaimer:* This is a thought experiment and should be treated as such.
> > Please weigh in with the good and bad of this idea...
> >
> > A couple of us have been discussing the idea of potentially replacing the
> > ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a license
> > issue because I think it is licensed under GPL, but for the sake of
> > discussion, let's assume we can overcome any license issues.
>
> VyOS is Debian based, much like the current VR. As long as it is not
> shipped with CloudStack, all fine.
>
> > I have spent some time recently with the VyOS and I have to admit, I was
> > pretty impressed. It is simple and intuitive and it gives you a lot more
> > options for auditing the configuration etc...
>
> I had the same "crazy" thoughts when I heard about VyOS the first time.
>
> When I looked at VyOS, the release cycle were not very frequent and the
> current stable release is still based on Debian 6 (EOL [1] since 02.16)
>
> However to me, it doesn't matter if it's VyOS or CloudLinux, or another
> solution.
>
> The question is more like what is wrong with the current VR and how can
> we make the VR great again. Things I would like to see:
>
> - VR must have a "clean", programmable, documented, API, supporting
> batch processing.
> - VR must be rock solid (minimal shell) state of the art, up to date,
> but small (Only contain things CloudStack needs, not more)
> - VR must be scale well (...) and support stateful HA
> - VR must be easy to upgrade (security) without downtimes.
>
> Christmas is soon... ;)
>
> René
>
> [1] https://www.debian.org/News/2016/20160212
>
>
Re: [DISCUSS] Replacing the VR
Posted by Rene Moser <ma...@renemoser.net>.
Hi
On 09/12/2016 10:20 PM, Will Stevens wrote:
> *Disclaimer:* This is a thought experiment and should be treated as such.
> Please weigh in with the good and bad of this idea...
>
> A couple of us have been discussing the idea of potentially replacing the
> ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a license
> issue because I think it is licensed under GPL, but for the sake of
> discussion, let's assume we can overcome any license issues.
VyOS is Debian based, much like the current VR. As long as it is not
shipped with CloudStack, all fine.
> I have spent some time recently with the VyOS and I have to admit, I was
> pretty impressed. It is simple and intuitive and it gives you a lot more
> options for auditing the configuration etc...
I had the same "crazy" thoughts when I heard about VyOS the first time.
When I looked at VyOS, the release cycle were not very frequent and the
current stable release is still based on Debian 6 (EOL [1] since 02.16)
However to me, it doesn't matter if it's VyOS or CloudLinux, or another
solution.
The question is more like what is wrong with the current VR and how can
we make the VR great again. Things I would like to see:
- VR must have a "clean", programmable, documented, API, supporting
batch processing.
- VR must be rock solid (minimal shell) state of the art, up to date,
but small (Only contain things CloudStack needs, not more)
- VR must be scale well (...) and support stateful HA
- VR must be easy to upgrade (security) without downtimes.
Christmas is soon... ;)
Ren�
[1] https://www.debian.org/News/2016/20160212
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <wi...@gmail.com>.
Ya. CloudRouter is interesting because it has a native api. For that reason
it was brought up as an alternative to VyOS in our internal discussions.
On Sep 13, 2016 5:23 AM, "Nux!" <nu...@li.nux.ro> wrote:
> Hi,
>
> I like the idea.
>
> Cloudrouter looks really promising, I'm not too keen on VyOS (it doesn't
> have a proper http api etc).
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Will Stevens" <wi...@gmail.com>
> > To: dev@cloudstack.apache.org
> > Sent: Monday, 12 September, 2016 21:20:11
> > Subject: [DISCUSS] Replacing the VR
>
> > *Disclaimer:* This is a thought experiment and should be treated as such.
> > Please weigh in with the good and bad of this idea...
> >
> > A couple of us have been discussing the idea of potentially replacing the
> > ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a license
> > issue because I think it is licensed under GPL, but for the sake of
> > discussion, let's assume we can overcome any license issues.
> >
> > I have spent some time recently with the VyOS and I have to admit, I was
> > pretty impressed. It is simple and intuitive and it gives you a lot more
> > options for auditing the configuration etc...
> >
> > Items of potential interest:
> > - Clean up our current VR script spaghetti to a simpler more auditable
> > configuration workflow.
> > - Gives a cleaner path for IPv6 support.
> > - Handles VPN configuration via the same configuration interface.
> > - Support for OSPF & BGP.
> > - VPN support through OpenVPN & StrongSwan.
> > - Easily supports HA (redundant routers) through VRRP.
> > - VXLAN support.
> > - Transaction based changes to the VR with rollback on error.
> >
> > Items that could be difficult to solve:
> > - Userdata password reset workflow and implementation.
> > - Upgrade process.
> >
> > The VyOS is not the only option if we were to consider this approach.
> > Another option, which I don't know as well, would be CloudRouter (AGPL
> > license) [2] which is purely API driven.
> >
> > Anyway, would love to hear your thoughts...
> >
> > Will
> >
> > [1] https://vyos.io/
> > [2] https://cloudrouter.org/
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
Ya, I can't find anything either. So that means that we would be working
with the API of each component, which I don't really like. It means we
don't have a single API endpoint to handle things like IPSec, etc (or even
an API at all for a lot of the functionality). :(
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Tue, Sep 13, 2016 at 3:52 PM, Marty Godsey <ma...@gonsource.com> wrote:
> Since Cloudrouter has a full distro for Opendaylight would you use that
> for the API? But I can't find anything about the Cloudrouter API if it does
> have one.
>
> Regards,
> Marty Godsey
> nSource Solutions
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: Tuesday, September 13, 2016 3:48 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> I can't seem to find any API documentation for CloudRouter. Maybe my
> Google foo is weak. Has anyone else found any usable docs on that?
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Tue, Sep 13, 2016 at 5:22 AM, Nux! <nu...@li.nux.ro> wrote:
>
> > Hi,
> >
> > I like the idea.
> >
> > Cloudrouter looks really promising, I'm not too keen on VyOS (it
> > doesn't have a proper http api etc).
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > ----- Original Message -----
> > > From: "Will Stevens" <wi...@gmail.com>
> > > To: dev@cloudstack.apache.org
> > > Sent: Monday, 12 September, 2016 21:20:11
> > > Subject: [DISCUSS] Replacing the VR
> >
> > > *Disclaimer:* This is a thought experiment and should be treated as
> such.
> > > Please weigh in with the good and bad of this idea...
> > >
> > > A couple of us have been discussing the idea of potentially
> > > replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
> > > There may be a license issue because I think it is licensed under
> > > GPL, but for the sake of discussion, let's assume we can overcome any
> license issues.
> > >
> > > I have spent some time recently with the VyOS and I have to admit, I
> > > was pretty impressed. It is simple and intuitive and it gives you a
> > > lot more options for auditing the configuration etc...
> > >
> > > Items of potential interest:
> > > - Clean up our current VR script spaghetti to a simpler more
> > > auditable configuration workflow.
> > > - Gives a cleaner path for IPv6 support.
> > > - Handles VPN configuration via the same configuration interface.
> > > - Support for OSPF & BGP.
> > > - VPN support through OpenVPN & StrongSwan.
> > > - Easily supports HA (redundant routers) through VRRP.
> > > - VXLAN support.
> > > - Transaction based changes to the VR with rollback on error.
> > >
> > > Items that could be difficult to solve:
> > > - Userdata password reset workflow and implementation.
> > > - Upgrade process.
> > >
> > > The VyOS is not the only option if we were to consider this approach.
> > > Another option, which I don't know as well, would be CloudRouter
> > > (AGPL
> > > license) [2] which is purely API driven.
> > >
> > > Anyway, would love to hear your thoughts...
> > >
> > > Will
> > >
> > > [1] https://vyos.io/
> > > [2] https://cloudrouter.org/
> >
>
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
Since Cloudrouter has a full distro for Opendaylight would you use that for the API? But I can't find anything about the Cloudrouter API if it does have one.
Regards,
Marty Godsey
nSource Solutions
-----Original Message-----
From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
Sent: Tuesday, September 13, 2016 3:48 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
I can't seem to find any API documentation for CloudRouter. Maybe my Google foo is weak. Has anyone else found any usable docs on that?
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_
On Tue, Sep 13, 2016 at 5:22 AM, Nux! <nu...@li.nux.ro> wrote:
> Hi,
>
> I like the idea.
>
> Cloudrouter looks really promising, I'm not too keen on VyOS (it
> doesn't have a proper http api etc).
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Will Stevens" <wi...@gmail.com>
> > To: dev@cloudstack.apache.org
> > Sent: Monday, 12 September, 2016 21:20:11
> > Subject: [DISCUSS] Replacing the VR
>
> > *Disclaimer:* This is a thought experiment and should be treated as such.
> > Please weigh in with the good and bad of this idea...
> >
> > A couple of us have been discussing the idea of potentially
> > replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
> > There may be a license issue because I think it is licensed under
> > GPL, but for the sake of discussion, let's assume we can overcome any license issues.
> >
> > I have spent some time recently with the VyOS and I have to admit, I
> > was pretty impressed. It is simple and intuitive and it gives you a
> > lot more options for auditing the configuration etc...
> >
> > Items of potential interest:
> > - Clean up our current VR script spaghetti to a simpler more
> > auditable configuration workflow.
> > - Gives a cleaner path for IPv6 support.
> > - Handles VPN configuration via the same configuration interface.
> > - Support for OSPF & BGP.
> > - VPN support through OpenVPN & StrongSwan.
> > - Easily supports HA (redundant routers) through VRRP.
> > - VXLAN support.
> > - Transaction based changes to the VR with rollback on error.
> >
> > Items that could be difficult to solve:
> > - Userdata password reset workflow and implementation.
> > - Upgrade process.
> >
> > The VyOS is not the only option if we were to consider this approach.
> > Another option, which I don't know as well, would be CloudRouter
> > (AGPL
> > license) [2] which is purely API driven.
> >
> > Anyway, would love to hear your thoughts...
> >
> > Will
> >
> > [1] https://vyos.io/
> > [2] https://cloudrouter.org/
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
I can't seem to find any API documentation for CloudRouter. Maybe my
Google foo is weak. Has anyone else found any usable docs on that?
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Tue, Sep 13, 2016 at 5:22 AM, Nux! <nu...@li.nux.ro> wrote:
> Hi,
>
> I like the idea.
>
> Cloudrouter looks really promising, I'm not too keen on VyOS (it doesn't
> have a proper http api etc).
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Will Stevens" <wi...@gmail.com>
> > To: dev@cloudstack.apache.org
> > Sent: Monday, 12 September, 2016 21:20:11
> > Subject: [DISCUSS] Replacing the VR
>
> > *Disclaimer:* This is a thought experiment and should be treated as such.
> > Please weigh in with the good and bad of this idea...
> >
> > A couple of us have been discussing the idea of potentially replacing the
> > ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a license
> > issue because I think it is licensed under GPL, but for the sake of
> > discussion, let's assume we can overcome any license issues.
> >
> > I have spent some time recently with the VyOS and I have to admit, I was
> > pretty impressed. It is simple and intuitive and it gives you a lot more
> > options for auditing the configuration etc...
> >
> > Items of potential interest:
> > - Clean up our current VR script spaghetti to a simpler more auditable
> > configuration workflow.
> > - Gives a cleaner path for IPv6 support.
> > - Handles VPN configuration via the same configuration interface.
> > - Support for OSPF & BGP.
> > - VPN support through OpenVPN & StrongSwan.
> > - Easily supports HA (redundant routers) through VRRP.
> > - VXLAN support.
> > - Transaction based changes to the VR with rollback on error.
> >
> > Items that could be difficult to solve:
> > - Userdata password reset workflow and implementation.
> > - Upgrade process.
> >
> > The VyOS is not the only option if we were to consider this approach.
> > Another option, which I don't know as well, would be CloudRouter (AGPL
> > license) [2] which is purely API driven.
> >
> > Anyway, would love to hear your thoughts...
> >
> > Will
> >
> > [1] https://vyos.io/
> > [2] https://cloudrouter.org/
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <wi...@gmail.com>.
I think there are other options for the vyos, but none are elegant from
what I have seen so far.
On Sep 13, 2016 1:14 PM, "Will Stevens" <wi...@gmail.com> wrote:
> Yes, not pretty, but this would probably be the best option for remote
> management of the vyos.
>
> https://github.com/vyos/python-vyos-mgmt
>
> On Sep 13, 2016 1:08 PM, "Simon Weller" <sw...@ena.com> wrote:
>
>> Yeah, today, the agent uses a script to inject CLI via ssh into the VR.
>> It's really (did I mention really) ugly.
>>
>> ________________________________
>> From: Marty Godsey <ma...@gonsource.com>
>> Sent: Tuesday, September 13, 2016 12:05 PM
>> To: dev@cloudstack.apache.org
>> Subject: RE: [DISCUSS] Replacing the VR
>>
>> So it looks like we are eliminating CloudRouter. To many missing or non
>> API managed features.
>>
>> But in reading VyOS, it also does not have a remote management API so
>> this would have to be a "SSH to CLI" job.
>>
>> If I am not mistaken is this how it's done now?
>>
>> -----Original Message-----
>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>> Behalf Of Will Stevens
>> Sent: Tuesday, September 13, 2016 12:58 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [DISCUSS] Replacing the VR
>>
>> Judging from this, it does not look like IPSec is managed via the API
>> though:
>> https://cloudrouter.atlassian.net/wiki/display/CPD/Bridging+
>> Public+Clouds+with+CloudRouter
>>
>> I believe it is also missing VXLAN and VRRP, but I have not dug into it
>> yet.
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>> @CloudOps_
>>
>> On Tue, Sep 13, 2016 at 12:53 PM, Marty Godsey <ma...@gonsource.com>
>> wrote:
>>
>> > It does.. Its under Secure Connectivity.
>> >
>> > -----Original Message-----
>> > From: Syed Ahmed [mailto:sahmed@cloudops.com]
>> > Sent: Tuesday, September 13, 2016 12:49 PM
>> > To: dev@cloudstack.apache.org
>> > Subject: Re: [DISCUSS] Replacing the VR
>> >
>> > Does CloudRouter provide VPN (site-site and client)? Looking from
>> > their website I don't seem to find it. Also, missing is VVRP for
>> > redundancy. Has anyone used it for this?
>> >
>> > On Tue, Sep 13, 2016 at 12:43 PM, Zaeem Arshad
>> > <za...@gmail.com>
>> > wrote:
>> >
>> > > +1 on cloudrouter. We have been looking at this as a potential
>> > > replacement/addon to our existing VRs.
>> > >
>> > > On Tue, Sep 13, 2016 at 9:07 PM, Will Stevens
>> > > <ws...@cloudops.com>
>> > > wrote:
>> > >
>> > > > yes, technically we should be able to just make a new VR
>> > > > implementation available and then when people create their network
>> > > > offerings, they just pick which VR implementation they want to use
>> > > > for the different capabilities.
>> > > >
>> > > > *Will STEVENS*
>> > > > Lead Developer
>> > > >
>> > > > *CloudOps* *| *Cloud Solutions Experts
>> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>> > > > tw @CloudOps_
>> > > >
>> > > > On Tue, Sep 13, 2016 at 11:41 AM, Dustin Wright <
>> > > > dwright@untangledtechnology.com> wrote:
>> > > >
>> > > > > I would like to see a "virtual router offering" in the UI which
>> > > > > lets
>> > > you
>> > > > > pick the legacy VR or your own. Probably a component of the
>> > > > > network offering. I've had many cases were I needed Mikrotik
>> > > > > RouterOS or
>> > > pfSense
>> > > > to
>> > > > > match a clients on-premise gear. ACS should find a way to stay
>> > > > > agnostic IMO.
>> > > > >
>> > > > > On Tue, Sep 13, 2016 at 11:36 AM, Marty Godsey
>> > > > > <ma...@gonsource.com>
>> > > > > wrote:
>> > > > >
>> > > > > > I like this idea as well. I would be willing to test both VyOS
>> > > > > > and Cloudrouter if you need an active service to test with.
>> > > > > > Specifically
>> > > I
>> > > > am
>> > > > > > looking to test IPv6 since I provide IPv6 /64 spaces to my
>> > > > > > customers
>> > > > and
>> > > > > I
>> > > > > > am having to provide it via an external router at the moment
>> > > > > > which
>> > > has
>> > > > a
>> > > > > > lot of manual configurations.
>> > > > > >
>> > > > > > Let me know if I can help in anyway.
>> > > > > >
>> > > > > > -----Original Message-----
>> > > > > > From: Will Stevens [mailto:williamstevens@gmail.com]
>> > > > > > Sent: Tuesday, September 13, 2016 7:21 AM
>> > > > > > To: dev@cloudstack.apache.org
>> > > > > > Subject: Re: [DISCUSS] Replacing the VR
>> > > > > >
>> > > > > > Ya. If we go this way, I like the approach of building the
>> > > integration
>> > > > > and
>> > > > > > putting it through its paces as a stand alone VR before we
>> > > > > > consider replacing the old VR and making it the default.
>> > > > > >
>> > > > > > On Sep 13, 2016 6:52 AM, "Jayapal Uradi" <
>> > > jayapal.uradi@accelerite.com
>> > > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi,
>> > > > > > >
>> > > > > > > Instead of replacing the VR in first place we should add
>> > > > > > > VyOS/cloudrouter as provider. Once it is stable, network
>> > > > > > > offerings
>> > > > (on
>> > > > > > > upgrade) can be updated to use it and we can drop the VR if
>> > > > > > > we want
>> > > > at
>> > > > > > that release onwards.
>> > > > > > >
>> > > > > > > VR is stabilized over a period of time and some of them are
>> > > > > > > running without issues. When we replicate the ACS VR
>> > > > > > > features in new
>> > > > solution
>> > > > > > > it takes some to find the missing pieces (hidden bugs).
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > > Jayapal
>> > > > > > >
>> > > > > > > > On Sep 13, 2016, at 2:52 PM, Nux! <
>> > > > > > >
>> > > > > > > > nux@li.nux.ro> wrote:
>> > > > > > > >
>> > > > > > > > Hi,
>> > > > > > > >
>> > > > > > > > I like the idea.
>> > > > > > > >
>> > > > > > > > Cloudrouter looks really promising, I'm not too keen on
>> > > > > > > > VyOS (it doesn't
>> > > > > > > have a proper http api etc).
>> > > > > > > >
>> > > > > > > > --
>> > > > > > > > Sent from the Delta quadrant using Borg technology!
>> > > > > > > >
>> > > > > > > > Nux!
>> > > > > > > > www.nux.ro
>> > > > > > > >
>> > > > > > > > ----- Original Message -----
>> > > > > > > >> From: "Will Stevens" <wi...@gmail.com>
>> > > > > > > >> To: dev@cloudstack.apache.org
>> > > > > > > >> Sent: Monday, 12 September, 2016 21:20:11
>> > > > > > > >> Subject: [DISCUSS] Replacing the VR
>> > > > > > > >
>> > > > > > > >> *Disclaimer:* This is a thought experiment and should be
>> > > > > > > >> treated
>> > > > as
>> > > > > > > such.
>> > > > > > > >> Please weigh in with the good and bad of this idea...
>> > > > > > > >>
>> > > > > > > >> A couple of us have been discussing the idea of
>> > > > > > > >> potentially replacing
>> > > > > > > the
>> > > > > > > >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There
>> > > > > > > >> may be
>> > > a
>> > > > > > > license
>> > > > > > > >> issue because I think it is licensed under GPL, but for
>> > > > > > > >> the sake
>> > > > of
>> > > > > > > >> discussion, let's assume we can overcome any license
>> issues.
>> > > > > > > >>
>> > > > > > > >> I have spent some time recently with the VyOS and I have
>> > > > > > > >> to
>> > > admit,
>> > > > > > > >> I was pretty impressed. It is simple and intuitive and
>> > > > > > > >> it gives you a lot
>> > > > > > > more
>> > > > > > > >> options for auditing the configuration etc...
>> > > > > > > >>
>> > > > > > > >> Items of potential interest:
>> > > > > > > >> - Clean up our current VR script spaghetti to a simpler
>> > > > > > > >> more auditable configuration workflow.
>> > > > > > > >> - Gives a cleaner path for IPv6 support.
>> > > > > > > >> - Handles VPN configuration via the same configuration
>> > > interface.
>> > > > > > > >> - Support for OSPF & BGP.
>> > > > > > > >> - VPN support through OpenVPN & StrongSwan.
>> > > > > > > >> - Easily supports HA (redundant routers) through VRRP.
>> > > > > > > >> - VXLAN support.
>> > > > > > > >> - Transaction based changes to the VR with rollback on
>> error.
>> > > > > > > >>
>> > > > > > > >> Items that could be difficult to solve:
>> > > > > > > >> - Userdata password reset workflow and implementation.
>> > > > > > > >> - Upgrade process.
>> > > > > > > >>
>> > > > > > > >> The VyOS is not the only option if we were to consider
>> > > > > > > >> this
>> > > > > approach.
>> > > > > > > >> Another option, which I don't know as well, would be
>> > > > > > > >> CloudRouter (AGPL
>> > > > > > > >> license) [2] which is purely API driven.
>> > > > > > > >>
>> > > > > > > >> Anyway, would love to hear your thoughts...
>> > > > > > > >>
>> > > > > > > >> Will
>> > > > > > > >>
>> > > > > > > >> [1] https://vyos.io/
>> > > > > > > >> [2] https://cloudrouter.org/
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > DISCLAIMER
>> > > > > > > ==========
>> > > > > > > This e-mail may contain privileged and confidential
>> > > > > > > information
>> > > which
>> > > > > > > is the property of Accelerite, a Persistent Systems business.
>> > > > > > > It is intended only for the use of the individual or entity
>> > > > > > > to which it
>> > > is
>> > > > > > > addressed. If you are not the intended recipient, you are
>> > > > > > > not authorized to read, retain, copy, print, distribute or
>> > > > > > > use this message. If you have received this communication in
>> > > > > > > error, please notify the sender and delete all copies of
>> > > > > > > this
>> > message.
>> > > Accelerite,
>> > > > a
>> > > > > > > Persistent Systems business does not accept any liability
>> > > > > > > for virus
>> > > > > > infected mails.
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <wi...@gmail.com>.
Yes, not pretty, but this would probably be the best option for remote
management of the vyos.
https://github.com/vyos/python-vyos-mgmt
On Sep 13, 2016 1:08 PM, "Simon Weller" <sw...@ena.com> wrote:
> Yeah, today, the agent uses a script to inject CLI via ssh into the VR.
> It's really (did I mention really) ugly.
>
> ________________________________
> From: Marty Godsey <ma...@gonsource.com>
> Sent: Tuesday, September 13, 2016 12:05 PM
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] Replacing the VR
>
> So it looks like we are eliminating CloudRouter. To many missing or non
> API managed features.
>
> But in reading VyOS, it also does not have a remote management API so this
> would have to be a "SSH to CLI" job.
>
> If I am not mistaken is this how it's done now?
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: Tuesday, September 13, 2016 12:58 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Judging from this, it does not look like IPSec is managed via the API
> though:
> https://cloudrouter.atlassian.net/wiki/display/CPD/Bridging+
> Public+Clouds+with+CloudRouter
>
> I believe it is also missing VXLAN and VRRP, but I have not dug into it
> yet.
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Tue, Sep 13, 2016 at 12:53 PM, Marty Godsey <ma...@gonsource.com>
> wrote:
>
> > It does.. Its under Secure Connectivity.
> >
> > -----Original Message-----
> > From: Syed Ahmed [mailto:sahmed@cloudops.com]
> > Sent: Tuesday, September 13, 2016 12:49 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] Replacing the VR
> >
> > Does CloudRouter provide VPN (site-site and client)? Looking from
> > their website I don't seem to find it. Also, missing is VVRP for
> > redundancy. Has anyone used it for this?
> >
> > On Tue, Sep 13, 2016 at 12:43 PM, Zaeem Arshad
> > <za...@gmail.com>
> > wrote:
> >
> > > +1 on cloudrouter. We have been looking at this as a potential
> > > replacement/addon to our existing VRs.
> > >
> > > On Tue, Sep 13, 2016 at 9:07 PM, Will Stevens
> > > <ws...@cloudops.com>
> > > wrote:
> > >
> > > > yes, technically we should be able to just make a new VR
> > > > implementation available and then when people create their network
> > > > offerings, they just pick which VR implementation they want to use
> > > > for the different capabilities.
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > *CloudOps* *| *Cloud Solutions Experts
> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > > > tw @CloudOps_
> > > >
> > > > On Tue, Sep 13, 2016 at 11:41 AM, Dustin Wright <
> > > > dwright@untangledtechnology.com> wrote:
> > > >
> > > > > I would like to see a "virtual router offering" in the UI which
> > > > > lets
> > > you
> > > > > pick the legacy VR or your own. Probably a component of the
> > > > > network offering. I've had many cases were I needed Mikrotik
> > > > > RouterOS or
> > > pfSense
> > > > to
> > > > > match a clients on-premise gear. ACS should find a way to stay
> > > > > agnostic IMO.
> > > > >
> > > > > On Tue, Sep 13, 2016 at 11:36 AM, Marty Godsey
> > > > > <ma...@gonsource.com>
> > > > > wrote:
> > > > >
> > > > > > I like this idea as well. I would be willing to test both VyOS
> > > > > > and Cloudrouter if you need an active service to test with.
> > > > > > Specifically
> > > I
> > > > am
> > > > > > looking to test IPv6 since I provide IPv6 /64 spaces to my
> > > > > > customers
> > > > and
> > > > > I
> > > > > > am having to provide it via an external router at the moment
> > > > > > which
> > > has
> > > > a
> > > > > > lot of manual configurations.
> > > > > >
> > > > > > Let me know if I can help in anyway.
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Will Stevens [mailto:williamstevens@gmail.com]
> > > > > > Sent: Tuesday, September 13, 2016 7:21 AM
> > > > > > To: dev@cloudstack.apache.org
> > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > >
> > > > > > Ya. If we go this way, I like the approach of building the
> > > integration
> > > > > and
> > > > > > putting it through its paces as a stand alone VR before we
> > > > > > consider replacing the old VR and making it the default.
> > > > > >
> > > > > > On Sep 13, 2016 6:52 AM, "Jayapal Uradi" <
> > > jayapal.uradi@accelerite.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Instead of replacing the VR in first place we should add
> > > > > > > VyOS/cloudrouter as provider. Once it is stable, network
> > > > > > > offerings
> > > > (on
> > > > > > > upgrade) can be updated to use it and we can drop the VR if
> > > > > > > we want
> > > > at
> > > > > > that release onwards.
> > > > > > >
> > > > > > > VR is stabilized over a period of time and some of them are
> > > > > > > running without issues. When we replicate the ACS VR
> > > > > > > features in new
> > > > solution
> > > > > > > it takes some to find the missing pieces (hidden bugs).
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Jayapal
> > > > > > >
> > > > > > > > On Sep 13, 2016, at 2:52 PM, Nux! <
> > > > > > >
> > > > > > > > nux@li.nux.ro> wrote:
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I like the idea.
> > > > > > > >
> > > > > > > > Cloudrouter looks really promising, I'm not too keen on
> > > > > > > > VyOS (it doesn't
> > > > > > > have a proper http api etc).
> > > > > > > >
> > > > > > > > --
> > > > > > > > Sent from the Delta quadrant using Borg technology!
> > > > > > > >
> > > > > > > > Nux!
> > > > > > > > www.nux.ro
> > > > > > > >
> > > > > > > > ----- Original Message -----
> > > > > > > >> From: "Will Stevens" <wi...@gmail.com>
> > > > > > > >> To: dev@cloudstack.apache.org
> > > > > > > >> Sent: Monday, 12 September, 2016 21:20:11
> > > > > > > >> Subject: [DISCUSS] Replacing the VR
> > > > > > > >
> > > > > > > >> *Disclaimer:* This is a thought experiment and should be
> > > > > > > >> treated
> > > > as
> > > > > > > such.
> > > > > > > >> Please weigh in with the good and bad of this idea...
> > > > > > > >>
> > > > > > > >> A couple of us have been discussing the idea of
> > > > > > > >> potentially replacing
> > > > > > > the
> > > > > > > >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There
> > > > > > > >> may be
> > > a
> > > > > > > license
> > > > > > > >> issue because I think it is licensed under GPL, but for
> > > > > > > >> the sake
> > > > of
> > > > > > > >> discussion, let's assume we can overcome any license issues.
> > > > > > > >>
> > > > > > > >> I have spent some time recently with the VyOS and I have
> > > > > > > >> to
> > > admit,
> > > > > > > >> I was pretty impressed. It is simple and intuitive and
> > > > > > > >> it gives you a lot
> > > > > > > more
> > > > > > > >> options for auditing the configuration etc...
> > > > > > > >>
> > > > > > > >> Items of potential interest:
> > > > > > > >> - Clean up our current VR script spaghetti to a simpler
> > > > > > > >> more auditable configuration workflow.
> > > > > > > >> - Gives a cleaner path for IPv6 support.
> > > > > > > >> - Handles VPN configuration via the same configuration
> > > interface.
> > > > > > > >> - Support for OSPF & BGP.
> > > > > > > >> - VPN support through OpenVPN & StrongSwan.
> > > > > > > >> - Easily supports HA (redundant routers) through VRRP.
> > > > > > > >> - VXLAN support.
> > > > > > > >> - Transaction based changes to the VR with rollback on
> error.
> > > > > > > >>
> > > > > > > >> Items that could be difficult to solve:
> > > > > > > >> - Userdata password reset workflow and implementation.
> > > > > > > >> - Upgrade process.
> > > > > > > >>
> > > > > > > >> The VyOS is not the only option if we were to consider
> > > > > > > >> this
> > > > > approach.
> > > > > > > >> Another option, which I don't know as well, would be
> > > > > > > >> CloudRouter (AGPL
> > > > > > > >> license) [2] which is purely API driven.
> > > > > > > >>
> > > > > > > >> Anyway, would love to hear your thoughts...
> > > > > > > >>
> > > > > > > >> Will
> > > > > > > >>
> > > > > > > >> [1] https://vyos.io/
> > > > > > > >> [2] https://cloudrouter.org/
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > DISCLAIMER
> > > > > > > ==========
> > > > > > > This e-mail may contain privileged and confidential
> > > > > > > information
> > > which
> > > > > > > is the property of Accelerite, a Persistent Systems business.
> > > > > > > It is intended only for the use of the individual or entity
> > > > > > > to which it
> > > is
> > > > > > > addressed. If you are not the intended recipient, you are
> > > > > > > not authorized to read, retain, copy, print, distribute or
> > > > > > > use this message. If you have received this communication in
> > > > > > > error, please notify the sender and delete all copies of
> > > > > > > this
> > message.
> > > Accelerite,
> > > > a
> > > > > > > Persistent Systems business does not accept any liability
> > > > > > > for virus
> > > > > > infected mails.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Simon Weller <sw...@ena.com>.
Yeah, today, the agent uses a script to inject CLI via ssh into the VR. It's really (did I mention really) ugly.
________________________________
From: Marty Godsey <ma...@gonsource.com>
Sent: Tuesday, September 13, 2016 12:05 PM
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] Replacing the VR
So it looks like we are eliminating CloudRouter. To many missing or non API managed features.
But in reading VyOS, it also does not have a remote management API so this would have to be a "SSH to CLI" job.
If I am not mistaken is this how it's done now?
-----Original Message-----
From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
Sent: Tuesday, September 13, 2016 12:58 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
Judging from this, it does not look like IPSec is managed via the API
though:
https://cloudrouter.atlassian.net/wiki/display/CPD/Bridging+Public+Clouds+with+CloudRouter
I believe it is also missing VXLAN and VRRP, but I have not dug into it yet.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_
On Tue, Sep 13, 2016 at 12:53 PM, Marty Godsey <ma...@gonsource.com> wrote:
> It does.. Its under Secure Connectivity.
>
> -----Original Message-----
> From: Syed Ahmed [mailto:sahmed@cloudops.com]
> Sent: Tuesday, September 13, 2016 12:49 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Does CloudRouter provide VPN (site-site and client)? Looking from
> their website I don't seem to find it. Also, missing is VVRP for
> redundancy. Has anyone used it for this?
>
> On Tue, Sep 13, 2016 at 12:43 PM, Zaeem Arshad
> <za...@gmail.com>
> wrote:
>
> > +1 on cloudrouter. We have been looking at this as a potential
> > replacement/addon to our existing VRs.
> >
> > On Tue, Sep 13, 2016 at 9:07 PM, Will Stevens
> > <ws...@cloudops.com>
> > wrote:
> >
> > > yes, technically we should be able to just make a new VR
> > > implementation available and then when people create their network
> > > offerings, they just pick which VR implementation they want to use
> > > for the different capabilities.
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > > tw @CloudOps_
> > >
> > > On Tue, Sep 13, 2016 at 11:41 AM, Dustin Wright <
> > > dwright@untangledtechnology.com> wrote:
> > >
> > > > I would like to see a "virtual router offering" in the UI which
> > > > lets
> > you
> > > > pick the legacy VR or your own. Probably a component of the
> > > > network offering. I've had many cases were I needed Mikrotik
> > > > RouterOS or
> > pfSense
> > > to
> > > > match a clients on-premise gear. ACS should find a way to stay
> > > > agnostic IMO.
> > > >
> > > > On Tue, Sep 13, 2016 at 11:36 AM, Marty Godsey
> > > > <ma...@gonsource.com>
> > > > wrote:
> > > >
> > > > > I like this idea as well. I would be willing to test both VyOS
> > > > > and Cloudrouter if you need an active service to test with.
> > > > > Specifically
> > I
> > > am
> > > > > looking to test IPv6 since I provide IPv6 /64 spaces to my
> > > > > customers
> > > and
> > > > I
> > > > > am having to provide it via an external router at the moment
> > > > > which
> > has
> > > a
> > > > > lot of manual configurations.
> > > > >
> > > > > Let me know if I can help in anyway.
> > > > >
> > > > > -----Original Message-----
> > > > > From: Will Stevens [mailto:williamstevens@gmail.com]
> > > > > Sent: Tuesday, September 13, 2016 7:21 AM
> > > > > To: dev@cloudstack.apache.org
> > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > >
> > > > > Ya. If we go this way, I like the approach of building the
> > integration
> > > > and
> > > > > putting it through its paces as a stand alone VR before we
> > > > > consider replacing the old VR and making it the default.
> > > > >
> > > > > On Sep 13, 2016 6:52 AM, "Jayapal Uradi" <
> > jayapal.uradi@accelerite.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Instead of replacing the VR in first place we should add
> > > > > > VyOS/cloudrouter as provider. Once it is stable, network
> > > > > > offerings
> > > (on
> > > > > > upgrade) can be updated to use it and we can drop the VR if
> > > > > > we want
> > > at
> > > > > that release onwards.
> > > > > >
> > > > > > VR is stabilized over a period of time and some of them are
> > > > > > running without issues. When we replicate the ACS VR
> > > > > > features in new
> > > solution
> > > > > > it takes some to find the missing pieces (hidden bugs).
> > > > > >
> > > > > > Thanks,
> > > > > > Jayapal
> > > > > >
> > > > > > > On Sep 13, 2016, at 2:52 PM, Nux! <
> > > > > >
> > > > > > > nux@li.nux.ro> wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I like the idea.
> > > > > > >
> > > > > > > Cloudrouter looks really promising, I'm not too keen on
> > > > > > > VyOS (it doesn't
> > > > > > have a proper http api etc).
> > > > > > >
> > > > > > > --
> > > > > > > Sent from the Delta quadrant using Borg technology!
> > > > > > >
> > > > > > > Nux!
> > > > > > > www.nux.ro
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > >> From: "Will Stevens" <wi...@gmail.com>
> > > > > > >> To: dev@cloudstack.apache.org
> > > > > > >> Sent: Monday, 12 September, 2016 21:20:11
> > > > > > >> Subject: [DISCUSS] Replacing the VR
> > > > > > >
> > > > > > >> *Disclaimer:* This is a thought experiment and should be
> > > > > > >> treated
> > > as
> > > > > > such.
> > > > > > >> Please weigh in with the good and bad of this idea...
> > > > > > >>
> > > > > > >> A couple of us have been discussing the idea of
> > > > > > >> potentially replacing
> > > > > > the
> > > > > > >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There
> > > > > > >> may be
> > a
> > > > > > license
> > > > > > >> issue because I think it is licensed under GPL, but for
> > > > > > >> the sake
> > > of
> > > > > > >> discussion, let's assume we can overcome any license issues.
> > > > > > >>
> > > > > > >> I have spent some time recently with the VyOS and I have
> > > > > > >> to
> > admit,
> > > > > > >> I was pretty impressed. It is simple and intuitive and
> > > > > > >> it gives you a lot
> > > > > > more
> > > > > > >> options for auditing the configuration etc...
> > > > > > >>
> > > > > > >> Items of potential interest:
> > > > > > >> - Clean up our current VR script spaghetti to a simpler
> > > > > > >> more auditable configuration workflow.
> > > > > > >> - Gives a cleaner path for IPv6 support.
> > > > > > >> - Handles VPN configuration via the same configuration
> > interface.
> > > > > > >> - Support for OSPF & BGP.
> > > > > > >> - VPN support through OpenVPN & StrongSwan.
> > > > > > >> - Easily supports HA (redundant routers) through VRRP.
> > > > > > >> - VXLAN support.
> > > > > > >> - Transaction based changes to the VR with rollback on error.
> > > > > > >>
> > > > > > >> Items that could be difficult to solve:
> > > > > > >> - Userdata password reset workflow and implementation.
> > > > > > >> - Upgrade process.
> > > > > > >>
> > > > > > >> The VyOS is not the only option if we were to consider
> > > > > > >> this
> > > > approach.
> > > > > > >> Another option, which I don't know as well, would be
> > > > > > >> CloudRouter (AGPL
> > > > > > >> license) [2] which is purely API driven.
> > > > > > >>
> > > > > > >> Anyway, would love to hear your thoughts...
> > > > > > >>
> > > > > > >> Will
> > > > > > >>
> > > > > > >> [1] https://vyos.io/
> > > > > > >> [2] https://cloudrouter.org/
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > DISCLAIMER
> > > > > > ==========
> > > > > > This e-mail may contain privileged and confidential
> > > > > > information
> > which
> > > > > > is the property of Accelerite, a Persistent Systems business.
> > > > > > It is intended only for the use of the individual or entity
> > > > > > to which it
> > is
> > > > > > addressed. If you are not the intended recipient, you are
> > > > > > not authorized to read, retain, copy, print, distribute or
> > > > > > use this message. If you have received this communication in
> > > > > > error, please notify the sender and delete all copies of
> > > > > > this
> message.
> > Accelerite,
> > > a
> > > > > > Persistent Systems business does not accept any liability
> > > > > > for virus
> > > > > infected mails.
> > > > > >
> > > > >
> > > >
> > >
> >
>
RE: [DISCUSS] Replacing the VR
Posted by Will Stevens <wi...@gmail.com>.
I am trying to find their api documentation to get a feel for what
cloudrouter can do via the API.
On Sep 13, 2016 1:05 PM, "Marty Godsey" <ma...@gonsource.com> wrote:
> So it looks like we are eliminating CloudRouter. To many missing or non
> API managed features.
>
> But in reading VyOS, it also does not have a remote management API so this
> would have to be a "SSH to CLI" job.
>
> If I am not mistaken is this how it's done now?
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: Tuesday, September 13, 2016 12:58 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Judging from this, it does not look like IPSec is managed via the API
> though:
> https://cloudrouter.atlassian.net/wiki/display/CPD/Bridging+
> Public+Clouds+with+CloudRouter
>
> I believe it is also missing VXLAN and VRRP, but I have not dug into it
> yet.
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Tue, Sep 13, 2016 at 12:53 PM, Marty Godsey <ma...@gonsource.com>
> wrote:
>
> > It does.. Its under Secure Connectivity.
> >
> > -----Original Message-----
> > From: Syed Ahmed [mailto:sahmed@cloudops.com]
> > Sent: Tuesday, September 13, 2016 12:49 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] Replacing the VR
> >
> > Does CloudRouter provide VPN (site-site and client)? Looking from
> > their website I don't seem to find it. Also, missing is VVRP for
> > redundancy. Has anyone used it for this?
> >
> > On Tue, Sep 13, 2016 at 12:43 PM, Zaeem Arshad
> > <za...@gmail.com>
> > wrote:
> >
> > > +1 on cloudrouter. We have been looking at this as a potential
> > > replacement/addon to our existing VRs.
> > >
> > > On Tue, Sep 13, 2016 at 9:07 PM, Will Stevens
> > > <ws...@cloudops.com>
> > > wrote:
> > >
> > > > yes, technically we should be able to just make a new VR
> > > > implementation available and then when people create their network
> > > > offerings, they just pick which VR implementation they want to use
> > > > for the different capabilities.
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > *CloudOps* *| *Cloud Solutions Experts
> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > > > tw @CloudOps_
> > > >
> > > > On Tue, Sep 13, 2016 at 11:41 AM, Dustin Wright <
> > > > dwright@untangledtechnology.com> wrote:
> > > >
> > > > > I would like to see a "virtual router offering" in the UI which
> > > > > lets
> > > you
> > > > > pick the legacy VR or your own. Probably a component of the
> > > > > network offering. I've had many cases were I needed Mikrotik
> > > > > RouterOS or
> > > pfSense
> > > > to
> > > > > match a clients on-premise gear. ACS should find a way to stay
> > > > > agnostic IMO.
> > > > >
> > > > > On Tue, Sep 13, 2016 at 11:36 AM, Marty Godsey
> > > > > <ma...@gonsource.com>
> > > > > wrote:
> > > > >
> > > > > > I like this idea as well. I would be willing to test both VyOS
> > > > > > and Cloudrouter if you need an active service to test with.
> > > > > > Specifically
> > > I
> > > > am
> > > > > > looking to test IPv6 since I provide IPv6 /64 spaces to my
> > > > > > customers
> > > > and
> > > > > I
> > > > > > am having to provide it via an external router at the moment
> > > > > > which
> > > has
> > > > a
> > > > > > lot of manual configurations.
> > > > > >
> > > > > > Let me know if I can help in anyway.
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Will Stevens [mailto:williamstevens@gmail.com]
> > > > > > Sent: Tuesday, September 13, 2016 7:21 AM
> > > > > > To: dev@cloudstack.apache.org
> > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > >
> > > > > > Ya. If we go this way, I like the approach of building the
> > > integration
> > > > > and
> > > > > > putting it through its paces as a stand alone VR before we
> > > > > > consider replacing the old VR and making it the default.
> > > > > >
> > > > > > On Sep 13, 2016 6:52 AM, "Jayapal Uradi" <
> > > jayapal.uradi@accelerite.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Instead of replacing the VR in first place we should add
> > > > > > > VyOS/cloudrouter as provider. Once it is stable, network
> > > > > > > offerings
> > > > (on
> > > > > > > upgrade) can be updated to use it and we can drop the VR if
> > > > > > > we want
> > > > at
> > > > > > that release onwards.
> > > > > > >
> > > > > > > VR is stabilized over a period of time and some of them are
> > > > > > > running without issues. When we replicate the ACS VR
> > > > > > > features in new
> > > > solution
> > > > > > > it takes some to find the missing pieces (hidden bugs).
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Jayapal
> > > > > > >
> > > > > > > > On Sep 13, 2016, at 2:52 PM, Nux! <
> > > > > > >
> > > > > > > > nux@li.nux.ro> wrote:
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I like the idea.
> > > > > > > >
> > > > > > > > Cloudrouter looks really promising, I'm not too keen on
> > > > > > > > VyOS (it doesn't
> > > > > > > have a proper http api etc).
> > > > > > > >
> > > > > > > > --
> > > > > > > > Sent from the Delta quadrant using Borg technology!
> > > > > > > >
> > > > > > > > Nux!
> > > > > > > > www.nux.ro
> > > > > > > >
> > > > > > > > ----- Original Message -----
> > > > > > > >> From: "Will Stevens" <wi...@gmail.com>
> > > > > > > >> To: dev@cloudstack.apache.org
> > > > > > > >> Sent: Monday, 12 September, 2016 21:20:11
> > > > > > > >> Subject: [DISCUSS] Replacing the VR
> > > > > > > >
> > > > > > > >> *Disclaimer:* This is a thought experiment and should be
> > > > > > > >> treated
> > > > as
> > > > > > > such.
> > > > > > > >> Please weigh in with the good and bad of this idea...
> > > > > > > >>
> > > > > > > >> A couple of us have been discussing the idea of
> > > > > > > >> potentially replacing
> > > > > > > the
> > > > > > > >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There
> > > > > > > >> may be
> > > a
> > > > > > > license
> > > > > > > >> issue because I think it is licensed under GPL, but for
> > > > > > > >> the sake
> > > > of
> > > > > > > >> discussion, let's assume we can overcome any license issues.
> > > > > > > >>
> > > > > > > >> I have spent some time recently with the VyOS and I have
> > > > > > > >> to
> > > admit,
> > > > > > > >> I was pretty impressed. It is simple and intuitive and
> > > > > > > >> it gives you a lot
> > > > > > > more
> > > > > > > >> options for auditing the configuration etc...
> > > > > > > >>
> > > > > > > >> Items of potential interest:
> > > > > > > >> - Clean up our current VR script spaghetti to a simpler
> > > > > > > >> more auditable configuration workflow.
> > > > > > > >> - Gives a cleaner path for IPv6 support.
> > > > > > > >> - Handles VPN configuration via the same configuration
> > > interface.
> > > > > > > >> - Support for OSPF & BGP.
> > > > > > > >> - VPN support through OpenVPN & StrongSwan.
> > > > > > > >> - Easily supports HA (redundant routers) through VRRP.
> > > > > > > >> - VXLAN support.
> > > > > > > >> - Transaction based changes to the VR with rollback on
> error.
> > > > > > > >>
> > > > > > > >> Items that could be difficult to solve:
> > > > > > > >> - Userdata password reset workflow and implementation.
> > > > > > > >> - Upgrade process.
> > > > > > > >>
> > > > > > > >> The VyOS is not the only option if we were to consider
> > > > > > > >> this
> > > > > approach.
> > > > > > > >> Another option, which I don't know as well, would be
> > > > > > > >> CloudRouter (AGPL
> > > > > > > >> license) [2] which is purely API driven.
> > > > > > > >>
> > > > > > > >> Anyway, would love to hear your thoughts...
> > > > > > > >>
> > > > > > > >> Will
> > > > > > > >>
> > > > > > > >> [1] https://vyos.io/
> > > > > > > >> [2] https://cloudrouter.org/
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > DISCLAIMER
> > > > > > > ==========
> > > > > > > This e-mail may contain privileged and confidential
> > > > > > > information
> > > which
> > > > > > > is the property of Accelerite, a Persistent Systems business.
> > > > > > > It is intended only for the use of the individual or entity
> > > > > > > to which it
> > > is
> > > > > > > addressed. If you are not the intended recipient, you are
> > > > > > > not authorized to read, retain, copy, print, distribute or
> > > > > > > use this message. If you have received this communication in
> > > > > > > error, please notify the sender and delete all copies of
> > > > > > > this
> > message.
> > > Accelerite,
> > > > a
> > > > > > > Persistent Systems business does not accept any liability
> > > > > > > for virus
> > > > > > infected mails.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
So it looks like we are eliminating CloudRouter. To many missing or non API managed features.
But in reading VyOS, it also does not have a remote management API so this would have to be a "SSH to CLI" job.
If I am not mistaken is this how it's done now?
-----Original Message-----
From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
Sent: Tuesday, September 13, 2016 12:58 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
Judging from this, it does not look like IPSec is managed via the API
though:
https://cloudrouter.atlassian.net/wiki/display/CPD/Bridging+Public+Clouds+with+CloudRouter
I believe it is also missing VXLAN and VRRP, but I have not dug into it yet.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_
On Tue, Sep 13, 2016 at 12:53 PM, Marty Godsey <ma...@gonsource.com> wrote:
> It does.. Its under Secure Connectivity.
>
> -----Original Message-----
> From: Syed Ahmed [mailto:sahmed@cloudops.com]
> Sent: Tuesday, September 13, 2016 12:49 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Does CloudRouter provide VPN (site-site and client)? Looking from
> their website I don't seem to find it. Also, missing is VVRP for
> redundancy. Has anyone used it for this?
>
> On Tue, Sep 13, 2016 at 12:43 PM, Zaeem Arshad
> <za...@gmail.com>
> wrote:
>
> > +1 on cloudrouter. We have been looking at this as a potential
> > replacement/addon to our existing VRs.
> >
> > On Tue, Sep 13, 2016 at 9:07 PM, Will Stevens
> > <ws...@cloudops.com>
> > wrote:
> >
> > > yes, technically we should be able to just make a new VR
> > > implementation available and then when people create their network
> > > offerings, they just pick which VR implementation they want to use
> > > for the different capabilities.
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > > tw @CloudOps_
> > >
> > > On Tue, Sep 13, 2016 at 11:41 AM, Dustin Wright <
> > > dwright@untangledtechnology.com> wrote:
> > >
> > > > I would like to see a "virtual router offering" in the UI which
> > > > lets
> > you
> > > > pick the legacy VR or your own. Probably a component of the
> > > > network offering. I've had many cases were I needed Mikrotik
> > > > RouterOS or
> > pfSense
> > > to
> > > > match a clients on-premise gear. ACS should find a way to stay
> > > > agnostic IMO.
> > > >
> > > > On Tue, Sep 13, 2016 at 11:36 AM, Marty Godsey
> > > > <ma...@gonsource.com>
> > > > wrote:
> > > >
> > > > > I like this idea as well. I would be willing to test both VyOS
> > > > > and Cloudrouter if you need an active service to test with.
> > > > > Specifically
> > I
> > > am
> > > > > looking to test IPv6 since I provide IPv6 /64 spaces to my
> > > > > customers
> > > and
> > > > I
> > > > > am having to provide it via an external router at the moment
> > > > > which
> > has
> > > a
> > > > > lot of manual configurations.
> > > > >
> > > > > Let me know if I can help in anyway.
> > > > >
> > > > > -----Original Message-----
> > > > > From: Will Stevens [mailto:williamstevens@gmail.com]
> > > > > Sent: Tuesday, September 13, 2016 7:21 AM
> > > > > To: dev@cloudstack.apache.org
> > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > >
> > > > > Ya. If we go this way, I like the approach of building the
> > integration
> > > > and
> > > > > putting it through its paces as a stand alone VR before we
> > > > > consider replacing the old VR and making it the default.
> > > > >
> > > > > On Sep 13, 2016 6:52 AM, "Jayapal Uradi" <
> > jayapal.uradi@accelerite.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Instead of replacing the VR in first place we should add
> > > > > > VyOS/cloudrouter as provider. Once it is stable, network
> > > > > > offerings
> > > (on
> > > > > > upgrade) can be updated to use it and we can drop the VR if
> > > > > > we want
> > > at
> > > > > that release onwards.
> > > > > >
> > > > > > VR is stabilized over a period of time and some of them are
> > > > > > running without issues. When we replicate the ACS VR
> > > > > > features in new
> > > solution
> > > > > > it takes some to find the missing pieces (hidden bugs).
> > > > > >
> > > > > > Thanks,
> > > > > > Jayapal
> > > > > >
> > > > > > > On Sep 13, 2016, at 2:52 PM, Nux! <
> > > > > >
> > > > > > > nux@li.nux.ro> wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I like the idea.
> > > > > > >
> > > > > > > Cloudrouter looks really promising, I'm not too keen on
> > > > > > > VyOS (it doesn't
> > > > > > have a proper http api etc).
> > > > > > >
> > > > > > > --
> > > > > > > Sent from the Delta quadrant using Borg technology!
> > > > > > >
> > > > > > > Nux!
> > > > > > > www.nux.ro
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > >> From: "Will Stevens" <wi...@gmail.com>
> > > > > > >> To: dev@cloudstack.apache.org
> > > > > > >> Sent: Monday, 12 September, 2016 21:20:11
> > > > > > >> Subject: [DISCUSS] Replacing the VR
> > > > > > >
> > > > > > >> *Disclaimer:* This is a thought experiment and should be
> > > > > > >> treated
> > > as
> > > > > > such.
> > > > > > >> Please weigh in with the good and bad of this idea...
> > > > > > >>
> > > > > > >> A couple of us have been discussing the idea of
> > > > > > >> potentially replacing
> > > > > > the
> > > > > > >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There
> > > > > > >> may be
> > a
> > > > > > license
> > > > > > >> issue because I think it is licensed under GPL, but for
> > > > > > >> the sake
> > > of
> > > > > > >> discussion, let's assume we can overcome any license issues.
> > > > > > >>
> > > > > > >> I have spent some time recently with the VyOS and I have
> > > > > > >> to
> > admit,
> > > > > > >> I was pretty impressed. It is simple and intuitive and
> > > > > > >> it gives you a lot
> > > > > > more
> > > > > > >> options for auditing the configuration etc...
> > > > > > >>
> > > > > > >> Items of potential interest:
> > > > > > >> - Clean up our current VR script spaghetti to a simpler
> > > > > > >> more auditable configuration workflow.
> > > > > > >> - Gives a cleaner path for IPv6 support.
> > > > > > >> - Handles VPN configuration via the same configuration
> > interface.
> > > > > > >> - Support for OSPF & BGP.
> > > > > > >> - VPN support through OpenVPN & StrongSwan.
> > > > > > >> - Easily supports HA (redundant routers) through VRRP.
> > > > > > >> - VXLAN support.
> > > > > > >> - Transaction based changes to the VR with rollback on error.
> > > > > > >>
> > > > > > >> Items that could be difficult to solve:
> > > > > > >> - Userdata password reset workflow and implementation.
> > > > > > >> - Upgrade process.
> > > > > > >>
> > > > > > >> The VyOS is not the only option if we were to consider
> > > > > > >> this
> > > > approach.
> > > > > > >> Another option, which I don't know as well, would be
> > > > > > >> CloudRouter (AGPL
> > > > > > >> license) [2] which is purely API driven.
> > > > > > >>
> > > > > > >> Anyway, would love to hear your thoughts...
> > > > > > >>
> > > > > > >> Will
> > > > > > >>
> > > > > > >> [1] https://vyos.io/
> > > > > > >> [2] https://cloudrouter.org/
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > DISCLAIMER
> > > > > > ==========
> > > > > > This e-mail may contain privileged and confidential
> > > > > > information
> > which
> > > > > > is the property of Accelerite, a Persistent Systems business.
> > > > > > It is intended only for the use of the individual or entity
> > > > > > to which it
> > is
> > > > > > addressed. If you are not the intended recipient, you are
> > > > > > not authorized to read, retain, copy, print, distribute or
> > > > > > use this message. If you have received this communication in
> > > > > > error, please notify the sender and delete all copies of
> > > > > > this
> message.
> > Accelerite,
> > > a
> > > > > > Persistent Systems business does not accept any liability
> > > > > > for virus
> > > > > infected mails.
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
Judging from this, it does not look like IPSec is managed via the API
though:
https://cloudrouter.atlassian.net/wiki/display/CPD/Bridging+Public+Clouds+with+CloudRouter
I believe it is also missing VXLAN and VRRP, but I have not dug into it yet.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Tue, Sep 13, 2016 at 12:53 PM, Marty Godsey <ma...@gonsource.com> wrote:
> It does.. Its under Secure Connectivity.
>
> -----Original Message-----
> From: Syed Ahmed [mailto:sahmed@cloudops.com]
> Sent: Tuesday, September 13, 2016 12:49 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Does CloudRouter provide VPN (site-site and client)? Looking from their
> website I don't seem to find it. Also, missing is VVRP for redundancy. Has
> anyone used it for this?
>
> On Tue, Sep 13, 2016 at 12:43 PM, Zaeem Arshad <za...@gmail.com>
> wrote:
>
> > +1 on cloudrouter. We have been looking at this as a potential
> > replacement/addon to our existing VRs.
> >
> > On Tue, Sep 13, 2016 at 9:07 PM, Will Stevens <ws...@cloudops.com>
> > wrote:
> >
> > > yes, technically we should be able to just make a new VR
> > > implementation available and then when people create their network
> > > offerings, they just pick which VR implementation they want to use
> > > for the different capabilities.
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > > tw @CloudOps_
> > >
> > > On Tue, Sep 13, 2016 at 11:41 AM, Dustin Wright <
> > > dwright@untangledtechnology.com> wrote:
> > >
> > > > I would like to see a "virtual router offering" in the UI which
> > > > lets
> > you
> > > > pick the legacy VR or your own. Probably a component of the
> > > > network offering. I've had many cases were I needed Mikrotik
> > > > RouterOS or
> > pfSense
> > > to
> > > > match a clients on-premise gear. ACS should find a way to stay
> > > > agnostic IMO.
> > > >
> > > > On Tue, Sep 13, 2016 at 11:36 AM, Marty Godsey
> > > > <ma...@gonsource.com>
> > > > wrote:
> > > >
> > > > > I like this idea as well. I would be willing to test both VyOS
> > > > > and Cloudrouter if you need an active service to test with.
> > > > > Specifically
> > I
> > > am
> > > > > looking to test IPv6 since I provide IPv6 /64 spaces to my
> > > > > customers
> > > and
> > > > I
> > > > > am having to provide it via an external router at the moment
> > > > > which
> > has
> > > a
> > > > > lot of manual configurations.
> > > > >
> > > > > Let me know if I can help in anyway.
> > > > >
> > > > > -----Original Message-----
> > > > > From: Will Stevens [mailto:williamstevens@gmail.com]
> > > > > Sent: Tuesday, September 13, 2016 7:21 AM
> > > > > To: dev@cloudstack.apache.org
> > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > >
> > > > > Ya. If we go this way, I like the approach of building the
> > integration
> > > > and
> > > > > putting it through its paces as a stand alone VR before we
> > > > > consider replacing the old VR and making it the default.
> > > > >
> > > > > On Sep 13, 2016 6:52 AM, "Jayapal Uradi" <
> > jayapal.uradi@accelerite.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Instead of replacing the VR in first place we should add
> > > > > > VyOS/cloudrouter as provider. Once it is stable, network
> > > > > > offerings
> > > (on
> > > > > > upgrade) can be updated to use it and we can drop the VR if we
> > > > > > want
> > > at
> > > > > that release onwards.
> > > > > >
> > > > > > VR is stabilized over a period of time and some of them are
> > > > > > running without issues. When we replicate the ACS VR features
> > > > > > in new
> > > solution
> > > > > > it takes some to find the missing pieces (hidden bugs).
> > > > > >
> > > > > > Thanks,
> > > > > > Jayapal
> > > > > >
> > > > > > > On Sep 13, 2016, at 2:52 PM, Nux! <
> > > > > >
> > > > > > > nux@li.nux.ro> wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I like the idea.
> > > > > > >
> > > > > > > Cloudrouter looks really promising, I'm not too keen on VyOS
> > > > > > > (it doesn't
> > > > > > have a proper http api etc).
> > > > > > >
> > > > > > > --
> > > > > > > Sent from the Delta quadrant using Borg technology!
> > > > > > >
> > > > > > > Nux!
> > > > > > > www.nux.ro
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > >> From: "Will Stevens" <wi...@gmail.com>
> > > > > > >> To: dev@cloudstack.apache.org
> > > > > > >> Sent: Monday, 12 September, 2016 21:20:11
> > > > > > >> Subject: [DISCUSS] Replacing the VR
> > > > > > >
> > > > > > >> *Disclaimer:* This is a thought experiment and should be
> > > > > > >> treated
> > > as
> > > > > > such.
> > > > > > >> Please weigh in with the good and bad of this idea...
> > > > > > >>
> > > > > > >> A couple of us have been discussing the idea of potentially
> > > > > > >> replacing
> > > > > > the
> > > > > > >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There
> > > > > > >> may be
> > a
> > > > > > license
> > > > > > >> issue because I think it is licensed under GPL, but for the
> > > > > > >> sake
> > > of
> > > > > > >> discussion, let's assume we can overcome any license issues.
> > > > > > >>
> > > > > > >> I have spent some time recently with the VyOS and I have to
> > admit,
> > > > > > >> I was pretty impressed. It is simple and intuitive and it
> > > > > > >> gives you a lot
> > > > > > more
> > > > > > >> options for auditing the configuration etc...
> > > > > > >>
> > > > > > >> Items of potential interest:
> > > > > > >> - Clean up our current VR script spaghetti to a simpler
> > > > > > >> more auditable configuration workflow.
> > > > > > >> - Gives a cleaner path for IPv6 support.
> > > > > > >> - Handles VPN configuration via the same configuration
> > interface.
> > > > > > >> - Support for OSPF & BGP.
> > > > > > >> - VPN support through OpenVPN & StrongSwan.
> > > > > > >> - Easily supports HA (redundant routers) through VRRP.
> > > > > > >> - VXLAN support.
> > > > > > >> - Transaction based changes to the VR with rollback on error.
> > > > > > >>
> > > > > > >> Items that could be difficult to solve:
> > > > > > >> - Userdata password reset workflow and implementation.
> > > > > > >> - Upgrade process.
> > > > > > >>
> > > > > > >> The VyOS is not the only option if we were to consider this
> > > > approach.
> > > > > > >> Another option, which I don't know as well, would be
> > > > > > >> CloudRouter (AGPL
> > > > > > >> license) [2] which is purely API driven.
> > > > > > >>
> > > > > > >> Anyway, would love to hear your thoughts...
> > > > > > >>
> > > > > > >> Will
> > > > > > >>
> > > > > > >> [1] https://vyos.io/
> > > > > > >> [2] https://cloudrouter.org/
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > DISCLAIMER
> > > > > > ==========
> > > > > > This e-mail may contain privileged and confidential
> > > > > > information
> > which
> > > > > > is the property of Accelerite, a Persistent Systems business.
> > > > > > It is intended only for the use of the individual or entity to
> > > > > > which it
> > is
> > > > > > addressed. If you are not the intended recipient, you are not
> > > > > > authorized to read, retain, copy, print, distribute or use
> > > > > > this message. If you have received this communication in
> > > > > > error, please notify the sender and delete all copies of this
> message.
> > Accelerite,
> > > a
> > > > > > Persistent Systems business does not accept any liability for
> > > > > > virus
> > > > > infected mails.
> > > > > >
> > > > >
> > > >
> > >
> >
>
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
It does.. Its under Secure Connectivity.
-----Original Message-----
From: Syed Ahmed [mailto:sahmed@cloudops.com]
Sent: Tuesday, September 13, 2016 12:49 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
Does CloudRouter provide VPN (site-site and client)? Looking from their website I don't seem to find it. Also, missing is VVRP for redundancy. Has anyone used it for this?
On Tue, Sep 13, 2016 at 12:43 PM, Zaeem Arshad <za...@gmail.com>
wrote:
> +1 on cloudrouter. We have been looking at this as a potential
> replacement/addon to our existing VRs.
>
> On Tue, Sep 13, 2016 at 9:07 PM, Will Stevens <ws...@cloudops.com>
> wrote:
>
> > yes, technically we should be able to just make a new VR
> > implementation available and then when people create their network
> > offerings, they just pick which VR implementation they want to use
> > for the different capabilities.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > tw @CloudOps_
> >
> > On Tue, Sep 13, 2016 at 11:41 AM, Dustin Wright <
> > dwright@untangledtechnology.com> wrote:
> >
> > > I would like to see a "virtual router offering" in the UI which
> > > lets
> you
> > > pick the legacy VR or your own. Probably a component of the
> > > network offering. I've had many cases were I needed Mikrotik
> > > RouterOS or
> pfSense
> > to
> > > match a clients on-premise gear. ACS should find a way to stay
> > > agnostic IMO.
> > >
> > > On Tue, Sep 13, 2016 at 11:36 AM, Marty Godsey
> > > <ma...@gonsource.com>
> > > wrote:
> > >
> > > > I like this idea as well. I would be willing to test both VyOS
> > > > and Cloudrouter if you need an active service to test with.
> > > > Specifically
> I
> > am
> > > > looking to test IPv6 since I provide IPv6 /64 spaces to my
> > > > customers
> > and
> > > I
> > > > am having to provide it via an external router at the moment
> > > > which
> has
> > a
> > > > lot of manual configurations.
> > > >
> > > > Let me know if I can help in anyway.
> > > >
> > > > -----Original Message-----
> > > > From: Will Stevens [mailto:williamstevens@gmail.com]
> > > > Sent: Tuesday, September 13, 2016 7:21 AM
> > > > To: dev@cloudstack.apache.org
> > > > Subject: Re: [DISCUSS] Replacing the VR
> > > >
> > > > Ya. If we go this way, I like the approach of building the
> integration
> > > and
> > > > putting it through its paces as a stand alone VR before we
> > > > consider replacing the old VR and making it the default.
> > > >
> > > > On Sep 13, 2016 6:52 AM, "Jayapal Uradi" <
> jayapal.uradi@accelerite.com
> > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Instead of replacing the VR in first place we should add
> > > > > VyOS/cloudrouter as provider. Once it is stable, network
> > > > > offerings
> > (on
> > > > > upgrade) can be updated to use it and we can drop the VR if we
> > > > > want
> > at
> > > > that release onwards.
> > > > >
> > > > > VR is stabilized over a period of time and some of them are
> > > > > running without issues. When we replicate the ACS VR features
> > > > > in new
> > solution
> > > > > it takes some to find the missing pieces (hidden bugs).
> > > > >
> > > > > Thanks,
> > > > > Jayapal
> > > > >
> > > > > > On Sep 13, 2016, at 2:52 PM, Nux! <
> > > > >
> > > > > > nux@li.nux.ro> wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I like the idea.
> > > > > >
> > > > > > Cloudrouter looks really promising, I'm not too keen on VyOS
> > > > > > (it doesn't
> > > > > have a proper http api etc).
> > > > > >
> > > > > > --
> > > > > > Sent from the Delta quadrant using Borg technology!
> > > > > >
> > > > > > Nux!
> > > > > > www.nux.ro
> > > > > >
> > > > > > ----- Original Message -----
> > > > > >> From: "Will Stevens" <wi...@gmail.com>
> > > > > >> To: dev@cloudstack.apache.org
> > > > > >> Sent: Monday, 12 September, 2016 21:20:11
> > > > > >> Subject: [DISCUSS] Replacing the VR
> > > > > >
> > > > > >> *Disclaimer:* This is a thought experiment and should be
> > > > > >> treated
> > as
> > > > > such.
> > > > > >> Please weigh in with the good and bad of this idea...
> > > > > >>
> > > > > >> A couple of us have been discussing the idea of potentially
> > > > > >> replacing
> > > > > the
> > > > > >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There
> > > > > >> may be
> a
> > > > > license
> > > > > >> issue because I think it is licensed under GPL, but for the
> > > > > >> sake
> > of
> > > > > >> discussion, let's assume we can overcome any license issues.
> > > > > >>
> > > > > >> I have spent some time recently with the VyOS and I have to
> admit,
> > > > > >> I was pretty impressed. It is simple and intuitive and it
> > > > > >> gives you a lot
> > > > > more
> > > > > >> options for auditing the configuration etc...
> > > > > >>
> > > > > >> Items of potential interest:
> > > > > >> - Clean up our current VR script spaghetti to a simpler
> > > > > >> more auditable configuration workflow.
> > > > > >> - Gives a cleaner path for IPv6 support.
> > > > > >> - Handles VPN configuration via the same configuration
> interface.
> > > > > >> - Support for OSPF & BGP.
> > > > > >> - VPN support through OpenVPN & StrongSwan.
> > > > > >> - Easily supports HA (redundant routers) through VRRP.
> > > > > >> - VXLAN support.
> > > > > >> - Transaction based changes to the VR with rollback on error.
> > > > > >>
> > > > > >> Items that could be difficult to solve:
> > > > > >> - Userdata password reset workflow and implementation.
> > > > > >> - Upgrade process.
> > > > > >>
> > > > > >> The VyOS is not the only option if we were to consider this
> > > approach.
> > > > > >> Another option, which I don't know as well, would be
> > > > > >> CloudRouter (AGPL
> > > > > >> license) [2] which is purely API driven.
> > > > > >>
> > > > > >> Anyway, would love to hear your thoughts...
> > > > > >>
> > > > > >> Will
> > > > > >>
> > > > > >> [1] https://vyos.io/
> > > > > >> [2] https://cloudrouter.org/
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > DISCLAIMER
> > > > > ==========
> > > > > This e-mail may contain privileged and confidential
> > > > > information
> which
> > > > > is the property of Accelerite, a Persistent Systems business.
> > > > > It is intended only for the use of the individual or entity to
> > > > > which it
> is
> > > > > addressed. If you are not the intended recipient, you are not
> > > > > authorized to read, retain, copy, print, distribute or use
> > > > > this message. If you have received this communication in
> > > > > error, please notify the sender and delete all copies of this message.
> Accelerite,
> > a
> > > > > Persistent Systems business does not accept any liability for
> > > > > virus
> > > > infected mails.
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Syed Ahmed <sa...@cloudops.com>.
Does CloudRouter provide VPN (site-site and client)? Looking from their
website I don't seem to find it. Also, missing is VVRP for redundancy. Has
anyone used it for this?
On Tue, Sep 13, 2016 at 12:43 PM, Zaeem Arshad <za...@gmail.com>
wrote:
> +1 on cloudrouter. We have been looking at this as a potential
> replacement/addon to our existing VRs.
>
> On Tue, Sep 13, 2016 at 9:07 PM, Will Stevens <ws...@cloudops.com>
> wrote:
>
> > yes, technically we should be able to just make a new VR implementation
> > available and then when people create their network offerings, they just
> > pick which VR implementation they want to use for the different
> > capabilities.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > w cloudops.com *|* tw @CloudOps_
> >
> > On Tue, Sep 13, 2016 at 11:41 AM, Dustin Wright <
> > dwright@untangledtechnology.com> wrote:
> >
> > > I would like to see a "virtual router offering" in the UI which lets
> you
> > > pick the legacy VR or your own. Probably a component of the network
> > > offering. I've had many cases were I needed Mikrotik RouterOS or
> pfSense
> > to
> > > match a clients on-premise gear. ACS should find a way to stay agnostic
> > > IMO.
> > >
> > > On Tue, Sep 13, 2016 at 11:36 AM, Marty Godsey <ma...@gonsource.com>
> > > wrote:
> > >
> > > > I like this idea as well. I would be willing to test both VyOS and
> > > > Cloudrouter if you need an active service to test with. Specifically
> I
> > am
> > > > looking to test IPv6 since I provide IPv6 /64 spaces to my customers
> > and
> > > I
> > > > am having to provide it via an external router at the moment which
> has
> > a
> > > > lot of manual configurations.
> > > >
> > > > Let me know if I can help in anyway.
> > > >
> > > > -----Original Message-----
> > > > From: Will Stevens [mailto:williamstevens@gmail.com]
> > > > Sent: Tuesday, September 13, 2016 7:21 AM
> > > > To: dev@cloudstack.apache.org
> > > > Subject: Re: [DISCUSS] Replacing the VR
> > > >
> > > > Ya. If we go this way, I like the approach of building the
> integration
> > > and
> > > > putting it through its paces as a stand alone VR before we consider
> > > > replacing the old VR and making it the default.
> > > >
> > > > On Sep 13, 2016 6:52 AM, "Jayapal Uradi" <
> jayapal.uradi@accelerite.com
> > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Instead of replacing the VR in first place we should add
> > > > > VyOS/cloudrouter as provider. Once it is stable, network offerings
> > (on
> > > > > upgrade) can be updated to use it and we can drop the VR if we want
> > at
> > > > that release onwards.
> > > > >
> > > > > VR is stabilized over a period of time and some of them are running
> > > > > without issues. When we replicate the ACS VR features in new
> > solution
> > > > > it takes some to find the missing pieces (hidden bugs).
> > > > >
> > > > > Thanks,
> > > > > Jayapal
> > > > >
> > > > > > On Sep 13, 2016, at 2:52 PM, Nux! <
> > > > >
> > > > > > nux@li.nux.ro> wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I like the idea.
> > > > > >
> > > > > > Cloudrouter looks really promising, I'm not too keen on VyOS (it
> > > > > > doesn't
> > > > > have a proper http api etc).
> > > > > >
> > > > > > --
> > > > > > Sent from the Delta quadrant using Borg technology!
> > > > > >
> > > > > > Nux!
> > > > > > www.nux.ro
> > > > > >
> > > > > > ----- Original Message -----
> > > > > >> From: "Will Stevens" <wi...@gmail.com>
> > > > > >> To: dev@cloudstack.apache.org
> > > > > >> Sent: Monday, 12 September, 2016 21:20:11
> > > > > >> Subject: [DISCUSS] Replacing the VR
> > > > > >
> > > > > >> *Disclaimer:* This is a thought experiment and should be treated
> > as
> > > > > such.
> > > > > >> Please weigh in with the good and bad of this idea...
> > > > > >>
> > > > > >> A couple of us have been discussing the idea of potentially
> > > > > >> replacing
> > > > > the
> > > > > >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be
> a
> > > > > license
> > > > > >> issue because I think it is licensed under GPL, but for the sake
> > of
> > > > > >> discussion, let's assume we can overcome any license issues.
> > > > > >>
> > > > > >> I have spent some time recently with the VyOS and I have to
> admit,
> > > > > >> I was pretty impressed. It is simple and intuitive and it gives
> > > > > >> you a lot
> > > > > more
> > > > > >> options for auditing the configuration etc...
> > > > > >>
> > > > > >> Items of potential interest:
> > > > > >> - Clean up our current VR script spaghetti to a simpler more
> > > > > >> auditable configuration workflow.
> > > > > >> - Gives a cleaner path for IPv6 support.
> > > > > >> - Handles VPN configuration via the same configuration
> interface.
> > > > > >> - Support for OSPF & BGP.
> > > > > >> - VPN support through OpenVPN & StrongSwan.
> > > > > >> - Easily supports HA (redundant routers) through VRRP.
> > > > > >> - VXLAN support.
> > > > > >> - Transaction based changes to the VR with rollback on error.
> > > > > >>
> > > > > >> Items that could be difficult to solve:
> > > > > >> - Userdata password reset workflow and implementation.
> > > > > >> - Upgrade process.
> > > > > >>
> > > > > >> The VyOS is not the only option if we were to consider this
> > > approach.
> > > > > >> Another option, which I don't know as well, would be CloudRouter
> > > > > >> (AGPL
> > > > > >> license) [2] which is purely API driven.
> > > > > >>
> > > > > >> Anyway, would love to hear your thoughts...
> > > > > >>
> > > > > >> Will
> > > > > >>
> > > > > >> [1] https://vyos.io/
> > > > > >> [2] https://cloudrouter.org/
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > DISCLAIMER
> > > > > ==========
> > > > > This e-mail may contain privileged and confidential information
> which
> > > > > is the property of Accelerite, a Persistent Systems business. It is
> > > > > intended only for the use of the individual or entity to which it
> is
> > > > > addressed. If you are not the intended recipient, you are not
> > > > > authorized to read, retain, copy, print, distribute or use this
> > > > > message. If you have received this communication in error, please
> > > > > notify the sender and delete all copies of this message.
> Accelerite,
> > a
> > > > > Persistent Systems business does not accept any liability for virus
> > > > infected mails.
> > > > >
> > > >
> > >
> >
>
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
I have spent all of 2 mins looking at Cloudrouter so it may do this and I haven’t found it but according to the feature list it doesn't support VXLANS. Am I missing it somewhere?
-----Original Message-----
From: Zaeem Arshad [mailto:zaeem.arshad@gmail.com]
Sent: Tuesday, September 13, 2016 12:43 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
+1 on cloudrouter. We have been looking at this as a potential
replacement/addon to our existing VRs.
On Tue, Sep 13, 2016 at 9:07 PM, Will Stevens <ws...@cloudops.com> wrote:
> yes, technically we should be able to just make a new VR
> implementation available and then when people create their network
> offerings, they just pick which VR implementation they want to use for
> the different capabilities.
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Tue, Sep 13, 2016 at 11:41 AM, Dustin Wright <
> dwright@untangledtechnology.com> wrote:
>
> > I would like to see a "virtual router offering" in the UI which lets
> > you pick the legacy VR or your own. Probably a component of the
> > network offering. I've had many cases were I needed Mikrotik
> > RouterOS or pfSense
> to
> > match a clients on-premise gear. ACS should find a way to stay
> > agnostic IMO.
> >
> > On Tue, Sep 13, 2016 at 11:36 AM, Marty Godsey <ma...@gonsource.com>
> > wrote:
> >
> > > I like this idea as well. I would be willing to test both VyOS and
> > > Cloudrouter if you need an active service to test with.
> > > Specifically I
> am
> > > looking to test IPv6 since I provide IPv6 /64 spaces to my
> > > customers
> and
> > I
> > > am having to provide it via an external router at the moment which
> > > has
> a
> > > lot of manual configurations.
> > >
> > > Let me know if I can help in anyway.
> > >
> > > -----Original Message-----
> > > From: Will Stevens [mailto:williamstevens@gmail.com]
> > > Sent: Tuesday, September 13, 2016 7:21 AM
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: [DISCUSS] Replacing the VR
> > >
> > > Ya. If we go this way, I like the approach of building the
> > > integration
> > and
> > > putting it through its paces as a stand alone VR before we
> > > consider replacing the old VR and making it the default.
> > >
> > > On Sep 13, 2016 6:52 AM, "Jayapal Uradi"
> > > <jayapal.uradi@accelerite.com
> >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Instead of replacing the VR in first place we should add
> > > > VyOS/cloudrouter as provider. Once it is stable, network
> > > > offerings
> (on
> > > > upgrade) can be updated to use it and we can drop the VR if we
> > > > want
> at
> > > that release onwards.
> > > >
> > > > VR is stabilized over a period of time and some of them are
> > > > running without issues. When we replicate the ACS VR features
> > > > in new
> solution
> > > > it takes some to find the missing pieces (hidden bugs).
> > > >
> > > > Thanks,
> > > > Jayapal
> > > >
> > > > > On Sep 13, 2016, at 2:52 PM, Nux! <
> > > >
> > > > > nux@li.nux.ro> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I like the idea.
> > > > >
> > > > > Cloudrouter looks really promising, I'm not too keen on VyOS
> > > > > (it doesn't
> > > > have a proper http api etc).
> > > > >
> > > > > --
> > > > > Sent from the Delta quadrant using Borg technology!
> > > > >
> > > > > Nux!
> > > > > www.nux.ro
> > > > >
> > > > > ----- Original Message -----
> > > > >> From: "Will Stevens" <wi...@gmail.com>
> > > > >> To: dev@cloudstack.apache.org
> > > > >> Sent: Monday, 12 September, 2016 21:20:11
> > > > >> Subject: [DISCUSS] Replacing the VR
> > > > >
> > > > >> *Disclaimer:* This is a thought experiment and should be
> > > > >> treated
> as
> > > > such.
> > > > >> Please weigh in with the good and bad of this idea...
> > > > >>
> > > > >> A couple of us have been discussing the idea of potentially
> > > > >> replacing
> > > > the
> > > > >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There may
> > > > >> be a
> > > > license
> > > > >> issue because I think it is licensed under GPL, but for the
> > > > >> sake
> of
> > > > >> discussion, let's assume we can overcome any license issues.
> > > > >>
> > > > >> I have spent some time recently with the VyOS and I have to
> > > > >> admit, I was pretty impressed. It is simple and intuitive
> > > > >> and it gives you a lot
> > > > more
> > > > >> options for auditing the configuration etc...
> > > > >>
> > > > >> Items of potential interest:
> > > > >> - Clean up our current VR script spaghetti to a simpler more
> > > > >> auditable configuration workflow.
> > > > >> - Gives a cleaner path for IPv6 support.
> > > > >> - Handles VPN configuration via the same configuration interface.
> > > > >> - Support for OSPF & BGP.
> > > > >> - VPN support through OpenVPN & StrongSwan.
> > > > >> - Easily supports HA (redundant routers) through VRRP.
> > > > >> - VXLAN support.
> > > > >> - Transaction based changes to the VR with rollback on error.
> > > > >>
> > > > >> Items that could be difficult to solve:
> > > > >> - Userdata password reset workflow and implementation.
> > > > >> - Upgrade process.
> > > > >>
> > > > >> The VyOS is not the only option if we were to consider this
> > approach.
> > > > >> Another option, which I don't know as well, would be
> > > > >> CloudRouter (AGPL
> > > > >> license) [2] which is purely API driven.
> > > > >>
> > > > >> Anyway, would love to hear your thoughts...
> > > > >>
> > > > >> Will
> > > > >>
> > > > >> [1] https://vyos.io/
> > > > >> [2] https://cloudrouter.org/
> > > >
> > > >
> > > >
> > > >
> > > > DISCLAIMER
> > > > ==========
> > > > This e-mail may contain privileged and confidential information
> > > > which is the property of Accelerite, a Persistent Systems
> > > > business. It is intended only for the use of the individual or
> > > > entity to which it is addressed. If you are not the intended
> > > > recipient, you are not authorized to read, retain, copy, print,
> > > > distribute or use this message. If you have received this
> > > > communication in error, please notify the sender and delete all
> > > > copies of this message. Accelerite,
> a
> > > > Persistent Systems business does not accept any liability for
> > > > virus
> > > infected mails.
> > > >
> > >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Zaeem Arshad <za...@gmail.com>.
+1 on cloudrouter. We have been looking at this as a potential
replacement/addon to our existing VRs.
On Tue, Sep 13, 2016 at 9:07 PM, Will Stevens <ws...@cloudops.com> wrote:
> yes, technically we should be able to just make a new VR implementation
> available and then when people create their network offerings, they just
> pick which VR implementation they want to use for the different
> capabilities.
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Tue, Sep 13, 2016 at 11:41 AM, Dustin Wright <
> dwright@untangledtechnology.com> wrote:
>
> > I would like to see a "virtual router offering" in the UI which lets you
> > pick the legacy VR or your own. Probably a component of the network
> > offering. I've had many cases were I needed Mikrotik RouterOS or pfSense
> to
> > match a clients on-premise gear. ACS should find a way to stay agnostic
> > IMO.
> >
> > On Tue, Sep 13, 2016 at 11:36 AM, Marty Godsey <ma...@gonsource.com>
> > wrote:
> >
> > > I like this idea as well. I would be willing to test both VyOS and
> > > Cloudrouter if you need an active service to test with. Specifically I
> am
> > > looking to test IPv6 since I provide IPv6 /64 spaces to my customers
> and
> > I
> > > am having to provide it via an external router at the moment which has
> a
> > > lot of manual configurations.
> > >
> > > Let me know if I can help in anyway.
> > >
> > > -----Original Message-----
> > > From: Will Stevens [mailto:williamstevens@gmail.com]
> > > Sent: Tuesday, September 13, 2016 7:21 AM
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: [DISCUSS] Replacing the VR
> > >
> > > Ya. If we go this way, I like the approach of building the integration
> > and
> > > putting it through its paces as a stand alone VR before we consider
> > > replacing the old VR and making it the default.
> > >
> > > On Sep 13, 2016 6:52 AM, "Jayapal Uradi" <jayapal.uradi@accelerite.com
> >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Instead of replacing the VR in first place we should add
> > > > VyOS/cloudrouter as provider. Once it is stable, network offerings
> (on
> > > > upgrade) can be updated to use it and we can drop the VR if we want
> at
> > > that release onwards.
> > > >
> > > > VR is stabilized over a period of time and some of them are running
> > > > without issues. When we replicate the ACS VR features in new
> solution
> > > > it takes some to find the missing pieces (hidden bugs).
> > > >
> > > > Thanks,
> > > > Jayapal
> > > >
> > > > > On Sep 13, 2016, at 2:52 PM, Nux! <
> > > >
> > > > > nux@li.nux.ro> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I like the idea.
> > > > >
> > > > > Cloudrouter looks really promising, I'm not too keen on VyOS (it
> > > > > doesn't
> > > > have a proper http api etc).
> > > > >
> > > > > --
> > > > > Sent from the Delta quadrant using Borg technology!
> > > > >
> > > > > Nux!
> > > > > www.nux.ro
> > > > >
> > > > > ----- Original Message -----
> > > > >> From: "Will Stevens" <wi...@gmail.com>
> > > > >> To: dev@cloudstack.apache.org
> > > > >> Sent: Monday, 12 September, 2016 21:20:11
> > > > >> Subject: [DISCUSS] Replacing the VR
> > > > >
> > > > >> *Disclaimer:* This is a thought experiment and should be treated
> as
> > > > such.
> > > > >> Please weigh in with the good and bad of this idea...
> > > > >>
> > > > >> A couple of us have been discussing the idea of potentially
> > > > >> replacing
> > > > the
> > > > >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a
> > > > license
> > > > >> issue because I think it is licensed under GPL, but for the sake
> of
> > > > >> discussion, let's assume we can overcome any license issues.
> > > > >>
> > > > >> I have spent some time recently with the VyOS and I have to admit,
> > > > >> I was pretty impressed. It is simple and intuitive and it gives
> > > > >> you a lot
> > > > more
> > > > >> options for auditing the configuration etc...
> > > > >>
> > > > >> Items of potential interest:
> > > > >> - Clean up our current VR script spaghetti to a simpler more
> > > > >> auditable configuration workflow.
> > > > >> - Gives a cleaner path for IPv6 support.
> > > > >> - Handles VPN configuration via the same configuration interface.
> > > > >> - Support for OSPF & BGP.
> > > > >> - VPN support through OpenVPN & StrongSwan.
> > > > >> - Easily supports HA (redundant routers) through VRRP.
> > > > >> - VXLAN support.
> > > > >> - Transaction based changes to the VR with rollback on error.
> > > > >>
> > > > >> Items that could be difficult to solve:
> > > > >> - Userdata password reset workflow and implementation.
> > > > >> - Upgrade process.
> > > > >>
> > > > >> The VyOS is not the only option if we were to consider this
> > approach.
> > > > >> Another option, which I don't know as well, would be CloudRouter
> > > > >> (AGPL
> > > > >> license) [2] which is purely API driven.
> > > > >>
> > > > >> Anyway, would love to hear your thoughts...
> > > > >>
> > > > >> Will
> > > > >>
> > > > >> [1] https://vyos.io/
> > > > >> [2] https://cloudrouter.org/
> > > >
> > > >
> > > >
> > > >
> > > > DISCLAIMER
> > > > ==========
> > > > This e-mail may contain privileged and confidential information which
> > > > is the property of Accelerite, a Persistent Systems business. It is
> > > > intended only for the use of the individual or entity to which it is
> > > > addressed. If you are not the intended recipient, you are not
> > > > authorized to read, retain, copy, print, distribute or use this
> > > > message. If you have received this communication in error, please
> > > > notify the sender and delete all copies of this message. Accelerite,
> a
> > > > Persistent Systems business does not accept any liability for virus
> > > infected mails.
> > > >
> > >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
yes, technically we should be able to just make a new VR implementation
available and then when people create their network offerings, they just
pick which VR implementation they want to use for the different
capabilities.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Tue, Sep 13, 2016 at 11:41 AM, Dustin Wright <
dwright@untangledtechnology.com> wrote:
> I would like to see a "virtual router offering" in the UI which lets you
> pick the legacy VR or your own. Probably a component of the network
> offering. I've had many cases were I needed Mikrotik RouterOS or pfSense to
> match a clients on-premise gear. ACS should find a way to stay agnostic
> IMO.
>
> On Tue, Sep 13, 2016 at 11:36 AM, Marty Godsey <ma...@gonsource.com>
> wrote:
>
> > I like this idea as well. I would be willing to test both VyOS and
> > Cloudrouter if you need an active service to test with. Specifically I am
> > looking to test IPv6 since I provide IPv6 /64 spaces to my customers and
> I
> > am having to provide it via an external router at the moment which has a
> > lot of manual configurations.
> >
> > Let me know if I can help in anyway.
> >
> > -----Original Message-----
> > From: Will Stevens [mailto:williamstevens@gmail.com]
> > Sent: Tuesday, September 13, 2016 7:21 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] Replacing the VR
> >
> > Ya. If we go this way, I like the approach of building the integration
> and
> > putting it through its paces as a stand alone VR before we consider
> > replacing the old VR and making it the default.
> >
> > On Sep 13, 2016 6:52 AM, "Jayapal Uradi" <ja...@accelerite.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Instead of replacing the VR in first place we should add
> > > VyOS/cloudrouter as provider. Once it is stable, network offerings (on
> > > upgrade) can be updated to use it and we can drop the VR if we want at
> > that release onwards.
> > >
> > > VR is stabilized over a period of time and some of them are running
> > > without issues. When we replicate the ACS VR features in new solution
> > > it takes some to find the missing pieces (hidden bugs).
> > >
> > > Thanks,
> > > Jayapal
> > >
> > > > On Sep 13, 2016, at 2:52 PM, Nux! <
> > >
> > > > nux@li.nux.ro> wrote:
> > > >
> > > > Hi,
> > > >
> > > > I like the idea.
> > > >
> > > > Cloudrouter looks really promising, I'm not too keen on VyOS (it
> > > > doesn't
> > > have a proper http api etc).
> > > >
> > > > --
> > > > Sent from the Delta quadrant using Borg technology!
> > > >
> > > > Nux!
> > > > www.nux.ro
> > > >
> > > > ----- Original Message -----
> > > >> From: "Will Stevens" <wi...@gmail.com>
> > > >> To: dev@cloudstack.apache.org
> > > >> Sent: Monday, 12 September, 2016 21:20:11
> > > >> Subject: [DISCUSS] Replacing the VR
> > > >
> > > >> *Disclaimer:* This is a thought experiment and should be treated as
> > > such.
> > > >> Please weigh in with the good and bad of this idea...
> > > >>
> > > >> A couple of us have been discussing the idea of potentially
> > > >> replacing
> > > the
> > > >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a
> > > license
> > > >> issue because I think it is licensed under GPL, but for the sake of
> > > >> discussion, let's assume we can overcome any license issues.
> > > >>
> > > >> I have spent some time recently with the VyOS and I have to admit,
> > > >> I was pretty impressed. It is simple and intuitive and it gives
> > > >> you a lot
> > > more
> > > >> options for auditing the configuration etc...
> > > >>
> > > >> Items of potential interest:
> > > >> - Clean up our current VR script spaghetti to a simpler more
> > > >> auditable configuration workflow.
> > > >> - Gives a cleaner path for IPv6 support.
> > > >> - Handles VPN configuration via the same configuration interface.
> > > >> - Support for OSPF & BGP.
> > > >> - VPN support through OpenVPN & StrongSwan.
> > > >> - Easily supports HA (redundant routers) through VRRP.
> > > >> - VXLAN support.
> > > >> - Transaction based changes to the VR with rollback on error.
> > > >>
> > > >> Items that could be difficult to solve:
> > > >> - Userdata password reset workflow and implementation.
> > > >> - Upgrade process.
> > > >>
> > > >> The VyOS is not the only option if we were to consider this
> approach.
> > > >> Another option, which I don't know as well, would be CloudRouter
> > > >> (AGPL
> > > >> license) [2] which is purely API driven.
> > > >>
> > > >> Anyway, would love to hear your thoughts...
> > > >>
> > > >> Will
> > > >>
> > > >> [1] https://vyos.io/
> > > >> [2] https://cloudrouter.org/
> > >
> > >
> > >
> > >
> > > DISCLAIMER
> > > ==========
> > > This e-mail may contain privileged and confidential information which
> > > is the property of Accelerite, a Persistent Systems business. It is
> > > intended only for the use of the individual or entity to which it is
> > > addressed. If you are not the intended recipient, you are not
> > > authorized to read, retain, copy, print, distribute or use this
> > > message. If you have received this communication in error, please
> > > notify the sender and delete all copies of this message. Accelerite, a
> > > Persistent Systems business does not accept any liability for virus
> > infected mails.
> > >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Dustin Wright <dw...@untangledtechnology.com>.
I would like to see a "virtual router offering" in the UI which lets you
pick the legacy VR or your own. Probably a component of the network
offering. I've had many cases were I needed Mikrotik RouterOS or pfSense to
match a clients on-premise gear. ACS should find a way to stay agnostic IMO.
On Tue, Sep 13, 2016 at 11:36 AM, Marty Godsey <ma...@gonsource.com> wrote:
> I like this idea as well. I would be willing to test both VyOS and
> Cloudrouter if you need an active service to test with. Specifically I am
> looking to test IPv6 since I provide IPv6 /64 spaces to my customers and I
> am having to provide it via an external router at the moment which has a
> lot of manual configurations.
>
> Let me know if I can help in anyway.
>
> -----Original Message-----
> From: Will Stevens [mailto:williamstevens@gmail.com]
> Sent: Tuesday, September 13, 2016 7:21 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Ya. If we go this way, I like the approach of building the integration and
> putting it through its paces as a stand alone VR before we consider
> replacing the old VR and making it the default.
>
> On Sep 13, 2016 6:52 AM, "Jayapal Uradi" <ja...@accelerite.com>
> wrote:
>
> > Hi,
> >
> > Instead of replacing the VR in first place we should add
> > VyOS/cloudrouter as provider. Once it is stable, network offerings (on
> > upgrade) can be updated to use it and we can drop the VR if we want at
> that release onwards.
> >
> > VR is stabilized over a period of time and some of them are running
> > without issues. When we replicate the ACS VR features in new solution
> > it takes some to find the missing pieces (hidden bugs).
> >
> > Thanks,
> > Jayapal
> >
> > > On Sep 13, 2016, at 2:52 PM, Nux! <
> >
> > > nux@li.nux.ro> wrote:
> > >
> > > Hi,
> > >
> > > I like the idea.
> > >
> > > Cloudrouter looks really promising, I'm not too keen on VyOS (it
> > > doesn't
> > have a proper http api etc).
> > >
> > > --
> > > Sent from the Delta quadrant using Borg technology!
> > >
> > > Nux!
> > > www.nux.ro
> > >
> > > ----- Original Message -----
> > >> From: "Will Stevens" <wi...@gmail.com>
> > >> To: dev@cloudstack.apache.org
> > >> Sent: Monday, 12 September, 2016 21:20:11
> > >> Subject: [DISCUSS] Replacing the VR
> > >
> > >> *Disclaimer:* This is a thought experiment and should be treated as
> > such.
> > >> Please weigh in with the good and bad of this idea...
> > >>
> > >> A couple of us have been discussing the idea of potentially
> > >> replacing
> > the
> > >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a
> > license
> > >> issue because I think it is licensed under GPL, but for the sake of
> > >> discussion, let's assume we can overcome any license issues.
> > >>
> > >> I have spent some time recently with the VyOS and I have to admit,
> > >> I was pretty impressed. It is simple and intuitive and it gives
> > >> you a lot
> > more
> > >> options for auditing the configuration etc...
> > >>
> > >> Items of potential interest:
> > >> - Clean up our current VR script spaghetti to a simpler more
> > >> auditable configuration workflow.
> > >> - Gives a cleaner path for IPv6 support.
> > >> - Handles VPN configuration via the same configuration interface.
> > >> - Support for OSPF & BGP.
> > >> - VPN support through OpenVPN & StrongSwan.
> > >> - Easily supports HA (redundant routers) through VRRP.
> > >> - VXLAN support.
> > >> - Transaction based changes to the VR with rollback on error.
> > >>
> > >> Items that could be difficult to solve:
> > >> - Userdata password reset workflow and implementation.
> > >> - Upgrade process.
> > >>
> > >> The VyOS is not the only option if we were to consider this approach.
> > >> Another option, which I don't know as well, would be CloudRouter
> > >> (AGPL
> > >> license) [2] which is purely API driven.
> > >>
> > >> Anyway, would love to hear your thoughts...
> > >>
> > >> Will
> > >>
> > >> [1] https://vyos.io/
> > >> [2] https://cloudrouter.org/
> >
> >
> >
> >
> > DISCLAIMER
> > ==========
> > This e-mail may contain privileged and confidential information which
> > is the property of Accelerite, a Persistent Systems business. It is
> > intended only for the use of the individual or entity to which it is
> > addressed. If you are not the intended recipient, you are not
> > authorized to read, retain, copy, print, distribute or use this
> > message. If you have received this communication in error, please
> > notify the sender and delete all copies of this message. Accelerite, a
> > Persistent Systems business does not accept any liability for virus
> infected mails.
> >
>
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
I like this idea as well. I would be willing to test both VyOS and Cloudrouter if you need an active service to test with. Specifically I am looking to test IPv6 since I provide IPv6 /64 spaces to my customers and I am having to provide it via an external router at the moment which has a lot of manual configurations.
Let me know if I can help in anyway.
-----Original Message-----
From: Will Stevens [mailto:williamstevens@gmail.com]
Sent: Tuesday, September 13, 2016 7:21 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
Ya. If we go this way, I like the approach of building the integration and putting it through its paces as a stand alone VR before we consider replacing the old VR and making it the default.
On Sep 13, 2016 6:52 AM, "Jayapal Uradi" <ja...@accelerite.com>
wrote:
> Hi,
>
> Instead of replacing the VR in first place we should add
> VyOS/cloudrouter as provider. Once it is stable, network offerings (on
> upgrade) can be updated to use it and we can drop the VR if we want at that release onwards.
>
> VR is stabilized over a period of time and some of them are running
> without issues. When we replicate the ACS VR features in new solution
> it takes some to find the missing pieces (hidden bugs).
>
> Thanks,
> Jayapal
>
> > On Sep 13, 2016, at 2:52 PM, Nux! <
>
> > nux@li.nux.ro> wrote:
> >
> > Hi,
> >
> > I like the idea.
> >
> > Cloudrouter looks really promising, I'm not too keen on VyOS (it
> > doesn't
> have a proper http api etc).
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > ----- Original Message -----
> >> From: "Will Stevens" <wi...@gmail.com>
> >> To: dev@cloudstack.apache.org
> >> Sent: Monday, 12 September, 2016 21:20:11
> >> Subject: [DISCUSS] Replacing the VR
> >
> >> *Disclaimer:* This is a thought experiment and should be treated as
> such.
> >> Please weigh in with the good and bad of this idea...
> >>
> >> A couple of us have been discussing the idea of potentially
> >> replacing
> the
> >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a
> license
> >> issue because I think it is licensed under GPL, but for the sake of
> >> discussion, let's assume we can overcome any license issues.
> >>
> >> I have spent some time recently with the VyOS and I have to admit,
> >> I was pretty impressed. It is simple and intuitive and it gives
> >> you a lot
> more
> >> options for auditing the configuration etc...
> >>
> >> Items of potential interest:
> >> - Clean up our current VR script spaghetti to a simpler more
> >> auditable configuration workflow.
> >> - Gives a cleaner path for IPv6 support.
> >> - Handles VPN configuration via the same configuration interface.
> >> - Support for OSPF & BGP.
> >> - VPN support through OpenVPN & StrongSwan.
> >> - Easily supports HA (redundant routers) through VRRP.
> >> - VXLAN support.
> >> - Transaction based changes to the VR with rollback on error.
> >>
> >> Items that could be difficult to solve:
> >> - Userdata password reset workflow and implementation.
> >> - Upgrade process.
> >>
> >> The VyOS is not the only option if we were to consider this approach.
> >> Another option, which I don't know as well, would be CloudRouter
> >> (AGPL
> >> license) [2] which is purely API driven.
> >>
> >> Anyway, would love to hear your thoughts...
> >>
> >> Will
> >>
> >> [1] https://vyos.io/
> >> [2] https://cloudrouter.org/
>
>
>
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which
> is the property of Accelerite, a Persistent Systems business. It is
> intended only for the use of the individual or entity to which it is
> addressed. If you are not the intended recipient, you are not
> authorized to read, retain, copy, print, distribute or use this
> message. If you have received this communication in error, please
> notify the sender and delete all copies of this message. Accelerite, a
> Persistent Systems business does not accept any liability for virus infected mails.
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <wi...@gmail.com>.
Ya. If we go this way, I like the approach of building the integration and
putting it through its paces as a stand alone VR before we consider
replacing the old VR and making it the default.
On Sep 13, 2016 6:52 AM, "Jayapal Uradi" <ja...@accelerite.com>
wrote:
> Hi,
>
> Instead of replacing the VR in first place we should add VyOS/cloudrouter
> as provider. Once it is stable, network offerings (on upgrade) can be
> updated to use it and we can drop the VR if we want at that release onwards.
>
> VR is stabilized over a period of time and some of them are running
> without issues. When we replicate the ACS VR features in new solution it
> takes some to find the missing pieces (hidden bugs).
>
> Thanks,
> Jayapal
>
> > On Sep 13, 2016, at 2:52 PM, Nux! <
>
> > nux@li.nux.ro> wrote:
> >
> > Hi,
> >
> > I like the idea.
> >
> > Cloudrouter looks really promising, I'm not too keen on VyOS (it doesn't
> have a proper http api etc).
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > ----- Original Message -----
> >> From: "Will Stevens" <wi...@gmail.com>
> >> To: dev@cloudstack.apache.org
> >> Sent: Monday, 12 September, 2016 21:20:11
> >> Subject: [DISCUSS] Replacing the VR
> >
> >> *Disclaimer:* This is a thought experiment and should be treated as
> such.
> >> Please weigh in with the good and bad of this idea...
> >>
> >> A couple of us have been discussing the idea of potentially replacing
> the
> >> ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a
> license
> >> issue because I think it is licensed under GPL, but for the sake of
> >> discussion, let's assume we can overcome any license issues.
> >>
> >> I have spent some time recently with the VyOS and I have to admit, I was
> >> pretty impressed. It is simple and intuitive and it gives you a lot
> more
> >> options for auditing the configuration etc...
> >>
> >> Items of potential interest:
> >> - Clean up our current VR script spaghetti to a simpler more auditable
> >> configuration workflow.
> >> - Gives a cleaner path for IPv6 support.
> >> - Handles VPN configuration via the same configuration interface.
> >> - Support for OSPF & BGP.
> >> - VPN support through OpenVPN & StrongSwan.
> >> - Easily supports HA (redundant routers) through VRRP.
> >> - VXLAN support.
> >> - Transaction based changes to the VR with rollback on error.
> >>
> >> Items that could be difficult to solve:
> >> - Userdata password reset workflow and implementation.
> >> - Upgrade process.
> >>
> >> The VyOS is not the only option if we were to consider this approach.
> >> Another option, which I don't know as well, would be CloudRouter (AGPL
> >> license) [2] which is purely API driven.
> >>
> >> Anyway, would love to hear your thoughts...
> >>
> >> Will
> >>
> >> [1] https://vyos.io/
> >> [2] https://cloudrouter.org/
>
>
>
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is
> the property of Accelerite, a Persistent Systems business. It is intended
> only for the use of the individual or entity to which it is addressed. If
> you are not the intended recipient, you are not authorized to read, retain,
> copy, print, distribute or use this message. If you have received this
> communication in error, please notify the sender and delete all copies of
> this message. Accelerite, a Persistent Systems business does not accept any
> liability for virus infected mails.
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
Those are good questions Marty. In talking to the guys, their 2.0 release
has been delayed a bit because of the issue with Debian 6 becoming EOL. So
the last year they have been porting everything to Debian 8. This will be
in 1.2.x, which is available but not fully tested yet. They expect it to
be stable in within about 6 months. I asked what the timeline for 2.0 was
and got a ballpark figure of 1.5 to 2 years out. It really depends on what
kind of resources they can get. They have started offering commercial
support in an effort to increase the funds coming into the project so they
can increase their velocity, but they are still understaffed.
I don't think it is out of the question to build an integration for VyOS,
but I expect 15-20% will have to be rewritten when 2.0 comes out. Still
not convinced this is the right approach, but it is an interesting option.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Fri, Sep 16, 2016 at 1:24 PM, Marty Godsey <ma...@gonsource.com> wrote:
> So based upon this discussion would it be prudent to wait on VyOS 2.0? The
> current VR is giving us issues but would the time invested in another
> "solution" be wasted especially if by the time another option is chose,
> then coded, then tested, then implemented and right as that time happened
> to be when VyOS 2.0 is released. Of course you said they are just in the
> scoping range so this could still be a year or more out.
>
> Thoughts?
>
> Regards,
> Marty Godsey
> nSource Solutions
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: Friday, September 16, 2016 10:31 AM
> To: dev@cloudstack.apache.org
> Cc: daniil@baturin.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> I just had a quick chat with a couple of the guys over on the VyOS chat.
> I have CC'ed one of them in case we have more licensing questions.
>
> So here is the status with the license "the code inherited from Vyatta and
> our modifications from it is GPLv2 (strict, not v2+). The config reading
> library is GPLv2 too, so anything that links to is is GPLv2.
> Some auxiliary components we made after the fork are more permissive,
> LGPLv2+ or MIT."
>
> They are currently in the process of scoping a redesign (VyOS 2.0), "we
> are planning a clean rewrite that will solve issues of the current config
> system".
> This will include the ability to configure via the API.
>
> If we have more questions for VyOS, they are very friendly and responsive,
> so we should be able to get answers.
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>
> > I agree with Will Ilya. There are so many problems with the VR right now.
> > Most of the outages we've had recently have somehow involved the VR.
> > We set custom iptables rules on the VR which can and have easily gone
> wrong.
> > Openswan is broken, Strongswan replacement still needs to be tested.
> > VVRP with redundant router still needs work, and not to mention the
> > problems we will have when we introduce IPv6 into the whole picture.
> >
> > I think the spirit of the discussion is to rely on a 3rd party to do
> > the networking for us (eg VyOS) and have us handle just the
> > orchestration. All the problems that I've described have already been
> > solved in VyOS. We also get the advantage of a potential wider
> > community to fix and maintain the VR and given our current development
> > velocity, it think it totally makes sense to look for a 3rd party option.
> >
> > -Syed
> >
> >
> > On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
> > wrote:
> >
> > > The VR has been biting us far too often recently, which is why we
> > > have started looking into alternative implementations.
> > >
> > > One of the things that is nice about potentially using the VyOS is
> > > that
> > it
> > > is based on Debian, so we should be able to run the other services
> > > that
> > we
> > > currently have like the password server and userdata on the VyOS.
> > > This means we would not have to change our architecture initially
> > > and could focus on only replacing the networking paths.
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > > tw @CloudOps_
> > >
> > > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> > >
> > > > The more this is discussed the more I think we should stick with
> > > > our
> > VR.
> > > >
> > > > All these other options either seem unfinished or with
> > > > incompatible license.
> > > >
> > > > VyOS looks the most promising so far, it's a serious, mature project.
> > > > Adopting it though means we'll have to microservice our way out of
> > > > it
> > > with
> > > > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
> > those
> > > > too. Imho this adds complexity we should void.
> > > >
> > > > --
> > > > Sent from the Delta quadrant using Borg technology!
> > > >
> > > > Nux!
> > > > www.nux.ro
> > > >
> > > > ----- Original Message -----
> > > > > From: "Will Stevens" <ws...@cloudops.com>
> > > > > To: dev@cloudstack.apache.org
> > > > > Sent: Thursday, 15 September, 2016 17:21:28
> > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > >
> > > > > Ya, we would need to add a daemon for VPN as well. Load
> > > > > balancing is another aspect which we will need to consider if we
> went this route.
> > > > > Something like https://traefik.io/ could potentially be a good
> > > > > fit
> > due
> > > > to
> > > > > its API driven configuration, but it may be more than what we need.
> > > > >
> > > > > We should probably try define which pieces make sense to be
> > > > > solved
> > > > together
> > > > > and which pieces would be best suited to be broken out.
> > > > >
> > > > > I think the network connectivity, routing and firewalling should
> > > probably
> > > > > all stay together since the majority of the tools we would
> > potentially
> > > > use
> > > > > would handle all of that together in a single implementation.
> > > > >
> > > > > The password server and userdata seems like a good option for
> > > > > being
> > > > broken
> > > > > out and handled independently (and probably rewritten completely
> > since
> > > > they
> > > > > currently have some issues).
> > > > >
> > > > > Load balancing is another that could warrant splitting out, but
> > > > > that depends on what direction we go and how we would be managing
> it.
> > DHCP
> > > > and
> > > > > DNS are others which could go either way.
> > > > >
> > > > > If we do split out services, I think we should consolidate as
> > > > > much as
> > > we
> > > > > can into each service we break out. Ideally a network packet
> > > > > would
> > > never
> > > > > hit more than one, maybe two, services. I don't think we should
> > > > > be splitting services 'just because', I think we need a valid
> > > > > case for splitting any service out because it adds complexity.
> > > > > Our project is already complex enough, we need to avoid adding
> > > > > complexity unless it
> > is
> > > > > really needed.
> > > > >
> > > > > Some more of my thoughts on this anyway...
> > > > >
> > > > > *Will STEVENS*
> > > > > Lead Developer
> > > > >
> > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> > > > > *|* tw @CloudOps_
> > > > >
> > > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
> > > wrote:
> > > > >
> > > > >> I do agree with you that this probably isn't the right place
> > > > >> the
> > > > password
> > > > >> service and user data.
> > > > >>
> > > > >>
> > > > >> Having said that, after taking a cursory look at the dev docs,
> > > > >> it
> > > > doesn't
> > > > >> seem that difficult to add new daemons:
> > https://opensnaproute.github.
> > > > >> io/docs/developer.html#creating-new-component
> > > > >>
> > > > >> <https://opensnaproute.github.io/docs/developer.html#
> > > > >> creating-new-component>
> > > > >>
> > > > >>
> > > > >> They've definitely build it with a microservices architecture
> > > > >> in
> > mind,
> > > > so
> > > > >> each individual feature is abstracted into it's own small
> > > > >> daemon
> > > > process.
> > > > >> We could just create a daemon for the password server and the
> > userdata
> > > > >> components if we really had to.
> > > > >>
> > > > >>
> > > > >> - Si
> > > > >>
> > > > >>
> > > > >> ________________________________
> > > > >> From: williamstevens@gmail.com <wi...@gmail.com> on
> > > > >> behalf
> > > of
> > > > >> Will Stevens <ws...@cloudops.com>
> > > > >> Sent: Thursday, September 15, 2016 9:17 AM
> > > > >> To: dev@cloudstack.apache.org
> > > > >> Subject: Re: [DISCUSS] Replacing the VR
> > > > >>
> > > > >> A big part of why I know about it is because it is written in Go.
> > :P
> > > > >>
> > > > >> Yes, it is definitely interesting for the routing and traffic
> > handling
> > > > >> aspects of the VR. We will likely have to rethink some of the
> > pieces
> > > a
> > > > >> little bit like the password server and userdata if we are to
> > > > >> adopt
> > a
> > > > >> different VR approach. This is where I think some of JohnB and
> > > > Chiradeep's
> > > > >> ideas make sense. In many ways, it does not make sense for the
> > device
> > > > >> handling routing and network traffic to also be responsible for
> > > > passwords
> > > > >> and userdata.
> > > > >>
> > > > >> *Will STEVENS*
> > > > >> Lead Developer
> > > > >>
> > > > >> *CloudOps* *| *Cloud Solutions Experts
> > > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> > > > >> *|* tw @CloudOps_
> > > > >>
> > > > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
> > > wrote:
> > > > >>
> > > > >> > I hadn't heard of Flexswitch until you mentioned it. It looks
> > pretty
> > > > >> cool!
> > > > >> > It even supports ONIE install.
> > > > >> >
> > > > >> > To be honest, the ipsec feature could be added, or we could
> > offload
> > > > it to
> > > > >> > separate vm if we needed to. The fact it is so feature rich
> > > > >> > from a
> > > > >> routing
> > > > >> > perspective (and all API driven) is really nice.
> > > > >> >
> > > > >> >
> > > > >> > Based on the roadmap, it looks like they plan to also support
> > > > >> capabilities
> > > > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
> > > > >> > will
> > be
> > > > huge
> > > > >> > for our carrier community that rely on these technologies to
> > > > >> > do
> > > > private
> > > > >> > gateway and inter-VPC interconnections today. We handle this
> > > > >> > stuff
> > > on
> > > > our
> > > > >> > ASRs right now with a vlan interconnect into the VR. Being
> > > > >> > able to
> > > do
> > > > >> MPLS
> > > > >> > all the way to the VR would be awesome.
> > > > >> >
> > > > >> >
> > > > >> > It also seems to be written in GO (a language here at ENA we
> > > > >> > know
> > > very
> > > > >> > well).
> > > > >> >
> > > > >> >
> > > > >> > - Si
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > ________________________________
> > > > >> > From: Will Stevens <wi...@gmail.com>
> > > > >> > Sent: Thursday, September 15, 2016 7:06 AM
> > > > >> > To: dev@cloudstack.apache.org
> > > > >> > Subject: RE: [DISCUSS] Replacing the VR
> > > > >> >
> > > > >> > Ya. I don't think it covers our whole use case, but what it
> > > > >> > does
> > > > cover is
> > > > >> > all api driven...
> > > > >> >
> > > > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
> > > wrote:
> > > > >> >
> > > > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it
> > > > >> > > was
> > not
> > > > >> > > intended to do IPSec.
> > > > >> > >
> > > > >> > > It seems as though VyOS is starting to look like the best
> > option.
> > > > >> > >
> > > > >> > > Regards,
> > > > >> > > Marty Godsey
> > > > >> > > nSource Solutions
> > > > >> > >
> > > > >> > > -----Original Message-----
> > > > >> > > From: williamstevens@gmail.com
> > > > >> > > [mailto:williamstevens@gmail.com
> > ]
> > > On
> > > > >> > > Behalf Of Will Stevens
> > > > >> > > Sent: Wednesday, September 14, 2016 11:06 PM
> > > > >> > > To: dev@cloudstack.apache.org
> > > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > > > >> > >
> > > > >> > > Or we could go completely crazy and go with something like
> > > > FlexSwitch
> > > > >> > from
> > > > >> > > SnapRoute
> > > > >> > > - http://www.snaproute.com/
> > > > >> > > - https://opensnaproute.github.io/docs/apis.html
> > > > >> > >
> > > > >> > > *Will STEVENS*
> > > > >> > > Lead Developer
> > > > >> > >
> > > > >> > > *CloudOps* *| *Cloud Solutions Experts
> > > > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > >> > > cloudops.com
> > > *|*
> > > > tw
> > > > >> > > @CloudOps_
> > > > >> > >
> > > > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> > > > wstevens@cloudops.com>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > I tend to agree with Syed and Marty. I am not sure what
> > > problems
> > > > are
> > > > >> > > > solved by splitting up the function of the VR into a
> > > > >> > > > bunch of
> > > > >> separate
> > > > >> > > > services. As Syed points out, the complexity added is
> > > > non-trivial.
> > > > >> > > > We now have to manage all the intercontainer networking
> > > > >> > > > as
> > well
> > > as
> > > > >> the
> > > > >> > > > orchestrated ACS networking.
> > > > >> > > >
> > > > >> > > > VyOS is interesting to me because it covers the majority
> > > > >> > > > of
> > our
> > > > use
> > > > >> > > > case with a single unified control plane. It also has
> > > > >> > > > good
> > > > support
> > > > >> > > > for extending features we care about, like IPv6, VXLAN,
> > > > >> > > > VRRP, transactions, etc...
> > > > >> > > >
> > > > >> > > > *Will STEVENS*
> > > > >> > > > Lead Developer
> > > > >> > > >
> > > > >> > > > *CloudOps* *| *Cloud Solutions Experts
> > > > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > cloudops.com
> > > > *|*
> > > > >> tw
> > > > >> > > > @CloudOps_
> > > > >> > > >
> > > > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> > > sahmed@cloudops.com>
> > > > >> > wrote:
> > > > >> > > >
> > > > >> > > >> Agree with Marty, adding Docker containers to the
> > > > >> > > >> picture
> > > > although
> > > > >> > > >> can make the VR more flexible but the added complexity
> > > > >> > > >> is
> > just
> > > > not
> > > > >> > > >> worth it. Not to mention we would need to take care of
> > > networking
> > > > >> > > >> each container manually and given that our iptable rules
> > > > >> > > >> are
> > > very
> > > > >> > > >> unstable at the moment I don't see a big value add.
> > > > >> > > >>
> > > > >> > > >> Vyos looks like a better solution to me. I know that it
> > > > >> > > >> does
> > > not
> > > > >> > > >> provide an api but it does fit the bill quite well
> > otherwise. I
> > > > >> > > >> specially like the fact that it has a transaction based
> > > > >> > > >> model
> > > and
> > > > >> you
> > > > >> > > >> can rollback changes if something goes wrong.
> > > > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> > > > marty@gonsource.com>
> > > > >> > > wrote:
> > > > >> > > >>
> > > > >> > > >> > Licensing aside, I think splitting the various
> > > > >> > > >> > functions
> > into
> > > > >> > > >> > containers is not a good route either. This will force
> > users
> > > to
> > > > >> > > >> > have to maintain
> > > > >> > > >> and
> > > > >> > > >> > use containers and adds complexity to the networking
> > aspects
> > > of
> > > > >> ACS.
> > > > >> > > >> > Complexity decreases stability. Now I understand the
> > argument
> > > > that
> > > > >> > > >> > a monolithic approach also brings its own set of
> > > > >> > > >> > issues but
> > > it
> > > > >> also
> > > > >> > > >> > simplifies it.
> > > > >> > > >> >
> > > > >> > > >> > Regards,
> > > > >> > > >> > Marty Godsey
> > > > >> > > >> > nSource Solutions
> > > > >> > > >> >
> > > > >> > > >> > -----Original Message-----
> > > > >> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> > > > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
> > > > >> > > >> > To: dev@cloudstack.apache.org
> > > > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR
> > > > >> > > >> >
> > > > >> > > >> > I rather doubt that the Cloudrouter will fit the needs
> > > > >> > > >> > of
> > the
> > > > >> > > >> > CloudStack project
> > > > >> > > >> > - it is AGPL licensed. Many enterprises will not
> > > > >> > > >> > touch
> > > > anything
> > > > >> > > >> > that
> > > > >> > > >> has
> > > > >> > > >> > AGPL
> > > > >> > > >> > - the github repo shows rather infrequent updates.
> > > > >> > > >> > Quite
> > > > likely
> > > > >> > > >> > they aren't considering the use cases of the
> > > > >> > > >> > CloudStack
> > > > community
> > > > >> > > >> >
> > > > >> > > >> > I'd back John B's comments on disaggregating the VR.
> > > > >> > > >> > Split
> > it
> > > > into
> > > > >> > > >> > many docker containers
> > > > >> > > >> > - password server
> > > > >> > > >> > - userdata server
> > > > >> > > >> > - DHCP / DNS
> > > > >> > > >> > - s2s VPN
> > > > >> > > >> > - RA VPN
> > > > >> > > >> > - intra-VPC routing and ACL
> > > > >> > > >> > - Port forwarding + NAT
> > > > >> > > >> > - FW
> > > > >> > > >> > - LB (public)
> > > > >> > > >> > - LB (internal),
> > > > >> > > >> > - secondary storage
> > > > >> > > >> > - agent
> > > > >> > > >> > Glue them together with docker compose files (one per
> > > > >> > > >> > use
> > > > case -
> > > > >> > > >> > basic zone, isolated, VPC, SSVM, etc).
> > > > >> > > >> >
> > > > >> > > >> > The VR image then becomes a JeOS + docker. You can
> > > > >> > > >> > test
> > each
> > > of
> > > > >> the
> > > > >> > > >> > components independently and fixing one bug in the
> > > > >> > > >> > field
> > (say
> > > > >> DHCP)
> > > > >> > > >> > is hitless to the other components. You don't need to
> > > > >> > > >> > build per-hypervisor VRs. You could even run on
> baremetal.
> > > > >> > > >> >
> > > > >> > > >> > Along the way you need to figure out how to
> > > > >> > > >> > - make the traffic traverse the containers that are
> > > > >> > > >> > needed
> > > to
> > > > be
> > > > >> > > >> > traversed (in most cases just 1)
> > > > >> > > >> > - bootstrap the router (how does it find its compose
> file?
> > > > where
> > > > >> > > >> > is the
> > > > >> > > >> > registry?)
> > > > >> > > >> > - rethink the command and control of the VR
> > > > >> > > >> > functions. SSH
> > > > works,
> > > > >> > > >> > but something more declarative, idempotent should be
> > > explored.
> > > > >> > > >> >
> > > > >> > > >> > As you do this, it becomes clearer which of the
> > > > >> > > >> > functions
> > can
> > > > be
> > > > >> > > >> > substituted by for example CloudRouter. Command and
> > > > >> > > >> > Control
> > > of
> > > > the
> > > > >> > > >> docker
> > > > >> > > >> > containers can be moved out to another container. Etc.
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> > > > >> > > >> > <ma...@gonsource.com>
> > > > >> > > >> > wrote:
> > > > >> > > >> >
> > > > >> > > >> > > This one does look nice. My biggest concern is the
> > > > >> > > >> > > lack
> > of
> > > > >> > > >> > > VXLANs. It seems that any of the ones we mentioned
> > > > >> > > >> > > do not
> > > > have
> > > > >> an
> > > > >> > > >> > > API so we may be stuck at the SSH method.
> > > > >> > > >> > >
> > > > >> > > >> > > Regards,
> > > > >> > > >> > > Marty Godsey
> > > > >> > > >> > > nSource Solutions
> > > > >> > > >> > >
> > > > >> > > >> > > -----Original Message-----
> > > > >> > > >> > > From: Abhinandan Prateek
> > > > >> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
> > > > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> > > > >> > > >> > > To: dev@cloudstack.apache.org
> > > > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > > > >> > > >> > >
> > > > >> > > >> > > Cloudrouter looks promising. These have potential to
> > > > >> > > >> > > save
> > > > future
> > > > >> > > >> > > engineering effort for example on ipv6 routing, OSPF
> etc.
> > > > >> > > >> > > And the best part is they come with test automation
> > > > framework.
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> > > > >> > > >> > > <ja...@accelerite.com>
> > > > >> > > >> > > wrote:
> > > > >> > > >> > >
> > > > >> > > >> > > >Hi,
> > > > >> > > >> > > >
> > > > >> > > >> > > >Instead of replacing the VR in first place we
> > > > >> > > >> > > >should add VyOS/cloudrouter
> > > > >> > > >> > > as provider. Once it is stable, network offerings
> > > > >> > > >> > > (on
> > > > upgrade)
> > > > >> > > >> > > can be updated to use it and we can drop the VR if
> > > > >> > > >> > > we
> > want
> > > at
> > > > >> > > >> > > that release
> > > > >> > > >> > onwards.
> > > > >> > > >> > > >
> > > > >> > > >> > > >VR is stabilized over a period of time and some of
> > > > >> > > >> > > >them
> > > are
> > > > >> > > >> > > >running
> > > > >> > > >> > > without issues. When we replicate the ACS VR
> > > > >> > > >> > > features in
> > > new
> > > > >> > > >> > > solution it takes some to find the missing pieces
> > > > >> > > >> > > (hidden
> > > > bugs).
> > > > >> > > >> > > >
> > > > >> > > >> > > >Thanks,
> > > > >> > > >> > > >Jayapal
> > > > >> > > >> > > >
> > > > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> > > > >> > > >> > > >
> > > > >> > > >> > > >> nux@li.nux.ro> wrote:
> > > > >> > > >> > > >>
> > > > >> > > >> > > >> Hi,
> > > > >> > > >> > > >>
> > > > >> > > >> > > >> I like the idea.
> > > > >> > > >> > > >>
> > > > >> > > >> > > >> Cloudrouter looks really promising, I'm not too
> > > > >> > > >> > > >> keen
> > on
> > > > VyOS
> > > > >> > > >> > > >> (it
> > > > >> > > >> > > doesn't have a proper http api etc).
> > > > >> > > >> > > >>
> > > > >> > > >> > > >> --
> > > > >> > > >> > > >> Sent from the Delta quadrant using Borg technology!
> > > > >> > > >> > > >>
> > > > >> > > >> > > >> Nux!
> > > > >> > > >> > > >> www.nux.ro
> > > > >> > > >> > > >>
> > > > >> > > >> > > >>
> > > > >> > > >> > > abhinandan.prateek@shapeblue.com
> > > > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
> > > > >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> > > > @shapeblue
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > > ----- Original Message -----
> > > > >> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> > > > >> > > >> > > >>> To: dev@cloudstack.apache.org
> > > > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> > > > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
> > > > >> > > >> > > >>
> > > > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and
> > > > >> > > >> > > >>> should
> > > be
> > > > >> > > >> > > >>> treated as
> > > > >> > > >> > > such.
> > > > >> > > >> > > >>> Please weigh in with the good and bad of this
> idea...
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> A couple of us have been discussing the idea of
> > > > potentially
> > > > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open
> > > > >> > > >> > > >>> Source
> > > > Vyatta
> > > > >> > VM).
> > > > >> > > >> > > >>> There may be a license issue because I think it
> > > > >> > > >> > > >>> is
> > > > licensed
> > > > >> > > >> > > >>> under GPL, but for the sake of discussion, let's
> > assume
> > > > we
> > > > >> > > >> > > >>> can overcome any
> > > > >> > > >> > > license issues.
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> I have spent some time recently with the VyOS
> > > > >> > > >> > > >>> and I
> > > have
> > > > to
> > > > >> > > >> > > >>> admit, I was pretty impressed. It is simple and
> > > > intuitive
> > > > >> > > >> > > >>> and it gives you a lot more options for auditing
> > > > >> > > >> > > >>> the
> > > > >> > > configuration etc...
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> Items of potential interest:
> > > > >> > > >> > > >>> - Clean up our current VR script spaghetti to a
> > simpler
> > > > more
> > > > >> > > >> > > >>> auditable configuration workflow.
> > > > >> > > >> > > >>> - Gives a cleaner path for IPv6 support.
> > > > >> > > >> > > >>> - Handles VPN configuration via the same
> > configuration
> > > > >> > > interface.
> > > > >> > > >> > > >>> - Support for OSPF & BGP.
> > > > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
> > > > >> > > >> > > >>> - Easily supports HA (redundant routers) through
> > VRRP.
> > > > >> > > >> > > >>> - VXLAN support.
> > > > >> > > >> > > >>> - Transaction based changes to the VR with
> > > > >> > > >> > > >>> rollback
> > on
> > > > >> error.
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> Items that could be difficult to solve:
> > > > >> > > >> > > >>> - Userdata password reset workflow and
> > implementation.
> > > > >> > > >> > > >>> - Upgrade process.
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> The VyOS is not the only option if we were to
> > consider
> > > > this
> > > > >> > > >> approach.
> > > > >> > > >> > > >>> Another option, which I don't know as well,
> > > > >> > > >> > > >>> would be CloudRouter (AGPL
> > > > >> > > >> > > >>> license) [2] which is purely API driven.
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> Anyway, would love to hear your thoughts...
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> Will
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> [1] https://vyos.io/ [2]
> > > > >> > > >> > > >>> https://cloudrouter.org/
> > > > >> > > >> > > >
> > > > >> > > >> > > >
> > > > >> > > >> > > >
> > > > >> > > >> > > >
> > > > >> > > >> > > >DISCLAIMER
> > > > >> > > >> > > >==========
> > > > >> > > >> > > >This e-mail may contain privileged and confidential
> > > > information
> > > > >> > > >> > > >which is
> > > > >> > > >> > > the property of Accelerite, a Persistent Systems
> > business.
> > > > It is
> > > > >> > > >> > > intended only for the use of the individual or
> > > > >> > > >> > > entity to
> > > > which
> > > > >> it
> > > > >> > > >> > > is addressed. If you are not the intended recipient,
> > > > >> > > >> > > you
> > > are
> > > > not
> > > > >> > > >> > > authorized to read, retain, copy, print, distribute
> > > > >> > > >> > > or
> > use
> > > > this
> > > > >> > > >> > > message. If you have received this communication in
> > error,
> > > > >> please
> > > > >> > > >> > > notify the sender and delete all copies of this
> message.
> > > > >> > > >> > > Accelerite, a Persistent Systems business does not
> > > > >> > > >> > > accept
> > > any
> > > > >> > > >> > > liability for virus
> > > > >> > > >> > infected mails.
> > > > >> > > >> > >
> > > > >> > > >> >
> > > > >> > > >>
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > >
> > >
> >
>
RE: [DISCUSS] Replacing the VR
Posted by Will Stevens <wi...@gmail.com>.
Well the community is in charge of the documentation, so all of us. My
colleague Pierre-Luc and I have spent quite a bit of time with the docs,
but we have not attacked this. There was an initiative earlier this year to
improve the docs, but I am not sure how far they got.
On Sep 22, 2016 2:37 PM, "Marty Godsey" <ma...@gonsource.com> wrote:
> Seems like we need someone to step up and document the process of this. I
> would offer however I am not a coder so I would not get far.
>
> How is "in charge" of documentation of ACS?
>
> Regards,
> Marty Godsey
>
> -----Original Message-----
> From: Matthew Smart [mailto:msmart@smartsoftwareinc.com]
> Sent: Thursday, September 22, 2016 2:35 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Thanks Will. That is the answer I expected tbh. But it never hurts to ask!
>
> Matthew Smart
> President
> Smart Software Solutions Inc.
> 108 S Pierre St.
> Pierre, SD 57501
>
> Phone: (605) 280-0383
> Skype: msmart13
> Email: msmart@smartsoftwareinc.com
>
> On 09/22/2016 01:24 PM, Will Stevens wrote:
> > Unfortunately there is not much documentation around the network
> > plugin functionality. When I wrote the Palo Alto integration I
> > basically figured out how to do it by reviewing existing plugins and
> just figuring it out.
> >
> > So if you were to begin to implement a new hardware firewall for
> > example, I would point you to the Palo Alto integration code [1] and
> > the functional spec [2] and then make myself available to try to
> > answer any questions you have (like how the NetworkGuru works, where
> > the different pieces are registered, etc)...
> >
> > Unfortunately it is not trivial, mainly because we don't have any
> > documentation to follow, but the plugin interface IS there. It just
> > requires people who have worked it in the past to offer guidance.
> >
> > [1]
> > https://github.com/apache/cloudstack/tree/master/plugins/network-eleme
> > nts/palo-alto
> > [2]
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Palo+Alto+Firew
> > all+Integration
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > @CloudOps_
> >
> > On Thu, Sep 22, 2016 at 1:53 PM, Matthew Smart
> > <ms...@smartsoftwareinc.com>
> > wrote:
> >
> >> Hey Murali,
> >>
> >> I have been reading through the API and other documentation to try to
> >> get a basic understanding of the network service offering abstraction
> >> methodology in CS. I have not dove into the code yet but before I did
> >> I thought I would try a different approach.
> >>
> >> Imagine I were to come to this list and say that I have a network
> >> offering that I sell and that I wanted to write whatever I needed to
> >> in order to integrate it as an offering in CloudStack. Is there some
> >> specific documentation and guidelines you would direct me to read in
> >> order to gather the knowledge necessary to create a cloudstack
> >> compatible interface for my product?
> >>
> >> I don't know the history but I see several products that have
> >> navigated this process (Nuage, Nicira, ...etc) and am wondering how a
> >> new provider would work with you guys to navigate that process. If
> >> this is too vague, we can pretend my new offering is a hardware
> firewall device.
> >>
> >> My goal here is to gain an understanding of how CS interacts with
> >> third party offerings underneath the hood. I have some thoughts (I
> >> think inline with Will Steven's brain dump and diagram) but want to
> >> make sure I am not suffering some misapprehensions about the
> >> architecture and, short of tracing code, was not successful at
> >> finding the information I needed to satisfy myself that I know how it
> is designed.
> >>
> >> Thanks,
> >>
> >> Matthew Smart
> >> President
> >> Smart Software Solutions Inc.
> >> 108 S Pierre St.
> >> Pierre, SD 57501
> >>
> >> Phone: (605) 280-0383
> >> Skype: msmart13
> >> Email: msmart@smartsoftwareinc.com
> >>
> >> On 09/20/2016 04:54 AM, Murali Reddy wrote:
> >>
> >>> Configuration management of network appliances particularly for
> >>> Cloud and NFV scenarios is still evolving area. Programmability is
> >>> the not the strength for even the most popular network operating
> >>> systems like IOS, JunoS etc. So its not surprising why CloudStack
> >>> integrates in a archaic way with stock linux for the VR.
> >>>
> >>> VR was never integral/hardcoded option in CloudStack. Its always
> >>> been a plugin. CloudStack network orchestration is well abstracted
> >>> and designed with vision to compose a network with different set of
> >>> providers for different services. Yes that vision is not fully
> >>> realised yet, and we don’t have true service function chains. That
> would be different discussion topic.
> >>>
> >>> I tend to agree with Simon, as alternate/interim option we can take
> >>> hard look whats causing the problems with current VR integration.
> >>> Personally, I think it would be easier we take a cue from
> >>> configuration managers and network configuration solutions out there
> >>> (for e.g promise theory based Cisco ACI) move to more declarative
> >>> model of expressing desired state of network configuration. Infact
> >>> current VR from 4.6, actually holds the desired state per service
> basis, seems to be in that direction.
> >>>
> >>> It does make sense to evaluate new appliances which can provide rich
> >>> semantics (like programmable API, declarative configuration,
> >>> versioning, commit/rollback etc), but will need significant
> >>> engineering effort and time to stabilise. We may make same mistakes
> >>> with integration of other appliance as well, if we fully don’t
> >>> understand the nature of the current problems with CloudStack core
> >>> and service provider interaction and current VR integration.
> >>>
> >>>
> >>> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
> >>>
> >>> I think our other option is to take a real look at what it would
> >>> take to
> >>>> fix the VR. In my opinion, a lot of the problems are related to the
> >>>> monolithic python code base and the fact nothing is actually
> separated.
> >>>>
> >>>> Secondly, the python scripts (and bash scripts) don't use any
> >>>> established libraries to complete tasks and instead shell out and
> >>>> run commands that are both hard to track and hard to parse on return.
> >>>>
> >>>>
> >>>> If we daemonized this, used a real api for Agent to VR
> >>>> communication, used common already existing libraries for the
> >>>> system service and network interactions and spent a bit of time
> >>>> separating out code into distinct modules, everything would behave a
> lot better.
> >>>>
> >>>>
> >>>> The pain and suffering is due to years and years of patches and
> >>>> constant shelling out to complete tasks in my opinion. If we spend
> >>>> time to rethink how we interact with the VR in general and we
> >>>> abstract the systems and networking stuff and use well known and
> >>>> stable libraries to do the work, the VR would be much easier to
> maintain.
> >>>>
> >>>>
> >>>> - Si
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> ________________________________
> >>>> From: Marty Godsey <ma...@gonsource.com>
> >>>> Sent: Friday, September 16, 2016 12:24 PM
> >>>> To: dev@cloudstack.apache.org
> >>>> Subject: RE: [DISCUSS] Replacing the VR
> >>>>
> >>>> So based upon this discussion would it be prudent to wait on VyOS 2.0?
> >>>> The current VR is giving us issues but would the time invested in
> >>>> another "solution" be wasted especially if by the time another
> >>>> option is chose, then coded, then tested, then implemented and
> >>>> right as that time happened to be when VyOS 2.0 is released. Of
> >>>> course you said they are just in the scoping range so this could
> still be a year or more out.
> >>>>
> >>>> Thoughts?
> >>>>
> >>>> Regards,
> >>>> Marty Godsey
> >>>> nSource Solutions
> >>>>
> >>>> -----Original Message-----
> >>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> >>>> Behalf Of Will Stevens
> >>>> Sent: Friday, September 16, 2016 10:31 AM
> >>>> To: dev@cloudstack.apache.org
> >>>> Cc: daniil@baturin.org
> >>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>
> >>>> I just had a quick chat with a couple of the guys over on the VyOS
> chat.
> >>>> I have CC'ed one of them in case we have more licensing questions.
> >>>>
> >>>> So here is the status with the license "the code inherited from
> >>>> Vyatta and our modifications from it is GPLv2 (strict, not v2+).
> >>>> The config reading library is GPLv2 too, so anything that links to is
> is GPLv2.
> >>>> Some auxiliary components we made after the fork are more
> >>>> permissive,
> >>>> LGPLv2+ or MIT."
> >>>>
> >>>> They are currently in the process of scoping a redesign (VyOS 2.0),
> >>>> "we are planning a clean rewrite that will solve issues of the
> >>>> current config system".
> >>>> This will include the ability to configure via the API.
> >>>>
> >>>> If we have more questions for VyOS, they are very friendly and
> >>>> responsive, so we should be able to get answers.
> >>>>
> >>>> *Will STEVENS*
> >>>> Lead Developer
> >>>>
> >>>> *CloudOps* *| *Cloud Solutions Experts
> >>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> >>>> tw @CloudOps_
> >>>>
> >>>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com>
> wrote:
> >>>>
> >>>> I agree with Will Ilya. There are so many problems with the VR right
> now.
> >>>>> Most of the outages we've had recently have somehow involved the VR.
> >>>>> We set custom iptables rules on the VR which can and have easily
> >>>>> gone wrong.
> >>>>> Openswan is broken, Strongswan replacement still needs to be tested.
> >>>>> VVRP with redundant router still needs work, and not to mention
> >>>>> the problems we will have when we introduce IPv6 into the whole
> picture.
> >>>>>
> >>>>> I think the spirit of the discussion is to rely on a 3rd party to
> >>>>> do the networking for us (eg VyOS) and have us handle just the
> >>>>> orchestration. All the problems that I've described have already
> >>>>> been solved in VyOS. We also get the advantage of a potential
> >>>>> wider community to fix and maintain the VR and given our current
> >>>>> development velocity, it think it totally makes sense to look for
> >>>>> a 3rd party option.
> >>>>>
> >>>>> -Syed
> >>>>>
> >>>>>
> >>>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens
> >>>>> <ws...@cloudops.com>
> >>>>> wrote:
> >>>>>
> >>>>> The VR has been biting us far too often recently, which is why we
> >>>>>> have started looking into alternative implementations.
> >>>>>>
> >>>>>> One of the things that is nice about potentially using the VyOS
> >>>>>> is that
> >>>>>>
> >>>>> it
> >>>>>
> >>>>>> is based on Debian, so we should be able to run the other
> >>>>>> services that
> >>>>>>
> >>>>> we
> >>>>>
> >>>>>> currently have like the password server and userdata on the VyOS.
> >>>>>> This means we would not have to change our architecture initially
> >>>>>> and could focus on only replacing the networking paths.
> >>>>>>
> >>>>>> *Will STEVENS*
> >>>>>> Lead Developer
> >>>>>>
> >>>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> >>>>>> *|* tw @CloudOps_
> >>>>>>
> >>>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> >>>>>>
> >>>>>> The more this is discussed the more I think we should stick with
> >>>>>>> our
> >>>>>>>
> >>>>>> VR.
> >>>>>> All these other options either seem unfinished or with
> >>>>>>> incompatible license.
> >>>>>>>
> >>>>>>> VyOS looks the most promising so far, it's a serious, mature
> project.
> >>>>>>> Adopting it though means we'll have to microservice our way out
> >>>>>>> of it
> >>>>>>>
> >>>>>> with
> >>>>>>
> >>>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS
> >>>>>>> serve
> >>>>>>>
> >>>>>> those
> >>>>>> too. Imho this adds complexity we should void.
> >>>>>>> --
> >>>>>>> Sent from the Delta quadrant using Borg technology!
> >>>>>>>
> >>>>>>> Nux!
> >>>>>>> www.nux.ro
> >>>>>>>
> >>>>>>> ----- Original Message -----
> >>>>>>>
> >>>>>>>> From: "Will Stevens" <ws...@cloudops.com>
> >>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
> >>>>>>>> Subject: Re: [DISCUSS] Replacing the VR Ya, we would need to
> >>>>>>>> add a daemon for VPN as well. Load balancing is another aspect
> >>>>>>>> which we will need to consider if we went this route.
> >>>>>>>> Something like https://traefik.io/ could potentially be a good
> >>>>>>>> fit
> >>>>>>>>
> >>>>>>> due
> >>>>>> to
> >>>>>>>> its API driven configuration, but it may be more than what we
> need.
> >>>>>>>>
> >>>>>>>> We should probably try define which pieces make sense to be
> >>>>>>>> solved
> >>>>>>>>
> >>>>>>> together
> >>>>>>>
> >>>>>>>> and which pieces would be best suited to be broken out.
> >>>>>>>>
> >>>>>>>> I think the network connectivity, routing and firewalling
> >>>>>>>> should
> >>>>>>>>
> >>>>>>> probably
> >>>>>>> all stay together since the majority of the tools we would
> >>>>>>> potentially
> >>>>>> use
> >>>>>>>> would handle all of that together in a single implementation.
> >>>>>>>>
> >>>>>>>> The password server and userdata seems like a good option for
> >>>>>>>> being
> >>>>>>>>
> >>>>>>> broken
> >>>>>>>
> >>>>>>>> out and handled independently (and probably rewritten
> >>>>>>>> completely
> >>>>>>>>
> >>>>>>> since
> >>>>>> they
> >>>>>>>> currently have some issues).
> >>>>>>>>
> >>>>>>>> Load balancing is another that could warrant splitting out, but
> >>>>>>>> that depends on what direction we go and how we would be managing
> it.
> >>>>>>>>
> >>>>>>> DHCP
> >>>>>> and
> >>>>>>>> DNS are others which could go either way.
> >>>>>>>>
> >>>>>>>> If we do split out services, I think we should consolidate as
> >>>>>>>> much as
> >>>>>>>>
> >>>>>>> we
> >>>>>>> can into each service we break out. Ideally a network packet
> >>>>>>>> would
> >>>>>>>>
> >>>>>>> never
> >>>>>>> hit more than one, maybe two, services. I don't think we should
> >>>>>>>> be splitting services 'just because', I think we need a valid
> >>>>>>>> case for splitting any service out because it adds complexity.
> >>>>>>>> Our project is already complex enough, we need to avoid adding
> >>>>>>>> complexity unless it
> >>>>>>>>
> >>>>>>> is
> >>>>>> really needed.
> >>>>>>>> Some more of my thoughts on this anyway...
> >>>>>>>>
> >>>>>>>> *Will STEVENS*
> >>>>>>>> Lead Developer
> >>>>>>>>
> >>>>>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> >>>>>>>> *|* tw @CloudOps_
> >>>>>>>>
> >>>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller
> >>>>>>>> <sw...@ena.com>
> >>>>>>>>
> >>>>>>> wrote:
> >>>>>>> I do agree with you that this probably isn't the right place
> >>>>>>>>> the
> >>>>>>>>>
> >>>>>>>> password
> >>>>>>>> service and user data.
> >>>>>>>>>
> >>>>>>>>> Having said that, after taking a cursory look at the dev docs,
> >>>>>>>>> it
> >>>>>>>>>
> >>>>>>>> doesn't
> >>>>>>>> seem that difficult to add new daemons:
> >>>>>>>> https://opensnaproute.github.
> >>>>>> io/docs/developer.html#creating-new-component
> >>>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
> >>>>>>>>> creating-new-component>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> They've definitely build it with a microservices architecture
> >>>>>>>>> in
> >>>>>>>>>
> >>>>>>>> mind,
> >>>>>> so
> >>>>>>>> each individual feature is abstracted into it's own small
> >>>>>>>>> daemon
> >>>>>>>>>
> >>>>>>>> process.
> >>>>>>>> We could just create a daemon for the password server and the
> >>>>>>>> userdata
> >>>>>> components if we really had to.
> >>>>>>>>>
> >>>>>>>>> - Si
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> ________________________________
> >>>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
> >>>>>>>>> behalf
> >>>>>>>>>
> >>>>>>>> of
> >>>>>>> Will Stevens <ws...@cloudops.com>
> >>>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
> >>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>>>
> >>>>>>>>> A big part of why I know about it is because it is written in Go.
> >>>>>>>>>
> >>>>>>>> :P
> >>>>>> Yes, it is definitely interesting for the routing and traffic
> >>>>>>>> handling
> >>>>>> aspects of the VR. We will likely have to rethink some of the
> >>>>>>>> pieces
> >>>>>> a
> >>>>>>
> >>>>>>> little bit like the password server and userdata if we are to
> >>>>>>>>> adopt
> >>>>>>>>>
> >>>>>>>> a
> >>>>>> different VR approach. This is where I think some of JohnB and
> >>>>>>>> Chiradeep's
> >>>>>>>> ideas make sense. In many ways, it does not make sense for the
> >>>>>>>> device
> >>>>>> handling routing and network traffic to also be responsible for
> >>>>>>>> passwords
> >>>>>>>> and userdata.
> >>>>>>>>> *Will STEVENS*
> >>>>>>>>> Lead Developer
> >>>>>>>>>
> >>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> >>>>>>>>> *|* tw @CloudOps_
> >>>>>>>>>
> >>>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller
> >>>>>>>>> <sw...@ena.com>
> >>>>>>>>>
> >>>>>>>> wrote:
> >>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
> >>>>>>>>> pretty
> >>>>>> cool!
> >>>>>>>>>> It even supports ONIE install.
> >>>>>>>>>>
> >>>>>>>>>> To be honest, the ipsec feature could be added, or we could
> >>>>>>>>>>
> >>>>>>>>> offload
> >>>>>> it to
> >>>>>>>> separate vm if we needed to. The fact it is so feature rich
> >>>>>>>>>> from a
> >>>>>>>>>>
> >>>>>>>>> routing
> >>>>>>>>>
> >>>>>>>>>> perspective (and all API driven) is really nice.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Based on the roadmap, it looks like they plan to also support
> >>>>>>>>>>
> >>>>>>>>> capabilities
> >>>>>>>>>
> >>>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
> >>>>>>>>>> will
> >>>>>>>>>>
> >>>>>>>>> be
> >>>>>> huge
> >>>>>>>> for our carrier community that rely on these technologies to
> >>>>>>>>>> do
> >>>>>>>>>>
> >>>>>>>>> private
> >>>>>>>> gateway and inter-VPC interconnections today. We handle this
> >>>>>>>>>> stuff
> >>>>>>>>>>
> >>>>>>>>> on
> >>>>>>> our
> >>>>>>>
> >>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
> >>>>>>>>>> able to
> >>>>>>>>>>
> >>>>>>>>> do
> >>>>>>> MPLS
> >>>>>>>>>> all the way to the VR would be awesome.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> It also seems to be written in GO (a language here at ENA we
> >>>>>>>>>> know
> >>>>>>>>>>
> >>>>>>>>> very
> >>>>>>> well).
> >>>>>>>>>>
> >>>>>>>>>> - Si
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> ________________________________
> >>>>>>>>>> From: Will Stevens <wi...@gmail.com>
> >>>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
> >>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
> >>>>>>>>>>
> >>>>>>>>>> Ya. I don't think it covers our whole use case, but what it
> >>>>>>>>>> does
> >>>>>>>>>>
> >>>>>>>>> cover is
> >>>>>>>> all api driven...
> >>>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
> >>>>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>> Though I don’t see VPN in Snaproute.. Makes sense since it
> >>>>>>>>>>> was
> >>>>>>>>>>>
> >>>>>>>>>> not
> >>>>>> intended to do IPSec.
> >>>>>>>>>>> It seems as though VyOS is starting to look like the best
> >>>>>>>>>>>
> >>>>>>>>>> option.
> >>>>>> Regards,
> >>>>>>>>>>> Marty Godsey
> >>>>>>>>>>> nSource Solutions
> >>>>>>>>>>>
> >>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>> From: williamstevens@gmail.com
> >>>>>>>>>>> [mailto:williamstevens@gmail.com
> >>>>>>>>>>>
> >>>>>>>>>> ]
> >>>>>> On
> >>>>>>
> >>>>>>> Behalf Of Will Stevens
> >>>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
> >>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>>>>>
> >>>>>>>>>>> Or we could go completely crazy and go with something like
> >>>>>>>>>>>
> >>>>>>>>>> FlexSwitch
> >>>>>>>> from
> >>>>>>>>>>> SnapRoute
> >>>>>>>>>>> - http://www.snaproute.com/
> >>>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
> >>>>>>>>>>>
> >>>>>>>>>>> *Will STEVENS*
> >>>>>>>>>>> Lead Developer
> >>>>>>>>>>>
> >>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> >>>>>>>>>>> cloudops.com
> >>>>>>>>>>>
> >>>>>>>>>> *|*
> >>>>>>> tw
> >>>>>>>
> >>>>>>>> @CloudOps_
> >>>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> >>>>>>>>>>>
> >>>>>>>>>> wstevens@cloudops.com>
> >>>>>>>> wrote:
> >>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
> >>>>>>>>>>> problems
> >>>>>>> are
> >>>>>>>
> >>>>>>>> solved by splitting up the function of the VR into a
> >>>>>>>>>>>> bunch of
> >>>>>>>>>>>>
> >>>>>>>>>>> separate
> >>>>>>>>>> services. As Syed points out, the complexity added is
> >>>>>>>>>>> non-trivial.
> >>>>>>>> We now have to manage all the intercontainer networking
> >>>>>>>>>>>> as
> >>>>>>>>>>>>
> >>>>>>>>>>> well
> >>>>>> as
> >>>>>>
> >>>>>>> the
> >>>>>>>>>> orchestrated ACS networking.
> >>>>>>>>>>>> VyOS is interesting to me because it covers the majority of
> >>>>>>>>>>>>
> >>>>>>>>>>> our
> >>>>>> use
> >>>>>>>> case with a single unified control plane. It also has
> >>>>>>>>>>>> good
> >>>>>>>>>>>>
> >>>>>>>>>>> support
> >>>>>>>> for extending features we care about, like IPv6, VXLAN,
> >>>>>>>>>>>> VRRP, transactions, etc...
> >>>>>>>>>>>>
> >>>>>>>>>>>> *Will STEVENS*
> >>>>>>>>>>>> Lead Developer
> >>>>>>>>>>>>
> >>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> >>>>>>>>>>>>
> >>>>>>>>>>> cloudops.com
> >>>>>> *|*
> >>>>>>>> tw
> >>>>>>>>>> @CloudOps_
> >>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> >>>>>>>>>>>>
> >>>>>>>>>>> sahmed@cloudops.com>
> >>>>>>> wrote:
> >>>>>>>>>>> Agree with Marty, adding Docker containers to the
> >>>>>>>>>>>>> picture
> >>>>>>>>>>>>>
> >>>>>>>>>>>> although
> >>>>>>>> can make the VR more flexible but the added complexity
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>
> >>>>>>>>>>>> just
> >>>>>> not
> >>>>>>>> worth it. Not to mention we would need to take care of
> >>>>>>>>>>>> networking
> >>>>>>> each container manually and given that our iptable rules
> >>>>>>>>>>>>> are
> >>>>>>>>>>>>>
> >>>>>>>>>>>> very
> >>>>>>> unstable at the moment I don't see a big value add.
> >>>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
> >>>>>>>>>>>>> does
> >>>>>>>>>>>>>
> >>>>>>>>>>>> not
> >>>>>>> provide an api but it does fit the bill quite well
> >>>>>>>>>>>> otherwise. I
> >>>>>> specially like the fact that it has a transaction based
> >>>>>>>>>>>>> model
> >>>>>>>>>>>>>
> >>>>>>>>>>>> and
> >>>>>>> you
> >>>>>>>>>> can rollback changes if something goes wrong.
> >>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> >>>>>>>>>>>>>
> >>>>>>>>>>>> marty@gonsource.com>
> >>>>>>>> wrote:
> >>>>>>>>>>>> Licensing aside, I think splitting the various
> >>>>>>>>>>>>>> functions
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> into
> >>>>>> containers is not a good route either. This will force
> >>>>>>>>>>>>> users
> >>>>>> to
> >>>>>>
> >>>>>>> have to maintain
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> use containers and adds complexity to the networking
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> aspects
> >>>>>> of
> >>>>>>
> >>>>>>> ACS.
> >>>>>>>>>> Complexity decreases stability. Now I understand the
> >>>>>>>>>>>>> argument
> >>>>>> that
> >>>>>>>> a monolithic approach also brings its own set of
> >>>>>>>>>>>>>> issues but
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> it
> >>>>>>> also
> >>>>>>>>>> simplifies it.
> >>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>> Marty Godsey
> >>>>>>>>>>>>>> nSource Solutions
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> >>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
> >>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs of
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> the
> >>>>>> CloudStack project
> >>>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not touch
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> anything
> >>>>>>>> that
> >>>>>>>>>>>>> has
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> AGPL
> >>>>>>>>>>>>>> - the github repo shows rather infrequent updates.
> >>>>>>>>>>>>>> Quite
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> likely
> >>>>>>>> they aren't considering the use cases of the
> >>>>>>>>>>>>>> CloudStack
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> community
> >>>>>>>> I'd back John B's comments on disaggregating the VR.
> >>>>>>>>>>>>>> Split
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> it
> >>>>>> into
> >>>>>>>> many docker containers
> >>>>>>>>>>>>>> - password server
> >>>>>>>>>>>>>> - userdata server
> >>>>>>>>>>>>>> - DHCP / DNS
> >>>>>>>>>>>>>> - s2s VPN
> >>>>>>>>>>>>>> - RA VPN
> >>>>>>>>>>>>>> - intra-VPC routing and ACL
> >>>>>>>>>>>>>> - Port forwarding + NAT
> >>>>>>>>>>>>>> - FW
> >>>>>>>>>>>>>> - LB (public)
> >>>>>>>>>>>>>> - LB (internal),
> >>>>>>>>>>>>>> - secondary storage
> >>>>>>>>>>>>>> - agent
> >>>>>>>>>>>>>> Glue them together with docker compose files (one per
> >>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> case -
> >>>>>>>> basic zone, isolated, VPC, SSVM, etc).
> >>>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can test
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> each
> >>>>>> of
> >>>>>>
> >>>>>>> the
> >>>>>>>>>> components independently and fixing one bug in the
> >>>>>>>>>>>>>> field
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> (say
> >>>>>> DHCP)
> >>>>>>>>>> is hitless to the other components. You don't need to
> >>>>>>>>>>>>>> build per-hypervisor VRs. You could even run on baremetal.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Along the way you need to figure out how to
> >>>>>>>>>>>>>> - make the traffic traverse the containers that are
> >>>>>>>>>>>>>> needed
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> to
> >>>>>>> be
> >>>>>>>
> >>>>>>>> traversed (in most cases just 1)
> >>>>>>>>>>>>>> - bootstrap the router (how does it find its compose
> file?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> where
> >>>>>>>> is the
> >>>>>>>>>>>>>> registry?)
> >>>>>>>>>>>>>> - rethink the command and control of the VR functions.
> >>>>>>>>>>>>>> SSH
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> works,
> >>>>>>>> but something more declarative, idempotent should be
> >>>>>>>>>>>>> explored.
> >>>>>>> As you do this, it becomes clearer which of the
> >>>>>>>>>>>>>> functions
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> can
> >>>>>> be
> >>>>>>>> substituted by for example CloudRouter. Command and
> >>>>>>>>>>>>>> Control
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> of
> >>>>>>> the
> >>>>>>>
> >>>>>>>> docker
> >>>>>>>>>>>>>> containers can be moved out to another container. Etc.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> >>>>>>>>>>>>>> <ma...@gonsource.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> This one does look nice. My biggest concern is the
> >>>>>>>>>>>>>>> lack
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> of
> >>>>>> VXLANs. It seems that any of the ones we mentioned
> >>>>>>>>>>>>>>> do not
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> have
> >>>>>>>> an
> >>>>>>>>>> API so we may be stuck at the SSH method.
> >>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>> Marty Godsey
> >>>>>>>>>>>>>>> nSource Solutions
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>>>>> From: Abhinandan Prateek
> >>>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
> >>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
> >>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
> >>>>>>>>>>>>>>> save
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> future
> >>>>>>>> engineering effort for example on ipv6 routing, OSPF etc.
> >>>>>>>>>>>>>>> And the best part is they come with test automation
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> framework.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
> >>>>>>>>>>>>>>> <ja...@accelerite.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>> Instead of replacing the VR in first place we should
> >>>>>>>>>>>>>>>> add VyOS/cloudrouter
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> as provider. Once it is stable, network offerings (on
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> upgrade)
> >>>>>>>> can be updated to use it and we can drop the VR if
> >>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> want
> >>>>>> at
> >>>>>>
> >>>>>>> that release
> >>>>>>>>>>>>>> onwards.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
> >>>>>>>>>>>>>>>> them
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> are
> >>>>>>> running
> >>>>>>>>>>>>>>> without issues. When we replicate the ACS VR features
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> new
> >>>>>>> solution it takes some to find the missing pieces
> >>>>>>>>>>>>>>> (hidden
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> bugs).
> >>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> Jayapal
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
> >>>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I like the idea.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too keen
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> on
> >>>>>> VyOS
> >>>>>>>> (it
> >>>>>>>>>>>>>>>> doesn't have a proper http api etc).
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Nux!
> >>>>>>>>>>>>>>>>> www.nux.ro
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
> >>>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
> >>>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> @shapeblue
> >>>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
> >>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
> >>>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
> >>>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and should
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> be
> >>>>>>> treated as
> >>>>>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>> Please weigh in with the good and bad of this idea...
> >>>>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> potentially
> >>>>>>>> replacing the ACS VR with the VyOS [1] (Open
> >>>>>>>>>>>>>>>>>> Source
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Vyatta
> >>>>>>>> VM).
> >>>>>>>>>>> There may be a license issue because I think it
> >>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> licensed
> >>>>>>>> under GPL, but for the sake of discussion, let's
> >>>>>>>>>>>>>>>>> assume
> >>>>>> we
> >>>>>>>> can overcome any
> >>>>>>>>>>>>>>>>> license issues.
> >>>>>>>>>>>>>>>> I have spent some time recently with the VyOS
> >>>>>>>>>>>>>>>>>> and I
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> have
> >>>>>>> to
> >>>>>>>
> >>>>>>>> admit, I was pretty impressed. It is simple and
> >>>>>>>>>>>>>>>>> intuitive
> >>>>>>>> and it gives you a lot more options for auditing
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> configuration etc...
> >>>>>>>>>>>> Items of potential interest:
> >>>>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> simpler
> >>>>>> more
> >>>>>>>> auditable configuration workflow.
> >>>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
> >>>>>>>>>>>>>>>>>> - Handles VPN configuration via the same
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> configuration
> >>>>>> interface.
> >>>>>>>>>>
>
>
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
Seems like we need someone to step up and document the process of this. I would offer however I am not a coder so I would not get far.
How is "in charge" of documentation of ACS?
Regards,
Marty Godsey
-----Original Message-----
From: Matthew Smart [mailto:msmart@smartsoftwareinc.com]
Sent: Thursday, September 22, 2016 2:35 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
Thanks Will. That is the answer I expected tbh. But it never hurts to ask!
Matthew Smart
President
Smart Software Solutions Inc.
108 S Pierre St.
Pierre, SD 57501
Phone: (605) 280-0383
Skype: msmart13
Email: msmart@smartsoftwareinc.com
On 09/22/2016 01:24 PM, Will Stevens wrote:
> Unfortunately there is not much documentation around the network
> plugin functionality. When I wrote the Palo Alto integration I
> basically figured out how to do it by reviewing existing plugins and just figuring it out.
>
> So if you were to begin to implement a new hardware firewall for
> example, I would point you to the Palo Alto integration code [1] and
> the functional spec [2] and then make myself available to try to
> answer any questions you have (like how the NetworkGuru works, where
> the different pieces are registered, etc)...
>
> Unfortunately it is not trivial, mainly because we don't have any
> documentation to follow, but the plugin interface IS there. It just
> requires people who have worked it in the past to offer guidance.
>
> [1]
> https://github.com/apache/cloudstack/tree/master/plugins/network-eleme
> nts/palo-alto
> [2]
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Palo+Alto+Firew
> all+Integration
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Thu, Sep 22, 2016 at 1:53 PM, Matthew Smart
> <ms...@smartsoftwareinc.com>
> wrote:
>
>> Hey Murali,
>>
>> I have been reading through the API and other documentation to try to
>> get a basic understanding of the network service offering abstraction
>> methodology in CS. I have not dove into the code yet but before I did
>> I thought I would try a different approach.
>>
>> Imagine I were to come to this list and say that I have a network
>> offering that I sell and that I wanted to write whatever I needed to
>> in order to integrate it as an offering in CloudStack. Is there some
>> specific documentation and guidelines you would direct me to read in
>> order to gather the knowledge necessary to create a cloudstack
>> compatible interface for my product?
>>
>> I don't know the history but I see several products that have
>> navigated this process (Nuage, Nicira, ...etc) and am wondering how a
>> new provider would work with you guys to navigate that process. If
>> this is too vague, we can pretend my new offering is a hardware firewall device.
>>
>> My goal here is to gain an understanding of how CS interacts with
>> third party offerings underneath the hood. I have some thoughts (I
>> think inline with Will Steven's brain dump and diagram) but want to
>> make sure I am not suffering some misapprehensions about the
>> architecture and, short of tracing code, was not successful at
>> finding the information I needed to satisfy myself that I know how it is designed.
>>
>> Thanks,
>>
>> Matthew Smart
>> President
>> Smart Software Solutions Inc.
>> 108 S Pierre St.
>> Pierre, SD 57501
>>
>> Phone: (605) 280-0383
>> Skype: msmart13
>> Email: msmart@smartsoftwareinc.com
>>
>> On 09/20/2016 04:54 AM, Murali Reddy wrote:
>>
>>> Configuration management of network appliances particularly for
>>> Cloud and NFV scenarios is still evolving area. Programmability is
>>> the not the strength for even the most popular network operating
>>> systems like IOS, JunoS etc. So its not surprising why CloudStack
>>> integrates in a archaic way with stock linux for the VR.
>>>
>>> VR was never integral/hardcoded option in CloudStack. Its always
>>> been a plugin. CloudStack network orchestration is well abstracted
>>> and designed with vision to compose a network with different set of
>>> providers for different services. Yes that vision is not fully
>>> realised yet, and we don’t have true service function chains. That would be different discussion topic.
>>>
>>> I tend to agree with Simon, as alternate/interim option we can take
>>> hard look whats causing the problems with current VR integration.
>>> Personally, I think it would be easier we take a cue from
>>> configuration managers and network configuration solutions out there
>>> (for e.g promise theory based Cisco ACI) move to more declarative
>>> model of expressing desired state of network configuration. Infact
>>> current VR from 4.6, actually holds the desired state per service basis, seems to be in that direction.
>>>
>>> It does make sense to evaluate new appliances which can provide rich
>>> semantics (like programmable API, declarative configuration,
>>> versioning, commit/rollback etc), but will need significant
>>> engineering effort and time to stabilise. We may make same mistakes
>>> with integration of other appliance as well, if we fully don’t
>>> understand the nature of the current problems with CloudStack core
>>> and service provider interaction and current VR integration.
>>>
>>>
>>> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>>>
>>> I think our other option is to take a real look at what it would
>>> take to
>>>> fix the VR. In my opinion, a lot of the problems are related to the
>>>> monolithic python code base and the fact nothing is actually separated.
>>>>
>>>> Secondly, the python scripts (and bash scripts) don't use any
>>>> established libraries to complete tasks and instead shell out and
>>>> run commands that are both hard to track and hard to parse on return.
>>>>
>>>>
>>>> If we daemonized this, used a real api for Agent to VR
>>>> communication, used common already existing libraries for the
>>>> system service and network interactions and spent a bit of time
>>>> separating out code into distinct modules, everything would behave a lot better.
>>>>
>>>>
>>>> The pain and suffering is due to years and years of patches and
>>>> constant shelling out to complete tasks in my opinion. If we spend
>>>> time to rethink how we interact with the VR in general and we
>>>> abstract the systems and networking stuff and use well known and
>>>> stable libraries to do the work, the VR would be much easier to maintain.
>>>>
>>>>
>>>> - Si
>>>>
>>>>
>>>>
>>>>
>>>> ________________________________
>>>> From: Marty Godsey <ma...@gonsource.com>
>>>> Sent: Friday, September 16, 2016 12:24 PM
>>>> To: dev@cloudstack.apache.org
>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>
>>>> So based upon this discussion would it be prudent to wait on VyOS 2.0?
>>>> The current VR is giving us issues but would the time invested in
>>>> another "solution" be wasted especially if by the time another
>>>> option is chose, then coded, then tested, then implemented and
>>>> right as that time happened to be when VyOS 2.0 is released. Of
>>>> course you said they are just in the scoping range so this could still be a year or more out.
>>>>
>>>> Thoughts?
>>>>
>>>> Regards,
>>>> Marty Godsey
>>>> nSource Solutions
>>>>
>>>> -----Original Message-----
>>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>>>> Behalf Of Will Stevens
>>>> Sent: Friday, September 16, 2016 10:31 AM
>>>> To: dev@cloudstack.apache.org
>>>> Cc: daniil@baturin.org
>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>
>>>> I just had a quick chat with a couple of the guys over on the VyOS chat.
>>>> I have CC'ed one of them in case we have more licensing questions.
>>>>
>>>> So here is the status with the license "the code inherited from
>>>> Vyatta and our modifications from it is GPLv2 (strict, not v2+).
>>>> The config reading library is GPLv2 too, so anything that links to is is GPLv2.
>>>> Some auxiliary components we made after the fork are more
>>>> permissive,
>>>> LGPLv2+ or MIT."
>>>>
>>>> They are currently in the process of scoping a redesign (VyOS 2.0),
>>>> "we are planning a clean rewrite that will solve issues of the
>>>> current config system".
>>>> This will include the ability to configure via the API.
>>>>
>>>> If we have more questions for VyOS, they are very friendly and
>>>> responsive, so we should be able to get answers.
>>>>
>>>> *Will STEVENS*
>>>> Lead Developer
>>>>
>>>> *CloudOps* *| *Cloud Solutions Experts
>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>> tw @CloudOps_
>>>>
>>>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>>>>
>>>> I agree with Will Ilya. There are so many problems with the VR right now.
>>>>> Most of the outages we've had recently have somehow involved the VR.
>>>>> We set custom iptables rules on the VR which can and have easily
>>>>> gone wrong.
>>>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>>>> VVRP with redundant router still needs work, and not to mention
>>>>> the problems we will have when we introduce IPv6 into the whole picture.
>>>>>
>>>>> I think the spirit of the discussion is to rely on a 3rd party to
>>>>> do the networking for us (eg VyOS) and have us handle just the
>>>>> orchestration. All the problems that I've described have already
>>>>> been solved in VyOS. We also get the advantage of a potential
>>>>> wider community to fix and maintain the VR and given our current
>>>>> development velocity, it think it totally makes sense to look for
>>>>> a 3rd party option.
>>>>>
>>>>> -Syed
>>>>>
>>>>>
>>>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens
>>>>> <ws...@cloudops.com>
>>>>> wrote:
>>>>>
>>>>> The VR has been biting us far too often recently, which is why we
>>>>>> have started looking into alternative implementations.
>>>>>>
>>>>>> One of the things that is nice about potentially using the VyOS
>>>>>> is that
>>>>>>
>>>>> it
>>>>>
>>>>>> is based on Debian, so we should be able to run the other
>>>>>> services that
>>>>>>
>>>>> we
>>>>>
>>>>>> currently have like the password server and userdata on the VyOS.
>>>>>> This means we would not have to change our architecture initially
>>>>>> and could focus on only replacing the networking paths.
>>>>>>
>>>>>> *Will STEVENS*
>>>>>> Lead Developer
>>>>>>
>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>> *|* tw @CloudOps_
>>>>>>
>>>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>>>
>>>>>> The more this is discussed the more I think we should stick with
>>>>>>> our
>>>>>>>
>>>>>> VR.
>>>>>> All these other options either seem unfinished or with
>>>>>>> incompatible license.
>>>>>>>
>>>>>>> VyOS looks the most promising so far, it's a serious, mature project.
>>>>>>> Adopting it though means we'll have to microservice our way out
>>>>>>> of it
>>>>>>>
>>>>>> with
>>>>>>
>>>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS
>>>>>>> serve
>>>>>>>
>>>>>> those
>>>>>> too. Imho this adds complexity we should void.
>>>>>>> --
>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>
>>>>>>> Nux!
>>>>>>> www.nux.ro
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>
>>>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR Ya, we would need to
>>>>>>>> add a daemon for VPN as well. Load balancing is another aspect
>>>>>>>> which we will need to consider if we went this route.
>>>>>>>> Something like https://traefik.io/ could potentially be a good
>>>>>>>> fit
>>>>>>>>
>>>>>>> due
>>>>>> to
>>>>>>>> its API driven configuration, but it may be more than what we need.
>>>>>>>>
>>>>>>>> We should probably try define which pieces make sense to be
>>>>>>>> solved
>>>>>>>>
>>>>>>> together
>>>>>>>
>>>>>>>> and which pieces would be best suited to be broken out.
>>>>>>>>
>>>>>>>> I think the network connectivity, routing and firewalling
>>>>>>>> should
>>>>>>>>
>>>>>>> probably
>>>>>>> all stay together since the majority of the tools we would
>>>>>>> potentially
>>>>>> use
>>>>>>>> would handle all of that together in a single implementation.
>>>>>>>>
>>>>>>>> The password server and userdata seems like a good option for
>>>>>>>> being
>>>>>>>>
>>>>>>> broken
>>>>>>>
>>>>>>>> out and handled independently (and probably rewritten
>>>>>>>> completely
>>>>>>>>
>>>>>>> since
>>>>>> they
>>>>>>>> currently have some issues).
>>>>>>>>
>>>>>>>> Load balancing is another that could warrant splitting out, but
>>>>>>>> that depends on what direction we go and how we would be managing it.
>>>>>>>>
>>>>>>> DHCP
>>>>>> and
>>>>>>>> DNS are others which could go either way.
>>>>>>>>
>>>>>>>> If we do split out services, I think we should consolidate as
>>>>>>>> much as
>>>>>>>>
>>>>>>> we
>>>>>>> can into each service we break out. Ideally a network packet
>>>>>>>> would
>>>>>>>>
>>>>>>> never
>>>>>>> hit more than one, maybe two, services. I don't think we should
>>>>>>>> be splitting services 'just because', I think we need a valid
>>>>>>>> case for splitting any service out because it adds complexity.
>>>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>>>> complexity unless it
>>>>>>>>
>>>>>>> is
>>>>>> really needed.
>>>>>>>> Some more of my thoughts on this anyway...
>>>>>>>>
>>>>>>>> *Will STEVENS*
>>>>>>>> Lead Developer
>>>>>>>>
>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>> *|* tw @CloudOps_
>>>>>>>>
>>>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller
>>>>>>>> <sw...@ena.com>
>>>>>>>>
>>>>>>> wrote:
>>>>>>> I do agree with you that this probably isn't the right place
>>>>>>>>> the
>>>>>>>>>
>>>>>>>> password
>>>>>>>> service and user data.
>>>>>>>>>
>>>>>>>>> Having said that, after taking a cursory look at the dev docs,
>>>>>>>>> it
>>>>>>>>>
>>>>>>>> doesn't
>>>>>>>> seem that difficult to add new daemons:
>>>>>>>> https://opensnaproute.github.
>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>>>> creating-new-component>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> They've definitely build it with a microservices architecture
>>>>>>>>> in
>>>>>>>>>
>>>>>>>> mind,
>>>>>> so
>>>>>>>> each individual feature is abstracted into it's own small
>>>>>>>>> daemon
>>>>>>>>>
>>>>>>>> process.
>>>>>>>> We could just create a daemon for the password server and the
>>>>>>>> userdata
>>>>>> components if we really had to.
>>>>>>>>>
>>>>>>>>> - Si
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ________________________________
>>>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
>>>>>>>>> behalf
>>>>>>>>>
>>>>>>>> of
>>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>
>>>>>>>>> A big part of why I know about it is because it is written in Go.
>>>>>>>>>
>>>>>>>> :P
>>>>>> Yes, it is definitely interesting for the routing and traffic
>>>>>>>> handling
>>>>>> aspects of the VR. We will likely have to rethink some of the
>>>>>>>> pieces
>>>>>> a
>>>>>>
>>>>>>> little bit like the password server and userdata if we are to
>>>>>>>>> adopt
>>>>>>>>>
>>>>>>>> a
>>>>>> different VR approach. This is where I think some of JohnB and
>>>>>>>> Chiradeep's
>>>>>>>> ideas make sense. In many ways, it does not make sense for the
>>>>>>>> device
>>>>>> handling routing and network traffic to also be responsible for
>>>>>>>> passwords
>>>>>>>> and userdata.
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>
>>>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller
>>>>>>>>> <sw...@ena.com>
>>>>>>>>>
>>>>>>>> wrote:
>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
>>>>>>>>> pretty
>>>>>> cool!
>>>>>>>>>> It even supports ONIE install.
>>>>>>>>>>
>>>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>>>>>>>>>
>>>>>>>>> offload
>>>>>> it to
>>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>>>> from a
>>>>>>>>>>
>>>>>>>>> routing
>>>>>>>>>
>>>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Based on the roadmap, it looks like they plan to also support
>>>>>>>>>>
>>>>>>>>> capabilities
>>>>>>>>>
>>>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>>>> will
>>>>>>>>>>
>>>>>>>>> be
>>>>>> huge
>>>>>>>> for our carrier community that rely on these technologies to
>>>>>>>>>> do
>>>>>>>>>>
>>>>>>>>> private
>>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>>>> stuff
>>>>>>>>>>
>>>>>>>>> on
>>>>>>> our
>>>>>>>
>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
>>>>>>>>>> able to
>>>>>>>>>>
>>>>>>>>> do
>>>>>>> MPLS
>>>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>>>> know
>>>>>>>>>>
>>>>>>>>> very
>>>>>>> well).
>>>>>>>>>>
>>>>>>>>>> - Si
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ________________________________
>>>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>>>
>>>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>>>> does
>>>>>>>>>>
>>>>>>>>> cover is
>>>>>>>> all api driven...
>>>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>> Though I don’t see VPN in Snaproute.. Makes sense since it
>>>>>>>>>>> was
>>>>>>>>>>>
>>>>>>>>>> not
>>>>>> intended to do IPSec.
>>>>>>>>>>> It seems as though VyOS is starting to look like the best
>>>>>>>>>>>
>>>>>>>>>> option.
>>>>>> Regards,
>>>>>>>>>>> Marty Godsey
>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>>>> [mailto:williamstevens@gmail.com
>>>>>>>>>>>
>>>>>>>>>> ]
>>>>>> On
>>>>>>
>>>>>>> Behalf Of Will Stevens
>>>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>
>>>>>>>>>>> Or we could go completely crazy and go with something like
>>>>>>>>>>>
>>>>>>>>>> FlexSwitch
>>>>>>>> from
>>>>>>>>>>> SnapRoute
>>>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>>>
>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>> Lead Developer
>>>>>>>>>>>
>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>> cloudops.com
>>>>>>>>>>>
>>>>>>>>>> *|*
>>>>>>> tw
>>>>>>>
>>>>>>>> @CloudOps_
>>>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>>>>>>>>
>>>>>>>>>> wstevens@cloudops.com>
>>>>>>>> wrote:
>>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
>>>>>>>>>>> problems
>>>>>>> are
>>>>>>>
>>>>>>>> solved by splitting up the function of the VR into a
>>>>>>>>>>>> bunch of
>>>>>>>>>>>>
>>>>>>>>>>> separate
>>>>>>>>>> services. As Syed points out, the complexity added is
>>>>>>>>>>> non-trivial.
>>>>>>>> We now have to manage all the intercontainer networking
>>>>>>>>>>>> as
>>>>>>>>>>>>
>>>>>>>>>>> well
>>>>>> as
>>>>>>
>>>>>>> the
>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>>> VyOS is interesting to me because it covers the majority of
>>>>>>>>>>>>
>>>>>>>>>>> our
>>>>>> use
>>>>>>>> case with a single unified control plane. It also has
>>>>>>>>>>>> good
>>>>>>>>>>>>
>>>>>>>>>>> support
>>>>>>>> for extending features we care about, like IPv6, VXLAN,
>>>>>>>>>>>> VRRP, transactions, etc...
>>>>>>>>>>>>
>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>
>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>>>
>>>>>>>>>>> cloudops.com
>>>>>> *|*
>>>>>>>> tw
>>>>>>>>>> @CloudOps_
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>>>>>>>>>>
>>>>>>>>>>> sahmed@cloudops.com>
>>>>>>> wrote:
>>>>>>>>>>> Agree with Marty, adding Docker containers to the
>>>>>>>>>>>>> picture
>>>>>>>>>>>>>
>>>>>>>>>>>> although
>>>>>>>> can make the VR more flexible but the added complexity
>>>>>>>>>>>>> is
>>>>>>>>>>>>>
>>>>>>>>>>>> just
>>>>>> not
>>>>>>>> worth it. Not to mention we would need to take care of
>>>>>>>>>>>> networking
>>>>>>> each container manually and given that our iptable rules
>>>>>>>>>>>>> are
>>>>>>>>>>>>>
>>>>>>>>>>>> very
>>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
>>>>>>>>>>>>> does
>>>>>>>>>>>>>
>>>>>>>>>>>> not
>>>>>>> provide an api but it does fit the bill quite well
>>>>>>>>>>>> otherwise. I
>>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>>>> model
>>>>>>>>>>>>>
>>>>>>>>>>>> and
>>>>>>> you
>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>>>>>>>>>>
>>>>>>>>>>>> marty@gonsource.com>
>>>>>>>> wrote:
>>>>>>>>>>>> Licensing aside, I think splitting the various
>>>>>>>>>>>>>> functions
>>>>>>>>>>>>>>
>>>>>>>>>>>>> into
>>>>>> containers is not a good route either. This will force
>>>>>>>>>>>>> users
>>>>>> to
>>>>>>
>>>>>>> have to maintain
>>>>>>>>>>>>> and
>>>>>>>>>>>>>
>>>>>>>>>>>>>> use containers and adds complexity to the networking
>>>>>>>>>>>>>>
>>>>>>>>>>>>> aspects
>>>>>> of
>>>>>>
>>>>>>> ACS.
>>>>>>>>>> Complexity decreases stability. Now I understand the
>>>>>>>>>>>>> argument
>>>>>> that
>>>>>>>> a monolithic approach also brings its own set of
>>>>>>>>>>>>>> issues but
>>>>>>>>>>>>>>
>>>>>>>>>>>>> it
>>>>>>> also
>>>>>>>>>> simplifies it.
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs of
>>>>>>>>>>>>>>
>>>>>>>>>>>>> the
>>>>>> CloudStack project
>>>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not touch
>>>>>>>>>>>>>>
>>>>>>>>>>>>> anything
>>>>>>>> that
>>>>>>>>>>>>> has
>>>>>>>>>>>>>
>>>>>>>>>>>>>> AGPL
>>>>>>>>>>>>>> - the github repo shows rather infrequent updates.
>>>>>>>>>>>>>> Quite
>>>>>>>>>>>>>>
>>>>>>>>>>>>> likely
>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>
>>>>>>>>>>>>> community
>>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>>>> Split
>>>>>>>>>>>>>>
>>>>>>>>>>>>> it
>>>>>> into
>>>>>>>> many docker containers
>>>>>>>>>>>>>> - password server
>>>>>>>>>>>>>> - userdata server
>>>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>>>> - FW
>>>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>>>> - agent
>>>>>>>>>>>>>> Glue them together with docker compose files (one per
>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>
>>>>>>>>>>>>> case -
>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can test
>>>>>>>>>>>>>>
>>>>>>>>>>>>> each
>>>>>> of
>>>>>>
>>>>>>> the
>>>>>>>>>> components independently and fixing one bug in the
>>>>>>>>>>>>>> field
>>>>>>>>>>>>>>
>>>>>>>>>>>>> (say
>>>>>> DHCP)
>>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>>>> build per-hypervisor VRs. You could even run on baremetal.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>>>> - make the traffic traverse the containers that are
>>>>>>>>>>>>>> needed
>>>>>>>>>>>>>>
>>>>>>>>>>>>> to
>>>>>>> be
>>>>>>>
>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>>>> - bootstrap the router (how does it find its compose file?
>>>>>>>>>>>>>>
>>>>>>>>>>>>> where
>>>>>>>> is the
>>>>>>>>>>>>>> registry?)
>>>>>>>>>>>>>> - rethink the command and control of the VR functions.
>>>>>>>>>>>>>> SSH
>>>>>>>>>>>>>>
>>>>>>>>>>>>> works,
>>>>>>>> but something more declarative, idempotent should be
>>>>>>>>>>>>> explored.
>>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>>>> functions
>>>>>>>>>>>>>>
>>>>>>>>>>>>> can
>>>>>> be
>>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>>>> Control
>>>>>>>>>>>>>>
>>>>>>>>>>>>> of
>>>>>>> the
>>>>>>>
>>>>>>>> docker
>>>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This one does look nice. My biggest concern is the
>>>>>>>>>>>>>>> lack
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> of
>>>>>> VXLANs. It seems that any of the ones we mentioned
>>>>>>>>>>>>>>> do not
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> have
>>>>>>>> an
>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>>>> save
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> future
>>>>>>>> engineering effort for example on ipv6 routing, OSPF etc.
>>>>>>>>>>>>>>> And the best part is they come with test automation
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> framework.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>> Instead of replacing the VR in first place we should
>>>>>>>>>>>>>>>> add VyOS/cloudrouter
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> as provider. Once it is stable, network offerings (on
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> upgrade)
>>>>>>>> can be updated to use it and we can drop the VR if
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> want
>>>>>> at
>>>>>>
>>>>>>> that release
>>>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>>>> them
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> are
>>>>>>> running
>>>>>>>>>>>>>>> without issues. When we replicate the ACS VR features
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> new
>>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>>>> (hidden
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> bugs).
>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too keen
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> on
>>>>>> VyOS
>>>>>>>> (it
>>>>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> @shapeblue
>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and should
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> be
>>>>>>> treated as
>>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>> Please weigh in with the good and bad of this idea...
>>>>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> potentially
>>>>>>>> replacing the ACS VR with the VyOS [1] (Open
>>>>>>>>>>>>>>>>>> Source
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Vyatta
>>>>>>>> VM).
>>>>>>>>>>> There may be a license issue because I think it
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> licensed
>>>>>>>> under GPL, but for the sake of discussion, let's
>>>>>>>>>>>>>>>>> assume
>>>>>> we
>>>>>>>> can overcome any
>>>>>>>>>>>>>>>>> license issues.
>>>>>>>>>>>>>>>> I have spent some time recently with the VyOS
>>>>>>>>>>>>>>>>>> and I
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> have
>>>>>>> to
>>>>>>>
>>>>>>>> admit, I was pretty impressed. It is simple and
>>>>>>>>>>>>>>>>> intuitive
>>>>>>>> and it gives you a lot more options for auditing
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> configuration etc...
>>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> simpler
>>>>>> more
>>>>>>>> auditable configuration workflow.
>>>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
>>>>>>>>>>>>>>>>>> - Handles VPN configuration via the same
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> configuration
>>>>>> interface.
>>>>>>>>>>
Re: [DISCUSS] Replacing the VR
Posted by Matthew Smart <ms...@smartsoftwareinc.com>.
Thanks Will. That is the answer I expected tbh. But it never hurts to ask!
Matthew Smart
President
Smart Software Solutions Inc.
108 S Pierre St.
Pierre, SD 57501
Phone: (605) 280-0383
Skype: msmart13
Email: msmart@smartsoftwareinc.com
On 09/22/2016 01:24 PM, Will Stevens wrote:
> Unfortunately there is not much documentation around the network plugin
> functionality. When I wrote the Palo Alto integration I basically figured
> out how to do it by reviewing existing plugins and just figuring it out.
>
> So if you were to begin to implement a new hardware firewall for example, I
> would point you to the Palo Alto integration code [1] and the functional
> spec [2] and then make myself available to try to answer any questions you
> have (like how the NetworkGuru works, where the different pieces are
> registered, etc)...
>
> Unfortunately it is not trivial, mainly because we don't have any
> documentation to follow, but the plugin interface IS there. It just
> requires people who have worked it in the past to offer guidance.
>
> [1]
> https://github.com/apache/cloudstack/tree/master/plugins/network-elements/palo-alto
> [2]
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Palo+Alto+Firewall+Integration
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Thu, Sep 22, 2016 at 1:53 PM, Matthew Smart <ms...@smartsoftwareinc.com>
> wrote:
>
>> Hey Murali,
>>
>> I have been reading through the API and other documentation to try to get
>> a basic understanding of the network service offering abstraction
>> methodology in CS. I have not dove into the code yet but before I did I
>> thought I would try a different approach.
>>
>> Imagine I were to come to this list and say that I have a network offering
>> that I sell and that I wanted to write whatever I needed to in order to
>> integrate it as an offering in CloudStack. Is there some specific
>> documentation and guidelines you would direct me to read in order to gather
>> the knowledge necessary to create a cloudstack compatible interface for my
>> product?
>>
>> I don't know the history but I see several products that have navigated
>> this process (Nuage, Nicira, ...etc) and am wondering how a new provider
>> would work with you guys to navigate that process. If this is too vague, we
>> can pretend my new offering is a hardware firewall device.
>>
>> My goal here is to gain an understanding of how CS interacts with third
>> party offerings underneath the hood. I have some thoughts (I think inline
>> with Will Steven's brain dump and diagram) but want to make sure I am not
>> suffering some misapprehensions about the architecture and, short of
>> tracing code, was not successful at finding the information I needed to
>> satisfy myself that I know how it is designed.
>>
>> Thanks,
>>
>> Matthew Smart
>> President
>> Smart Software Solutions Inc.
>> 108 S Pierre St.
>> Pierre, SD 57501
>>
>> Phone: (605) 280-0383
>> Skype: msmart13
>> Email: msmart@smartsoftwareinc.com
>>
>> On 09/20/2016 04:54 AM, Murali Reddy wrote:
>>
>>> Configuration management of network appliances particularly for Cloud and
>>> NFV scenarios is still evolving area. Programmability is the not the
>>> strength for even the most popular network operating systems like IOS,
>>> JunoS etc. So its not surprising why CloudStack integrates in a archaic way
>>> with stock linux for the VR.
>>>
>>> VR was never integral/hardcoded option in CloudStack. Its always been a
>>> plugin. CloudStack network orchestration is well abstracted and designed
>>> with vision to compose a network with different set of providers for
>>> different services. Yes that vision is not fully realised yet, and we don\u2019t
>>> have true service function chains. That would be different discussion topic.
>>>
>>> I tend to agree with Simon, as alternate/interim option we can take hard
>>> look whats causing the problems with current VR integration. Personally, I
>>> think it would be easier we take a cue from configuration managers and
>>> network configuration solutions out there (for e.g promise theory based
>>> Cisco ACI) move to more declarative model of expressing desired state of
>>> network configuration. Infact current VR from 4.6, actually holds the
>>> desired state per service basis, seems to be in that direction.
>>>
>>> It does make sense to evaluate new appliances which can provide rich
>>> semantics (like programmable API, declarative configuration, versioning,
>>> commit/rollback etc), but will need significant engineering effort and time
>>> to stabilise. We may make same mistakes with integration of other appliance
>>> as well, if we fully don\u2019t understand the nature of the current problems
>>> with CloudStack core and service provider interaction and current VR
>>> integration.
>>>
>>>
>>> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>>>
>>> I think our other option is to take a real look at what it would take to
>>>> fix the VR. In my opinion, a lot of the problems are related to the
>>>> monolithic python code base and the fact nothing is actually separated.
>>>>
>>>> Secondly, the python scripts (and bash scripts) don't use any
>>>> established libraries to complete tasks and instead shell out and run
>>>> commands that are both hard to track and hard to parse on return.
>>>>
>>>>
>>>> If we daemonized this, used a real api for Agent to VR communication,
>>>> used common already existing libraries for the system service and network
>>>> interactions and spent a bit of time separating out code into distinct
>>>> modules, everything would behave a lot better.
>>>>
>>>>
>>>> The pain and suffering is due to years and years of patches and constant
>>>> shelling out to complete tasks in my opinion. If we spend time to rethink
>>>> how we interact with the VR in general and we abstract the systems and
>>>> networking stuff and use well known and stable libraries to do the work,
>>>> the VR would be much easier to maintain.
>>>>
>>>>
>>>> - Si
>>>>
>>>>
>>>>
>>>>
>>>> ________________________________
>>>> From: Marty Godsey <ma...@gonsource.com>
>>>> Sent: Friday, September 16, 2016 12:24 PM
>>>> To: dev@cloudstack.apache.org
>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>
>>>> So based upon this discussion would it be prudent to wait on VyOS 2.0?
>>>> The current VR is giving us issues but would the time invested in another
>>>> "solution" be wasted especially if by the time another option is chose,
>>>> then coded, then tested, then implemented and right as that time happened
>>>> to be when VyOS 2.0 is released. Of course you said they are just in the
>>>> scoping range so this could still be a year or more out.
>>>>
>>>> Thoughts?
>>>>
>>>> Regards,
>>>> Marty Godsey
>>>> nSource Solutions
>>>>
>>>> -----Original Message-----
>>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>>>> Behalf Of Will Stevens
>>>> Sent: Friday, September 16, 2016 10:31 AM
>>>> To: dev@cloudstack.apache.org
>>>> Cc: daniil@baturin.org
>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>
>>>> I just had a quick chat with a couple of the guys over on the VyOS chat.
>>>> I have CC'ed one of them in case we have more licensing questions.
>>>>
>>>> So here is the status with the license "the code inherited from Vyatta
>>>> and our modifications from it is GPLv2 (strict, not v2+). The config
>>>> reading library is GPLv2 too, so anything that links to is is GPLv2.
>>>> Some auxiliary components we made after the fork are more permissive,
>>>> LGPLv2+ or MIT."
>>>>
>>>> They are currently in the process of scoping a redesign (VyOS 2.0), "we
>>>> are planning a clean rewrite that will solve issues of the current config
>>>> system".
>>>> This will include the ability to configure via the API.
>>>>
>>>> If we have more questions for VyOS, they are very friendly and
>>>> responsive, so we should be able to get answers.
>>>>
>>>> *Will STEVENS*
>>>> Lead Developer
>>>>
>>>> *CloudOps* *| *Cloud Solutions Experts
>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>>>> @CloudOps_
>>>>
>>>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>>>>
>>>> I agree with Will Ilya. There are so many problems with the VR right now.
>>>>> Most of the outages we've had recently have somehow involved the VR.
>>>>> We set custom iptables rules on the VR which can and have easily gone
>>>>> wrong.
>>>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>>>> VVRP with redundant router still needs work, and not to mention the
>>>>> problems we will have when we introduce IPv6 into the whole picture.
>>>>>
>>>>> I think the spirit of the discussion is to rely on a 3rd party to do
>>>>> the networking for us (eg VyOS) and have us handle just the
>>>>> orchestration. All the problems that I've described have already been
>>>>> solved in VyOS. We also get the advantage of a potential wider
>>>>> community to fix and maintain the VR and given our current development
>>>>> velocity, it think it totally makes sense to look for a 3rd party
>>>>> option.
>>>>>
>>>>> -Syed
>>>>>
>>>>>
>>>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
>>>>> wrote:
>>>>>
>>>>> The VR has been biting us far too often recently, which is why we
>>>>>> have started looking into alternative implementations.
>>>>>>
>>>>>> One of the things that is nice about potentially using the VyOS is
>>>>>> that
>>>>>>
>>>>> it
>>>>>
>>>>>> is based on Debian, so we should be able to run the other services
>>>>>> that
>>>>>>
>>>>> we
>>>>>
>>>>>> currently have like the password server and userdata on the VyOS.
>>>>>> This means we would not have to change our architecture initially
>>>>>> and could focus on only replacing the networking paths.
>>>>>>
>>>>>> *Will STEVENS*
>>>>>> Lead Developer
>>>>>>
>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>>>> tw @CloudOps_
>>>>>>
>>>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>>>
>>>>>> The more this is discussed the more I think we should stick with
>>>>>>> our
>>>>>>>
>>>>>> VR.
>>>>>> All these other options either seem unfinished or with
>>>>>>> incompatible license.
>>>>>>>
>>>>>>> VyOS looks the most promising so far, it's a serious, mature project.
>>>>>>> Adopting it though means we'll have to microservice our way out of
>>>>>>> it
>>>>>>>
>>>>>> with
>>>>>>
>>>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>>>>>>>
>>>>>> those
>>>>>> too. Imho this adds complexity we should void.
>>>>>>> --
>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>
>>>>>>> Nux!
>>>>>>> www.nux.ro
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>
>>>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>> Ya, we would need to add a daemon for VPN as well. Load
>>>>>>>> balancing is another aspect which we will need to consider if we
>>>>>>>> went this route.
>>>>>>>> Something like https://traefik.io/ could potentially be a good
>>>>>>>> fit
>>>>>>>>
>>>>>>> due
>>>>>> to
>>>>>>>> its API driven configuration, but it may be more than what we need.
>>>>>>>>
>>>>>>>> We should probably try define which pieces make sense to be
>>>>>>>> solved
>>>>>>>>
>>>>>>> together
>>>>>>>
>>>>>>>> and which pieces would be best suited to be broken out.
>>>>>>>>
>>>>>>>> I think the network connectivity, routing and firewalling should
>>>>>>>>
>>>>>>> probably
>>>>>>> all stay together since the majority of the tools we would
>>>>>>> potentially
>>>>>> use
>>>>>>>> would handle all of that together in a single implementation.
>>>>>>>>
>>>>>>>> The password server and userdata seems like a good option for
>>>>>>>> being
>>>>>>>>
>>>>>>> broken
>>>>>>>
>>>>>>>> out and handled independently (and probably rewritten completely
>>>>>>>>
>>>>>>> since
>>>>>> they
>>>>>>>> currently have some issues).
>>>>>>>>
>>>>>>>> Load balancing is another that could warrant splitting out, but
>>>>>>>> that depends on what direction we go and how we would be managing it.
>>>>>>>>
>>>>>>> DHCP
>>>>>> and
>>>>>>>> DNS are others which could go either way.
>>>>>>>>
>>>>>>>> If we do split out services, I think we should consolidate as
>>>>>>>> much as
>>>>>>>>
>>>>>>> we
>>>>>>> can into each service we break out. Ideally a network packet
>>>>>>>> would
>>>>>>>>
>>>>>>> never
>>>>>>> hit more than one, maybe two, services. I don't think we should
>>>>>>>> be splitting services 'just because', I think we need a valid
>>>>>>>> case for splitting any service out because it adds complexity.
>>>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>>>> complexity unless it
>>>>>>>>
>>>>>>> is
>>>>>> really needed.
>>>>>>>> Some more of my thoughts on this anyway...
>>>>>>>>
>>>>>>>> *Will STEVENS*
>>>>>>>> Lead Developer
>>>>>>>>
>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>> *|* tw @CloudOps_
>>>>>>>>
>>>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>>>>>>>>
>>>>>>> wrote:
>>>>>>> I do agree with you that this probably isn't the right place
>>>>>>>>> the
>>>>>>>>>
>>>>>>>> password
>>>>>>>> service and user data.
>>>>>>>>>
>>>>>>>>> Having said that, after taking a cursory look at the dev docs,
>>>>>>>>> it
>>>>>>>>>
>>>>>>>> doesn't
>>>>>>>> seem that difficult to add new daemons:
>>>>>>>> https://opensnaproute.github.
>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>>>> creating-new-component>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> They've definitely build it with a microservices architecture
>>>>>>>>> in
>>>>>>>>>
>>>>>>>> mind,
>>>>>> so
>>>>>>>> each individual feature is abstracted into it's own small
>>>>>>>>> daemon
>>>>>>>>>
>>>>>>>> process.
>>>>>>>> We could just create a daemon for the password server and the
>>>>>>>> userdata
>>>>>> components if we really had to.
>>>>>>>>>
>>>>>>>>> - Si
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ________________________________
>>>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
>>>>>>>>> behalf
>>>>>>>>>
>>>>>>>> of
>>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>
>>>>>>>>> A big part of why I know about it is because it is written in Go.
>>>>>>>>>
>>>>>>>> :P
>>>>>> Yes, it is definitely interesting for the routing and traffic
>>>>>>>> handling
>>>>>> aspects of the VR. We will likely have to rethink some of the
>>>>>>>> pieces
>>>>>> a
>>>>>>
>>>>>>> little bit like the password server and userdata if we are to
>>>>>>>>> adopt
>>>>>>>>>
>>>>>>>> a
>>>>>> different VR approach. This is where I think some of JohnB and
>>>>>>>> Chiradeep's
>>>>>>>> ideas make sense. In many ways, it does not make sense for the
>>>>>>>> device
>>>>>> handling routing and network traffic to also be responsible for
>>>>>>>> passwords
>>>>>>>> and userdata.
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>
>>>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>>>>>>>>>
>>>>>>>> wrote:
>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
>>>>>>>>> pretty
>>>>>> cool!
>>>>>>>>>> It even supports ONIE install.
>>>>>>>>>>
>>>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>>>>>>>>>
>>>>>>>>> offload
>>>>>> it to
>>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>>>> from a
>>>>>>>>>>
>>>>>>>>> routing
>>>>>>>>>
>>>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Based on the roadmap, it looks like they plan to also support
>>>>>>>>>>
>>>>>>>>> capabilities
>>>>>>>>>
>>>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>>>> will
>>>>>>>>>>
>>>>>>>>> be
>>>>>> huge
>>>>>>>> for our carrier community that rely on these technologies to
>>>>>>>>>> do
>>>>>>>>>>
>>>>>>>>> private
>>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>>>> stuff
>>>>>>>>>>
>>>>>>>>> on
>>>>>>> our
>>>>>>>
>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
>>>>>>>>>> able to
>>>>>>>>>>
>>>>>>>>> do
>>>>>>> MPLS
>>>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>>>> know
>>>>>>>>>>
>>>>>>>>> very
>>>>>>> well).
>>>>>>>>>>
>>>>>>>>>> - Si
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ________________________________
>>>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>>>
>>>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>>>> does
>>>>>>>>>>
>>>>>>>>> cover is
>>>>>>>> all api driven...
>>>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>> Though I don\u2019t see VPN in Snaproute.. Makes sense since it
>>>>>>>>>>> was
>>>>>>>>>>>
>>>>>>>>>> not
>>>>>> intended to do IPSec.
>>>>>>>>>>> It seems as though VyOS is starting to look like the best
>>>>>>>>>>>
>>>>>>>>>> option.
>>>>>> Regards,
>>>>>>>>>>> Marty Godsey
>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>>>> [mailto:williamstevens@gmail.com
>>>>>>>>>>>
>>>>>>>>>> ]
>>>>>> On
>>>>>>
>>>>>>> Behalf Of Will Stevens
>>>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>
>>>>>>>>>>> Or we could go completely crazy and go with something like
>>>>>>>>>>>
>>>>>>>>>> FlexSwitch
>>>>>>>> from
>>>>>>>>>>> SnapRoute
>>>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>>>
>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>> Lead Developer
>>>>>>>>>>>
>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>> cloudops.com
>>>>>>>>>>>
>>>>>>>>>> *|*
>>>>>>> tw
>>>>>>>
>>>>>>>> @CloudOps_
>>>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>>>>>>>>
>>>>>>>>>> wstevens@cloudops.com>
>>>>>>>> wrote:
>>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
>>>>>>>>>>> problems
>>>>>>> are
>>>>>>>
>>>>>>>> solved by splitting up the function of the VR into a
>>>>>>>>>>>> bunch of
>>>>>>>>>>>>
>>>>>>>>>>> separate
>>>>>>>>>> services. As Syed points out, the complexity added is
>>>>>>>>>>> non-trivial.
>>>>>>>> We now have to manage all the intercontainer networking
>>>>>>>>>>>> as
>>>>>>>>>>>>
>>>>>>>>>>> well
>>>>>> as
>>>>>>
>>>>>>> the
>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>>> VyOS is interesting to me because it covers the majority
>>>>>>>>>>>> of
>>>>>>>>>>>>
>>>>>>>>>>> our
>>>>>> use
>>>>>>>> case with a single unified control plane. It also has
>>>>>>>>>>>> good
>>>>>>>>>>>>
>>>>>>>>>>> support
>>>>>>>> for extending features we care about, like IPv6, VXLAN,
>>>>>>>>>>>> VRRP, transactions, etc...
>>>>>>>>>>>>
>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>
>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>>>
>>>>>>>>>>> cloudops.com
>>>>>> *|*
>>>>>>>> tw
>>>>>>>>>> @CloudOps_
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>>>>>>>>>>
>>>>>>>>>>> sahmed@cloudops.com>
>>>>>>> wrote:
>>>>>>>>>>> Agree with Marty, adding Docker containers to the
>>>>>>>>>>>>> picture
>>>>>>>>>>>>>
>>>>>>>>>>>> although
>>>>>>>> can make the VR more flexible but the added complexity
>>>>>>>>>>>>> is
>>>>>>>>>>>>>
>>>>>>>>>>>> just
>>>>>> not
>>>>>>>> worth it. Not to mention we would need to take care of
>>>>>>>>>>>> networking
>>>>>>> each container manually and given that our iptable rules
>>>>>>>>>>>>> are
>>>>>>>>>>>>>
>>>>>>>>>>>> very
>>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
>>>>>>>>>>>>> does
>>>>>>>>>>>>>
>>>>>>>>>>>> not
>>>>>>> provide an api but it does fit the bill quite well
>>>>>>>>>>>> otherwise. I
>>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>>>> model
>>>>>>>>>>>>>
>>>>>>>>>>>> and
>>>>>>> you
>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>>>>>>>>>>
>>>>>>>>>>>> marty@gonsource.com>
>>>>>>>> wrote:
>>>>>>>>>>>> Licensing aside, I think splitting the various
>>>>>>>>>>>>>> functions
>>>>>>>>>>>>>>
>>>>>>>>>>>>> into
>>>>>> containers is not a good route either. This will force
>>>>>>>>>>>>> users
>>>>>> to
>>>>>>
>>>>>>> have to maintain
>>>>>>>>>>>>> and
>>>>>>>>>>>>>
>>>>>>>>>>>>>> use containers and adds complexity to the networking
>>>>>>>>>>>>>>
>>>>>>>>>>>>> aspects
>>>>>> of
>>>>>>
>>>>>>> ACS.
>>>>>>>>>> Complexity decreases stability. Now I understand the
>>>>>>>>>>>>> argument
>>>>>> that
>>>>>>>> a monolithic approach also brings its own set of
>>>>>>>>>>>>>> issues but
>>>>>>>>>>>>>>
>>>>>>>>>>>>> it
>>>>>>> also
>>>>>>>>>> simplifies it.
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>
>>>>>>>>>>>>> the
>>>>>> CloudStack project
>>>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
>>>>>>>>>>>>>> touch
>>>>>>>>>>>>>>
>>>>>>>>>>>>> anything
>>>>>>>> that
>>>>>>>>>>>>> has
>>>>>>>>>>>>>
>>>>>>>>>>>>>> AGPL
>>>>>>>>>>>>>> - the github repo shows rather infrequent updates.
>>>>>>>>>>>>>> Quite
>>>>>>>>>>>>>>
>>>>>>>>>>>>> likely
>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>
>>>>>>>>>>>>> community
>>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>>>> Split
>>>>>>>>>>>>>>
>>>>>>>>>>>>> it
>>>>>> into
>>>>>>>> many docker containers
>>>>>>>>>>>>>> - password server
>>>>>>>>>>>>>> - userdata server
>>>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>>>> - FW
>>>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>>>> - agent
>>>>>>>>>>>>>> Glue them together with docker compose files (one per
>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>
>>>>>>>>>>>>> case -
>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
>>>>>>>>>>>>>> test
>>>>>>>>>>>>>>
>>>>>>>>>>>>> each
>>>>>> of
>>>>>>
>>>>>>> the
>>>>>>>>>> components independently and fixing one bug in the
>>>>>>>>>>>>>> field
>>>>>>>>>>>>>>
>>>>>>>>>>>>> (say
>>>>>> DHCP)
>>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>>>> build per-hypervisor VRs. You could even run on baremetal.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>>>> - make the traffic traverse the containers that are
>>>>>>>>>>>>>> needed
>>>>>>>>>>>>>>
>>>>>>>>>>>>> to
>>>>>>> be
>>>>>>>
>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>>>> - bootstrap the router (how does it find its compose file?
>>>>>>>>>>>>>>
>>>>>>>>>>>>> where
>>>>>>>> is the
>>>>>>>>>>>>>> registry?)
>>>>>>>>>>>>>> - rethink the command and control of the VR
>>>>>>>>>>>>>> functions. SSH
>>>>>>>>>>>>>>
>>>>>>>>>>>>> works,
>>>>>>>> but something more declarative, idempotent should be
>>>>>>>>>>>>> explored.
>>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>>>> functions
>>>>>>>>>>>>>>
>>>>>>>>>>>>> can
>>>>>> be
>>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>>>> Control
>>>>>>>>>>>>>>
>>>>>>>>>>>>> of
>>>>>>> the
>>>>>>>
>>>>>>>> docker
>>>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This one does look nice. My biggest concern is the
>>>>>>>>>>>>>>> lack
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> of
>>>>>> VXLANs. It seems that any of the ones we mentioned
>>>>>>>>>>>>>>> do not
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> have
>>>>>>>> an
>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>>>> save
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> future
>>>>>>>> engineering effort for example on ipv6 routing, OSPF etc.
>>>>>>>>>>>>>>> And the best part is they come with test automation
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> framework.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>> Instead of replacing the VR in first place we
>>>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> as provider. Once it is stable, network offerings
>>>>>>>>>>>>>>> (on
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> upgrade)
>>>>>>>> can be updated to use it and we can drop the VR if
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> want
>>>>>> at
>>>>>>
>>>>>>> that release
>>>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>>>> them
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> are
>>>>>>> running
>>>>>>>>>>>>>>> without issues. When we replicate the ACS VR
>>>>>>>>>>>>>>> features in
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> new
>>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>>>> (hidden
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> bugs).
>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
>>>>>>>>>>>>>>>>> keen
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> on
>>>>>> VyOS
>>>>>>>> (it
>>>>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> @shapeblue
>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> be
>>>>>>> treated as
>>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>> Please weigh in with the good and bad of this idea...
>>>>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> potentially
>>>>>>>> replacing the ACS VR with the VyOS [1] (Open
>>>>>>>>>>>>>>>>>> Source
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Vyatta
>>>>>>>> VM).
>>>>>>>>>>> There may be a license issue because I think it
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> licensed
>>>>>>>> under GPL, but for the sake of discussion, let's
>>>>>>>>>>>>>>>>> assume
>>>>>> we
>>>>>>>> can overcome any
>>>>>>>>>>>>>>>>> license issues.
>>>>>>>>>>>>>>>> I have spent some time recently with the VyOS
>>>>>>>>>>>>>>>>>> and I
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> have
>>>>>>> to
>>>>>>>
>>>>>>>> admit, I was pretty impressed. It is simple and
>>>>>>>>>>>>>>>>> intuitive
>>>>>>>> and it gives you a lot more options for auditing
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> configuration etc...
>>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> simpler
>>>>>> more
>>>>>>>> auditable configuration workflow.
>>>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
>>>>>>>>>>>>>>>>>> - Handles VPN configuration via the same
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> configuration
>>>>>> interface.
>>>>>>>>>>
Re: [DISCUSS] Replacing the VR
Posted by Matthew Smart <ms...@smartsoftwareinc.com>.
Sounds like a plan. It fits pretty well with my current priority items.
I am helping Vyos beta test their Debian 8 version. I am sure you will
be hearing from me as I dig into it and need advice!
Matthew Smart
President
Smart Software Solutions Inc.
108 S Pierre St.
Pierre, SD 57501
Phone: (605) 280-0383
Skype: msmart13
Email: msmart@smartsoftwareinc.com
On 09/28/2016 07:00 AM, Will Stevens wrote:
> \u200bHey Matthew,
> Yes, I completely agree with your approach. Building a plugin is
> definitely the best first step with VyOS. Working on that will also get
> your head into the way that CloudStack handles network orchestration, which
> will be very valuable going forward as well.
>
> Yes, what you have highlighted about the VyOS having to be an external
> device is accurate for the current plugin system. I think it is probably
> possible to have the plugin system actually create a virtual machine with a
> VyOS template, but I have not tried that and I have not seen an example of
> that.
>
> For now, you can review the Palo Alto Networks integration. It is
> relatively clean for what you would need. I wrote that plugin, so just ask
> me if you have any questions. Currently that plugin only supports Isolated
> Guest Networks and not VPC Networks. I am not sure if there is an example
> anywhere that handles VPC Networks with the plugin system because most of
> the external firewall devices don't handle overlapping IP space (which is
> used in VPCs). You can get around that by making the VyOS 'Dedicated' so
> it can't be shared between more than one VPC, but that will make the
> management of the external VyOS boxes a much bigger pain.
>
> If you have questions about any of this, just let us know and I will help
> you out.
>
> In summary, I think you have the right approach by focusing on a plugin and
> we can expand on it later.
>
> Cheers,
>
> Will
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Tue, Sep 27, 2016 at 7:13 PM, Matthew Smart <ms...@smartsoftwareinc.com>
> wrote:
>
>> Will,
>>
>> I think that would be very helpful to me at least and for posterity for
>> sure. I am in the process of rolling out my first production deployment of
>> Cloudstack so I have been busier than expected (plus I have been jumping
>> back and forth between different offices). What I intended to propose was
>> that we tackle this using an intermediary step. Instead of jumping in and
>> replacing the VR wholesale from the start of this initiative what if we
>> look at integrating Vyos as a plugin in the same manner as other network
>> offering plugins. Replacing such a core and vital component of the
>> Cloudstack infrastructure as the VR should be met with a great deal of
>> caution and with careful thought and planning. Such a replacement would
>> require a nontrivial amount of effort by core contributors to be successful
>> and it is questionable whether such an initiative is the area where they
>> feel their time is best invested.
>>
>> In my opinion, using the current plugin capabilities of Cloudstack as a
>> first step would have the benefit of giving the devs a proof of concept
>> that can be used to evaluate Vyos as a potential replacement for the
>> current VR without touching the current core networking functionality at
>> all. Additionally, the code needed to integrate Vyos as a plugin would, I
>> assume, be directly reusable if the decision is made to go forward with a
>> Vyos based VR refactor. So we are not duplicating or wasting effort either.
>> Plus, it would serve as an excellent process during which to document the
>> network offering plugin architecture.
>>
>> On the downside to this approach, my understanding would be that the Vyos
>> deploy in such an environment would have to be external to the cloudstack
>> ecosystem it supports (either a bare metal install or a non cloudstack
>> managed VM). Personally, for POC purposes I do not see this as a huge
>> stumbling block since I expect that anyone who works on cloudstack can
>> deploy a VM using Virsh or some similar hypervisor interface.
>>
>> What do you think of that approach? What percentage of the functions
>> current embodied by the VR are abstracted in such a way as to be
>> offloadable to a plugin? Would such a plugin be feature complete enough to
>> represent a proof of concept for a potential Vyos based VR?
>>
>> Thanks,
>>
>>
>> Matthew Smart
>> President
>> Smart Software Solutions Inc.
>> 108 S Pierre St.
>> Pierre, SD 57501
>>
>> Phone: (605) 280-0383
>> Skype: msmart13
>> Email: msmart@smartsoftwareinc.com
>>
>> On 09/26/2016 03:01 PM, Will Stevens wrote:
>>
>>> I feel like I have squashed this discussion with my potential approach to
>>> handling this. Maybe we should just pickup this discussion assuming I
>>> didn't post that. :P
>>>
>>> Regarding the docs. I have considered building a stubbed example network
>>> plugin and then documenting how you would take that stub and build on it.
>>> Would that be interesting?
>>>
>>> *Will STEVENS*
>>> Lead Developer
>>>
>>> *CloudOps* *| *Cloud Solutions Experts
>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>> w cloudops.com *|* tw @CloudOps_
>>>
>>> On Fri, Sep 23, 2016 at 12:15 AM, Murali Reddy <mu...@gmail.com>
>>> wrote:
>>>
>>> Matthew,
>>>> Please see https://cwiki.apache.org/confluence/display/CLOUDSTACK/
>>>> Extending+CloudStack+Networking
>>>>
>>>> Thanks,
>>>> Murali
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 22/09/16, 11:23 PM, "Matthew Smart" <ms...@smartsoftwareinc.com>
>>>> wrote:
>>>>
>>>> Hey Murali,
>>>>> I have been reading through the API and other documentation to try to
>>>>> get a basic understanding of the network service offering abstraction
>>>>> methodology in CS. I have not dove into the code yet but before I did I
>>>>> thought I would try a different approach.
>>>>>
>>>>> Imagine I were to come to this list and say that I have a network
>>>>> offering that I sell and that I wanted to write whatever I needed to in
>>>>> order to integrate it as an offering in CloudStack. Is there some
>>>>> specific documentation and guidelines you would direct me to read in
>>>>> order to gather the knowledge necessary to create a cloudstack
>>>>> compatible interface for my product?
>>>>>
>>>>> I don't know the history but I see several products that have navigated
>>>>> this process (Nuage, Nicira, ...etc) and am wondering how a new provider
>>>>> would work with you guys to navigate that process. If this is too vague,
>>>>> we can pretend my new offering is a hardware firewall device.
>>>>>
>>>>> My goal here is to gain an understanding of how CS interacts with third
>>>>> party offerings underneath the hood. I have some thoughts (I think
>>>>> inline with Will Steven's brain dump and diagram) but want to make sure
>>>>> I am not suffering some misapprehensions about the architecture and,
>>>>> short of tracing code, was not successful at finding the information I
>>>>> needed to satisfy myself that I know how it is designed.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Matthew Smart
>>>>> President
>>>>> Smart Software Solutions Inc.
>>>>> 108 S Pierre St.
>>>>> Pierre, SD 57501
>>>>>
>>>>> Phone: (605) 280-0383
>>>>> Skype: msmart13
>>>>> Email: msmart@smartsoftwareinc.com
>>>>>
>>>>> On 09/20/2016 04:54 AM, Murali Reddy wrote:
>>>>>
>>>>>> Configuration management of network appliances particularly for Cloud
>>>>>>
>>>>> and NFV scenarios is still evolving area. Programmability is the not the
>>>> strength for even the most popular network operating systems like IOS,
>>>> JunoS etc. So its not surprising why CloudStack integrates in a archaic
>>>> way
>>>> with stock linux for the VR.
>>>>
>>>>> VR was never integral/hardcoded option in CloudStack. Its always been a
>>>>> plugin. CloudStack network orchestration is well abstracted and designed
>>>> with vision to compose a network with different set of providers for
>>>> different services. Yes that vision is not fully realised yet, and we
>>>> don\u2019t
>>>> have true service function chains. That would be different discussion
>>>> topic.
>>>>
>>>>> I tend to agree with Simon, as alternate/interim option we can take
>>>>> hard look whats causing the problems with current VR integration.
>>>> Personally, I think it would be easier we take a cue from configuration
>>>> managers and network configuration solutions out there (for e.g promise
>>>> theory based Cisco ACI) move to more declarative model of expressing
>>>> desired state of network configuration. Infact current VR from 4.6,
>>>> actually holds the desired state per service basis, seems to be in that
>>>> direction.
>>>>
>>>>> It does make sense to evaluate new appliances which can provide rich
>>>>> semantics (like programmable API, declarative configuration, versioning,
>>>> commit/rollback etc), but will need significant engineering effort and
>>>> time
>>>> to stabilise. We may make same mistakes with integration of other
>>>> appliance
>>>> as well, if we fully don\u2019t understand the nature of the current problems
>>>> with CloudStack core and service provider interaction and current VR
>>>> integration.
>>>>
>>>>>> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>>>>>>
>>>>>> I think our other option is to take a real look at what it would take
>>>>>> to fix the VR. In my opinion, a lot of the problems are related to the
>>>> monolithic python code base and the fact nothing is actually separated.
>>>>
>>>>> Secondly, the python scripts (and bash scripts) don't use any
>>>>>> established libraries to complete tasks and instead shell out and run
>>>> commands that are both hard to track and hard to parse on return.
>>>>
>>>>>>> If we daemonized this, used a real api for Agent to VR communication,
>>>>>>>
>>>>>> used common already existing libraries for the system service and
>>>> network
>>>> interactions and spent a bit of time separating out code into distinct
>>>> modules, everything would behave a lot better.
>>>>
>>>>>>> The pain and suffering is due to years and years of patches and
>>>>>>>
>>>>>> constant shelling out to complete tasks in my opinion. If we spend
>>>> time to
>>>> rethink how we interact with the VR in general and we abstract the
>>>> systems
>>>> and networking stuff and use well known and stable libraries to do the
>>>> work, the VR would be much easier to maintain.
>>>>
>>>>>>> - Si
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ________________________________
>>>>>>> From: Marty Godsey <ma...@gonsource.com>
>>>>>>> Sent: Friday, September 16, 2016 12:24 PM
>>>>>>> To: dev@cloudstack.apache.org
>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>
>>>>>>> So based upon this discussion would it be prudent to wait on VyOS 2.0?
>>>>>>>
>>>>>> The current VR is giving us issues but would the time invested in
>>>> another
>>>> "solution" be wasted especially if by the time another option is chose,
>>>> then coded, then tested, then implemented and right as that time happened
>>>> to be when VyOS 2.0 is released. Of course you said they are just in the
>>>> scoping range so this could still be a year or more out.
>>>>
>>>>> Thoughts?
>>>>>>> Regards,
>>>>>>> Marty Godsey
>>>>>>> nSource Solutions
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>>>>>>>
>>>>>> Behalf Of Will Stevens
>>>>> Sent: Friday, September 16, 2016 10:31 AM
>>>>>>> To: dev@cloudstack.apache.org
>>>>>>> Cc: daniil@baturin.org
>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>
>>>>>>> I just had a quick chat with a couple of the guys over on the VyOS
>>>>>>>
>>>>>> chat.
>>>>> I have CC'ed one of them in case we have more licensing questions.
>>>>>>> So here is the status with the license "the code inherited from Vyatta
>>>>>>>
>>>>>> and our modifications from it is GPLv2 (strict, not v2+). The config
>>>> reading library is GPLv2 too, so anything that links to is is GPLv2.
>>>>
>>>>> Some auxiliary components we made after the fork are more permissive,
>>>>>>> LGPLv2+ or MIT."
>>>>>>>
>>>>>>> They are currently in the process of scoping a redesign (VyOS 2.0),
>>>>>>>
>>>>>> "we are planning a clean rewrite that will solve issues of the current
>>>> config system".
>>>>
>>>>> This will include the ability to configure via the API.
>>>>>>> If we have more questions for VyOS, they are very friendly and
>>>>>>>
>>>>>> responsive, so we should be able to get answers.
>>>>> *Will STEVENS*
>>>>>>> Lead Developer
>>>>>>>
>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>>>>>>>
>>>>>> @CloudOps_
>>>>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com>
>>>>>> wrote:
>>>>> I agree with Will Ilya. There are so many problems with the VR right
>>>>>>> now.
>>>>> Most of the outages we've had recently have somehow involved the VR.
>>>>>>>> We set custom iptables rules on the VR which can and have easily gone
>>>>>>>>
>>>>>>> wrong.
>>>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>>>>>>> VVRP with redundant router still needs work, and not to mention the
>>>>>>>> problems we will have when we introduce IPv6 into the whole picture.
>>>>>>>>
>>>>>>>> I think the spirit of the discussion is to rely on a 3rd party to do
>>>>>>>> the networking for us (eg VyOS) and have us handle just the
>>>>>>>> orchestration. All the problems that I've described have already been
>>>>>>>> solved in VyOS. We also get the advantage of a potential wider
>>>>>>>> community to fix and maintain the VR and given our current
>>>>>>>> development
>>>>>>>> velocity, it think it totally makes sense to look for a 3rd party
>>>>>>>>
>>>>>>> option.
>>>>> -Syed
>>>>>>>>
>>>>>>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <wstevens@cloudops.com
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> The VR has been biting us far too often recently, which is why we
>>>>>>>>> have started looking into alternative implementations.
>>>>>>>>>
>>>>>>>>> One of the things that is nice about potentially using the VyOS is
>>>>>>>>> that
>>>>>>>>>
>>>>>>>> it
>>>>>>>>
>>>>>>>>> is based on Debian, so we should be able to run the other services
>>>>>>>>> that
>>>>>>>>>
>>>>>>>> we
>>>>>>>>
>>>>>>>>> currently have like the password server and userdata on the VyOS.
>>>>>>>>> This means we would not have to change our architecture initially
>>>>>>>>> and could focus on only replacing the networking paths.
>>>>>>>>>
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>>>>>>> tw @CloudOps_
>>>>>>>>>
>>>>>>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>>>>>>
>>>>>>>>> The more this is discussed the more I think we should stick with
>>>>>>>>>> our
>>>>>>>>>>
>>>>>>>>> VR.
>>>>>>>>> All these other options either seem unfinished or with
>>>>>>>>>> incompatible license.
>>>>>>>>>>
>>>>>>>>>> VyOS looks the most promising so far, it's a serious, mature
>>>>>>>>>>
>>>>>>>>> project.
>>>>> Adopting it though means we'll have to microservice our way out of
>>>>>>>>>> it
>>>>>>>>>>
>>>>>>>>> with
>>>>>>>>>
>>>>>>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>>>>>>>>>>
>>>>>>>>> those
>>>>>>>>> too. Imho this adds complexity we should void.
>>>>>>>>>> --
>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>
>>>>>>>>>> Nux!
>>>>>>>>>> www.nux.ro
>>>>>>>>>>
>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>
>>>>>>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>> Ya, we would need to add a daemon for VPN as well. Load
>>>>>>>>>>> balancing is another aspect which we will need to consider if we
>>>>>>>>>>>
>>>>>>>>>> went this route.
>>>>> Something like https://traefik.io/ could potentially be a good
>>>>>>>>>>> fit
>>>>>>>>>>>
>>>>>>>>>> due
>>>>>>>>> to
>>>>>>>>>>> its API driven configuration, but it may be more than what we
>>>>>>>>>>> need.
>>>>>>>>>>>
>>>>>>>>>>> We should probably try define which pieces make sense to be
>>>>>>>>>>> solved
>>>>>>>>>>>
>>>>>>>>>> together
>>>>>>>>>>
>>>>>>>>>>> and which pieces would be best suited to be broken out.
>>>>>>>>>>>
>>>>>>>>>>> I think the network connectivity, routing and firewalling should
>>>>>>>>>>>
>>>>>>>>>> probably
>>>>>>>>>> all stay together since the majority of the tools we would
>>>>>>>>>> potentially
>>>>>>>>> use
>>>>>>>>>>> would handle all of that together in a single implementation.
>>>>>>>>>>>
>>>>>>>>>>> The password server and userdata seems like a good option for
>>>>>>>>>>> being
>>>>>>>>>>>
>>>>>>>>>> broken
>>>>>>>>>>
>>>>>>>>>>> out and handled independently (and probably rewritten completely
>>>>>>>>>>>
>>>>>>>>>> since
>>>>>>>>> they
>>>>>>>>>>> currently have some issues).
>>>>>>>>>>>
>>>>>>>>>>> Load balancing is another that could warrant splitting out, but
>>>>>>>>>>> that depends on what direction we go and how we would be managing
>>>>>>>>>>>
>>>>>>>>>> it.
>>>>> DHCP
>>>>>>>>> and
>>>>>>>>>>> DNS are others which could go either way.
>>>>>>>>>>>
>>>>>>>>>>> If we do split out services, I think we should consolidate as
>>>>>>>>>>> much as
>>>>>>>>>>>
>>>>>>>>>> we
>>>>>>>>>> can into each service we break out. Ideally a network packet
>>>>>>>>>>> would
>>>>>>>>>>>
>>>>>>>>>> never
>>>>>>>>>> hit more than one, maybe two, services. I don't think we should
>>>>>>>>>>> be splitting services 'just because', I think we need a valid
>>>>>>>>>>> case for splitting any service out because it adds complexity.
>>>>>>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>>>>>>> complexity unless it
>>>>>>>>>>>
>>>>>>>>>> is
>>>>>>>>> really needed.
>>>>>>>>>>> Some more of my thoughts on this anyway...
>>>>>>>>>>>
>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>> Lead Developer
>>>>>>>>>>>
>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>> I do agree with you that this probably isn't the right place
>>>>>>>>>>>> the
>>>>>>>>>>>>
>>>>>>>>>>> password
>>>>>>>>>>> service and user data.
>>>>>>>>>>>>
>>>>>>>>>>>> Having said that, after taking a cursory look at the dev docs,
>>>>>>>>>>>> it
>>>>>>>>>>>>
>>>>>>>>>>> doesn't
>>>>>>>>>>> seem that difficult to add new daemons:
>>>>>>>>>>> https://opensnaproute.github.
>>>>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>>>>>>> creating-new-component>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> They've definitely build it with a microservices architecture
>>>>>>>>>>>> in
>>>>>>>>>>>>
>>>>>>>>>>> mind,
>>>>>>>>> so
>>>>>>>>>>> each individual feature is abstracted into it's own small
>>>>>>>>>>>> daemon
>>>>>>>>>>>>
>>>>>>>>>>> process.
>>>>>>>>>>> We could just create a daemon for the password server and the
>>>>>>>>>>> userdata
>>>>>>>>> components if we really had to.
>>>>>>>>>>>>
>>>>>>>>>>>> - Si
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ________________________________
>>>>>>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
>>>>>>>>>>>> behalf
>>>>>>>>>>>>
>>>>>>>>>>> of
>>>>>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>
>>>>>>>>>>>> A big part of why I know about it is because it is written in Go.
>>>>>>>>>>>>
>>>>>>>>>>> :P
>>>>>>>>> Yes, it is definitely interesting for the routing and traffic
>>>>>>>>>>> handling
>>>>>>>>> aspects of the VR. We will likely have to rethink some of the
>>>>>>>>>>> pieces
>>>>>>>>> a
>>>>>>>>>
>>>>>>>>>> little bit like the password server and userdata if we are to
>>>>>>>>>>>> adopt
>>>>>>>>>>>>
>>>>>>>>>>> a
>>>>>>>>> different VR approach. This is where I think some of JohnB and
>>>>>>>>>>> Chiradeep's
>>>>>>>>>>> ideas make sense. In many ways, it does not make sense for the
>>>>>>>>>>> device
>>>>>>>>> handling routing and network traffic to also be responsible for
>>>>>>>>>>> passwords
>>>>>>>>>>> and userdata.
>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>
>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>>>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
>>>>>>>>>>>> pretty
>>>>>>>>> cool!
>>>>>>>>>>>>> It even supports ONIE install.
>>>>>>>>>>>>>
>>>>>>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>>>>>>>>>>>>
>>>>>>>>>>>> offload
>>>>>>>>> it to
>>>>>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>>>>>>> from a
>>>>>>>>>>>>>
>>>>>>>>>>>> routing
>>>>>>>>>>>>
>>>>>>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Based on the roadmap, it looks like they plan to also support
>>>>>>>>>>>>>
>>>>>>>>>>>> capabilities
>>>>>>>>>>>>
>>>>>>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>>>>>>> will
>>>>>>>>>>>>>
>>>>>>>>>>>> be
>>>>>>>>> huge
>>>>>>>>>>> for our carrier community that rely on these technologies to
>>>>>>>>>>>>> do
>>>>>>>>>>>>>
>>>>>>>>>>>> private
>>>>>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>>>>>>> stuff
>>>>>>>>>>>>>
>>>>>>>>>>>> on
>>>>>>>>>> our
>>>>>>>>>>
>>>>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
>>>>>>>>>>>>> able to
>>>>>>>>>>>>>
>>>>>>>>>>>> do
>>>>>>>>>> MPLS
>>>>>>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>>>>>>> know
>>>>>>>>>>>>>
>>>>>>>>>>>> very
>>>>>>>>>> well).
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Si
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ________________________________
>>>>>>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>>>>>>> does
>>>>>>>>>>>>>
>>>>>>>>>>>> cover is
>>>>>>>>>>> all api driven...
>>>>>>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>>>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>> Though I don\u2019t see VPN in Snaproute.. Makes sense since it
>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>
>>>>>>>>>>>>> not
>>>>>>>>> intended to do IPSec.
>>>>>>>>>>>>>> It seems as though VyOS is starting to look like the best
>>>>>>>>>>>>>>
>>>>>>>>>>>>> option.
>>>>>>>>> Regards,
>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>>>>>>> [mailto:williamstevens@gmail.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>> ]
>>>>>>>>> On
>>>>>>>>>
>>>>>>>>>> Behalf Of Will Stevens
>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Or we could go completely crazy and go with something like
>>>>>>>>>>>>>>
>>>>>>>>>>>>> FlexSwitch
>>>>>>>>>>> from
>>>>>>>>>>>>>> SnapRoute
>>>>>>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>>>>> cloudops.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>> *|*
>>>>>>>>>> tw
>>>>>>>>>>
>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>>>>>>>>>>>
>>>>>>>>>>>>> wstevens@cloudops.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
>>>>>>>>>>>>>> problems
>>>>>>>>>> are
>>>>>>>>>>
>>>>>>>>>>> solved by splitting up the function of the VR into a
>>>>>>>>>>>>>>> bunch of
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> separate
>>>>>>>>>>>>> services. As Syed points out, the complexity added is
>>>>>>>>>>>>>> non-trivial.
>>>>>>>>>>> We now have to manage all the intercontainer networking
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> well
>>>>>>>>> as
>>>>>>>>>
>>>>>>>>>> the
>>>>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>>>>>> VyOS is interesting to me because it covers the majority
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> our
>>>>>>>>> use
>>>>>>>>>>> case with a single unified control plane. It also has
>>>>>>>>>>>>>>> good
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> support
>>>>>>>>>>> for extending features we care about, like IPv6, VXLAN,
>>>>>>>>>>>>>>> VRRP, transactions, etc...
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> cloudops.com
>>>>>>>>> *|*
>>>>>>>>>>> tw
>>>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> sahmed@cloudops.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>> Agree with Marty, adding Docker containers to the
>>>>>>>>>>>>>>>> picture
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> although
>>>>>>>>>>> can make the VR more flexible but the added complexity
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> just
>>>>>>>>> not
>>>>>>>>>>> worth it. Not to mention we would need to take care of
>>>>>>>>>>>>>>> networking
>>>>>>>>>> each container manually and given that our iptable rules
>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> very
>>>>>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
>>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> not
>>>>>>>>>> provide an api but it does fit the bill quite well
>>>>>>>>>>>>>>> otherwise. I
>>>>>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>>>>>>> model
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and
>>>>>>>>>> you
>>>>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> marty@gonsource.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> Licensing aside, I think splitting the various
>>>>>>>>>>>>>>>>> functions
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> into
>>>>>>>>> containers is not a good route either. This will force
>>>>>>>>>>>>>>>> users
>>>>>>>>> to
>>>>>>>>>
>>>>>>>>>> have to maintain
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> use containers and adds complexity to the networking
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> aspects
>>>>>>>>> of
>>>>>>>>>
>>>>>>>>>> ACS.
>>>>>>>>>>>>> Complexity decreases stability. Now I understand the
>>>>>>>>>>>>>>>> argument
>>>>>>>>> that
>>>>>>>>>>> a monolithic approach also brings its own set of
>>>>>>>>>>>>>>>>> issues but
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> it
>>>>>>>>>> also
>>>>>>>>>>>>> simplifies it.
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> the
>>>>>>>>> CloudStack project
>>>>>>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
>>>>>>>>>>>>>>>>> touch
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> anything
>>>>>>>>>>> that
>>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> AGPL
>>>>>>>>>>>>>>>>> - the github repo shows rather infrequent updates.
>>>>>>>>>>>>>>>>> Quite
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> likely
>>>>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> community
>>>>>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>>>>>>> Split
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> it
>>>>>>>>> into
>>>>>>>>>>> many docker containers
>>>>>>>>>>>>>>>>> - password server
>>>>>>>>>>>>>>>>> - userdata server
>>>>>>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>>>>>>> - FW
>>>>>>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>>>>>>> - agent
>>>>>>>>>>>>>>>>> Glue them together with docker compose files (one per
>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> case -
>>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
>>>>>>>>>>>>>>>>> test
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> each
>>>>>>>>> of
>>>>>>>>>
>>>>>>>>>> the
>>>>>>>>>>>>> components independently and fixing one bug in the
>>>>>>>>>>>>>>>>> field
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> (say
>>>>>>>>> DHCP)
>>>>>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>>>>>>> build per-hypervisor VRs. You could even run on baremetal.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>>>>>>> - make the traffic traverse the containers that are
>>>>>>>>>>>>>>>>> needed
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> to
>>>>>>>>>> be
>>>>>>>>>>
>>>>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>>>>>>> - bootstrap the router (how does it find its compose
>>>>>>>>>>>>>>>>> file?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> where
>>>>>>>>>>> is the
>>>>>>>>>>>>>>>>> registry?)
>>>>>>>>>>>>>>>>> - rethink the command and control of the VR
>>>>>>>>>>>>>>>>> functions. SSH
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> works,
>>>>>>>>>>> but something more declarative, idempotent should be
>>>>>>>>>>>>>>>> explored.
>>>>>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>>>>>>> functions
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> can
>>>>>>>>> be
>>>>>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>>>>>>> Control
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> of
>>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>> docker
>>>>>>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This one does look nice. My biggest concern is the
>>>>>>>>>>>>>>>>>> lack
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> of
>>>>>>>>> VXLANs. It seems that any of the ones we mentioned
>>>>>>>>>>>>>>>>>> do not
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> have
>>>>>>>>>>> an
>>>>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>>>>>>> save
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> future
>>>>>>>>>>> engineering effort for example on ipv6 routing, OSPF etc.
>>>>>>>>>>>>>>>>>> And the best part is they come with test automation
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> framework.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>> Instead of replacing the VR in first place we
>>>>>>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> as provider. Once it is stable, network offerings
>>>>>>>>>>>>>>>>>> (on
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> upgrade)
>>>>>>>>>>> can be updated to use it and we can drop the VR if
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> want
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>>> that release
>>>>>>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>>>>>>> them
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> are
>>>>>>>>>> running
>>>>>>>>>>>>>>>>>> without issues. When we replicate the ACS VR
>>>>>>>>>>>>>>>>>> features in
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> new
>>>>>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>>>>>>> (hidden
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> bugs).
>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
>>>>>>>>>>>>>>>>>>>> keen
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>> VyOS
>>>>>>>>>>> (it
>>>>>>>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> @shapeblue
>>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>> treated as
>>>>>>>>>>>>>>>>>>>>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
Hey Matthew,
Yes, I completely agree with your approach. Building a plugin is
definitely the best first step with VyOS. Working on that will also get
your head into the way that CloudStack handles network orchestration, which
will be very valuable going forward as well.
Yes, what you have highlighted about the VyOS having to be an external
device is accurate for the current plugin system. I think it is probably
possible to have the plugin system actually create a virtual machine with a
VyOS template, but I have not tried that and I have not seen an example of
that.
For now, you can review the Palo Alto Networks integration. It is
relatively clean for what you would need. I wrote that plugin, so just ask
me if you have any questions. Currently that plugin only supports Isolated
Guest Networks and not VPC Networks. I am not sure if there is an example
anywhere that handles VPC Networks with the plugin system because most of
the external firewall devices don't handle overlapping IP space (which is
used in VPCs). You can get around that by making the VyOS 'Dedicated' so
it can't be shared between more than one VPC, but that will make the
management of the external VyOS boxes a much bigger pain.
If you have questions about any of this, just let us know and I will help
you out.
In summary, I think you have the right approach by focusing on a plugin and
we can expand on it later.
Cheers,
Will
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Tue, Sep 27, 2016 at 7:13 PM, Matthew Smart <ms...@smartsoftwareinc.com>
wrote:
> Will,
>
> I think that would be very helpful to me at least and for posterity for
> sure. I am in the process of rolling out my first production deployment of
> Cloudstack so I have been busier than expected (plus I have been jumping
> back and forth between different offices). What I intended to propose was
> that we tackle this using an intermediary step. Instead of jumping in and
> replacing the VR wholesale from the start of this initiative what if we
> look at integrating Vyos as a plugin in the same manner as other network
> offering plugins. Replacing such a core and vital component of the
> Cloudstack infrastructure as the VR should be met with a great deal of
> caution and with careful thought and planning. Such a replacement would
> require a nontrivial amount of effort by core contributors to be successful
> and it is questionable whether such an initiative is the area where they
> feel their time is best invested.
>
> In my opinion, using the current plugin capabilities of Cloudstack as a
> first step would have the benefit of giving the devs a proof of concept
> that can be used to evaluate Vyos as a potential replacement for the
> current VR without touching the current core networking functionality at
> all. Additionally, the code needed to integrate Vyos as a plugin would, I
> assume, be directly reusable if the decision is made to go forward with a
> Vyos based VR refactor. So we are not duplicating or wasting effort either.
> Plus, it would serve as an excellent process during which to document the
> network offering plugin architecture.
>
> On the downside to this approach, my understanding would be that the Vyos
> deploy in such an environment would have to be external to the cloudstack
> ecosystem it supports (either a bare metal install or a non cloudstack
> managed VM). Personally, for POC purposes I do not see this as a huge
> stumbling block since I expect that anyone who works on cloudstack can
> deploy a VM using Virsh or some similar hypervisor interface.
>
> What do you think of that approach? What percentage of the functions
> current embodied by the VR are abstracted in such a way as to be
> offloadable to a plugin? Would such a plugin be feature complete enough to
> represent a proof of concept for a potential Vyos based VR?
>
> Thanks,
>
>
> Matthew Smart
> President
> Smart Software Solutions Inc.
> 108 S Pierre St.
> Pierre, SD 57501
>
> Phone: (605) 280-0383
> Skype: msmart13
> Email: msmart@smartsoftwareinc.com
>
> On 09/26/2016 03:01 PM, Will Stevens wrote:
>
>> I feel like I have squashed this discussion with my potential approach to
>> handling this. Maybe we should just pickup this discussion assuming I
>> didn't post that. :P
>>
>> Regarding the docs. I have considered building a stubbed example network
>> plugin and then documenting how you would take that stub and build on it.
>> Would that be interesting?
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> w cloudops.com *|* tw @CloudOps_
>>
>> On Fri, Sep 23, 2016 at 12:15 AM, Murali Reddy <mu...@gmail.com>
>> wrote:
>>
>> Matthew,
>>>
>>> Please see https://cwiki.apache.org/confluence/display/CLOUDSTACK/
>>> Extending+CloudStack+Networking
>>>
>>> Thanks,
>>> Murali
>>>
>>>
>>>
>>>
>>>
>>> On 22/09/16, 11:23 PM, "Matthew Smart" <ms...@smartsoftwareinc.com>
>>> wrote:
>>>
>>> Hey Murali,
>>>>
>>>> I have been reading through the API and other documentation to try to
>>>> get a basic understanding of the network service offering abstraction
>>>> methodology in CS. I have not dove into the code yet but before I did I
>>>> thought I would try a different approach.
>>>>
>>>> Imagine I were to come to this list and say that I have a network
>>>> offering that I sell and that I wanted to write whatever I needed to in
>>>> order to integrate it as an offering in CloudStack. Is there some
>>>> specific documentation and guidelines you would direct me to read in
>>>> order to gather the knowledge necessary to create a cloudstack
>>>> compatible interface for my product?
>>>>
>>>> I don't know the history but I see several products that have navigated
>>>> this process (Nuage, Nicira, ...etc) and am wondering how a new provider
>>>> would work with you guys to navigate that process. If this is too vague,
>>>> we can pretend my new offering is a hardware firewall device.
>>>>
>>>> My goal here is to gain an understanding of how CS interacts with third
>>>> party offerings underneath the hood. I have some thoughts (I think
>>>> inline with Will Steven's brain dump and diagram) but want to make sure
>>>> I am not suffering some misapprehensions about the architecture and,
>>>> short of tracing code, was not successful at finding the information I
>>>> needed to satisfy myself that I know how it is designed.
>>>>
>>>> Thanks,
>>>>
>>>> Matthew Smart
>>>> President
>>>> Smart Software Solutions Inc.
>>>> 108 S Pierre St.
>>>> Pierre, SD 57501
>>>>
>>>> Phone: (605) 280-0383
>>>> Skype: msmart13
>>>> Email: msmart@smartsoftwareinc.com
>>>>
>>>> On 09/20/2016 04:54 AM, Murali Reddy wrote:
>>>>
>>>>> Configuration management of network appliances particularly for Cloud
>>>>>
>>>> and NFV scenarios is still evolving area. Programmability is the not the
>>> strength for even the most popular network operating systems like IOS,
>>> JunoS etc. So its not surprising why CloudStack integrates in a archaic
>>> way
>>> with stock linux for the VR.
>>>
>>>> VR was never integral/hardcoded option in CloudStack. Its always been a
>>>>>
>>>> plugin. CloudStack network orchestration is well abstracted and designed
>>> with vision to compose a network with different set of providers for
>>> different services. Yes that vision is not fully realised yet, and we
>>> don’t
>>> have true service function chains. That would be different discussion
>>> topic.
>>>
>>>> I tend to agree with Simon, as alternate/interim option we can take
>>>>>
>>>> hard look whats causing the problems with current VR integration.
>>> Personally, I think it would be easier we take a cue from configuration
>>> managers and network configuration solutions out there (for e.g promise
>>> theory based Cisco ACI) move to more declarative model of expressing
>>> desired state of network configuration. Infact current VR from 4.6,
>>> actually holds the desired state per service basis, seems to be in that
>>> direction.
>>>
>>>> It does make sense to evaluate new appliances which can provide rich
>>>>>
>>>> semantics (like programmable API, declarative configuration, versioning,
>>> commit/rollback etc), but will need significant engineering effort and
>>> time
>>> to stabilise. We may make same mistakes with integration of other
>>> appliance
>>> as well, if we fully don’t understand the nature of the current problems
>>> with CloudStack core and service provider interaction and current VR
>>> integration.
>>>
>>>>
>>>>> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>>>>>
>>>>> I think our other option is to take a real look at what it would take
>>>>>>
>>>>> to fix the VR. In my opinion, a lot of the problems are related to the
>>> monolithic python code base and the fact nothing is actually separated.
>>>
>>>> Secondly, the python scripts (and bash scripts) don't use any
>>>>>>
>>>>> established libraries to complete tasks and instead shell out and run
>>> commands that are both hard to track and hard to parse on return.
>>>
>>>>
>>>>>> If we daemonized this, used a real api for Agent to VR communication,
>>>>>>
>>>>> used common already existing libraries for the system service and
>>> network
>>> interactions and spent a bit of time separating out code into distinct
>>> modules, everything would behave a lot better.
>>>
>>>>
>>>>>> The pain and suffering is due to years and years of patches and
>>>>>>
>>>>> constant shelling out to complete tasks in my opinion. If we spend
>>> time to
>>> rethink how we interact with the VR in general and we abstract the
>>> systems
>>> and networking stuff and use well known and stable libraries to do the
>>> work, the VR would be much easier to maintain.
>>>
>>>>
>>>>>> - Si
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ________________________________
>>>>>> From: Marty Godsey <ma...@gonsource.com>
>>>>>> Sent: Friday, September 16, 2016 12:24 PM
>>>>>> To: dev@cloudstack.apache.org
>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>
>>>>>> So based upon this discussion would it be prudent to wait on VyOS 2.0?
>>>>>>
>>>>> The current VR is giving us issues but would the time invested in
>>> another
>>> "solution" be wasted especially if by the time another option is chose,
>>> then coded, then tested, then implemented and right as that time happened
>>> to be when VyOS 2.0 is released. Of course you said they are just in the
>>> scoping range so this could still be a year or more out.
>>>
>>>> Thoughts?
>>>>>>
>>>>>> Regards,
>>>>>> Marty Godsey
>>>>>> nSource Solutions
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>>>>>>
>>>>> Behalf Of Will Stevens
>>>
>>>> Sent: Friday, September 16, 2016 10:31 AM
>>>>>> To: dev@cloudstack.apache.org
>>>>>> Cc: daniil@baturin.org
>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>
>>>>>> I just had a quick chat with a couple of the guys over on the VyOS
>>>>>>
>>>>> chat.
>>>
>>>> I have CC'ed one of them in case we have more licensing questions.
>>>>>>
>>>>>> So here is the status with the license "the code inherited from Vyatta
>>>>>>
>>>>> and our modifications from it is GPLv2 (strict, not v2+). The config
>>> reading library is GPLv2 too, so anything that links to is is GPLv2.
>>>
>>>> Some auxiliary components we made after the fork are more permissive,
>>>>>> LGPLv2+ or MIT."
>>>>>>
>>>>>> They are currently in the process of scoping a redesign (VyOS 2.0),
>>>>>>
>>>>> "we are planning a clean rewrite that will solve issues of the current
>>> config system".
>>>
>>>> This will include the ability to configure via the API.
>>>>>>
>>>>>> If we have more questions for VyOS, they are very friendly and
>>>>>>
>>>>> responsive, so we should be able to get answers.
>>>
>>>> *Will STEVENS*
>>>>>> Lead Developer
>>>>>>
>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>>>>>>
>>>>> @CloudOps_
>>>
>>>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com>
>>>>>>
>>>>> wrote:
>>>
>>>> I agree with Will Ilya. There are so many problems with the VR right
>>>>>>>
>>>>>> now.
>>>
>>>> Most of the outages we've had recently have somehow involved the VR.
>>>>>>> We set custom iptables rules on the VR which can and have easily gone
>>>>>>>
>>>>>> wrong.
>>>
>>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>>>>>> VVRP with redundant router still needs work, and not to mention the
>>>>>>> problems we will have when we introduce IPv6 into the whole picture.
>>>>>>>
>>>>>>> I think the spirit of the discussion is to rely on a 3rd party to do
>>>>>>> the networking for us (eg VyOS) and have us handle just the
>>>>>>> orchestration. All the problems that I've described have already been
>>>>>>> solved in VyOS. We also get the advantage of a potential wider
>>>>>>> community to fix and maintain the VR and given our current
>>>>>>> development
>>>>>>> velocity, it think it totally makes sense to look for a 3rd party
>>>>>>>
>>>>>> option.
>>>
>>>> -Syed
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <wstevens@cloudops.com
>>>>>>> >
>>>>>>> wrote:
>>>>>>>
>>>>>>> The VR has been biting us far too often recently, which is why we
>>>>>>>> have started looking into alternative implementations.
>>>>>>>>
>>>>>>>> One of the things that is nice about potentially using the VyOS is
>>>>>>>> that
>>>>>>>>
>>>>>>> it
>>>>>>>
>>>>>>>> is based on Debian, so we should be able to run the other services
>>>>>>>> that
>>>>>>>>
>>>>>>> we
>>>>>>>
>>>>>>>> currently have like the password server and userdata on the VyOS.
>>>>>>>> This means we would not have to change our architecture initially
>>>>>>>> and could focus on only replacing the networking paths.
>>>>>>>>
>>>>>>>> *Will STEVENS*
>>>>>>>> Lead Developer
>>>>>>>>
>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>>>>>> tw @CloudOps_
>>>>>>>>
>>>>>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>>>>>
>>>>>>>> The more this is discussed the more I think we should stick with
>>>>>>>>> our
>>>>>>>>>
>>>>>>>> VR.
>>>>>>>
>>>>>>>> All these other options either seem unfinished or with
>>>>>>>>> incompatible license.
>>>>>>>>>
>>>>>>>>> VyOS looks the most promising so far, it's a serious, mature
>>>>>>>>>
>>>>>>>> project.
>>>
>>>> Adopting it though means we'll have to microservice our way out of
>>>>>>>>> it
>>>>>>>>>
>>>>>>>> with
>>>>>>>>
>>>>>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>>>>>>>>>
>>>>>>>> those
>>>>>>>
>>>>>>>> too. Imho this adds complexity we should void.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>
>>>>>>>>> Nux!
>>>>>>>>> www.nux.ro
>>>>>>>>>
>>>>>>>>> ----- Original Message -----
>>>>>>>>>
>>>>>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>> Ya, we would need to add a daemon for VPN as well. Load
>>>>>>>>>> balancing is another aspect which we will need to consider if we
>>>>>>>>>>
>>>>>>>>> went this route.
>>>
>>>> Something like https://traefik.io/ could potentially be a good
>>>>>>>>>> fit
>>>>>>>>>>
>>>>>>>>> due
>>>>>>>
>>>>>>>> to
>>>>>>>>>
>>>>>>>>>> its API driven configuration, but it may be more than what we
>>>>>>>>>> need.
>>>>>>>>>>
>>>>>>>>>> We should probably try define which pieces make sense to be
>>>>>>>>>> solved
>>>>>>>>>>
>>>>>>>>> together
>>>>>>>>>
>>>>>>>>>> and which pieces would be best suited to be broken out.
>>>>>>>>>>
>>>>>>>>>> I think the network connectivity, routing and firewalling should
>>>>>>>>>>
>>>>>>>>> probably
>>>>>>>>
>>>>>>>>> all stay together since the majority of the tools we would
>>>>>>>>>>
>>>>>>>>> potentially
>>>>>>>
>>>>>>>> use
>>>>>>>>>
>>>>>>>>>> would handle all of that together in a single implementation.
>>>>>>>>>>
>>>>>>>>>> The password server and userdata seems like a good option for
>>>>>>>>>> being
>>>>>>>>>>
>>>>>>>>> broken
>>>>>>>>>
>>>>>>>>>> out and handled independently (and probably rewritten completely
>>>>>>>>>>
>>>>>>>>> since
>>>>>>>
>>>>>>>> they
>>>>>>>>>
>>>>>>>>>> currently have some issues).
>>>>>>>>>>
>>>>>>>>>> Load balancing is another that could warrant splitting out, but
>>>>>>>>>> that depends on what direction we go and how we would be managing
>>>>>>>>>>
>>>>>>>>> it.
>>>
>>>> DHCP
>>>>>>>
>>>>>>>> and
>>>>>>>>>
>>>>>>>>>> DNS are others which could go either way.
>>>>>>>>>>
>>>>>>>>>> If we do split out services, I think we should consolidate as
>>>>>>>>>> much as
>>>>>>>>>>
>>>>>>>>> we
>>>>>>>>
>>>>>>>>> can into each service we break out. Ideally a network packet
>>>>>>>>>> would
>>>>>>>>>>
>>>>>>>>> never
>>>>>>>>
>>>>>>>>> hit more than one, maybe two, services. I don't think we should
>>>>>>>>>> be splitting services 'just because', I think we need a valid
>>>>>>>>>> case for splitting any service out because it adds complexity.
>>>>>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>>>>>> complexity unless it
>>>>>>>>>>
>>>>>>>>> is
>>>>>>>
>>>>>>>> really needed.
>>>>>>>>>>
>>>>>>>>>> Some more of my thoughts on this anyway...
>>>>>>>>>>
>>>>>>>>>> *Will STEVENS*
>>>>>>>>>> Lead Developer
>>>>>>>>>>
>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I do agree with you that this probably isn't the right place
>>>>>>>>>>> the
>>>>>>>>>>>
>>>>>>>>>> password
>>>>>>>>>
>>>>>>>>>> service and user data.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Having said that, after taking a cursory look at the dev docs,
>>>>>>>>>>> it
>>>>>>>>>>>
>>>>>>>>>> doesn't
>>>>>>>>>
>>>>>>>>>> seem that difficult to add new daemons:
>>>>>>>>>>>
>>>>>>>>>> https://opensnaproute.github.
>>>>>>>
>>>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>>>>>
>>>>>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>>>>>> creating-new-component>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> They've definitely build it with a microservices architecture
>>>>>>>>>>> in
>>>>>>>>>>>
>>>>>>>>>> mind,
>>>>>>>
>>>>>>>> so
>>>>>>>>>
>>>>>>>>>> each individual feature is abstracted into it's own small
>>>>>>>>>>> daemon
>>>>>>>>>>>
>>>>>>>>>> process.
>>>>>>>>>
>>>>>>>>>> We could just create a daemon for the password server and the
>>>>>>>>>>>
>>>>>>>>>> userdata
>>>>>>>
>>>>>>>> components if we really had to.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> - Si
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ________________________________
>>>>>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
>>>>>>>>>>> behalf
>>>>>>>>>>>
>>>>>>>>>> of
>>>>>>>>
>>>>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>
>>>>>>>>>>> A big part of why I know about it is because it is written in Go.
>>>>>>>>>>>
>>>>>>>>>> :P
>>>>>>>
>>>>>>>> Yes, it is definitely interesting for the routing and traffic
>>>>>>>>>>>
>>>>>>>>>> handling
>>>>>>>
>>>>>>>> aspects of the VR. We will likely have to rethink some of the
>>>>>>>>>>>
>>>>>>>>>> pieces
>>>>>>>
>>>>>>>> a
>>>>>>>>
>>>>>>>>> little bit like the password server and userdata if we are to
>>>>>>>>>>> adopt
>>>>>>>>>>>
>>>>>>>>>> a
>>>>>>>
>>>>>>>> different VR approach. This is where I think some of JohnB and
>>>>>>>>>>>
>>>>>>>>>> Chiradeep's
>>>>>>>>>
>>>>>>>>>> ideas make sense. In many ways, it does not make sense for the
>>>>>>>>>>>
>>>>>>>>>> device
>>>>>>>
>>>>>>>> handling routing and network traffic to also be responsible for
>>>>>>>>>>>
>>>>>>>>>> passwords
>>>>>>>>>
>>>>>>>>>> and userdata.
>>>>>>>>>>>
>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>> Lead Developer
>>>>>>>>>>>
>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
>>>>>>>>>>>>
>>>>>>>>>>> pretty
>>>>>>>
>>>>>>>> cool!
>>>>>>>>>>>
>>>>>>>>>>>> It even supports ONIE install.
>>>>>>>>>>>>
>>>>>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>>>>>>>>>>>
>>>>>>>>>>> offload
>>>>>>>
>>>>>>>> it to
>>>>>>>>>
>>>>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>>>>>> from a
>>>>>>>>>>>>
>>>>>>>>>>> routing
>>>>>>>>>>>
>>>>>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Based on the roadmap, it looks like they plan to also support
>>>>>>>>>>>>
>>>>>>>>>>> capabilities
>>>>>>>>>>>
>>>>>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>>>>>> will
>>>>>>>>>>>>
>>>>>>>>>>> be
>>>>>>>
>>>>>>>> huge
>>>>>>>>>
>>>>>>>>>> for our carrier community that rely on these technologies to
>>>>>>>>>>>> do
>>>>>>>>>>>>
>>>>>>>>>>> private
>>>>>>>>>
>>>>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>>>>>> stuff
>>>>>>>>>>>>
>>>>>>>>>>> on
>>>>>>>>
>>>>>>>>> our
>>>>>>>>>
>>>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
>>>>>>>>>>>> able to
>>>>>>>>>>>>
>>>>>>>>>>> do
>>>>>>>>
>>>>>>>>> MPLS
>>>>>>>>>>>
>>>>>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>>>>>> know
>>>>>>>>>>>>
>>>>>>>>>>> very
>>>>>>>>
>>>>>>>>> well).
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> - Si
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ________________________________
>>>>>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>>>>>
>>>>>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>>>>>> does
>>>>>>>>>>>>
>>>>>>>>>>> cover is
>>>>>>>>>
>>>>>>>>>> all api driven...
>>>>>>>>>>>>
>>>>>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>>>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Though I don’t see VPN in Snaproute.. Makes sense since it
>>>>>>>>>>>>> was
>>>>>>>>>>>>>
>>>>>>>>>>>> not
>>>>>>>
>>>>>>>> intended to do IPSec.
>>>>>>>>>>>>>
>>>>>>>>>>>>> It seems as though VyOS is starting to look like the best
>>>>>>>>>>>>>
>>>>>>>>>>>> option.
>>>>>>>
>>>>>>>> Regards,
>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>>>>>> [mailto:williamstevens@gmail.com
>>>>>>>>>>>>>
>>>>>>>>>>>> ]
>>>>>>>
>>>>>>>> On
>>>>>>>>
>>>>>>>>> Behalf Of Will Stevens
>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>
>>>>>>>>>>>>> Or we could go completely crazy and go with something like
>>>>>>>>>>>>>
>>>>>>>>>>>> FlexSwitch
>>>>>>>>>
>>>>>>>>>> from
>>>>>>>>>>>>
>>>>>>>>>>>>> SnapRoute
>>>>>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>>
>>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>>>> cloudops.com
>>>>>>>>>>>>>
>>>>>>>>>>>> *|*
>>>>>>>>
>>>>>>>>> tw
>>>>>>>>>
>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>>>>>>>>>>
>>>>>>>>>>>> wstevens@cloudops.com>
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
>>>>>>>>>>>>>>
>>>>>>>>>>>>> problems
>>>>>>>>
>>>>>>>>> are
>>>>>>>>>
>>>>>>>>>> solved by splitting up the function of the VR into a
>>>>>>>>>>>>>> bunch of
>>>>>>>>>>>>>>
>>>>>>>>>>>>> separate
>>>>>>>>>>>
>>>>>>>>>>>> services. As Syed points out, the complexity added is
>>>>>>>>>>>>>>
>>>>>>>>>>>>> non-trivial.
>>>>>>>>>
>>>>>>>>>> We now have to manage all the intercontainer networking
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>
>>>>>>>>>>>>> well
>>>>>>>
>>>>>>>> as
>>>>>>>>
>>>>>>>>> the
>>>>>>>>>>>
>>>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> VyOS is interesting to me because it covers the majority
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>
>>>>>>>>>>>>> our
>>>>>>>
>>>>>>>> use
>>>>>>>>>
>>>>>>>>>> case with a single unified control plane. It also has
>>>>>>>>>>>>>> good
>>>>>>>>>>>>>>
>>>>>>>>>>>>> support
>>>>>>>>>
>>>>>>>>>> for extending features we care about, like IPv6, VXLAN,
>>>>>>>>>>>>>> VRRP, transactions, etc...
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>>>>>
>>>>>>>>>>>>> cloudops.com
>>>>>>>
>>>>>>>> *|*
>>>>>>>>>
>>>>>>>>>> tw
>>>>>>>>>>>
>>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>>>>>>>>>>>>
>>>>>>>>>>>>> sahmed@cloudops.com>
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Agree with Marty, adding Docker containers to the
>>>>>>>>>>>>>>> picture
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> although
>>>>>>>>>
>>>>>>>>>> can make the VR more flexible but the added complexity
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> just
>>>>>>>
>>>>>>>> not
>>>>>>>>>
>>>>>>>>>> worth it. Not to mention we would need to take care of
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> networking
>>>>>>>>
>>>>>>>>> each container manually and given that our iptable rules
>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> very
>>>>>>>>
>>>>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> not
>>>>>>>>
>>>>>>>>> provide an api but it does fit the bill quite well
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> otherwise. I
>>>>>>>
>>>>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>>>>>> model
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> and
>>>>>>>>
>>>>>>>>> you
>>>>>>>>>>>
>>>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> marty@gonsource.com>
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Licensing aside, I think splitting the various
>>>>>>>>>>>>>>>> functions
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> into
>>>>>>>
>>>>>>>> containers is not a good route either. This will force
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> users
>>>>>>>
>>>>>>>> to
>>>>>>>>
>>>>>>>>> have to maintain
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> use containers and adds complexity to the networking
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> aspects
>>>>>>>
>>>>>>>> of
>>>>>>>>
>>>>>>>>> ACS.
>>>>>>>>>>>
>>>>>>>>>>>> Complexity decreases stability. Now I understand the
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> argument
>>>>>>>
>>>>>>>> that
>>>>>>>>>
>>>>>>>>>> a monolithic approach also brings its own set of
>>>>>>>>>>>>>>>> issues but
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> it
>>>>>>>>
>>>>>>>>> also
>>>>>>>>>>>
>>>>>>>>>>>> simplifies it.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> the
>>>>>>>
>>>>>>>> CloudStack project
>>>>>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
>>>>>>>>>>>>>>>> touch
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> anything
>>>>>>>>>
>>>>>>>>>> that
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> AGPL
>>>>>>>>>>>>>>>> - the github repo shows rather infrequent updates.
>>>>>>>>>>>>>>>> Quite
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> likely
>>>>>>>>>
>>>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> community
>>>>>>>>>
>>>>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>>>>>> Split
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> it
>>>>>>>
>>>>>>>> into
>>>>>>>>>
>>>>>>>>>> many docker containers
>>>>>>>>>>>>>>>> - password server
>>>>>>>>>>>>>>>> - userdata server
>>>>>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>>>>>> - FW
>>>>>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>>>>>> - agent
>>>>>>>>>>>>>>>> Glue them together with docker compose files (one per
>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> case -
>>>>>>>>>
>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
>>>>>>>>>>>>>>>> test
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> each
>>>>>>>
>>>>>>>> of
>>>>>>>>
>>>>>>>>> the
>>>>>>>>>>>
>>>>>>>>>>>> components independently and fixing one bug in the
>>>>>>>>>>>>>>>> field
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> (say
>>>>>>>
>>>>>>>> DHCP)
>>>>>>>>>>>
>>>>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>>>>>> build per-hypervisor VRs. You could even run on baremetal.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>>>>>> - make the traffic traverse the containers that are
>>>>>>>>>>>>>>>> needed
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> to
>>>>>>>>
>>>>>>>>> be
>>>>>>>>>
>>>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>>>>>> - bootstrap the router (how does it find its compose
>>>>>>>>>>>>>>>> file?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> where
>>>>>>>>>
>>>>>>>>>> is the
>>>>>>>>>>>>>>>> registry?)
>>>>>>>>>>>>>>>> - rethink the command and control of the VR
>>>>>>>>>>>>>>>> functions. SSH
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> works,
>>>>>>>>>
>>>>>>>>>> but something more declarative, idempotent should be
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> explored.
>>>>>>>>
>>>>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>>>>>> functions
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> can
>>>>>>>
>>>>>>>> be
>>>>>>>>>
>>>>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>>>>>> Control
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> of
>>>>>>>>
>>>>>>>>> the
>>>>>>>>>
>>>>>>>>>> docker
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This one does look nice. My biggest concern is the
>>>>>>>>>>>>>>>>> lack
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> of
>>>>>>>
>>>>>>>> VXLANs. It seems that any of the ones we mentioned
>>>>>>>>>>>>>>>>> do not
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> have
>>>>>>>>>
>>>>>>>>>> an
>>>>>>>>>>>
>>>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>>>>>> save
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> future
>>>>>>>>>
>>>>>>>>>> engineering effort for example on ipv6 routing, OSPF etc.
>>>>>>>>>>>>>>>>> And the best part is they come with test automation
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> framework.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Instead of replacing the VR in first place we
>>>>>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> as provider. Once it is stable, network offerings
>>>>>>>>>>>>>>>>> (on
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> upgrade)
>>>>>>>>>
>>>>>>>>>> can be updated to use it and we can drop the VR if
>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> want
>>>>>>>
>>>>>>>> at
>>>>>>>>
>>>>>>>>> that release
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>>>>>> them
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> are
>>>>>>>>
>>>>>>>>> running
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> without issues. When we replicate the ACS VR
>>>>>>>>>>>>>>>>> features in
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> new
>>>>>>>>
>>>>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>>>>>> (hidden
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> bugs).
>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
>>>>>>>>>>>>>>>>>>> keen
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> on
>>>>>>>
>>>>>>>> VyOS
>>>>>>>>>
>>>>>>>>>> (it
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> @shapeblue
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> be
>>>>>>>>
>>>>>>>>> treated as
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
Matthew,
From an Engineers viewpoint and someone who has a production deployment of CloudStack now offering VPC services to providers, I think the approach of a plugin is a wise move. As you mentioned, a full blown replacement of the VR, since it is such a vital piece of ACS, would require extensive work to make sure that it is not only implemented in such a manner as to support future changes more easily but also in a way that provides a rock solid platform when upgraded since al the VRs would need to be upgraded. By doing it as a plugin, this allows providers to "try" out VyOS first in a non-destructive manner and easily roll back if it proves to not be a good move.
So I agree with the plugin route.
Regards,
Marty Godsey
-----Original Message-----
From: Matthew Smart [mailto:msmart@smartsoftwareinc.com]
Sent: Tuesday, September 27, 2016 7:13 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
Will,
I think that would be very helpful to me at least and for posterity for sure. I am in the process of rolling out my first production deployment of Cloudstack so I have been busier than expected (plus I have been jumping back and forth between different offices). What I intended to propose was that we tackle this using an intermediary step. Instead of jumping in and replacing the VR wholesale from the start of this initiative what if we look at integrating Vyos as a plugin in the same manner as other network offering plugins. Replacing such a core and vital component of the Cloudstack infrastructure as the VR should be met with a great deal of caution and with careful thought and planning. Such a replacement would require a nontrivial amount of effort by core contributors to be successful and it is questionable whether such an initiative is the area where they feel their time is best invested.
In my opinion, using the current plugin capabilities of Cloudstack as a first step would have the benefit of giving the devs a proof of concept that can be used to evaluate Vyos as a potential replacement for the current VR without touching the current core networking functionality at all. Additionally, the code needed to integrate Vyos as a plugin would, I assume, be directly reusable if the decision is made to go forward with a Vyos based VR refactor. So we are not duplicating or wasting effort either. Plus, it would serve as an excellent process during which to document the network offering plugin architecture.
On the downside to this approach, my understanding would be that the Vyos deploy in such an environment would have to be external to the cloudstack ecosystem it supports (either a bare metal install or a non cloudstack managed VM). Personally, for POC purposes I do not see this as a huge stumbling block since I expect that anyone who works on cloudstack can deploy a VM using Virsh or some similar hypervisor interface.
What do you think of that approach? What percentage of the functions current embodied by the VR are abstracted in such a way as to be offloadable to a plugin? Would such a plugin be feature complete enough to represent a proof of concept for a potential Vyos based VR?
Thanks,
Matthew Smart
President
Smart Software Solutions Inc.
108 S Pierre St.
Pierre, SD 57501
Phone: (605) 280-0383
Skype: msmart13
Email: msmart@smartsoftwareinc.com
On 09/26/2016 03:01 PM, Will Stevens wrote:
> I feel like I have squashed this discussion with my potential approach
> to handling this. Maybe we should just pickup this discussion
> assuming I didn't post that. :P
>
> Regarding the docs. I have considered building a stubbed example
> network plugin and then documenting how you would take that stub and build on it.
> Would that be interesting?
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Fri, Sep 23, 2016 at 12:15 AM, Murali Reddy
> <mu...@gmail.com>
> wrote:
>
>> Matthew,
>>
>> Please see https://cwiki.apache.org/confluence/display/CLOUDSTACK/
>> Extending+CloudStack+Networking
>>
>> Thanks,
>> Murali
>>
>>
>>
>>
>>
>> On 22/09/16, 11:23 PM, "Matthew Smart" <ms...@smartsoftwareinc.com>
>> wrote:
>>
>>> Hey Murali,
>>>
>>> I have been reading through the API and other documentation to try
>>> to get a basic understanding of the network service offering
>>> abstraction methodology in CS. I have not dove into the code yet but
>>> before I did I thought I would try a different approach.
>>>
>>> Imagine I were to come to this list and say that I have a network
>>> offering that I sell and that I wanted to write whatever I needed to
>>> in order to integrate it as an offering in CloudStack. Is there some
>>> specific documentation and guidelines you would direct me to read in
>>> order to gather the knowledge necessary to create a cloudstack
>>> compatible interface for my product?
>>>
>>> I don't know the history but I see several products that have
>>> navigated this process (Nuage, Nicira, ...etc) and am wondering how
>>> a new provider would work with you guys to navigate that process. If
>>> this is too vague, we can pretend my new offering is a hardware firewall device.
>>>
>>> My goal here is to gain an understanding of how CS interacts with
>>> third party offerings underneath the hood. I have some thoughts (I
>>> think inline with Will Steven's brain dump and diagram) but want to
>>> make sure I am not suffering some misapprehensions about the
>>> architecture and, short of tracing code, was not successful at
>>> finding the information I needed to satisfy myself that I know how it is designed.
>>>
>>> Thanks,
>>>
>>> Matthew Smart
>>> President
>>> Smart Software Solutions Inc.
>>> 108 S Pierre St.
>>> Pierre, SD 57501
>>>
>>> Phone: (605) 280-0383
>>> Skype: msmart13
>>> Email: msmart@smartsoftwareinc.com
>>>
>>> On 09/20/2016 04:54 AM, Murali Reddy wrote:
>>>> Configuration management of network appliances particularly for
>>>> Cloud
>> and NFV scenarios is still evolving area. Programmability is the not
>> the strength for even the most popular network operating systems like
>> IOS, JunoS etc. So its not surprising why CloudStack integrates in a
>> archaic way with stock linux for the VR.
>>>> VR was never integral/hardcoded option in CloudStack. Its always
>>>> been a
>> plugin. CloudStack network orchestration is well abstracted and
>> designed with vision to compose a network with different set of
>> providers for different services. Yes that vision is not fully
>> realised yet, and we don’t have true service function chains. That would be different discussion topic.
>>>> I tend to agree with Simon, as alternate/interim option we can take
>> hard look whats causing the problems with current VR integration.
>> Personally, I think it would be easier we take a cue from
>> configuration managers and network configuration solutions out there
>> (for e.g promise theory based Cisco ACI) move to more declarative
>> model of expressing desired state of network configuration. Infact
>> current VR from 4.6, actually holds the desired state per service
>> basis, seems to be in that direction.
>>>> It does make sense to evaluate new appliances which can provide
>>>> rich
>> semantics (like programmable API, declarative configuration,
>> versioning, commit/rollback etc), but will need significant
>> engineering effort and time to stabilise. We may make same mistakes
>> with integration of other appliance as well, if we fully don’t
>> understand the nature of the current problems with CloudStack core
>> and service provider interaction and current VR integration.
>>>>
>>>> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>>>>
>>>>> I think our other option is to take a real look at what it would
>>>>> take
>> to fix the VR. In my opinion, a lot of the problems are related to
>> the monolithic python code base and the fact nothing is actually separated.
>>>>> Secondly, the python scripts (and bash scripts) don't use any
>> established libraries to complete tasks and instead shell out and run
>> commands that are both hard to track and hard to parse on return.
>>>>>
>>>>> If we daemonized this, used a real api for Agent to VR
>>>>> communication,
>> used common already existing libraries for the system service and
>> network interactions and spent a bit of time separating out code into
>> distinct modules, everything would behave a lot better.
>>>>>
>>>>> The pain and suffering is due to years and years of patches and
>> constant shelling out to complete tasks in my opinion. If we spend
>> time to rethink how we interact with the VR in general and we
>> abstract the systems and networking stuff and use well known and
>> stable libraries to do the work, the VR would be much easier to maintain.
>>>>>
>>>>> - Si
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ________________________________
>>>>> From: Marty Godsey <ma...@gonsource.com>
>>>>> Sent: Friday, September 16, 2016 12:24 PM
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>
>>>>> So based upon this discussion would it be prudent to wait on VyOS 2.0?
>> The current VR is giving us issues but would the time invested in
>> another "solution" be wasted especially if by the time another option
>> is chose, then coded, then tested, then implemented and right as that
>> time happened to be when VyOS 2.0 is released. Of course you said
>> they are just in the scoping range so this could still be a year or more out.
>>>>> Thoughts?
>>>>>
>>>>> Regards,
>>>>> Marty Godsey
>>>>> nSource Solutions
>>>>>
>>>>> -----Original Message-----
>>>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com]
>>>>> On
>> Behalf Of Will Stevens
>>>>> Sent: Friday, September 16, 2016 10:31 AM
>>>>> To: dev@cloudstack.apache.org
>>>>> Cc: daniil@baturin.org
>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>
>>>>> I just had a quick chat with a couple of the guys over on the VyOS
>> chat.
>>>>> I have CC'ed one of them in case we have more licensing questions.
>>>>>
>>>>> So here is the status with the license "the code inherited from
>>>>> Vyatta
>> and our modifications from it is GPLv2 (strict, not v2+). The config
>> reading library is GPLv2 too, so anything that links to is is GPLv2.
>>>>> Some auxiliary components we made after the fork are more
>>>>> permissive,
>>>>> LGPLv2+ or MIT."
>>>>>
>>>>> They are currently in the process of scoping a redesign (VyOS
>>>>> 2.0),
>> "we are planning a clean rewrite that will solve issues of the
>> current config system".
>>>>> This will include the ability to configure via the API.
>>>>>
>>>>> If we have more questions for VyOS, they are very friendly and
>> responsive, so we should be able to get answers.
>>>>> *Will STEVENS*
>>>>> Lead Developer
>>>>>
>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>>> tw
>> @CloudOps_
>>>>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com>
>> wrote:
>>>>>> I agree with Will Ilya. There are so many problems with the VR
>>>>>> right
>> now.
>>>>>> Most of the outages we've had recently have somehow involved the VR.
>>>>>> We set custom iptables rules on the VR which can and have easily
>>>>>> gone
>> wrong.
>>>>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>>>>> VVRP with redundant router still needs work, and not to mention
>>>>>> the problems we will have when we introduce IPv6 into the whole picture.
>>>>>>
>>>>>> I think the spirit of the discussion is to rely on a 3rd party to
>>>>>> do the networking for us (eg VyOS) and have us handle just the
>>>>>> orchestration. All the problems that I've described have already
>>>>>> been solved in VyOS. We also get the advantage of a potential
>>>>>> wider community to fix and maintain the VR and given our current
>>>>>> development velocity, it think it totally makes sense to look for
>>>>>> a 3rd party
>> option.
>>>>>> -Syed
>>>>>>
>>>>>>
>>>>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens
>>>>>> <ws...@cloudops.com>
>>>>>> wrote:
>>>>>>
>>>>>>> The VR has been biting us far too often recently, which is why
>>>>>>> we have started looking into alternative implementations.
>>>>>>>
>>>>>>> One of the things that is nice about potentially using the VyOS
>>>>>>> is that
>>>>>> it
>>>>>>> is based on Debian, so we should be able to run the other
>>>>>>> services that
>>>>>> we
>>>>>>> currently have like the password server and userdata on the VyOS.
>>>>>>> This means we would not have to change our architecture
>>>>>>> initially and could focus on only replacing the networking paths.
>>>>>>>
>>>>>>> *Will STEVENS*
>>>>>>> Lead Developer
>>>>>>>
>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>> *|* tw @CloudOps_
>>>>>>>
>>>>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>>>>
>>>>>>>> The more this is discussed the more I think we should stick
>>>>>>>> with our
>>>>>> VR.
>>>>>>>> All these other options either seem unfinished or with
>>>>>>>> incompatible license.
>>>>>>>>
>>>>>>>> VyOS looks the most promising so far, it's a serious, mature
>> project.
>>>>>>>> Adopting it though means we'll have to microservice our way out
>>>>>>>> of it
>>>>>>> with
>>>>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS
>>>>>>>> serve
>>>>>> those
>>>>>>>> too. Imho this adds complexity we should void.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>
>>>>>>>> Nux!
>>>>>>>> www.nux.ro
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR Ya, we would need to
>>>>>>>>> add a daemon for VPN as well. Load balancing is another
>>>>>>>>> aspect which we will need to consider if we
>> went this route.
>>>>>>>>> Something like https://traefik.io/ could potentially be a good
>>>>>>>>> fit
>>>>>> due
>>>>>>>> to
>>>>>>>>> its API driven configuration, but it may be more than what we need.
>>>>>>>>>
>>>>>>>>> We should probably try define which pieces make sense to be
>>>>>>>>> solved
>>>>>>>> together
>>>>>>>>> and which pieces would be best suited to be broken out.
>>>>>>>>>
>>>>>>>>> I think the network connectivity, routing and firewalling
>>>>>>>>> should
>>>>>>> probably
>>>>>>>>> all stay together since the majority of the tools we would
>>>>>> potentially
>>>>>>>> use
>>>>>>>>> would handle all of that together in a single implementation.
>>>>>>>>>
>>>>>>>>> The password server and userdata seems like a good option for
>>>>>>>>> being
>>>>>>>> broken
>>>>>>>>> out and handled independently (and probably rewritten
>>>>>>>>> completely
>>>>>> since
>>>>>>>> they
>>>>>>>>> currently have some issues).
>>>>>>>>>
>>>>>>>>> Load balancing is another that could warrant splitting out,
>>>>>>>>> but that depends on what direction we go and how we would be
>>>>>>>>> managing
>> it.
>>>>>> DHCP
>>>>>>>> and
>>>>>>>>> DNS are others which could go either way.
>>>>>>>>>
>>>>>>>>> If we do split out services, I think we should consolidate as
>>>>>>>>> much as
>>>>>>> we
>>>>>>>>> can into each service we break out. Ideally a network packet
>>>>>>>>> would
>>>>>>> never
>>>>>>>>> hit more than one, maybe two, services. I don't think we
>>>>>>>>> should be splitting services 'just because', I think we need a
>>>>>>>>> valid case for splitting any service out because it adds complexity.
>>>>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>>>>> complexity unless it
>>>>>> is
>>>>>>>>> really needed.
>>>>>>>>>
>>>>>>>>> Some more of my thoughts on this anyway...
>>>>>>>>>
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>
>>>>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller
>>>>>>>>> <sw...@ena.com>
>>>>>>> wrote:
>>>>>>>>>> I do agree with you that this probably isn't the right place
>>>>>>>>>> the
>>>>>>>> password
>>>>>>>>>> service and user data.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Having said that, after taking a cursory look at the dev
>>>>>>>>>> docs, it
>>>>>>>> doesn't
>>>>>>>>>> seem that difficult to add new daemons:
>>>>>> https://opensnaproute.github.
>>>>>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>>>>
>>>>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>>>>> creating-new-component>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> They've definitely build it with a microservices architecture
>>>>>>>>>> in
>>>>>> mind,
>>>>>>>> so
>>>>>>>>>> each individual feature is abstracted into it's own small
>>>>>>>>>> daemon
>>>>>>>> process.
>>>>>>>>>> We could just create a daemon for the password server and the
>>>>>> userdata
>>>>>>>>>> components if we really had to.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Si
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ________________________________
>>>>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
>>>>>>>>>> behalf
>>>>>>> of
>>>>>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>
>>>>>>>>>> A big part of why I know about it is because it is written in Go.
>>>>>> :P
>>>>>>>>>> Yes, it is definitely interesting for the routing and traffic
>>>>>> handling
>>>>>>>>>> aspects of the VR. We will likely have to rethink some of
>>>>>>>>>> the
>>>>>> pieces
>>>>>>> a
>>>>>>>>>> little bit like the password server and userdata if we are to
>>>>>>>>>> adopt
>>>>>> a
>>>>>>>>>> different VR approach. This is where I think some of JohnB
>>>>>>>>>> and
>>>>>>>> Chiradeep's
>>>>>>>>>> ideas make sense. In many ways, it does not make sense for
>>>>>>>>>> the
>>>>>> device
>>>>>>>>>> handling routing and network traffic to also be responsible
>>>>>>>>>> for
>>>>>>>> passwords
>>>>>>>>>> and userdata.
>>>>>>>>>>
>>>>>>>>>> *Will STEVENS*
>>>>>>>>>> Lead Developer
>>>>>>>>>>
>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>> cloudops.com
>>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller
>>>>>>>>>> <sw...@ena.com>
>>>>>>> wrote:
>>>>>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It
>>>>>>>>>>> looks
>>>>>> pretty
>>>>>>>>>> cool!
>>>>>>>>>>> It even supports ONIE install.
>>>>>>>>>>>
>>>>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>>>>> offload
>>>>>>>> it to
>>>>>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>>>>> from a
>>>>>>>>>> routing
>>>>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Based on the roadmap, it looks like they plan to also
>>>>>>>>>>> support
>>>>>>>>>> capabilities
>>>>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>>>>> will
>>>>>> be
>>>>>>>> huge
>>>>>>>>>>> for our carrier community that rely on these technologies to
>>>>>>>>>>> do
>>>>>>>> private
>>>>>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>>>>> stuff
>>>>>>> on
>>>>>>>> our
>>>>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
>>>>>>>>>>> able to
>>>>>>> do
>>>>>>>>>> MPLS
>>>>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>>>>> know
>>>>>>> very
>>>>>>>>>>> well).
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> - Si
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ________________________________
>>>>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>>>>
>>>>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>>>>> does
>>>>>>>> cover is
>>>>>>>>>>> all api driven...
>>>>>>>>>>>
>>>>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey"
>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>> wrote:
>>>>>>>>>>>> Though I don’t see VPN in Snaproute.. Makes sense since it
>>>>>>>>>>>> was
>>>>>> not
>>>>>>>>>>>> intended to do IPSec.
>>>>>>>>>>>>
>>>>>>>>>>>> It seems as though VyOS is starting to look like the best
>>>>>> option.
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>>>>> [mailto:williamstevens@gmail.com
>>>>>> ]
>>>>>>> On
>>>>>>>>>>>> Behalf Of Will Stevens
>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>
>>>>>>>>>>>> Or we could go completely crazy and go with something like
>>>>>>>> FlexSwitch
>>>>>>>>>>> from
>>>>>>>>>>>> SnapRoute
>>>>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>>>>
>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>
>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>>> cloudops.com
>>>>>>> *|*
>>>>>>>> tw
>>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>>>>> wstevens@cloudops.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
>>>>>>> problems
>>>>>>>> are
>>>>>>>>>>>>> solved by splitting up the function of the VR into a bunch
>>>>>>>>>>>>> of
>>>>>>>>>> separate
>>>>>>>>>>>>> services. As Syed points out, the complexity added is
>>>>>>>> non-trivial.
>>>>>>>>>>>>> We now have to manage all the intercontainer networking as
>>>>>> well
>>>>>>> as
>>>>>>>>>> the
>>>>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>>>>
>>>>>>>>>>>>> VyOS is interesting to me because it covers the majority
>>>>>>>>>>>>> of
>>>>>> our
>>>>>>>> use
>>>>>>>>>>>>> case with a single unified control plane. It also has
>>>>>>>>>>>>> good
>>>>>>>> support
>>>>>>>>>>>>> for extending features we care about, like IPv6, VXLAN,
>>>>>>>>>>>>> VRRP, transactions, etc...
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>>
>>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>> cloudops.com
>>>>>>>> *|*
>>>>>>>>>> tw
>>>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>>>>> sahmed@cloudops.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> Agree with Marty, adding Docker containers to the picture
>>>>>>>> although
>>>>>>>>>>>>>> can make the VR more flexible but the added complexity is
>>>>>> just
>>>>>>>> not
>>>>>>>>>>>>>> worth it. Not to mention we would need to take care of
>>>>>>> networking
>>>>>>>>>>>>>> each container manually and given that our iptable rules
>>>>>>>>>>>>>> are
>>>>>>> very
>>>>>>>>>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
>>>>>>>>>>>>>> does
>>>>>>> not
>>>>>>>>>>>>>> provide an api but it does fit the bill quite well
>>>>>> otherwise. I
>>>>>>>>>>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>>>>> model
>>>>>>> and
>>>>>>>>>> you
>>>>>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>>>>> marty@gonsource.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> Licensing aside, I think splitting the various functions
>>>>>> into
>>>>>>>>>>>>>>> containers is not a good route either. This will force
>>>>>> users
>>>>>>> to
>>>>>>>>>>>>>>> have to maintain
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> use containers and adds complexity to the networking
>>>>>> aspects
>>>>>>> of
>>>>>>>>>> ACS.
>>>>>>>>>>>>>>> Complexity decreases stability. Now I understand the
>>>>>> argument
>>>>>>>> that
>>>>>>>>>>>>>>> a monolithic approach also brings its own set of issues
>>>>>>>>>>>>>>> but
>>>>>>> it
>>>>>>>>>> also
>>>>>>>>>>>>>>> simplifies it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
>>>>>>>>>>>>>>> of
>>>>>> the
>>>>>>>>>>>>>>> CloudStack project
>>>>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
>>>>>>>>>>>>>>> touch
>>>>>>>> anything
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>> AGPL
>>>>>>>>>>>>>>> - the github repo shows rather infrequent updates.
>>>>>>>>>>>>>>> Quite
>>>>>>>> likely
>>>>>>>>>>>>>>> they aren't considering the use cases of the CloudStack
>>>>>>>> community
>>>>>>>>>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>>>>> Split
>>>>>> it
>>>>>>>> into
>>>>>>>>>>>>>>> many docker containers
>>>>>>>>>>>>>>> - password server
>>>>>>>>>>>>>>> - userdata server
>>>>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>>>>> - FW
>>>>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>>>>> - agent
>>>>>>>>>>>>>>> Glue them together with docker compose files (one per
>>>>>>>>>>>>>>> use
>>>>>>>> case -
>>>>>>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can test
>>>>>> each
>>>>>>> of
>>>>>>>>>> the
>>>>>>>>>>>>>>> components independently and fixing one bug in the field
>>>>>> (say
>>>>>>>>>> DHCP)
>>>>>>>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>>>>> build per-hypervisor VRs. You could even run on baremetal.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>>>>> - make the traffic traverse the containers that are
>>>>>>>>>>>>>>> needed
>>>>>>> to
>>>>>>>> be
>>>>>>>>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>>>>> - bootstrap the router (how does it find its compose file?
>>>>>>>> where
>>>>>>>>>>>>>>> is the
>>>>>>>>>>>>>>> registry?)
>>>>>>>>>>>>>>> - rethink the command and control of the VR
>>>>>>>>>>>>>>> functions. SSH
>>>>>>>> works,
>>>>>>>>>>>>>>> but something more declarative, idempotent should be
>>>>>>> explored.
>>>>>>>>>>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>>>>> functions
>>>>>> can
>>>>>>>> be
>>>>>>>>>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>>>>> Control
>>>>>>> of
>>>>>>>> the
>>>>>>>>>>>>>> docker
>>>>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This one does look nice. My biggest concern is the lack
>>>>>> of
>>>>>>>>>>>>>>>> VXLANs. It seems that any of the ones we mentioned do
>>>>>>>>>>>>>>>> not
>>>>>>>> have
>>>>>>>>>> an
>>>>>>>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>>>>> save
>>>>>>>> future
>>>>>>>>>>>>>>>> engineering effort for example on ipv6 routing, OSPF etc.
>>>>>>>>>>>>>>>> And the best part is they come with test automation
>>>>>>>> framework.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Instead of replacing the VR in first place we should
>>>>>>>>>>>>>>>>> add VyOS/cloudrouter
>>>>>>>>>>>>>>>> as provider. Once it is stable, network offerings (on
>>>>>>>> upgrade)
>>>>>>>>>>>>>>>> can be updated to use it and we can drop the VR if we
>>>>>> want
>>>>>>> at
>>>>>>>>>>>>>>>> that release
>>>>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>>>>> them
>>>>>>> are
>>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>> without issues. When we replicate the ACS VR features
>>>>>>>>>>>>>>>> in
>>>>>>> new
>>>>>>>>>>>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>>>>> (hidden
>>>>>>>> bugs).
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! < nux@li.nux.ro>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too keen
>>>>>> on
>>>>>>>> VyOS
>>>>>>>>>>>>>>>>>> (it
>>>>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>>>>>>> @shapeblue
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
>>>>>>>>>>>>>>>>>>> should
>>>>>>> be
>>>>>>>>>>>>>>>>>>> treated as
>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>> Please weigh in with the good and bad of this idea...
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
>>>>>>>> potentially
>>>>>>>>>>>>>>>>>>> replacing the ACS VR with the VyOS [1] (Open Source
>>>>>>>> Vyatta
>>>>>>>>>>> VM).
>>>>>>>>>>>>>>>>>>> There may be a license issue because I think it is
>>>>>>>> licensed
>>>>>>>>>>>>>>>>>>> under GPL, but for the sake of discussion, let's
>>>>>> assume
>>>>>>>> we
>>>>>>>>>>>>>>>>>>> can overcome any
>>>>>>>>>>>>>>>> license issues.
>>>>>>>>>>>>>>>>>>> I have spent some time recently with the VyOS and I
>>>>>>> have
>>>>>>>> to
>>>>>>>>>>>>>>>>>>> admit, I was pretty impressed. It is simple and
>>>>>>>> intuitive
>>>>>>>>>>>>>>>>>>> and it gives you a lot more options for auditing the
>>>>>>>>>>>> configuration etc...
>>>>>>>>>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
>>>>>> simpler
>>>>>>>> more
>>>>>>>>>>>>>>>>>>> auditable configuration workflow.
>>>>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
>>>>>>>>>>>>>>>>>>> - Handles VPN configuration via the same
>>>>>> configuration
>>>>>>>>>>>> interface.
>>>>>>>>>>>>>>>>>>> - Support for OSPF & BGP.
>>>>>>>>>>>>>>>>>>> - VPN support through OpenVPN & StrongSwan.
>>>>>>>>>>>>>>>>>>> - Easily supports HA (redundant routers) through
>>>>>> VRRP.
>>>>>>>>>>>>>>>>>>> - VXLAN support.
>>>>>>>>>>>>>>>>>>> - Transaction based changes to the VR with rollback
>>>>>> on
>>>>>>>>>> error.
>>>>>>>>>>>>>>>>>>> Items that could be difficult to solve:
>>>>>>>>>>>>>>>>>>> - Userdata password reset workflow and
>>>>>> implementation.
>>>>>>>>>>>>>>>>>>> - Upgrade process.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The VyOS is not the only option if we were to
>>>>>> consider
>>>>>>>> this
>>>>>>>>>>>>>> approach.
>>>>>>>>>>>>>>>>>>> Another option, which I don't know as well, would be
>>>>>>>>>>>>>>>>>>> CloudRouter (AGPL
>>>>>>>>>>>>>>>>>>> license) [2] which is purely API driven.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Anyway, would love to hear your thoughts...
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [1] https://vyos.io/ [2] https://cloudrouter.org/
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> DISCLAIMER
>>>>>>>>>>>>>>>>> ==========
>>>>>>>>>>>>>>>>> This e-mail may contain privileged and confidential
>>>>>>>> information
>>>>>>>>>>>>>>>>> which is
>>>>>>>>>>>>>>>> the property of Accelerite, a Persistent Systems
>>>>>> business.
>>>>>>>> It is
>>>>>>>>>>>>>>>> intended only for the use of the individual or entity
>>>>>>>>>>>>>>>> to
>>>>>>>> which
>>>>>>>>>> it
>>>>>>>>>>>>>>>> is addressed. If you are not the intended recipient,
>>>>>>>>>>>>>>>> you
>>>>>>> are
>>>>>>>> not
>>>>>>>>>>>>>>>> authorized to read, retain, copy, print, distribute or
>>>>>> use
>>>>>>>> this
>>>>>>>>>>>>>>>> message. If you have received this communication in
>>>>>> error,
>>>>>>>>>> please
>>>>>>>>>>>>>>>> notify the sender and delete all copies of this message.
>>>>>>>>>>>>>>>> Accelerite, a Persistent Systems business does not
>>>>>>>>>>>>>>>> accept
>>>>>>> any
>>>>>>>>>>>>>>>> liability for virus
>>>>>>>>>>>>>>> infected mails.
>>
Re: [DISCUSS] Replacing the VR
Posted by Matthew Smart <ms...@smartsoftwareinc.com>.
Will,
I think that would be very helpful to me at least and for posterity for
sure. I am in the process of rolling out my first production deployment
of Cloudstack so I have been busier than expected (plus I have been
jumping back and forth between different offices). What I intended to
propose was that we tackle this using an intermediary step. Instead of
jumping in and replacing the VR wholesale from the start of this
initiative what if we look at integrating Vyos as a plugin in the same
manner as other network offering plugins. Replacing such a core and
vital component of the Cloudstack infrastructure as the VR should be met
with a great deal of caution and with careful thought and planning. Such
a replacement would require a nontrivial amount of effort by core
contributors to be successful and it is questionable whether such an
initiative is the area where they feel their time is best invested.
In my opinion, using the current plugin capabilities of Cloudstack as a
first step would have the benefit of giving the devs a proof of concept
that can be used to evaluate Vyos as a potential replacement for the
current VR without touching the current core networking functionality at
all. Additionally, the code needed to integrate Vyos as a plugin would,
I assume, be directly reusable if the decision is made to go forward
with a Vyos based VR refactor. So we are not duplicating or wasting
effort either. Plus, it would serve as an excellent process during which
to document the network offering plugin architecture.
On the downside to this approach, my understanding would be that the
Vyos deploy in such an environment would have to be external to the
cloudstack ecosystem it supports (either a bare metal install or a non
cloudstack managed VM). Personally, for POC purposes I do not see this
as a huge stumbling block since I expect that anyone who works on
cloudstack can deploy a VM using Virsh or some similar hypervisor interface.
What do you think of that approach? What percentage of the functions
current embodied by the VR are abstracted in such a way as to be
offloadable to a plugin? Would such a plugin be feature complete enough
to represent a proof of concept for a potential Vyos based VR?
Thanks,
Matthew Smart
President
Smart Software Solutions Inc.
108 S Pierre St.
Pierre, SD 57501
Phone: (605) 280-0383
Skype: msmart13
Email: msmart@smartsoftwareinc.com
On 09/26/2016 03:01 PM, Will Stevens wrote:
> I feel like I have squashed this discussion with my potential approach to
> handling this. Maybe we should just pickup this discussion assuming I
> didn't post that. :P
>
> Regarding the docs. I have considered building a stubbed example network
> plugin and then documenting how you would take that stub and build on it.
> Would that be interesting?
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Fri, Sep 23, 2016 at 12:15 AM, Murali Reddy <mu...@gmail.com>
> wrote:
>
>> Matthew,
>>
>> Please see https://cwiki.apache.org/confluence/display/CLOUDSTACK/
>> Extending+CloudStack+Networking
>>
>> Thanks,
>> Murali
>>
>>
>>
>>
>>
>> On 22/09/16, 11:23 PM, "Matthew Smart" <ms...@smartsoftwareinc.com>
>> wrote:
>>
>>> Hey Murali,
>>>
>>> I have been reading through the API and other documentation to try to
>>> get a basic understanding of the network service offering abstraction
>>> methodology in CS. I have not dove into the code yet but before I did I
>>> thought I would try a different approach.
>>>
>>> Imagine I were to come to this list and say that I have a network
>>> offering that I sell and that I wanted to write whatever I needed to in
>>> order to integrate it as an offering in CloudStack. Is there some
>>> specific documentation and guidelines you would direct me to read in
>>> order to gather the knowledge necessary to create a cloudstack
>>> compatible interface for my product?
>>>
>>> I don't know the history but I see several products that have navigated
>>> this process (Nuage, Nicira, ...etc) and am wondering how a new provider
>>> would work with you guys to navigate that process. If this is too vague,
>>> we can pretend my new offering is a hardware firewall device.
>>>
>>> My goal here is to gain an understanding of how CS interacts with third
>>> party offerings underneath the hood. I have some thoughts (I think
>>> inline with Will Steven's brain dump and diagram) but want to make sure
>>> I am not suffering some misapprehensions about the architecture and,
>>> short of tracing code, was not successful at finding the information I
>>> needed to satisfy myself that I know how it is designed.
>>>
>>> Thanks,
>>>
>>> Matthew Smart
>>> President
>>> Smart Software Solutions Inc.
>>> 108 S Pierre St.
>>> Pierre, SD 57501
>>>
>>> Phone: (605) 280-0383
>>> Skype: msmart13
>>> Email: msmart@smartsoftwareinc.com
>>>
>>> On 09/20/2016 04:54 AM, Murali Reddy wrote:
>>>> Configuration management of network appliances particularly for Cloud
>> and NFV scenarios is still evolving area. Programmability is the not the
>> strength for even the most popular network operating systems like IOS,
>> JunoS etc. So its not surprising why CloudStack integrates in a archaic way
>> with stock linux for the VR.
>>>> VR was never integral/hardcoded option in CloudStack. Its always been a
>> plugin. CloudStack network orchestration is well abstracted and designed
>> with vision to compose a network with different set of providers for
>> different services. Yes that vision is not fully realised yet, and we don\u2019t
>> have true service function chains. That would be different discussion topic.
>>>> I tend to agree with Simon, as alternate/interim option we can take
>> hard look whats causing the problems with current VR integration.
>> Personally, I think it would be easier we take a cue from configuration
>> managers and network configuration solutions out there (for e.g promise
>> theory based Cisco ACI) move to more declarative model of expressing
>> desired state of network configuration. Infact current VR from 4.6,
>> actually holds the desired state per service basis, seems to be in that
>> direction.
>>>> It does make sense to evaluate new appliances which can provide rich
>> semantics (like programmable API, declarative configuration, versioning,
>> commit/rollback etc), but will need significant engineering effort and time
>> to stabilise. We may make same mistakes with integration of other appliance
>> as well, if we fully don\u2019t understand the nature of the current problems
>> with CloudStack core and service provider interaction and current VR
>> integration.
>>>>
>>>> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>>>>
>>>>> I think our other option is to take a real look at what it would take
>> to fix the VR. In my opinion, a lot of the problems are related to the
>> monolithic python code base and the fact nothing is actually separated.
>>>>> Secondly, the python scripts (and bash scripts) don't use any
>> established libraries to complete tasks and instead shell out and run
>> commands that are both hard to track and hard to parse on return.
>>>>>
>>>>> If we daemonized this, used a real api for Agent to VR communication,
>> used common already existing libraries for the system service and network
>> interactions and spent a bit of time separating out code into distinct
>> modules, everything would behave a lot better.
>>>>>
>>>>> The pain and suffering is due to years and years of patches and
>> constant shelling out to complete tasks in my opinion. If we spend time to
>> rethink how we interact with the VR in general and we abstract the systems
>> and networking stuff and use well known and stable libraries to do the
>> work, the VR would be much easier to maintain.
>>>>>
>>>>> - Si
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ________________________________
>>>>> From: Marty Godsey <ma...@gonsource.com>
>>>>> Sent: Friday, September 16, 2016 12:24 PM
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>
>>>>> So based upon this discussion would it be prudent to wait on VyOS 2.0?
>> The current VR is giving us issues but would the time invested in another
>> "solution" be wasted especially if by the time another option is chose,
>> then coded, then tested, then implemented and right as that time happened
>> to be when VyOS 2.0 is released. Of course you said they are just in the
>> scoping range so this could still be a year or more out.
>>>>> Thoughts?
>>>>>
>>>>> Regards,
>>>>> Marty Godsey
>>>>> nSource Solutions
>>>>>
>>>>> -----Original Message-----
>>>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>> Behalf Of Will Stevens
>>>>> Sent: Friday, September 16, 2016 10:31 AM
>>>>> To: dev@cloudstack.apache.org
>>>>> Cc: daniil@baturin.org
>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>
>>>>> I just had a quick chat with a couple of the guys over on the VyOS
>> chat.
>>>>> I have CC'ed one of them in case we have more licensing questions.
>>>>>
>>>>> So here is the status with the license "the code inherited from Vyatta
>> and our modifications from it is GPLv2 (strict, not v2+). The config
>> reading library is GPLv2 too, so anything that links to is is GPLv2.
>>>>> Some auxiliary components we made after the fork are more permissive,
>>>>> LGPLv2+ or MIT."
>>>>>
>>>>> They are currently in the process of scoping a redesign (VyOS 2.0),
>> "we are planning a clean rewrite that will solve issues of the current
>> config system".
>>>>> This will include the ability to configure via the API.
>>>>>
>>>>> If we have more questions for VyOS, they are very friendly and
>> responsive, so we should be able to get answers.
>>>>> *Will STEVENS*
>>>>> Lead Developer
>>>>>
>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>> @CloudOps_
>>>>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com>
>> wrote:
>>>>>> I agree with Will Ilya. There are so many problems with the VR right
>> now.
>>>>>> Most of the outages we've had recently have somehow involved the VR.
>>>>>> We set custom iptables rules on the VR which can and have easily gone
>> wrong.
>>>>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>>>>> VVRP with redundant router still needs work, and not to mention the
>>>>>> problems we will have when we introduce IPv6 into the whole picture.
>>>>>>
>>>>>> I think the spirit of the discussion is to rely on a 3rd party to do
>>>>>> the networking for us (eg VyOS) and have us handle just the
>>>>>> orchestration. All the problems that I've described have already been
>>>>>> solved in VyOS. We also get the advantage of a potential wider
>>>>>> community to fix and maintain the VR and given our current development
>>>>>> velocity, it think it totally makes sense to look for a 3rd party
>> option.
>>>>>> -Syed
>>>>>>
>>>>>>
>>>>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
>>>>>> wrote:
>>>>>>
>>>>>>> The VR has been biting us far too often recently, which is why we
>>>>>>> have started looking into alternative implementations.
>>>>>>>
>>>>>>> One of the things that is nice about potentially using the VyOS is
>>>>>>> that
>>>>>> it
>>>>>>> is based on Debian, so we should be able to run the other services
>>>>>>> that
>>>>>> we
>>>>>>> currently have like the password server and userdata on the VyOS.
>>>>>>> This means we would not have to change our architecture initially
>>>>>>> and could focus on only replacing the networking paths.
>>>>>>>
>>>>>>> *Will STEVENS*
>>>>>>> Lead Developer
>>>>>>>
>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>>>>> tw @CloudOps_
>>>>>>>
>>>>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>>>>
>>>>>>>> The more this is discussed the more I think we should stick with
>>>>>>>> our
>>>>>> VR.
>>>>>>>> All these other options either seem unfinished or with
>>>>>>>> incompatible license.
>>>>>>>>
>>>>>>>> VyOS looks the most promising so far, it's a serious, mature
>> project.
>>>>>>>> Adopting it though means we'll have to microservice our way out of
>>>>>>>> it
>>>>>>> with
>>>>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>>>>>> those
>>>>>>>> too. Imho this adds complexity we should void.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>
>>>>>>>> Nux!
>>>>>>>> www.nux.ro
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>> Ya, we would need to add a daemon for VPN as well. Load
>>>>>>>>> balancing is another aspect which we will need to consider if we
>> went this route.
>>>>>>>>> Something like https://traefik.io/ could potentially be a good
>>>>>>>>> fit
>>>>>> due
>>>>>>>> to
>>>>>>>>> its API driven configuration, but it may be more than what we need.
>>>>>>>>>
>>>>>>>>> We should probably try define which pieces make sense to be
>>>>>>>>> solved
>>>>>>>> together
>>>>>>>>> and which pieces would be best suited to be broken out.
>>>>>>>>>
>>>>>>>>> I think the network connectivity, routing and firewalling should
>>>>>>> probably
>>>>>>>>> all stay together since the majority of the tools we would
>>>>>> potentially
>>>>>>>> use
>>>>>>>>> would handle all of that together in a single implementation.
>>>>>>>>>
>>>>>>>>> The password server and userdata seems like a good option for
>>>>>>>>> being
>>>>>>>> broken
>>>>>>>>> out and handled independently (and probably rewritten completely
>>>>>> since
>>>>>>>> they
>>>>>>>>> currently have some issues).
>>>>>>>>>
>>>>>>>>> Load balancing is another that could warrant splitting out, but
>>>>>>>>> that depends on what direction we go and how we would be managing
>> it.
>>>>>> DHCP
>>>>>>>> and
>>>>>>>>> DNS are others which could go either way.
>>>>>>>>>
>>>>>>>>> If we do split out services, I think we should consolidate as
>>>>>>>>> much as
>>>>>>> we
>>>>>>>>> can into each service we break out. Ideally a network packet
>>>>>>>>> would
>>>>>>> never
>>>>>>>>> hit more than one, maybe two, services. I don't think we should
>>>>>>>>> be splitting services 'just because', I think we need a valid
>>>>>>>>> case for splitting any service out because it adds complexity.
>>>>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>>>>> complexity unless it
>>>>>> is
>>>>>>>>> really needed.
>>>>>>>>>
>>>>>>>>> Some more of my thoughts on this anyway...
>>>>>>>>>
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>
>>>>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>>>>>>> wrote:
>>>>>>>>>> I do agree with you that this probably isn't the right place
>>>>>>>>>> the
>>>>>>>> password
>>>>>>>>>> service and user data.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Having said that, after taking a cursory look at the dev docs,
>>>>>>>>>> it
>>>>>>>> doesn't
>>>>>>>>>> seem that difficult to add new daemons:
>>>>>> https://opensnaproute.github.
>>>>>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>>>>
>>>>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>>>>> creating-new-component>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> They've definitely build it with a microservices architecture
>>>>>>>>>> in
>>>>>> mind,
>>>>>>>> so
>>>>>>>>>> each individual feature is abstracted into it's own small
>>>>>>>>>> daemon
>>>>>>>> process.
>>>>>>>>>> We could just create a daemon for the password server and the
>>>>>> userdata
>>>>>>>>>> components if we really had to.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Si
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ________________________________
>>>>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
>>>>>>>>>> behalf
>>>>>>> of
>>>>>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>
>>>>>>>>>> A big part of why I know about it is because it is written in Go.
>>>>>> :P
>>>>>>>>>> Yes, it is definitely interesting for the routing and traffic
>>>>>> handling
>>>>>>>>>> aspects of the VR. We will likely have to rethink some of the
>>>>>> pieces
>>>>>>> a
>>>>>>>>>> little bit like the password server and userdata if we are to
>>>>>>>>>> adopt
>>>>>> a
>>>>>>>>>> different VR approach. This is where I think some of JohnB and
>>>>>>>> Chiradeep's
>>>>>>>>>> ideas make sense. In many ways, it does not make sense for the
>>>>>> device
>>>>>>>>>> handling routing and network traffic to also be responsible for
>>>>>>>> passwords
>>>>>>>>>> and userdata.
>>>>>>>>>>
>>>>>>>>>> *Will STEVENS*
>>>>>>>>>> Lead Developer
>>>>>>>>>>
>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>>>>>>> wrote:
>>>>>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
>>>>>> pretty
>>>>>>>>>> cool!
>>>>>>>>>>> It even supports ONIE install.
>>>>>>>>>>>
>>>>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>>>>> offload
>>>>>>>> it to
>>>>>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>>>>> from a
>>>>>>>>>> routing
>>>>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Based on the roadmap, it looks like they plan to also support
>>>>>>>>>> capabilities
>>>>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>>>>> will
>>>>>> be
>>>>>>>> huge
>>>>>>>>>>> for our carrier community that rely on these technologies to
>>>>>>>>>>> do
>>>>>>>> private
>>>>>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>>>>> stuff
>>>>>>> on
>>>>>>>> our
>>>>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
>>>>>>>>>>> able to
>>>>>>> do
>>>>>>>>>> MPLS
>>>>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>>>>> know
>>>>>>> very
>>>>>>>>>>> well).
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> - Si
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ________________________________
>>>>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>>>>
>>>>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>>>>> does
>>>>>>>> cover is
>>>>>>>>>>> all api driven...
>>>>>>>>>>>
>>>>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>>>>>>> wrote:
>>>>>>>>>>>> Though I don\u2019t see VPN in Snaproute.. Makes sense since it
>>>>>>>>>>>> was
>>>>>> not
>>>>>>>>>>>> intended to do IPSec.
>>>>>>>>>>>>
>>>>>>>>>>>> It seems as though VyOS is starting to look like the best
>>>>>> option.
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>>>>> [mailto:williamstevens@gmail.com
>>>>>> ]
>>>>>>> On
>>>>>>>>>>>> Behalf Of Will Stevens
>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>
>>>>>>>>>>>> Or we could go completely crazy and go with something like
>>>>>>>> FlexSwitch
>>>>>>>>>>> from
>>>>>>>>>>>> SnapRoute
>>>>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>>>>
>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>
>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>>> cloudops.com
>>>>>>> *|*
>>>>>>>> tw
>>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>>>>> wstevens@cloudops.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
>>>>>>> problems
>>>>>>>> are
>>>>>>>>>>>>> solved by splitting up the function of the VR into a
>>>>>>>>>>>>> bunch of
>>>>>>>>>> separate
>>>>>>>>>>>>> services. As Syed points out, the complexity added is
>>>>>>>> non-trivial.
>>>>>>>>>>>>> We now have to manage all the intercontainer networking
>>>>>>>>>>>>> as
>>>>>> well
>>>>>>> as
>>>>>>>>>> the
>>>>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>>>>
>>>>>>>>>>>>> VyOS is interesting to me because it covers the majority
>>>>>>>>>>>>> of
>>>>>> our
>>>>>>>> use
>>>>>>>>>>>>> case with a single unified control plane. It also has
>>>>>>>>>>>>> good
>>>>>>>> support
>>>>>>>>>>>>> for extending features we care about, like IPv6, VXLAN,
>>>>>>>>>>>>> VRRP, transactions, etc...
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>>
>>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>> cloudops.com
>>>>>>>> *|*
>>>>>>>>>> tw
>>>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>>>>> sahmed@cloudops.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> Agree with Marty, adding Docker containers to the
>>>>>>>>>>>>>> picture
>>>>>>>> although
>>>>>>>>>>>>>> can make the VR more flexible but the added complexity
>>>>>>>>>>>>>> is
>>>>>> just
>>>>>>>> not
>>>>>>>>>>>>>> worth it. Not to mention we would need to take care of
>>>>>>> networking
>>>>>>>>>>>>>> each container manually and given that our iptable rules
>>>>>>>>>>>>>> are
>>>>>>> very
>>>>>>>>>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
>>>>>>>>>>>>>> does
>>>>>>> not
>>>>>>>>>>>>>> provide an api but it does fit the bill quite well
>>>>>> otherwise. I
>>>>>>>>>>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>>>>> model
>>>>>>> and
>>>>>>>>>> you
>>>>>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>>>>> marty@gonsource.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> Licensing aside, I think splitting the various
>>>>>>>>>>>>>>> functions
>>>>>> into
>>>>>>>>>>>>>>> containers is not a good route either. This will force
>>>>>> users
>>>>>>> to
>>>>>>>>>>>>>>> have to maintain
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> use containers and adds complexity to the networking
>>>>>> aspects
>>>>>>> of
>>>>>>>>>> ACS.
>>>>>>>>>>>>>>> Complexity decreases stability. Now I understand the
>>>>>> argument
>>>>>>>> that
>>>>>>>>>>>>>>> a monolithic approach also brings its own set of
>>>>>>>>>>>>>>> issues but
>>>>>>> it
>>>>>>>>>> also
>>>>>>>>>>>>>>> simplifies it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
>>>>>>>>>>>>>>> of
>>>>>> the
>>>>>>>>>>>>>>> CloudStack project
>>>>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
>>>>>>>>>>>>>>> touch
>>>>>>>> anything
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>> AGPL
>>>>>>>>>>>>>>> - the github repo shows rather infrequent updates.
>>>>>>>>>>>>>>> Quite
>>>>>>>> likely
>>>>>>>>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>>>>> CloudStack
>>>>>>>> community
>>>>>>>>>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>>>>> Split
>>>>>> it
>>>>>>>> into
>>>>>>>>>>>>>>> many docker containers
>>>>>>>>>>>>>>> - password server
>>>>>>>>>>>>>>> - userdata server
>>>>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>>>>> - FW
>>>>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>>>>> - agent
>>>>>>>>>>>>>>> Glue them together with docker compose files (one per
>>>>>>>>>>>>>>> use
>>>>>>>> case -
>>>>>>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
>>>>>>>>>>>>>>> test
>>>>>> each
>>>>>>> of
>>>>>>>>>> the
>>>>>>>>>>>>>>> components independently and fixing one bug in the
>>>>>>>>>>>>>>> field
>>>>>> (say
>>>>>>>>>> DHCP)
>>>>>>>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>>>>> build per-hypervisor VRs. You could even run on baremetal.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>>>>> - make the traffic traverse the containers that are
>>>>>>>>>>>>>>> needed
>>>>>>> to
>>>>>>>> be
>>>>>>>>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>>>>> - bootstrap the router (how does it find its compose file?
>>>>>>>> where
>>>>>>>>>>>>>>> is the
>>>>>>>>>>>>>>> registry?)
>>>>>>>>>>>>>>> - rethink the command and control of the VR
>>>>>>>>>>>>>>> functions. SSH
>>>>>>>> works,
>>>>>>>>>>>>>>> but something more declarative, idempotent should be
>>>>>>> explored.
>>>>>>>>>>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>>>>> functions
>>>>>> can
>>>>>>>> be
>>>>>>>>>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>>>>> Control
>>>>>>> of
>>>>>>>> the
>>>>>>>>>>>>>> docker
>>>>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This one does look nice. My biggest concern is the
>>>>>>>>>>>>>>>> lack
>>>>>> of
>>>>>>>>>>>>>>>> VXLANs. It seems that any of the ones we mentioned
>>>>>>>>>>>>>>>> do not
>>>>>>>> have
>>>>>>>>>> an
>>>>>>>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>>>>> save
>>>>>>>> future
>>>>>>>>>>>>>>>> engineering effort for example on ipv6 routing, OSPF etc.
>>>>>>>>>>>>>>>> And the best part is they come with test automation
>>>>>>>> framework.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Instead of replacing the VR in first place we
>>>>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>>>>> as provider. Once it is stable, network offerings
>>>>>>>>>>>>>>>> (on
>>>>>>>> upgrade)
>>>>>>>>>>>>>>>> can be updated to use it and we can drop the VR if
>>>>>>>>>>>>>>>> we
>>>>>> want
>>>>>>> at
>>>>>>>>>>>>>>>> that release
>>>>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>>>>> them
>>>>>>> are
>>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>> without issues. When we replicate the ACS VR
>>>>>>>>>>>>>>>> features in
>>>>>>> new
>>>>>>>>>>>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>>>>> (hidden
>>>>>>>> bugs).
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
>>>>>>>>>>>>>>>>>> keen
>>>>>> on
>>>>>>>> VyOS
>>>>>>>>>>>>>>>>>> (it
>>>>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>>>>>>> @shapeblue
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
>>>>>>>>>>>>>>>>>>> should
>>>>>>> be
>>>>>>>>>>>>>>>>>>> treated as
>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>> Please weigh in with the good and bad of this idea...
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
>>>>>>>> potentially
>>>>>>>>>>>>>>>>>>> replacing the ACS VR with the VyOS [1] (Open
>>>>>>>>>>>>>>>>>>> Source
>>>>>>>> Vyatta
>>>>>>>>>>> VM).
>>>>>>>>>>>>>>>>>>> There may be a license issue because I think it
>>>>>>>>>>>>>>>>>>> is
>>>>>>>> licensed
>>>>>>>>>>>>>>>>>>> under GPL, but for the sake of discussion, let's
>>>>>> assume
>>>>>>>> we
>>>>>>>>>>>>>>>>>>> can overcome any
>>>>>>>>>>>>>>>> license issues.
>>>>>>>>>>>>>>>>>>> I have spent some time recently with the VyOS
>>>>>>>>>>>>>>>>>>> and I
>>>>>>> have
>>>>>>>> to
>>>>>>>>>>>>>>>>>>> admit, I was pretty impressed. It is simple and
>>>>>>>> intuitive
>>>>>>>>>>>>>>>>>>> and it gives you a lot more options for auditing
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>> configuration etc...
>>>>>>>>>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
>>>>>> simpler
>>>>>>>> more
>>>>>>>>>>>>>>>>>>> auditable configuration workflow.
>>>>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
>>>>>>>>>>>>>>>>>>> - Handles VPN configuration via the same
>>>>>> configuration
>>>>>>>>>>>> interface.
>>>>>>>>>>>>>>>>>>> - Support for OSPF & BGP.
>>>>>>>>>>>>>>>>>>> - VPN support through OpenVPN & StrongSwan.
>>>>>>>>>>>>>>>>>>> - Easily supports HA (redundant routers) through
>>>>>> VRRP.
>>>>>>>>>>>>>>>>>>> - VXLAN support.
>>>>>>>>>>>>>>>>>>> - Transaction based changes to the VR with
>>>>>>>>>>>>>>>>>>> rollback
>>>>>> on
>>>>>>>>>> error.
>>>>>>>>>>>>>>>>>>> Items that could be difficult to solve:
>>>>>>>>>>>>>>>>>>> - Userdata password reset workflow and
>>>>>> implementation.
>>>>>>>>>>>>>>>>>>> - Upgrade process.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The VyOS is not the only option if we were to
>>>>>> consider
>>>>>>>> this
>>>>>>>>>>>>>> approach.
>>>>>>>>>>>>>>>>>>> Another option, which I don't know as well,
>>>>>>>>>>>>>>>>>>> would be CloudRouter (AGPL
>>>>>>>>>>>>>>>>>>> license) [2] which is purely API driven.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Anyway, would love to hear your thoughts...
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [1] https://vyos.io/ [2]
>>>>>>>>>>>>>>>>>>> https://cloudrouter.org/
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> DISCLAIMER
>>>>>>>>>>>>>>>>> ==========
>>>>>>>>>>>>>>>>> This e-mail may contain privileged and confidential
>>>>>>>> information
>>>>>>>>>>>>>>>>> which is
>>>>>>>>>>>>>>>> the property of Accelerite, a Persistent Systems
>>>>>> business.
>>>>>>>> It is
>>>>>>>>>>>>>>>> intended only for the use of the individual or
>>>>>>>>>>>>>>>> entity to
>>>>>>>> which
>>>>>>>>>> it
>>>>>>>>>>>>>>>> is addressed. If you are not the intended recipient,
>>>>>>>>>>>>>>>> you
>>>>>>> are
>>>>>>>> not
>>>>>>>>>>>>>>>> authorized to read, retain, copy, print, distribute
>>>>>>>>>>>>>>>> or
>>>>>> use
>>>>>>>> this
>>>>>>>>>>>>>>>> message. If you have received this communication in
>>>>>> error,
>>>>>>>>>> please
>>>>>>>>>>>>>>>> notify the sender and delete all copies of this message.
>>>>>>>>>>>>>>>> Accelerite, a Persistent Systems business does not
>>>>>>>>>>>>>>>> accept
>>>>>>> any
>>>>>>>>>>>>>>>> liability for virus
>>>>>>>>>>>>>>> infected mails.
>>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
I feel like I have squashed this discussion with my potential approach to
handling this. Maybe we should just pickup this discussion assuming I
didn't post that. :P
Regarding the docs. I have considered building a stubbed example network
plugin and then documenting how you would take that stub and build on it.
Would that be interesting?
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Fri, Sep 23, 2016 at 12:15 AM, Murali Reddy <mu...@gmail.com>
wrote:
> Matthew,
>
> Please see https://cwiki.apache.org/confluence/display/CLOUDSTACK/
> Extending+CloudStack+Networking
>
> Thanks,
> Murali
>
>
>
>
>
> On 22/09/16, 11:23 PM, "Matthew Smart" <ms...@smartsoftwareinc.com>
> wrote:
>
> >Hey Murali,
> >
> >I have been reading through the API and other documentation to try to
> >get a basic understanding of the network service offering abstraction
> >methodology in CS. I have not dove into the code yet but before I did I
> >thought I would try a different approach.
> >
> >Imagine I were to come to this list and say that I have a network
> >offering that I sell and that I wanted to write whatever I needed to in
> >order to integrate it as an offering in CloudStack. Is there some
> >specific documentation and guidelines you would direct me to read in
> >order to gather the knowledge necessary to create a cloudstack
> >compatible interface for my product?
> >
> >I don't know the history but I see several products that have navigated
> >this process (Nuage, Nicira, ...etc) and am wondering how a new provider
> >would work with you guys to navigate that process. If this is too vague,
> >we can pretend my new offering is a hardware firewall device.
> >
> >My goal here is to gain an understanding of how CS interacts with third
> >party offerings underneath the hood. I have some thoughts (I think
> >inline with Will Steven's brain dump and diagram) but want to make sure
> >I am not suffering some misapprehensions about the architecture and,
> >short of tracing code, was not successful at finding the information I
> >needed to satisfy myself that I know how it is designed.
> >
> >Thanks,
> >
> >Matthew Smart
> >President
> >Smart Software Solutions Inc.
> >108 S Pierre St.
> >Pierre, SD 57501
> >
> >Phone: (605) 280-0383
> >Skype: msmart13
> >Email: msmart@smartsoftwareinc.com
> >
> >On 09/20/2016 04:54 AM, Murali Reddy wrote:
> >> Configuration management of network appliances particularly for Cloud
> and NFV scenarios is still evolving area. Programmability is the not the
> strength for even the most popular network operating systems like IOS,
> JunoS etc. So its not surprising why CloudStack integrates in a archaic way
> with stock linux for the VR.
> >>
> >> VR was never integral/hardcoded option in CloudStack. Its always been a
> plugin. CloudStack network orchestration is well abstracted and designed
> with vision to compose a network with different set of providers for
> different services. Yes that vision is not fully realised yet, and we don’t
> have true service function chains. That would be different discussion topic.
> >>
> >> I tend to agree with Simon, as alternate/interim option we can take
> hard look whats causing the problems with current VR integration.
> Personally, I think it would be easier we take a cue from configuration
> managers and network configuration solutions out there (for e.g promise
> theory based Cisco ACI) move to more declarative model of expressing
> desired state of network configuration. Infact current VR from 4.6,
> actually holds the desired state per service basis, seems to be in that
> direction.
> >>
> >> It does make sense to evaluate new appliances which can provide rich
> semantics (like programmable API, declarative configuration, versioning,
> commit/rollback etc), but will need significant engineering effort and time
> to stabilise. We may make same mistakes with integration of other appliance
> as well, if we fully don’t understand the nature of the current problems
> with CloudStack core and service provider interaction and current VR
> integration.
> >>
> >>
> >> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
> >>
> >>> I think our other option is to take a real look at what it would take
> to fix the VR. In my opinion, a lot of the problems are related to the
> monolithic python code base and the fact nothing is actually separated.
> >>>
> >>> Secondly, the python scripts (and bash scripts) don't use any
> established libraries to complete tasks and instead shell out and run
> commands that are both hard to track and hard to parse on return.
> >>>
> >>>
> >>> If we daemonized this, used a real api for Agent to VR communication,
> used common already existing libraries for the system service and network
> interactions and spent a bit of time separating out code into distinct
> modules, everything would behave a lot better.
> >>>
> >>>
> >>> The pain and suffering is due to years and years of patches and
> constant shelling out to complete tasks in my opinion. If we spend time to
> rethink how we interact with the VR in general and we abstract the systems
> and networking stuff and use well known and stable libraries to do the
> work, the VR would be much easier to maintain.
> >>>
> >>>
> >>> - Si
> >>>
> >>>
> >>>
> >>>
> >>> ________________________________
> >>> From: Marty Godsey <ma...@gonsource.com>
> >>> Sent: Friday, September 16, 2016 12:24 PM
> >>> To: dev@cloudstack.apache.org
> >>> Subject: RE: [DISCUSS] Replacing the VR
> >>>
> >>> So based upon this discussion would it be prudent to wait on VyOS 2.0?
> The current VR is giving us issues but would the time invested in another
> "solution" be wasted especially if by the time another option is chose,
> then coded, then tested, then implemented and right as that time happened
> to be when VyOS 2.0 is released. Of course you said they are just in the
> scoping range so this could still be a year or more out.
> >>>
> >>> Thoughts?
> >>>
> >>> Regards,
> >>> Marty Godsey
> >>> nSource Solutions
> >>>
> >>> -----Original Message-----
> >>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> >>> Sent: Friday, September 16, 2016 10:31 AM
> >>> To: dev@cloudstack.apache.org
> >>> Cc: daniil@baturin.org
> >>> Subject: Re: [DISCUSS] Replacing the VR
> >>>
> >>> I just had a quick chat with a couple of the guys over on the VyOS
> chat.
> >>> I have CC'ed one of them in case we have more licensing questions.
> >>>
> >>> So here is the status with the license "the code inherited from Vyatta
> and our modifications from it is GPLv2 (strict, not v2+). The config
> reading library is GPLv2 too, so anything that links to is is GPLv2.
> >>> Some auxiliary components we made after the fork are more permissive,
> >>> LGPLv2+ or MIT."
> >>>
> >>> They are currently in the process of scoping a redesign (VyOS 2.0),
> "we are planning a clean rewrite that will solve issues of the current
> config system".
> >>> This will include the ability to configure via the API.
> >>>
> >>> If we have more questions for VyOS, they are very friendly and
> responsive, so we should be able to get answers.
> >>>
> >>> *Will STEVENS*
> >>> Lead Developer
> >>>
> >>> *CloudOps* *| *Cloud Solutions Experts
> >>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
> >>>
> >>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com>
> wrote:
> >>>
> >>>> I agree with Will Ilya. There are so many problems with the VR right
> now.
> >>>> Most of the outages we've had recently have somehow involved the VR.
> >>>> We set custom iptables rules on the VR which can and have easily gone
> wrong.
> >>>> Openswan is broken, Strongswan replacement still needs to be tested.
> >>>> VVRP with redundant router still needs work, and not to mention the
> >>>> problems we will have when we introduce IPv6 into the whole picture.
> >>>>
> >>>> I think the spirit of the discussion is to rely on a 3rd party to do
> >>>> the networking for us (eg VyOS) and have us handle just the
> >>>> orchestration. All the problems that I've described have already been
> >>>> solved in VyOS. We also get the advantage of a potential wider
> >>>> community to fix and maintain the VR and given our current development
> >>>> velocity, it think it totally makes sense to look for a 3rd party
> option.
> >>>>
> >>>> -Syed
> >>>>
> >>>>
> >>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
> >>>> wrote:
> >>>>
> >>>>> The VR has been biting us far too often recently, which is why we
> >>>>> have started looking into alternative implementations.
> >>>>>
> >>>>> One of the things that is nice about potentially using the VyOS is
> >>>>> that
> >>>> it
> >>>>> is based on Debian, so we should be able to run the other services
> >>>>> that
> >>>> we
> >>>>> currently have like the password server and userdata on the VyOS.
> >>>>> This means we would not have to change our architecture initially
> >>>>> and could focus on only replacing the networking paths.
> >>>>>
> >>>>> *Will STEVENS*
> >>>>> Lead Developer
> >>>>>
> >>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> >>>>> tw @CloudOps_
> >>>>>
> >>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> >>>>>
> >>>>>> The more this is discussed the more I think we should stick with
> >>>>>> our
> >>>> VR.
> >>>>>> All these other options either seem unfinished or with
> >>>>>> incompatible license.
> >>>>>>
> >>>>>> VyOS looks the most promising so far, it's a serious, mature
> project.
> >>>>>> Adopting it though means we'll have to microservice our way out of
> >>>>>> it
> >>>>> with
> >>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
> >>>> those
> >>>>>> too. Imho this adds complexity we should void.
> >>>>>>
> >>>>>> --
> >>>>>> Sent from the Delta quadrant using Borg technology!
> >>>>>>
> >>>>>> Nux!
> >>>>>> www.nux.ro
> >>>>>>
> >>>>>> ----- Original Message -----
> >>>>>>> From: "Will Stevens" <ws...@cloudops.com>
> >>>>>>> To: dev@cloudstack.apache.org
> >>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
> >>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>> Ya, we would need to add a daemon for VPN as well. Load
> >>>>>>> balancing is another aspect which we will need to consider if we
> went this route.
> >>>>>>> Something like https://traefik.io/ could potentially be a good
> >>>>>>> fit
> >>>> due
> >>>>>> to
> >>>>>>> its API driven configuration, but it may be more than what we need.
> >>>>>>>
> >>>>>>> We should probably try define which pieces make sense to be
> >>>>>>> solved
> >>>>>> together
> >>>>>>> and which pieces would be best suited to be broken out.
> >>>>>>>
> >>>>>>> I think the network connectivity, routing and firewalling should
> >>>>> probably
> >>>>>>> all stay together since the majority of the tools we would
> >>>> potentially
> >>>>>> use
> >>>>>>> would handle all of that together in a single implementation.
> >>>>>>>
> >>>>>>> The password server and userdata seems like a good option for
> >>>>>>> being
> >>>>>> broken
> >>>>>>> out and handled independently (and probably rewritten completely
> >>>> since
> >>>>>> they
> >>>>>>> currently have some issues).
> >>>>>>>
> >>>>>>> Load balancing is another that could warrant splitting out, but
> >>>>>>> that depends on what direction we go and how we would be managing
> it.
> >>>> DHCP
> >>>>>> and
> >>>>>>> DNS are others which could go either way.
> >>>>>>>
> >>>>>>> If we do split out services, I think we should consolidate as
> >>>>>>> much as
> >>>>> we
> >>>>>>> can into each service we break out. Ideally a network packet
> >>>>>>> would
> >>>>> never
> >>>>>>> hit more than one, maybe two, services. I don't think we should
> >>>>>>> be splitting services 'just because', I think we need a valid
> >>>>>>> case for splitting any service out because it adds complexity.
> >>>>>>> Our project is already complex enough, we need to avoid adding
> >>>>>>> complexity unless it
> >>>> is
> >>>>>>> really needed.
> >>>>>>>
> >>>>>>> Some more of my thoughts on this anyway...
> >>>>>>>
> >>>>>>> *Will STEVENS*
> >>>>>>> Lead Developer
> >>>>>>>
> >>>>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> >>>>>>> *|* tw @CloudOps_
> >>>>>>>
> >>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
> >>>>> wrote:
> >>>>>>>> I do agree with you that this probably isn't the right place
> >>>>>>>> the
> >>>>>> password
> >>>>>>>> service and user data.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Having said that, after taking a cursory look at the dev docs,
> >>>>>>>> it
> >>>>>> doesn't
> >>>>>>>> seem that difficult to add new daemons:
> >>>> https://opensnaproute.github.
> >>>>>>>> io/docs/developer.html#creating-new-component
> >>>>>>>>
> >>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
> >>>>>>>> creating-new-component>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> They've definitely build it with a microservices architecture
> >>>>>>>> in
> >>>> mind,
> >>>>>> so
> >>>>>>>> each individual feature is abstracted into it's own small
> >>>>>>>> daemon
> >>>>>> process.
> >>>>>>>> We could just create a daemon for the password server and the
> >>>> userdata
> >>>>>>>> components if we really had to.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> - Si
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> ________________________________
> >>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
> >>>>>>>> behalf
> >>>>> of
> >>>>>>>> Will Stevens <ws...@cloudops.com>
> >>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
> >>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>>
> >>>>>>>> A big part of why I know about it is because it is written in Go.
> >>>> :P
> >>>>>>>> Yes, it is definitely interesting for the routing and traffic
> >>>> handling
> >>>>>>>> aspects of the VR. We will likely have to rethink some of the
> >>>> pieces
> >>>>> a
> >>>>>>>> little bit like the password server and userdata if we are to
> >>>>>>>> adopt
> >>>> a
> >>>>>>>> different VR approach. This is where I think some of JohnB and
> >>>>>> Chiradeep's
> >>>>>>>> ideas make sense. In many ways, it does not make sense for the
> >>>> device
> >>>>>>>> handling routing and network traffic to also be responsible for
> >>>>>> passwords
> >>>>>>>> and userdata.
> >>>>>>>>
> >>>>>>>> *Will STEVENS*
> >>>>>>>> Lead Developer
> >>>>>>>>
> >>>>>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> >>>>>>>> *|* tw @CloudOps_
> >>>>>>>>
> >>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
> >>>>> wrote:
> >>>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
> >>>> pretty
> >>>>>>>> cool!
> >>>>>>>>> It even supports ONIE install.
> >>>>>>>>>
> >>>>>>>>> To be honest, the ipsec feature could be added, or we could
> >>>> offload
> >>>>>> it to
> >>>>>>>>> separate vm if we needed to. The fact it is so feature rich
> >>>>>>>>> from a
> >>>>>>>> routing
> >>>>>>>>> perspective (and all API driven) is really nice.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Based on the roadmap, it looks like they plan to also support
> >>>>>>>> capabilities
> >>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
> >>>>>>>>> will
> >>>> be
> >>>>>> huge
> >>>>>>>>> for our carrier community that rely on these technologies to
> >>>>>>>>> do
> >>>>>> private
> >>>>>>>>> gateway and inter-VPC interconnections today. We handle this
> >>>>>>>>> stuff
> >>>>> on
> >>>>>> our
> >>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
> >>>>>>>>> able to
> >>>>> do
> >>>>>>>> MPLS
> >>>>>>>>> all the way to the VR would be awesome.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> It also seems to be written in GO (a language here at ENA we
> >>>>>>>>> know
> >>>>> very
> >>>>>>>>> well).
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> - Si
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> ________________________________
> >>>>>>>>> From: Will Stevens <wi...@gmail.com>
> >>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
> >>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
> >>>>>>>>>
> >>>>>>>>> Ya. I don't think it covers our whole use case, but what it
> >>>>>>>>> does
> >>>>>> cover is
> >>>>>>>>> all api driven...
> >>>>>>>>>
> >>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
> >>>>> wrote:
> >>>>>>>>>> Though I don’t see VPN in Snaproute.. Makes sense since it
> >>>>>>>>>> was
> >>>> not
> >>>>>>>>>> intended to do IPSec.
> >>>>>>>>>>
> >>>>>>>>>> It seems as though VyOS is starting to look like the best
> >>>> option.
> >>>>>>>>>> Regards,
> >>>>>>>>>> Marty Godsey
> >>>>>>>>>> nSource Solutions
> >>>>>>>>>>
> >>>>>>>>>> -----Original Message-----
> >>>>>>>>>> From: williamstevens@gmail.com
> >>>>>>>>>> [mailto:williamstevens@gmail.com
> >>>> ]
> >>>>> On
> >>>>>>>>>> Behalf Of Will Stevens
> >>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
> >>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>>>>
> >>>>>>>>>> Or we could go completely crazy and go with something like
> >>>>>> FlexSwitch
> >>>>>>>>> from
> >>>>>>>>>> SnapRoute
> >>>>>>>>>> - http://www.snaproute.com/
> >>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
> >>>>>>>>>>
> >>>>>>>>>> *Will STEVENS*
> >>>>>>>>>> Lead Developer
> >>>>>>>>>>
> >>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> >>>>>>>>>> cloudops.com
> >>>>> *|*
> >>>>>> tw
> >>>>>>>>>> @CloudOps_
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> >>>>>> wstevens@cloudops.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
> >>>>> problems
> >>>>>> are
> >>>>>>>>>>> solved by splitting up the function of the VR into a
> >>>>>>>>>>> bunch of
> >>>>>>>> separate
> >>>>>>>>>>> services. As Syed points out, the complexity added is
> >>>>>> non-trivial.
> >>>>>>>>>>> We now have to manage all the intercontainer networking
> >>>>>>>>>>> as
> >>>> well
> >>>>> as
> >>>>>>>> the
> >>>>>>>>>>> orchestrated ACS networking.
> >>>>>>>>>>>
> >>>>>>>>>>> VyOS is interesting to me because it covers the majority
> >>>>>>>>>>> of
> >>>> our
> >>>>>> use
> >>>>>>>>>>> case with a single unified control plane. It also has
> >>>>>>>>>>> good
> >>>>>> support
> >>>>>>>>>>> for extending features we care about, like IPv6, VXLAN,
> >>>>>>>>>>> VRRP, transactions, etc...
> >>>>>>>>>>>
> >>>>>>>>>>> *Will STEVENS*
> >>>>>>>>>>> Lead Developer
> >>>>>>>>>>>
> >>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> >>>> cloudops.com
> >>>>>> *|*
> >>>>>>>> tw
> >>>>>>>>>>> @CloudOps_
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> >>>>> sahmed@cloudops.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>> Agree with Marty, adding Docker containers to the
> >>>>>>>>>>>> picture
> >>>>>> although
> >>>>>>>>>>>> can make the VR more flexible but the added complexity
> >>>>>>>>>>>> is
> >>>> just
> >>>>>> not
> >>>>>>>>>>>> worth it. Not to mention we would need to take care of
> >>>>> networking
> >>>>>>>>>>>> each container manually and given that our iptable rules
> >>>>>>>>>>>> are
> >>>>> very
> >>>>>>>>>>>> unstable at the moment I don't see a big value add.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
> >>>>>>>>>>>> does
> >>>>> not
> >>>>>>>>>>>> provide an api but it does fit the bill quite well
> >>>> otherwise. I
> >>>>>>>>>>>> specially like the fact that it has a transaction based
> >>>>>>>>>>>> model
> >>>>> and
> >>>>>>>> you
> >>>>>>>>>>>> can rollback changes if something goes wrong.
> >>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> >>>>>> marty@gonsource.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>> Licensing aside, I think splitting the various
> >>>>>>>>>>>>> functions
> >>>> into
> >>>>>>>>>>>>> containers is not a good route either. This will force
> >>>> users
> >>>>> to
> >>>>>>>>>>>>> have to maintain
> >>>>>>>>>>>> and
> >>>>>>>>>>>>> use containers and adds complexity to the networking
> >>>> aspects
> >>>>> of
> >>>>>>>> ACS.
> >>>>>>>>>>>>> Complexity decreases stability. Now I understand the
> >>>> argument
> >>>>>> that
> >>>>>>>>>>>>> a monolithic approach also brings its own set of
> >>>>>>>>>>>>> issues but
> >>>>> it
> >>>>>>>> also
> >>>>>>>>>>>>> simplifies it.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>> Marty Godsey
> >>>>>>>>>>>>> nSource Solutions
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> >>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
> >>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
> >>>>>>>>>>>>> of
> >>>> the
> >>>>>>>>>>>>> CloudStack project
> >>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
> >>>>>>>>>>>>> touch
> >>>>>> anything
> >>>>>>>>>>>>> that
> >>>>>>>>>>>> has
> >>>>>>>>>>>>> AGPL
> >>>>>>>>>>>>> - the github repo shows rather infrequent updates.
> >>>>>>>>>>>>> Quite
> >>>>>> likely
> >>>>>>>>>>>>> they aren't considering the use cases of the
> >>>>>>>>>>>>> CloudStack
> >>>>>> community
> >>>>>>>>>>>>> I'd back John B's comments on disaggregating the VR.
> >>>>>>>>>>>>> Split
> >>>> it
> >>>>>> into
> >>>>>>>>>>>>> many docker containers
> >>>>>>>>>>>>> - password server
> >>>>>>>>>>>>> - userdata server
> >>>>>>>>>>>>> - DHCP / DNS
> >>>>>>>>>>>>> - s2s VPN
> >>>>>>>>>>>>> - RA VPN
> >>>>>>>>>>>>> - intra-VPC routing and ACL
> >>>>>>>>>>>>> - Port forwarding + NAT
> >>>>>>>>>>>>> - FW
> >>>>>>>>>>>>> - LB (public)
> >>>>>>>>>>>>> - LB (internal),
> >>>>>>>>>>>>> - secondary storage
> >>>>>>>>>>>>> - agent
> >>>>>>>>>>>>> Glue them together with docker compose files (one per
> >>>>>>>>>>>>> use
> >>>>>> case -
> >>>>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
> >>>>>>>>>>>>> test
> >>>> each
> >>>>> of
> >>>>>>>> the
> >>>>>>>>>>>>> components independently and fixing one bug in the
> >>>>>>>>>>>>> field
> >>>> (say
> >>>>>>>> DHCP)
> >>>>>>>>>>>>> is hitless to the other components. You don't need to
> >>>>>>>>>>>>> build per-hypervisor VRs. You could even run on baremetal.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Along the way you need to figure out how to
> >>>>>>>>>>>>> - make the traffic traverse the containers that are
> >>>>>>>>>>>>> needed
> >>>>> to
> >>>>>> be
> >>>>>>>>>>>>> traversed (in most cases just 1)
> >>>>>>>>>>>>> - bootstrap the router (how does it find its compose file?
> >>>>>> where
> >>>>>>>>>>>>> is the
> >>>>>>>>>>>>> registry?)
> >>>>>>>>>>>>> - rethink the command and control of the VR
> >>>>>>>>>>>>> functions. SSH
> >>>>>> works,
> >>>>>>>>>>>>> but something more declarative, idempotent should be
> >>>>> explored.
> >>>>>>>>>>>>> As you do this, it becomes clearer which of the
> >>>>>>>>>>>>> functions
> >>>> can
> >>>>>> be
> >>>>>>>>>>>>> substituted by for example CloudRouter. Command and
> >>>>>>>>>>>>> Control
> >>>>> of
> >>>>>> the
> >>>>>>>>>>>> docker
> >>>>>>>>>>>>> containers can be moved out to another container. Etc.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> >>>>>>>>>>>>> <ma...@gonsource.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> This one does look nice. My biggest concern is the
> >>>>>>>>>>>>>> lack
> >>>> of
> >>>>>>>>>>>>>> VXLANs. It seems that any of the ones we mentioned
> >>>>>>>>>>>>>> do not
> >>>>>> have
> >>>>>>>> an
> >>>>>>>>>>>>>> API so we may be stuck at the SSH method.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>> Marty Godsey
> >>>>>>>>>>>>>> nSource Solutions
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>>>> From: Abhinandan Prateek
> >>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
> >>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
> >>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
> >>>>>>>>>>>>>> save
> >>>>>> future
> >>>>>>>>>>>>>> engineering effort for example on ipv6 routing, OSPF etc.
> >>>>>>>>>>>>>> And the best part is they come with test automation
> >>>>>> framework.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
> >>>>>>>>>>>>>> <ja...@accelerite.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Instead of replacing the VR in first place we
> >>>>>>>>>>>>>>> should add VyOS/cloudrouter
> >>>>>>>>>>>>>> as provider. Once it is stable, network offerings
> >>>>>>>>>>>>>> (on
> >>>>>> upgrade)
> >>>>>>>>>>>>>> can be updated to use it and we can drop the VR if
> >>>>>>>>>>>>>> we
> >>>> want
> >>>>> at
> >>>>>>>>>>>>>> that release
> >>>>>>>>>>>>> onwards.
> >>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
> >>>>>>>>>>>>>>> them
> >>>>> are
> >>>>>>>>>>>>>>> running
> >>>>>>>>>>>>>> without issues. When we replicate the ACS VR
> >>>>>>>>>>>>>> features in
> >>>>> new
> >>>>>>>>>>>>>> solution it takes some to find the missing pieces
> >>>>>>>>>>>>>> (hidden
> >>>>>> bugs).
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> Jayapal
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
> >>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I like the idea.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
> >>>>>>>>>>>>>>>> keen
> >>>> on
> >>>>>> VyOS
> >>>>>>>>>>>>>>>> (it
> >>>>>>>>>>>>>> doesn't have a proper http api etc).
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Nux!
> >>>>>>>>>>>>>>>> www.nux.ro
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
> >>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
> >>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> >>>>>> @shapeblue
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
> >>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
> >>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
> >>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
> >>>>>>>>>>>>>>>>> should
> >>>>> be
> >>>>>>>>>>>>>>>>> treated as
> >>>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>>> Please weigh in with the good and bad of this idea...
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
> >>>>>> potentially
> >>>>>>>>>>>>>>>>> replacing the ACS VR with the VyOS [1] (Open
> >>>>>>>>>>>>>>>>> Source
> >>>>>> Vyatta
> >>>>>>>>> VM).
> >>>>>>>>>>>>>>>>> There may be a license issue because I think it
> >>>>>>>>>>>>>>>>> is
> >>>>>> licensed
> >>>>>>>>>>>>>>>>> under GPL, but for the sake of discussion, let's
> >>>> assume
> >>>>>> we
> >>>>>>>>>>>>>>>>> can overcome any
> >>>>>>>>>>>>>> license issues.
> >>>>>>>>>>>>>>>>> I have spent some time recently with the VyOS
> >>>>>>>>>>>>>>>>> and I
> >>>>> have
> >>>>>> to
> >>>>>>>>>>>>>>>>> admit, I was pretty impressed. It is simple and
> >>>>>> intuitive
> >>>>>>>>>>>>>>>>> and it gives you a lot more options for auditing
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>> configuration etc...
> >>>>>>>>>>>>>>>>> Items of potential interest:
> >>>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
> >>>> simpler
> >>>>>> more
> >>>>>>>>>>>>>>>>> auditable configuration workflow.
> >>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
> >>>>>>>>>>>>>>>>> - Handles VPN configuration via the same
> >>>> configuration
> >>>>>>>>>> interface.
> >>>>>>>>>>>>>>>>> - Support for OSPF & BGP.
> >>>>>>>>>>>>>>>>> - VPN support through OpenVPN & StrongSwan.
> >>>>>>>>>>>>>>>>> - Easily supports HA (redundant routers) through
> >>>> VRRP.
> >>>>>>>>>>>>>>>>> - VXLAN support.
> >>>>>>>>>>>>>>>>> - Transaction based changes to the VR with
> >>>>>>>>>>>>>>>>> rollback
> >>>> on
> >>>>>>>> error.
> >>>>>>>>>>>>>>>>> Items that could be difficult to solve:
> >>>>>>>>>>>>>>>>> - Userdata password reset workflow and
> >>>> implementation.
> >>>>>>>>>>>>>>>>> - Upgrade process.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> The VyOS is not the only option if we were to
> >>>> consider
> >>>>>> this
> >>>>>>>>>>>> approach.
> >>>>>>>>>>>>>>>>> Another option, which I don't know as well,
> >>>>>>>>>>>>>>>>> would be CloudRouter (AGPL
> >>>>>>>>>>>>>>>>> license) [2] which is purely API driven.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Anyway, would love to hear your thoughts...
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Will
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> [1] https://vyos.io/ [2]
> >>>>>>>>>>>>>>>>> https://cloudrouter.org/
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> DISCLAIMER
> >>>>>>>>>>>>>>> ==========
> >>>>>>>>>>>>>>> This e-mail may contain privileged and confidential
> >>>>>> information
> >>>>>>>>>>>>>>> which is
> >>>>>>>>>>>>>> the property of Accelerite, a Persistent Systems
> >>>> business.
> >>>>>> It is
> >>>>>>>>>>>>>> intended only for the use of the individual or
> >>>>>>>>>>>>>> entity to
> >>>>>> which
> >>>>>>>> it
> >>>>>>>>>>>>>> is addressed. If you are not the intended recipient,
> >>>>>>>>>>>>>> you
> >>>>> are
> >>>>>> not
> >>>>>>>>>>>>>> authorized to read, retain, copy, print, distribute
> >>>>>>>>>>>>>> or
> >>>> use
> >>>>>> this
> >>>>>>>>>>>>>> message. If you have received this communication in
> >>>> error,
> >>>>>>>> please
> >>>>>>>>>>>>>> notify the sender and delete all copies of this message.
> >>>>>>>>>>>>>> Accelerite, a Persistent Systems business does not
> >>>>>>>>>>>>>> accept
> >>>>> any
> >>>>>>>>>>>>>> liability for virus
> >>>>>>>>>>>>> infected mails.
> >>>>>>>>>>>
> >
>
>
Re: [DISCUSS] Replacing the VR
Posted by Murali Reddy <mu...@gmail.com>.
Matthew,
Please see https://cwiki.apache.org/confluence/display/CLOUDSTACK/Extending+CloudStack+Networking
Thanks,
Murali
On 22/09/16, 11:23 PM, "Matthew Smart" <ms...@smartsoftwareinc.com> wrote:
>Hey Murali,
>
>I have been reading through the API and other documentation to try to
>get a basic understanding of the network service offering abstraction
>methodology in CS. I have not dove into the code yet but before I did I
>thought I would try a different approach.
>
>Imagine I were to come to this list and say that I have a network
>offering that I sell and that I wanted to write whatever I needed to in
>order to integrate it as an offering in CloudStack. Is there some
>specific documentation and guidelines you would direct me to read in
>order to gather the knowledge necessary to create a cloudstack
>compatible interface for my product?
>
>I don't know the history but I see several products that have navigated
>this process (Nuage, Nicira, ...etc) and am wondering how a new provider
>would work with you guys to navigate that process. If this is too vague,
>we can pretend my new offering is a hardware firewall device.
>
>My goal here is to gain an understanding of how CS interacts with third
>party offerings underneath the hood. I have some thoughts (I think
>inline with Will Steven's brain dump and diagram) but want to make sure
>I am not suffering some misapprehensions about the architecture and,
>short of tracing code, was not successful at finding the information I
>needed to satisfy myself that I know how it is designed.
>
>Thanks,
>
>Matthew Smart
>President
>Smart Software Solutions Inc.
>108 S Pierre St.
>Pierre, SD 57501
>
>Phone: (605) 280-0383
>Skype: msmart13
>Email: msmart@smartsoftwareinc.com
>
>On 09/20/2016 04:54 AM, Murali Reddy wrote:
>> Configuration management of network appliances particularly for Cloud and NFV scenarios is still evolving area. Programmability is the not the strength for even the most popular network operating systems like IOS, JunoS etc. So its not surprising why CloudStack integrates in a archaic way with stock linux for the VR.
>>
>> VR was never integral/hardcoded option in CloudStack. Its always been a plugin. CloudStack network orchestration is well abstracted and designed with vision to compose a network with different set of providers for different services. Yes that vision is not fully realised yet, and we don’t have true service function chains. That would be different discussion topic.
>>
>> I tend to agree with Simon, as alternate/interim option we can take hard look whats causing the problems with current VR integration. Personally, I think it would be easier we take a cue from configuration managers and network configuration solutions out there (for e.g promise theory based Cisco ACI) move to more declarative model of expressing desired state of network configuration. Infact current VR from 4.6, actually holds the desired state per service basis, seems to be in that direction.
>>
>> It does make sense to evaluate new appliances which can provide rich semantics (like programmable API, declarative configuration, versioning, commit/rollback etc), but will need significant engineering effort and time to stabilise. We may make same mistakes with integration of other appliance as well, if we fully don’t understand the nature of the current problems with CloudStack core and service provider interaction and current VR integration.
>>
>>
>> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>>
>>> I think our other option is to take a real look at what it would take to fix the VR. In my opinion, a lot of the problems are related to the monolithic python code base and the fact nothing is actually separated.
>>>
>>> Secondly, the python scripts (and bash scripts) don't use any established libraries to complete tasks and instead shell out and run commands that are both hard to track and hard to parse on return.
>>>
>>>
>>> If we daemonized this, used a real api for Agent to VR communication, used common already existing libraries for the system service and network interactions and spent a bit of time separating out code into distinct modules, everything would behave a lot better.
>>>
>>>
>>> The pain and suffering is due to years and years of patches and constant shelling out to complete tasks in my opinion. If we spend time to rethink how we interact with the VR in general and we abstract the systems and networking stuff and use well known and stable libraries to do the work, the VR would be much easier to maintain.
>>>
>>>
>>> - Si
>>>
>>>
>>>
>>>
>>> ________________________________
>>> From: Marty Godsey <ma...@gonsource.com>
>>> Sent: Friday, September 16, 2016 12:24 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: RE: [DISCUSS] Replacing the VR
>>>
>>> So based upon this discussion would it be prudent to wait on VyOS 2.0? The current VR is giving us issues but would the time invested in another "solution" be wasted especially if by the time another option is chose, then coded, then tested, then implemented and right as that time happened to be when VyOS 2.0 is released. Of course you said they are just in the scoping range so this could still be a year or more out.
>>>
>>> Thoughts?
>>>
>>> Regards,
>>> Marty Godsey
>>> nSource Solutions
>>>
>>> -----Original Message-----
>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
>>> Sent: Friday, September 16, 2016 10:31 AM
>>> To: dev@cloudstack.apache.org
>>> Cc: daniil@baturin.org
>>> Subject: Re: [DISCUSS] Replacing the VR
>>>
>>> I just had a quick chat with a couple of the guys over on the VyOS chat.
>>> I have CC'ed one of them in case we have more licensing questions.
>>>
>>> So here is the status with the license "the code inherited from Vyatta and our modifications from it is GPLv2 (strict, not v2+). The config reading library is GPLv2 too, so anything that links to is is GPLv2.
>>> Some auxiliary components we made after the fork are more permissive,
>>> LGPLv2+ or MIT."
>>>
>>> They are currently in the process of scoping a redesign (VyOS 2.0), "we are planning a clean rewrite that will solve issues of the current config system".
>>> This will include the ability to configure via the API.
>>>
>>> If we have more questions for VyOS, they are very friendly and responsive, so we should be able to get answers.
>>>
>>> *Will STEVENS*
>>> Lead Developer
>>>
>>> *CloudOps* *| *Cloud Solutions Experts
>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_
>>>
>>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>>>
>>>> I agree with Will Ilya. There are so many problems with the VR right now.
>>>> Most of the outages we've had recently have somehow involved the VR.
>>>> We set custom iptables rules on the VR which can and have easily gone wrong.
>>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>>> VVRP with redundant router still needs work, and not to mention the
>>>> problems we will have when we introduce IPv6 into the whole picture.
>>>>
>>>> I think the spirit of the discussion is to rely on a 3rd party to do
>>>> the networking for us (eg VyOS) and have us handle just the
>>>> orchestration. All the problems that I've described have already been
>>>> solved in VyOS. We also get the advantage of a potential wider
>>>> community to fix and maintain the VR and given our current development
>>>> velocity, it think it totally makes sense to look for a 3rd party option.
>>>>
>>>> -Syed
>>>>
>>>>
>>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
>>>> wrote:
>>>>
>>>>> The VR has been biting us far too often recently, which is why we
>>>>> have started looking into alternative implementations.
>>>>>
>>>>> One of the things that is nice about potentially using the VyOS is
>>>>> that
>>>> it
>>>>> is based on Debian, so we should be able to run the other services
>>>>> that
>>>> we
>>>>> currently have like the password server and userdata on the VyOS.
>>>>> This means we would not have to change our architecture initially
>>>>> and could focus on only replacing the networking paths.
>>>>>
>>>>> *Will STEVENS*
>>>>> Lead Developer
>>>>>
>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>>> tw @CloudOps_
>>>>>
>>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>>
>>>>>> The more this is discussed the more I think we should stick with
>>>>>> our
>>>> VR.
>>>>>> All these other options either seem unfinished or with
>>>>>> incompatible license.
>>>>>>
>>>>>> VyOS looks the most promising so far, it's a serious, mature project.
>>>>>> Adopting it though means we'll have to microservice our way out of
>>>>>> it
>>>>> with
>>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>>>> those
>>>>>> too. Imho this adds complexity we should void.
>>>>>>
>>>>>> --
>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>
>>>>>> Nux!
>>>>>> www.nux.ro
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>>> To: dev@cloudstack.apache.org
>>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>> Ya, we would need to add a daemon for VPN as well. Load
>>>>>>> balancing is another aspect which we will need to consider if we went this route.
>>>>>>> Something like https://traefik.io/ could potentially be a good
>>>>>>> fit
>>>> due
>>>>>> to
>>>>>>> its API driven configuration, but it may be more than what we need.
>>>>>>>
>>>>>>> We should probably try define which pieces make sense to be
>>>>>>> solved
>>>>>> together
>>>>>>> and which pieces would be best suited to be broken out.
>>>>>>>
>>>>>>> I think the network connectivity, routing and firewalling should
>>>>> probably
>>>>>>> all stay together since the majority of the tools we would
>>>> potentially
>>>>>> use
>>>>>>> would handle all of that together in a single implementation.
>>>>>>>
>>>>>>> The password server and userdata seems like a good option for
>>>>>>> being
>>>>>> broken
>>>>>>> out and handled independently (and probably rewritten completely
>>>> since
>>>>>> they
>>>>>>> currently have some issues).
>>>>>>>
>>>>>>> Load balancing is another that could warrant splitting out, but
>>>>>>> that depends on what direction we go and how we would be managing it.
>>>> DHCP
>>>>>> and
>>>>>>> DNS are others which could go either way.
>>>>>>>
>>>>>>> If we do split out services, I think we should consolidate as
>>>>>>> much as
>>>>> we
>>>>>>> can into each service we break out. Ideally a network packet
>>>>>>> would
>>>>> never
>>>>>>> hit more than one, maybe two, services. I don't think we should
>>>>>>> be splitting services 'just because', I think we need a valid
>>>>>>> case for splitting any service out because it adds complexity.
>>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>>> complexity unless it
>>>> is
>>>>>>> really needed.
>>>>>>>
>>>>>>> Some more of my thoughts on this anyway...
>>>>>>>
>>>>>>> *Will STEVENS*
>>>>>>> Lead Developer
>>>>>>>
>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>> *|* tw @CloudOps_
>>>>>>>
>>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>>>>> wrote:
>>>>>>>> I do agree with you that this probably isn't the right place
>>>>>>>> the
>>>>>> password
>>>>>>>> service and user data.
>>>>>>>>
>>>>>>>>
>>>>>>>> Having said that, after taking a cursory look at the dev docs,
>>>>>>>> it
>>>>>> doesn't
>>>>>>>> seem that difficult to add new daemons:
>>>> https://opensnaproute.github.
>>>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>>
>>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>>> creating-new-component>
>>>>>>>>
>>>>>>>>
>>>>>>>> They've definitely build it with a microservices architecture
>>>>>>>> in
>>>> mind,
>>>>>> so
>>>>>>>> each individual feature is abstracted into it's own small
>>>>>>>> daemon
>>>>>> process.
>>>>>>>> We could just create a daemon for the password server and the
>>>> userdata
>>>>>>>> components if we really had to.
>>>>>>>>
>>>>>>>>
>>>>>>>> - Si
>>>>>>>>
>>>>>>>>
>>>>>>>> ________________________________
>>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
>>>>>>>> behalf
>>>>> of
>>>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>
>>>>>>>> A big part of why I know about it is because it is written in Go.
>>>> :P
>>>>>>>> Yes, it is definitely interesting for the routing and traffic
>>>> handling
>>>>>>>> aspects of the VR. We will likely have to rethink some of the
>>>> pieces
>>>>> a
>>>>>>>> little bit like the password server and userdata if we are to
>>>>>>>> adopt
>>>> a
>>>>>>>> different VR approach. This is where I think some of JohnB and
>>>>>> Chiradeep's
>>>>>>>> ideas make sense. In many ways, it does not make sense for the
>>>> device
>>>>>>>> handling routing and network traffic to also be responsible for
>>>>>> passwords
>>>>>>>> and userdata.
>>>>>>>>
>>>>>>>> *Will STEVENS*
>>>>>>>> Lead Developer
>>>>>>>>
>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>> *|* tw @CloudOps_
>>>>>>>>
>>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>>>>> wrote:
>>>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
>>>> pretty
>>>>>>>> cool!
>>>>>>>>> It even supports ONIE install.
>>>>>>>>>
>>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>>> offload
>>>>>> it to
>>>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>>> from a
>>>>>>>> routing
>>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Based on the roadmap, it looks like they plan to also support
>>>>>>>> capabilities
>>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>>> will
>>>> be
>>>>>> huge
>>>>>>>>> for our carrier community that rely on these technologies to
>>>>>>>>> do
>>>>>> private
>>>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>>> stuff
>>>>> on
>>>>>> our
>>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
>>>>>>>>> able to
>>>>> do
>>>>>>>> MPLS
>>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>>> know
>>>>> very
>>>>>>>>> well).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> - Si
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ________________________________
>>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>>
>>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>>> does
>>>>>> cover is
>>>>>>>>> all api driven...
>>>>>>>>>
>>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>>>>> wrote:
>>>>>>>>>> Though I don’t see VPN in Snaproute.. Makes sense since it
>>>>>>>>>> was
>>>> not
>>>>>>>>>> intended to do IPSec.
>>>>>>>>>>
>>>>>>>>>> It seems as though VyOS is starting to look like the best
>>>> option.
>>>>>>>>>> Regards,
>>>>>>>>>> Marty Godsey
>>>>>>>>>> nSource Solutions
>>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>>> [mailto:williamstevens@gmail.com
>>>> ]
>>>>> On
>>>>>>>>>> Behalf Of Will Stevens
>>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>
>>>>>>>>>> Or we could go completely crazy and go with something like
>>>>>> FlexSwitch
>>>>>>>>> from
>>>>>>>>>> SnapRoute
>>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>>
>>>>>>>>>> *Will STEVENS*
>>>>>>>>>> Lead Developer
>>>>>>>>>>
>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>> cloudops.com
>>>>> *|*
>>>>>> tw
>>>>>>>>>> @CloudOps_
>>>>>>>>>>
>>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>>> wstevens@cloudops.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
>>>>> problems
>>>>>> are
>>>>>>>>>>> solved by splitting up the function of the VR into a
>>>>>>>>>>> bunch of
>>>>>>>> separate
>>>>>>>>>>> services. As Syed points out, the complexity added is
>>>>>> non-trivial.
>>>>>>>>>>> We now have to manage all the intercontainer networking
>>>>>>>>>>> as
>>>> well
>>>>> as
>>>>>>>> the
>>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>>
>>>>>>>>>>> VyOS is interesting to me because it covers the majority
>>>>>>>>>>> of
>>>> our
>>>>>> use
>>>>>>>>>>> case with a single unified control plane. It also has
>>>>>>>>>>> good
>>>>>> support
>>>>>>>>>>> for extending features we care about, like IPv6, VXLAN,
>>>>>>>>>>> VRRP, transactions, etc...
>>>>>>>>>>>
>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>> Lead Developer
>>>>>>>>>>>
>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>> cloudops.com
>>>>>> *|*
>>>>>>>> tw
>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>>> sahmed@cloudops.com>
>>>>>>>>> wrote:
>>>>>>>>>>>> Agree with Marty, adding Docker containers to the
>>>>>>>>>>>> picture
>>>>>> although
>>>>>>>>>>>> can make the VR more flexible but the added complexity
>>>>>>>>>>>> is
>>>> just
>>>>>> not
>>>>>>>>>>>> worth it. Not to mention we would need to take care of
>>>>> networking
>>>>>>>>>>>> each container manually and given that our iptable rules
>>>>>>>>>>>> are
>>>>> very
>>>>>>>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>>
>>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
>>>>>>>>>>>> does
>>>>> not
>>>>>>>>>>>> provide an api but it does fit the bill quite well
>>>> otherwise. I
>>>>>>>>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>>> model
>>>>> and
>>>>>>>> you
>>>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>>> marty@gonsource.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>> Licensing aside, I think splitting the various
>>>>>>>>>>>>> functions
>>>> into
>>>>>>>>>>>>> containers is not a good route either. This will force
>>>> users
>>>>> to
>>>>>>>>>>>>> have to maintain
>>>>>>>>>>>> and
>>>>>>>>>>>>> use containers and adds complexity to the networking
>>>> aspects
>>>>> of
>>>>>>>> ACS.
>>>>>>>>>>>>> Complexity decreases stability. Now I understand the
>>>> argument
>>>>>> that
>>>>>>>>>>>>> a monolithic approach also brings its own set of
>>>>>>>>>>>>> issues but
>>>>> it
>>>>>>>> also
>>>>>>>>>>>>> simplifies it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>
>>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
>>>>>>>>>>>>> of
>>>> the
>>>>>>>>>>>>> CloudStack project
>>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
>>>>>>>>>>>>> touch
>>>>>> anything
>>>>>>>>>>>>> that
>>>>>>>>>>>> has
>>>>>>>>>>>>> AGPL
>>>>>>>>>>>>> - the github repo shows rather infrequent updates.
>>>>>>>>>>>>> Quite
>>>>>> likely
>>>>>>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>>> CloudStack
>>>>>> community
>>>>>>>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>>> Split
>>>> it
>>>>>> into
>>>>>>>>>>>>> many docker containers
>>>>>>>>>>>>> - password server
>>>>>>>>>>>>> - userdata server
>>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>>> - FW
>>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>>> - agent
>>>>>>>>>>>>> Glue them together with docker compose files (one per
>>>>>>>>>>>>> use
>>>>>> case -
>>>>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>>
>>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
>>>>>>>>>>>>> test
>>>> each
>>>>> of
>>>>>>>> the
>>>>>>>>>>>>> components independently and fixing one bug in the
>>>>>>>>>>>>> field
>>>> (say
>>>>>>>> DHCP)
>>>>>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>>> build per-hypervisor VRs. You could even run on baremetal.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>>> - make the traffic traverse the containers that are
>>>>>>>>>>>>> needed
>>>>> to
>>>>>> be
>>>>>>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>>> - bootstrap the router (how does it find its compose file?
>>>>>> where
>>>>>>>>>>>>> is the
>>>>>>>>>>>>> registry?)
>>>>>>>>>>>>> - rethink the command and control of the VR
>>>>>>>>>>>>> functions. SSH
>>>>>> works,
>>>>>>>>>>>>> but something more declarative, idempotent should be
>>>>> explored.
>>>>>>>>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>>> functions
>>>> can
>>>>>> be
>>>>>>>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>>> Control
>>>>> of
>>>>>> the
>>>>>>>>>>>> docker
>>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> This one does look nice. My biggest concern is the
>>>>>>>>>>>>>> lack
>>>> of
>>>>>>>>>>>>>> VXLANs. It seems that any of the ones we mentioned
>>>>>>>>>>>>>> do not
>>>>>> have
>>>>>>>> an
>>>>>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>>> save
>>>>>> future
>>>>>>>>>>>>>> engineering effort for example on ipv6 routing, OSPF etc.
>>>>>>>>>>>>>> And the best part is they come with test automation
>>>>>> framework.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Instead of replacing the VR in first place we
>>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>>> as provider. Once it is stable, network offerings
>>>>>>>>>>>>>> (on
>>>>>> upgrade)
>>>>>>>>>>>>>> can be updated to use it and we can drop the VR if
>>>>>>>>>>>>>> we
>>>> want
>>>>> at
>>>>>>>>>>>>>> that release
>>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>>> them
>>>>> are
>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>> without issues. When we replicate the ACS VR
>>>>>>>>>>>>>> features in
>>>>> new
>>>>>>>>>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>>> (hidden
>>>>>> bugs).
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
>>>>>>>>>>>>>>>> keen
>>>> on
>>>>>> VyOS
>>>>>>>>>>>>>>>> (it
>>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>>>>> @shapeblue
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
>>>>>>>>>>>>>>>>> should
>>>>> be
>>>>>>>>>>>>>>>>> treated as
>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>> Please weigh in with the good and bad of this idea...
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
>>>>>> potentially
>>>>>>>>>>>>>>>>> replacing the ACS VR with the VyOS [1] (Open
>>>>>>>>>>>>>>>>> Source
>>>>>> Vyatta
>>>>>>>>> VM).
>>>>>>>>>>>>>>>>> There may be a license issue because I think it
>>>>>>>>>>>>>>>>> is
>>>>>> licensed
>>>>>>>>>>>>>>>>> under GPL, but for the sake of discussion, let's
>>>> assume
>>>>>> we
>>>>>>>>>>>>>>>>> can overcome any
>>>>>>>>>>>>>> license issues.
>>>>>>>>>>>>>>>>> I have spent some time recently with the VyOS
>>>>>>>>>>>>>>>>> and I
>>>>> have
>>>>>> to
>>>>>>>>>>>>>>>>> admit, I was pretty impressed. It is simple and
>>>>>> intuitive
>>>>>>>>>>>>>>>>> and it gives you a lot more options for auditing
>>>>>>>>>>>>>>>>> the
>>>>>>>>>> configuration etc...
>>>>>>>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
>>>> simpler
>>>>>> more
>>>>>>>>>>>>>>>>> auditable configuration workflow.
>>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
>>>>>>>>>>>>>>>>> - Handles VPN configuration via the same
>>>> configuration
>>>>>>>>>> interface.
>>>>>>>>>>>>>>>>> - Support for OSPF & BGP.
>>>>>>>>>>>>>>>>> - VPN support through OpenVPN & StrongSwan.
>>>>>>>>>>>>>>>>> - Easily supports HA (redundant routers) through
>>>> VRRP.
>>>>>>>>>>>>>>>>> - VXLAN support.
>>>>>>>>>>>>>>>>> - Transaction based changes to the VR with
>>>>>>>>>>>>>>>>> rollback
>>>> on
>>>>>>>> error.
>>>>>>>>>>>>>>>>> Items that could be difficult to solve:
>>>>>>>>>>>>>>>>> - Userdata password reset workflow and
>>>> implementation.
>>>>>>>>>>>>>>>>> - Upgrade process.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The VyOS is not the only option if we were to
>>>> consider
>>>>>> this
>>>>>>>>>>>> approach.
>>>>>>>>>>>>>>>>> Another option, which I don't know as well,
>>>>>>>>>>>>>>>>> would be CloudRouter (AGPL
>>>>>>>>>>>>>>>>> license) [2] which is purely API driven.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Anyway, would love to hear your thoughts...
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> [1] https://vyos.io/ [2]
>>>>>>>>>>>>>>>>> https://cloudrouter.org/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> DISCLAIMER
>>>>>>>>>>>>>>> ==========
>>>>>>>>>>>>>>> This e-mail may contain privileged and confidential
>>>>>> information
>>>>>>>>>>>>>>> which is
>>>>>>>>>>>>>> the property of Accelerite, a Persistent Systems
>>>> business.
>>>>>> It is
>>>>>>>>>>>>>> intended only for the use of the individual or
>>>>>>>>>>>>>> entity to
>>>>>> which
>>>>>>>> it
>>>>>>>>>>>>>> is addressed. If you are not the intended recipient,
>>>>>>>>>>>>>> you
>>>>> are
>>>>>> not
>>>>>>>>>>>>>> authorized to read, retain, copy, print, distribute
>>>>>>>>>>>>>> or
>>>> use
>>>>>> this
>>>>>>>>>>>>>> message. If you have received this communication in
>>>> error,
>>>>>>>> please
>>>>>>>>>>>>>> notify the sender and delete all copies of this message.
>>>>>>>>>>>>>> Accelerite, a Persistent Systems business does not
>>>>>>>>>>>>>> accept
>>>>> any
>>>>>>>>>>>>>> liability for virus
>>>>>>>>>>>>> infected mails.
>>>>>>>>>>>
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
Unfortunately there is not much documentation around the network plugin
functionality. When I wrote the Palo Alto integration I basically figured
out how to do it by reviewing existing plugins and just figuring it out.
So if you were to begin to implement a new hardware firewall for example, I
would point you to the Palo Alto integration code [1] and the functional
spec [2] and then make myself available to try to answer any questions you
have (like how the NetworkGuru works, where the different pieces are
registered, etc)...
Unfortunately it is not trivial, mainly because we don't have any
documentation to follow, but the plugin interface IS there. It just
requires people who have worked it in the past to offer guidance.
[1]
https://github.com/apache/cloudstack/tree/master/plugins/network-elements/palo-alto
[2]
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Palo+Alto+Firewall+Integration
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Thu, Sep 22, 2016 at 1:53 PM, Matthew Smart <ms...@smartsoftwareinc.com>
wrote:
> Hey Murali,
>
> I have been reading through the API and other documentation to try to get
> a basic understanding of the network service offering abstraction
> methodology in CS. I have not dove into the code yet but before I did I
> thought I would try a different approach.
>
> Imagine I were to come to this list and say that I have a network offering
> that I sell and that I wanted to write whatever I needed to in order to
> integrate it as an offering in CloudStack. Is there some specific
> documentation and guidelines you would direct me to read in order to gather
> the knowledge necessary to create a cloudstack compatible interface for my
> product?
>
> I don't know the history but I see several products that have navigated
> this process (Nuage, Nicira, ...etc) and am wondering how a new provider
> would work with you guys to navigate that process. If this is too vague, we
> can pretend my new offering is a hardware firewall device.
>
> My goal here is to gain an understanding of how CS interacts with third
> party offerings underneath the hood. I have some thoughts (I think inline
> with Will Steven's brain dump and diagram) but want to make sure I am not
> suffering some misapprehensions about the architecture and, short of
> tracing code, was not successful at finding the information I needed to
> satisfy myself that I know how it is designed.
>
> Thanks,
>
> Matthew Smart
> President
> Smart Software Solutions Inc.
> 108 S Pierre St.
> Pierre, SD 57501
>
> Phone: (605) 280-0383
> Skype: msmart13
> Email: msmart@smartsoftwareinc.com
>
> On 09/20/2016 04:54 AM, Murali Reddy wrote:
>
>> Configuration management of network appliances particularly for Cloud and
>> NFV scenarios is still evolving area. Programmability is the not the
>> strength for even the most popular network operating systems like IOS,
>> JunoS etc. So its not surprising why CloudStack integrates in a archaic way
>> with stock linux for the VR.
>>
>> VR was never integral/hardcoded option in CloudStack. Its always been a
>> plugin. CloudStack network orchestration is well abstracted and designed
>> with vision to compose a network with different set of providers for
>> different services. Yes that vision is not fully realised yet, and we don’t
>> have true service function chains. That would be different discussion topic.
>>
>> I tend to agree with Simon, as alternate/interim option we can take hard
>> look whats causing the problems with current VR integration. Personally, I
>> think it would be easier we take a cue from configuration managers and
>> network configuration solutions out there (for e.g promise theory based
>> Cisco ACI) move to more declarative model of expressing desired state of
>> network configuration. Infact current VR from 4.6, actually holds the
>> desired state per service basis, seems to be in that direction.
>>
>> It does make sense to evaluate new appliances which can provide rich
>> semantics (like programmable API, declarative configuration, versioning,
>> commit/rollback etc), but will need significant engineering effort and time
>> to stabilise. We may make same mistakes with integration of other appliance
>> as well, if we fully don’t understand the nature of the current problems
>> with CloudStack core and service provider interaction and current VR
>> integration.
>>
>>
>> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>>
>> I think our other option is to take a real look at what it would take to
>>> fix the VR. In my opinion, a lot of the problems are related to the
>>> monolithic python code base and the fact nothing is actually separated.
>>>
>>> Secondly, the python scripts (and bash scripts) don't use any
>>> established libraries to complete tasks and instead shell out and run
>>> commands that are both hard to track and hard to parse on return.
>>>
>>>
>>> If we daemonized this, used a real api for Agent to VR communication,
>>> used common already existing libraries for the system service and network
>>> interactions and spent a bit of time separating out code into distinct
>>> modules, everything would behave a lot better.
>>>
>>>
>>> The pain and suffering is due to years and years of patches and constant
>>> shelling out to complete tasks in my opinion. If we spend time to rethink
>>> how we interact with the VR in general and we abstract the systems and
>>> networking stuff and use well known and stable libraries to do the work,
>>> the VR would be much easier to maintain.
>>>
>>>
>>> - Si
>>>
>>>
>>>
>>>
>>> ________________________________
>>> From: Marty Godsey <ma...@gonsource.com>
>>> Sent: Friday, September 16, 2016 12:24 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: RE: [DISCUSS] Replacing the VR
>>>
>>> So based upon this discussion would it be prudent to wait on VyOS 2.0?
>>> The current VR is giving us issues but would the time invested in another
>>> "solution" be wasted especially if by the time another option is chose,
>>> then coded, then tested, then implemented and right as that time happened
>>> to be when VyOS 2.0 is released. Of course you said they are just in the
>>> scoping range so this could still be a year or more out.
>>>
>>> Thoughts?
>>>
>>> Regards,
>>> Marty Godsey
>>> nSource Solutions
>>>
>>> -----Original Message-----
>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>>> Behalf Of Will Stevens
>>> Sent: Friday, September 16, 2016 10:31 AM
>>> To: dev@cloudstack.apache.org
>>> Cc: daniil@baturin.org
>>> Subject: Re: [DISCUSS] Replacing the VR
>>>
>>> I just had a quick chat with a couple of the guys over on the VyOS chat.
>>> I have CC'ed one of them in case we have more licensing questions.
>>>
>>> So here is the status with the license "the code inherited from Vyatta
>>> and our modifications from it is GPLv2 (strict, not v2+). The config
>>> reading library is GPLv2 too, so anything that links to is is GPLv2.
>>> Some auxiliary components we made after the fork are more permissive,
>>> LGPLv2+ or MIT."
>>>
>>> They are currently in the process of scoping a redesign (VyOS 2.0), "we
>>> are planning a clean rewrite that will solve issues of the current config
>>> system".
>>> This will include the ability to configure via the API.
>>>
>>> If we have more questions for VyOS, they are very friendly and
>>> responsive, so we should be able to get answers.
>>>
>>> *Will STEVENS*
>>> Lead Developer
>>>
>>> *CloudOps* *| *Cloud Solutions Experts
>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>>> @CloudOps_
>>>
>>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>>>
>>> I agree with Will Ilya. There are so many problems with the VR right now.
>>>> Most of the outages we've had recently have somehow involved the VR.
>>>> We set custom iptables rules on the VR which can and have easily gone
>>>> wrong.
>>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>>> VVRP with redundant router still needs work, and not to mention the
>>>> problems we will have when we introduce IPv6 into the whole picture.
>>>>
>>>> I think the spirit of the discussion is to rely on a 3rd party to do
>>>> the networking for us (eg VyOS) and have us handle just the
>>>> orchestration. All the problems that I've described have already been
>>>> solved in VyOS. We also get the advantage of a potential wider
>>>> community to fix and maintain the VR and given our current development
>>>> velocity, it think it totally makes sense to look for a 3rd party
>>>> option.
>>>>
>>>> -Syed
>>>>
>>>>
>>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
>>>> wrote:
>>>>
>>>> The VR has been biting us far too often recently, which is why we
>>>>> have started looking into alternative implementations.
>>>>>
>>>>> One of the things that is nice about potentially using the VyOS is
>>>>> that
>>>>>
>>>> it
>>>>
>>>>> is based on Debian, so we should be able to run the other services
>>>>> that
>>>>>
>>>> we
>>>>
>>>>> currently have like the password server and userdata on the VyOS.
>>>>> This means we would not have to change our architecture initially
>>>>> and could focus on only replacing the networking paths.
>>>>>
>>>>> *Will STEVENS*
>>>>> Lead Developer
>>>>>
>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>>> tw @CloudOps_
>>>>>
>>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>>
>>>>> The more this is discussed the more I think we should stick with
>>>>>> our
>>>>>>
>>>>> VR.
>>>>
>>>>> All these other options either seem unfinished or with
>>>>>> incompatible license.
>>>>>>
>>>>>> VyOS looks the most promising so far, it's a serious, mature project.
>>>>>> Adopting it though means we'll have to microservice our way out of
>>>>>> it
>>>>>>
>>>>> with
>>>>>
>>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>>>>>>
>>>>> those
>>>>
>>>>> too. Imho this adds complexity we should void.
>>>>>>
>>>>>> --
>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>
>>>>>> Nux!
>>>>>> www.nux.ro
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>
>>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>>> To: dev@cloudstack.apache.org
>>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>> Ya, we would need to add a daemon for VPN as well. Load
>>>>>>> balancing is another aspect which we will need to consider if we
>>>>>>> went this route.
>>>>>>> Something like https://traefik.io/ could potentially be a good
>>>>>>> fit
>>>>>>>
>>>>>> due
>>>>
>>>>> to
>>>>>>
>>>>>>> its API driven configuration, but it may be more than what we need.
>>>>>>>
>>>>>>> We should probably try define which pieces make sense to be
>>>>>>> solved
>>>>>>>
>>>>>> together
>>>>>>
>>>>>>> and which pieces would be best suited to be broken out.
>>>>>>>
>>>>>>> I think the network connectivity, routing and firewalling should
>>>>>>>
>>>>>> probably
>>>>>
>>>>>> all stay together since the majority of the tools we would
>>>>>>>
>>>>>> potentially
>>>>
>>>>> use
>>>>>>
>>>>>>> would handle all of that together in a single implementation.
>>>>>>>
>>>>>>> The password server and userdata seems like a good option for
>>>>>>> being
>>>>>>>
>>>>>> broken
>>>>>>
>>>>>>> out and handled independently (and probably rewritten completely
>>>>>>>
>>>>>> since
>>>>
>>>>> they
>>>>>>
>>>>>>> currently have some issues).
>>>>>>>
>>>>>>> Load balancing is another that could warrant splitting out, but
>>>>>>> that depends on what direction we go and how we would be managing it.
>>>>>>>
>>>>>> DHCP
>>>>
>>>>> and
>>>>>>
>>>>>>> DNS are others which could go either way.
>>>>>>>
>>>>>>> If we do split out services, I think we should consolidate as
>>>>>>> much as
>>>>>>>
>>>>>> we
>>>>>
>>>>>> can into each service we break out. Ideally a network packet
>>>>>>> would
>>>>>>>
>>>>>> never
>>>>>
>>>>>> hit more than one, maybe two, services. I don't think we should
>>>>>>> be splitting services 'just because', I think we need a valid
>>>>>>> case for splitting any service out because it adds complexity.
>>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>>> complexity unless it
>>>>>>>
>>>>>> is
>>>>
>>>>> really needed.
>>>>>>>
>>>>>>> Some more of my thoughts on this anyway...
>>>>>>>
>>>>>>> *Will STEVENS*
>>>>>>> Lead Developer
>>>>>>>
>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>> *|* tw @CloudOps_
>>>>>>>
>>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>>>>>>>
>>>>>> wrote:
>>>>>
>>>>>> I do agree with you that this probably isn't the right place
>>>>>>>> the
>>>>>>>>
>>>>>>> password
>>>>>>
>>>>>>> service and user data.
>>>>>>>>
>>>>>>>>
>>>>>>>> Having said that, after taking a cursory look at the dev docs,
>>>>>>>> it
>>>>>>>>
>>>>>>> doesn't
>>>>>>
>>>>>>> seem that difficult to add new daemons:
>>>>>>>>
>>>>>>> https://opensnaproute.github.
>>>>
>>>>> io/docs/developer.html#creating-new-component
>>>>>>>>
>>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>>> creating-new-component>
>>>>>>>>
>>>>>>>>
>>>>>>>> They've definitely build it with a microservices architecture
>>>>>>>> in
>>>>>>>>
>>>>>>> mind,
>>>>
>>>>> so
>>>>>>
>>>>>>> each individual feature is abstracted into it's own small
>>>>>>>> daemon
>>>>>>>>
>>>>>>> process.
>>>>>>
>>>>>>> We could just create a daemon for the password server and the
>>>>>>>>
>>>>>>> userdata
>>>>
>>>>> components if we really had to.
>>>>>>>>
>>>>>>>>
>>>>>>>> - Si
>>>>>>>>
>>>>>>>>
>>>>>>>> ________________________________
>>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
>>>>>>>> behalf
>>>>>>>>
>>>>>>> of
>>>>>
>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>
>>>>>>>> A big part of why I know about it is because it is written in Go.
>>>>>>>>
>>>>>>> :P
>>>>
>>>>> Yes, it is definitely interesting for the routing and traffic
>>>>>>>>
>>>>>>> handling
>>>>
>>>>> aspects of the VR. We will likely have to rethink some of the
>>>>>>>>
>>>>>>> pieces
>>>>
>>>>> a
>>>>>
>>>>>> little bit like the password server and userdata if we are to
>>>>>>>> adopt
>>>>>>>>
>>>>>>> a
>>>>
>>>>> different VR approach. This is where I think some of JohnB and
>>>>>>>>
>>>>>>> Chiradeep's
>>>>>>
>>>>>>> ideas make sense. In many ways, it does not make sense for the
>>>>>>>>
>>>>>>> device
>>>>
>>>>> handling routing and network traffic to also be responsible for
>>>>>>>>
>>>>>>> passwords
>>>>>>
>>>>>>> and userdata.
>>>>>>>>
>>>>>>>> *Will STEVENS*
>>>>>>>> Lead Developer
>>>>>>>>
>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>> *|* tw @CloudOps_
>>>>>>>>
>>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>>>>>>>>
>>>>>>> wrote:
>>>>>
>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
>>>>>>>>>
>>>>>>>> pretty
>>>>
>>>>> cool!
>>>>>>>>
>>>>>>>>> It even supports ONIE install.
>>>>>>>>>
>>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>>>>>>>>
>>>>>>>> offload
>>>>
>>>>> it to
>>>>>>
>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>>> from a
>>>>>>>>>
>>>>>>>> routing
>>>>>>>>
>>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Based on the roadmap, it looks like they plan to also support
>>>>>>>>>
>>>>>>>> capabilities
>>>>>>>>
>>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>>> will
>>>>>>>>>
>>>>>>>> be
>>>>
>>>>> huge
>>>>>>
>>>>>>> for our carrier community that rely on these technologies to
>>>>>>>>> do
>>>>>>>>>
>>>>>>>> private
>>>>>>
>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>>> stuff
>>>>>>>>>
>>>>>>>> on
>>>>>
>>>>>> our
>>>>>>
>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
>>>>>>>>> able to
>>>>>>>>>
>>>>>>>> do
>>>>>
>>>>>> MPLS
>>>>>>>>
>>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>>> know
>>>>>>>>>
>>>>>>>> very
>>>>>
>>>>>> well).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> - Si
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ________________________________
>>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>>
>>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>>> does
>>>>>>>>>
>>>>>>>> cover is
>>>>>>
>>>>>>> all api driven...
>>>>>>>>>
>>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>>>>>>>>>
>>>>>>>> wrote:
>>>>>
>>>>>> Though I don’t see VPN in Snaproute.. Makes sense since it
>>>>>>>>>> was
>>>>>>>>>>
>>>>>>>>> not
>>>>
>>>>> intended to do IPSec.
>>>>>>>>>>
>>>>>>>>>> It seems as though VyOS is starting to look like the best
>>>>>>>>>>
>>>>>>>>> option.
>>>>
>>>>> Regards,
>>>>>>>>>> Marty Godsey
>>>>>>>>>> nSource Solutions
>>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>>> [mailto:williamstevens@gmail.com
>>>>>>>>>>
>>>>>>>>> ]
>>>>
>>>>> On
>>>>>
>>>>>> Behalf Of Will Stevens
>>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>
>>>>>>>>>> Or we could go completely crazy and go with something like
>>>>>>>>>>
>>>>>>>>> FlexSwitch
>>>>>>
>>>>>>> from
>>>>>>>>>
>>>>>>>>>> SnapRoute
>>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>>
>>>>>>>>>> *Will STEVENS*
>>>>>>>>>> Lead Developer
>>>>>>>>>>
>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>> cloudops.com
>>>>>>>>>>
>>>>>>>>> *|*
>>>>>
>>>>>> tw
>>>>>>
>>>>>>> @CloudOps_
>>>>>>>>>>
>>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>>>>>>>
>>>>>>>>> wstevens@cloudops.com>
>>>>>>
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
>>>>>>>>>>>
>>>>>>>>>> problems
>>>>>
>>>>>> are
>>>>>>
>>>>>>> solved by splitting up the function of the VR into a
>>>>>>>>>>> bunch of
>>>>>>>>>>>
>>>>>>>>>> separate
>>>>>>>>
>>>>>>>>> services. As Syed points out, the complexity added is
>>>>>>>>>>>
>>>>>>>>>> non-trivial.
>>>>>>
>>>>>>> We now have to manage all the intercontainer networking
>>>>>>>>>>> as
>>>>>>>>>>>
>>>>>>>>>> well
>>>>
>>>>> as
>>>>>
>>>>>> the
>>>>>>>>
>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>>
>>>>>>>>>>> VyOS is interesting to me because it covers the majority
>>>>>>>>>>> of
>>>>>>>>>>>
>>>>>>>>>> our
>>>>
>>>>> use
>>>>>>
>>>>>>> case with a single unified control plane. It also has
>>>>>>>>>>> good
>>>>>>>>>>>
>>>>>>>>>> support
>>>>>>
>>>>>>> for extending features we care about, like IPv6, VXLAN,
>>>>>>>>>>> VRRP, transactions, etc...
>>>>>>>>>>>
>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>> Lead Developer
>>>>>>>>>>>
>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>>
>>>>>>>>>> cloudops.com
>>>>
>>>>> *|*
>>>>>>
>>>>>>> tw
>>>>>>>>
>>>>>>>>> @CloudOps_
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>>>>>>>>>
>>>>>>>>>> sahmed@cloudops.com>
>>>>>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Agree with Marty, adding Docker containers to the
>>>>>>>>>>>> picture
>>>>>>>>>>>>
>>>>>>>>>>> although
>>>>>>
>>>>>>> can make the VR more flexible but the added complexity
>>>>>>>>>>>> is
>>>>>>>>>>>>
>>>>>>>>>>> just
>>>>
>>>>> not
>>>>>>
>>>>>>> worth it. Not to mention we would need to take care of
>>>>>>>>>>>>
>>>>>>>>>>> networking
>>>>>
>>>>>> each container manually and given that our iptable rules
>>>>>>>>>>>> are
>>>>>>>>>>>>
>>>>>>>>>>> very
>>>>>
>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>>
>>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
>>>>>>>>>>>> does
>>>>>>>>>>>>
>>>>>>>>>>> not
>>>>>
>>>>>> provide an api but it does fit the bill quite well
>>>>>>>>>>>>
>>>>>>>>>>> otherwise. I
>>>>
>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>>> model
>>>>>>>>>>>>
>>>>>>>>>>> and
>>>>>
>>>>>> you
>>>>>>>>
>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>>>>>>>>>
>>>>>>>>>>> marty@gonsource.com>
>>>>>>
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Licensing aside, I think splitting the various
>>>>>>>>>>>>> functions
>>>>>>>>>>>>>
>>>>>>>>>>>> into
>>>>
>>>>> containers is not a good route either. This will force
>>>>>>>>>>>>>
>>>>>>>>>>>> users
>>>>
>>>>> to
>>>>>
>>>>>> have to maintain
>>>>>>>>>>>>>
>>>>>>>>>>>> and
>>>>>>>>>>>>
>>>>>>>>>>>>> use containers and adds complexity to the networking
>>>>>>>>>>>>>
>>>>>>>>>>>> aspects
>>>>
>>>>> of
>>>>>
>>>>>> ACS.
>>>>>>>>
>>>>>>>>> Complexity decreases stability. Now I understand the
>>>>>>>>>>>>>
>>>>>>>>>>>> argument
>>>>
>>>>> that
>>>>>>
>>>>>>> a monolithic approach also brings its own set of
>>>>>>>>>>>>> issues but
>>>>>>>>>>>>>
>>>>>>>>>>>> it
>>>>>
>>>>>> also
>>>>>>>>
>>>>>>>>> simplifies it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>
>>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
>>>>>>>>>>>>> of
>>>>>>>>>>>>>
>>>>>>>>>>>> the
>>>>
>>>>> CloudStack project
>>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
>>>>>>>>>>>>> touch
>>>>>>>>>>>>>
>>>>>>>>>>>> anything
>>>>>>
>>>>>>> that
>>>>>>>>>>>>>
>>>>>>>>>>>> has
>>>>>>>>>>>>
>>>>>>>>>>>>> AGPL
>>>>>>>>>>>>> - the github repo shows rather infrequent updates.
>>>>>>>>>>>>> Quite
>>>>>>>>>>>>>
>>>>>>>>>>>> likely
>>>>>>
>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>
>>>>>>>>>>>> community
>>>>>>
>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>>> Split
>>>>>>>>>>>>>
>>>>>>>>>>>> it
>>>>
>>>>> into
>>>>>>
>>>>>>> many docker containers
>>>>>>>>>>>>> - password server
>>>>>>>>>>>>> - userdata server
>>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>>> - FW
>>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>>> - agent
>>>>>>>>>>>>> Glue them together with docker compose files (one per
>>>>>>>>>>>>> use
>>>>>>>>>>>>>
>>>>>>>>>>>> case -
>>>>>>
>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>>
>>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
>>>>>>>>>>>>> test
>>>>>>>>>>>>>
>>>>>>>>>>>> each
>>>>
>>>>> of
>>>>>
>>>>>> the
>>>>>>>>
>>>>>>>>> components independently and fixing one bug in the
>>>>>>>>>>>>> field
>>>>>>>>>>>>>
>>>>>>>>>>>> (say
>>>>
>>>>> DHCP)
>>>>>>>>
>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>>> build per-hypervisor VRs. You could even run on baremetal.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>>> - make the traffic traverse the containers that are
>>>>>>>>>>>>> needed
>>>>>>>>>>>>>
>>>>>>>>>>>> to
>>>>>
>>>>>> be
>>>>>>
>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>>> - bootstrap the router (how does it find its compose file?
>>>>>>>>>>>>>
>>>>>>>>>>>> where
>>>>>>
>>>>>>> is the
>>>>>>>>>>>>> registry?)
>>>>>>>>>>>>> - rethink the command and control of the VR
>>>>>>>>>>>>> functions. SSH
>>>>>>>>>>>>>
>>>>>>>>>>>> works,
>>>>>>
>>>>>>> but something more declarative, idempotent should be
>>>>>>>>>>>>>
>>>>>>>>>>>> explored.
>>>>>
>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>>> functions
>>>>>>>>>>>>>
>>>>>>>>>>>> can
>>>>
>>>>> be
>>>>>>
>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>>> Control
>>>>>>>>>>>>>
>>>>>>>>>>>> of
>>>>>
>>>>>> the
>>>>>>
>>>>>>> docker
>>>>>>>>>>>>
>>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> This one does look nice. My biggest concern is the
>>>>>>>>>>>>>> lack
>>>>>>>>>>>>>>
>>>>>>>>>>>>> of
>>>>
>>>>> VXLANs. It seems that any of the ones we mentioned
>>>>>>>>>>>>>> do not
>>>>>>>>>>>>>>
>>>>>>>>>>>>> have
>>>>>>
>>>>>>> an
>>>>>>>>
>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>>> save
>>>>>>>>>>>>>>
>>>>>>>>>>>>> future
>>>>>>
>>>>>>> engineering effort for example on ipv6 routing, OSPF etc.
>>>>>>>>>>>>>> And the best part is they come with test automation
>>>>>>>>>>>>>>
>>>>>>>>>>>>> framework.
>>>>>>
>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Instead of replacing the VR in first place we
>>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> as provider. Once it is stable, network offerings
>>>>>>>>>>>>>> (on
>>>>>>>>>>>>>>
>>>>>>>>>>>>> upgrade)
>>>>>>
>>>>>>> can be updated to use it and we can drop the VR if
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>
>>>>>>>>>>>>> want
>>>>
>>>>> at
>>>>>
>>>>>> that release
>>>>>>>>>>>>>>
>>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>>> them
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> are
>>>>>
>>>>>> running
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> without issues. When we replicate the ACS VR
>>>>>>>>>>>>>> features in
>>>>>>>>>>>>>>
>>>>>>>>>>>>> new
>>>>>
>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>>> (hidden
>>>>>>>>>>>>>>
>>>>>>>>>>>>> bugs).
>>>>>>
>>>>>>> Thanks,
>>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
>>>>>>>>>>>>>>>> keen
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> on
>>>>
>>>>> VyOS
>>>>>>
>>>>>>> (it
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>>>>>>>>>>>>>
>>>>>>>>>>>>> @shapeblue
>>>>>>
>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> be
>>>>>
>>>>>> treated as
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please weigh in with the good and bad of this idea...
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> potentially
>>>>>>
>>>>>>> replacing the ACS VR with the VyOS [1] (Open
>>>>>>>>>>>>>>>>> Source
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Vyatta
>>>>>>
>>>>>>> VM).
>>>>>>>>>
>>>>>>>>>> There may be a license issue because I think it
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> licensed
>>>>>>
>>>>>>> under GPL, but for the sake of discussion, let's
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> assume
>>>>
>>>>> we
>>>>>>
>>>>>>> can overcome any
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> license issues.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have spent some time recently with the VyOS
>>>>>>>>>>>>>>>>> and I
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> have
>>>>>
>>>>>> to
>>>>>>
>>>>>>> admit, I was pretty impressed. It is simple and
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> intuitive
>>>>>>
>>>>>>> and it gives you a lot more options for auditing
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> configuration etc...
>>>>>>>>>>
>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> simpler
>>>>
>>>>> more
>>>>>>
>>>>>>> auditable configuration workflow.
>>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
>>>>>>>>>>>>>>>>> - Handles VPN configuration via the same
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> configuration
>>>>
>>>>> interface.
>>>>>>>>>
>>>>>>>>>
Re: [DISCUSS] Replacing the VR
Posted by Matthew Smart <ms...@smartsoftwareinc.com>.
Hey Murali,
I have been reading through the API and other documentation to try to
get a basic understanding of the network service offering abstraction
methodology in CS. I have not dove into the code yet but before I did I
thought I would try a different approach.
Imagine I were to come to this list and say that I have a network
offering that I sell and that I wanted to write whatever I needed to in
order to integrate it as an offering in CloudStack. Is there some
specific documentation and guidelines you would direct me to read in
order to gather the knowledge necessary to create a cloudstack
compatible interface for my product?
I don't know the history but I see several products that have navigated
this process (Nuage, Nicira, ...etc) and am wondering how a new provider
would work with you guys to navigate that process. If this is too vague,
we can pretend my new offering is a hardware firewall device.
My goal here is to gain an understanding of how CS interacts with third
party offerings underneath the hood. I have some thoughts (I think
inline with Will Steven's brain dump and diagram) but want to make sure
I am not suffering some misapprehensions about the architecture and,
short of tracing code, was not successful at finding the information I
needed to satisfy myself that I know how it is designed.
Thanks,
Matthew Smart
President
Smart Software Solutions Inc.
108 S Pierre St.
Pierre, SD 57501
Phone: (605) 280-0383
Skype: msmart13
Email: msmart@smartsoftwareinc.com
On 09/20/2016 04:54 AM, Murali Reddy wrote:
> Configuration management of network appliances particularly for Cloud and NFV scenarios is still evolving area. Programmability is the not the strength for even the most popular network operating systems like IOS, JunoS etc. So its not surprising why CloudStack integrates in a archaic way with stock linux for the VR.
>
> VR was never integral/hardcoded option in CloudStack. Its always been a plugin. CloudStack network orchestration is well abstracted and designed with vision to compose a network with different set of providers for different services. Yes that vision is not fully realised yet, and we don\u2019t have true service function chains. That would be different discussion topic.
>
> I tend to agree with Simon, as alternate/interim option we can take hard look whats causing the problems with current VR integration. Personally, I think it would be easier we take a cue from configuration managers and network configuration solutions out there (for e.g promise theory based Cisco ACI) move to more declarative model of expressing desired state of network configuration. Infact current VR from 4.6, actually holds the desired state per service basis, seems to be in that direction.
>
> It does make sense to evaluate new appliances which can provide rich semantics (like programmable API, declarative configuration, versioning, commit/rollback etc), but will need significant engineering effort and time to stabilise. We may make same mistakes with integration of other appliance as well, if we fully don\u2019t understand the nature of the current problems with CloudStack core and service provider interaction and current VR integration.
>
>
> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>
>> I think our other option is to take a real look at what it would take to fix the VR. In my opinion, a lot of the problems are related to the monolithic python code base and the fact nothing is actually separated.
>>
>> Secondly, the python scripts (and bash scripts) don't use any established libraries to complete tasks and instead shell out and run commands that are both hard to track and hard to parse on return.
>>
>>
>> If we daemonized this, used a real api for Agent to VR communication, used common already existing libraries for the system service and network interactions and spent a bit of time separating out code into distinct modules, everything would behave a lot better.
>>
>>
>> The pain and suffering is due to years and years of patches and constant shelling out to complete tasks in my opinion. If we spend time to rethink how we interact with the VR in general and we abstract the systems and networking stuff and use well known and stable libraries to do the work, the VR would be much easier to maintain.
>>
>>
>> - Si
>>
>>
>>
>>
>> ________________________________
>> From: Marty Godsey <ma...@gonsource.com>
>> Sent: Friday, September 16, 2016 12:24 PM
>> To: dev@cloudstack.apache.org
>> Subject: RE: [DISCUSS] Replacing the VR
>>
>> So based upon this discussion would it be prudent to wait on VyOS 2.0? The current VR is giving us issues but would the time invested in another "solution" be wasted especially if by the time another option is chose, then coded, then tested, then implemented and right as that time happened to be when VyOS 2.0 is released. Of course you said they are just in the scoping range so this could still be a year or more out.
>>
>> Thoughts?
>>
>> Regards,
>> Marty Godsey
>> nSource Solutions
>>
>> -----Original Message-----
>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
>> Sent: Friday, September 16, 2016 10:31 AM
>> To: dev@cloudstack.apache.org
>> Cc: daniil@baturin.org
>> Subject: Re: [DISCUSS] Replacing the VR
>>
>> I just had a quick chat with a couple of the guys over on the VyOS chat.
>> I have CC'ed one of them in case we have more licensing questions.
>>
>> So here is the status with the license "the code inherited from Vyatta and our modifications from it is GPLv2 (strict, not v2+). The config reading library is GPLv2 too, so anything that links to is is GPLv2.
>> Some auxiliary components we made after the fork are more permissive,
>> LGPLv2+ or MIT."
>>
>> They are currently in the process of scoping a redesign (VyOS 2.0), "we are planning a clean rewrite that will solve issues of the current config system".
>> This will include the ability to configure via the API.
>>
>> If we have more questions for VyOS, they are very friendly and responsive, so we should be able to get answers.
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_
>>
>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>>
>>> I agree with Will Ilya. There are so many problems with the VR right now.
>>> Most of the outages we've had recently have somehow involved the VR.
>>> We set custom iptables rules on the VR which can and have easily gone wrong.
>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>> VVRP with redundant router still needs work, and not to mention the
>>> problems we will have when we introduce IPv6 into the whole picture.
>>>
>>> I think the spirit of the discussion is to rely on a 3rd party to do
>>> the networking for us (eg VyOS) and have us handle just the
>>> orchestration. All the problems that I've described have already been
>>> solved in VyOS. We also get the advantage of a potential wider
>>> community to fix and maintain the VR and given our current development
>>> velocity, it think it totally makes sense to look for a 3rd party option.
>>>
>>> -Syed
>>>
>>>
>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
>>> wrote:
>>>
>>>> The VR has been biting us far too often recently, which is why we
>>>> have started looking into alternative implementations.
>>>>
>>>> One of the things that is nice about potentially using the VyOS is
>>>> that
>>> it
>>>> is based on Debian, so we should be able to run the other services
>>>> that
>>> we
>>>> currently have like the password server and userdata on the VyOS.
>>>> This means we would not have to change our architecture initially
>>>> and could focus on only replacing the networking paths.
>>>>
>>>> *Will STEVENS*
>>>> Lead Developer
>>>>
>>>> *CloudOps* *| *Cloud Solutions Experts
>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>> tw @CloudOps_
>>>>
>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>
>>>>> The more this is discussed the more I think we should stick with
>>>>> our
>>> VR.
>>>>> All these other options either seem unfinished or with
>>>>> incompatible license.
>>>>>
>>>>> VyOS looks the most promising so far, it's a serious, mature project.
>>>>> Adopting it though means we'll have to microservice our way out of
>>>>> it
>>>> with
>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>>> those
>>>>> too. Imho this adds complexity we should void.
>>>>>
>>>>> --
>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>
>>>>> Nux!
>>>>> www.nux.ro
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>> To: dev@cloudstack.apache.org
>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>> Ya, we would need to add a daemon for VPN as well. Load
>>>>>> balancing is another aspect which we will need to consider if we went this route.
>>>>>> Something like https://traefik.io/ could potentially be a good
>>>>>> fit
>>> due
>>>>> to
>>>>>> its API driven configuration, but it may be more than what we need.
>>>>>>
>>>>>> We should probably try define which pieces make sense to be
>>>>>> solved
>>>>> together
>>>>>> and which pieces would be best suited to be broken out.
>>>>>>
>>>>>> I think the network connectivity, routing and firewalling should
>>>> probably
>>>>>> all stay together since the majority of the tools we would
>>> potentially
>>>>> use
>>>>>> would handle all of that together in a single implementation.
>>>>>>
>>>>>> The password server and userdata seems like a good option for
>>>>>> being
>>>>> broken
>>>>>> out and handled independently (and probably rewritten completely
>>> since
>>>>> they
>>>>>> currently have some issues).
>>>>>>
>>>>>> Load balancing is another that could warrant splitting out, but
>>>>>> that depends on what direction we go and how we would be managing it.
>>> DHCP
>>>>> and
>>>>>> DNS are others which could go either way.
>>>>>>
>>>>>> If we do split out services, I think we should consolidate as
>>>>>> much as
>>>> we
>>>>>> can into each service we break out. Ideally a network packet
>>>>>> would
>>>> never
>>>>>> hit more than one, maybe two, services. I don't think we should
>>>>>> be splitting services 'just because', I think we need a valid
>>>>>> case for splitting any service out because it adds complexity.
>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>> complexity unless it
>>> is
>>>>>> really needed.
>>>>>>
>>>>>> Some more of my thoughts on this anyway...
>>>>>>
>>>>>> *Will STEVENS*
>>>>>> Lead Developer
>>>>>>
>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>> *|* tw @CloudOps_
>>>>>>
>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>>>> wrote:
>>>>>>> I do agree with you that this probably isn't the right place
>>>>>>> the
>>>>> password
>>>>>>> service and user data.
>>>>>>>
>>>>>>>
>>>>>>> Having said that, after taking a cursory look at the dev docs,
>>>>>>> it
>>>>> doesn't
>>>>>>> seem that difficult to add new daemons:
>>> https://opensnaproute.github.
>>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>
>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>> creating-new-component>
>>>>>>>
>>>>>>>
>>>>>>> They've definitely build it with a microservices architecture
>>>>>>> in
>>> mind,
>>>>> so
>>>>>>> each individual feature is abstracted into it's own small
>>>>>>> daemon
>>>>> process.
>>>>>>> We could just create a daemon for the password server and the
>>> userdata
>>>>>>> components if we really had to.
>>>>>>>
>>>>>>>
>>>>>>> - Si
>>>>>>>
>>>>>>>
>>>>>>> ________________________________
>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
>>>>>>> behalf
>>>> of
>>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>> To: dev@cloudstack.apache.org
>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>
>>>>>>> A big part of why I know about it is because it is written in Go.
>>> :P
>>>>>>> Yes, it is definitely interesting for the routing and traffic
>>> handling
>>>>>>> aspects of the VR. We will likely have to rethink some of the
>>> pieces
>>>> a
>>>>>>> little bit like the password server and userdata if we are to
>>>>>>> adopt
>>> a
>>>>>>> different VR approach. This is where I think some of JohnB and
>>>>> Chiradeep's
>>>>>>> ideas make sense. In many ways, it does not make sense for the
>>> device
>>>>>>> handling routing and network traffic to also be responsible for
>>>>> passwords
>>>>>>> and userdata.
>>>>>>>
>>>>>>> *Will STEVENS*
>>>>>>> Lead Developer
>>>>>>>
>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>> *|* tw @CloudOps_
>>>>>>>
>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>>>> wrote:
>>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
>>> pretty
>>>>>>> cool!
>>>>>>>> It even supports ONIE install.
>>>>>>>>
>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>> offload
>>>>> it to
>>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>> from a
>>>>>>> routing
>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>
>>>>>>>>
>>>>>>>> Based on the roadmap, it looks like they plan to also support
>>>>>>> capabilities
>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>> will
>>> be
>>>>> huge
>>>>>>>> for our carrier community that rely on these technologies to
>>>>>>>> do
>>>>> private
>>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>> stuff
>>>> on
>>>>> our
>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
>>>>>>>> able to
>>>> do
>>>>>>> MPLS
>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>
>>>>>>>>
>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>> know
>>>> very
>>>>>>>> well).
>>>>>>>>
>>>>>>>>
>>>>>>>> - Si
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ________________________________
>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>
>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>> does
>>>>> cover is
>>>>>>>> all api driven...
>>>>>>>>
>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>>>> wrote:
>>>>>>>>> Though I don\u2019t see VPN in Snaproute.. Makes sense since it
>>>>>>>>> was
>>> not
>>>>>>>>> intended to do IPSec.
>>>>>>>>>
>>>>>>>>> It seems as though VyOS is starting to look like the best
>>> option.
>>>>>>>>> Regards,
>>>>>>>>> Marty Godsey
>>>>>>>>> nSource Solutions
>>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>> [mailto:williamstevens@gmail.com
>>> ]
>>>> On
>>>>>>>>> Behalf Of Will Stevens
>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>
>>>>>>>>> Or we could go completely crazy and go with something like
>>>>> FlexSwitch
>>>>>>>> from
>>>>>>>>> SnapRoute
>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>> cloudops.com
>>>> *|*
>>>>> tw
>>>>>>>>> @CloudOps_
>>>>>>>>>
>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>> wstevens@cloudops.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
>>>> problems
>>>>> are
>>>>>>>>>> solved by splitting up the function of the VR into a
>>>>>>>>>> bunch of
>>>>>>> separate
>>>>>>>>>> services. As Syed points out, the complexity added is
>>>>> non-trivial.
>>>>>>>>>> We now have to manage all the intercontainer networking
>>>>>>>>>> as
>>> well
>>>> as
>>>>>>> the
>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>
>>>>>>>>>> VyOS is interesting to me because it covers the majority
>>>>>>>>>> of
>>> our
>>>>> use
>>>>>>>>>> case with a single unified control plane. It also has
>>>>>>>>>> good
>>>>> support
>>>>>>>>>> for extending features we care about, like IPv6, VXLAN,
>>>>>>>>>> VRRP, transactions, etc...
>>>>>>>>>>
>>>>>>>>>> *Will STEVENS*
>>>>>>>>>> Lead Developer
>>>>>>>>>>
>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>> cloudops.com
>>>>> *|*
>>>>>>> tw
>>>>>>>>>> @CloudOps_
>>>>>>>>>>
>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>> sahmed@cloudops.com>
>>>>>>>> wrote:
>>>>>>>>>>> Agree with Marty, adding Docker containers to the
>>>>>>>>>>> picture
>>>>> although
>>>>>>>>>>> can make the VR more flexible but the added complexity
>>>>>>>>>>> is
>>> just
>>>>> not
>>>>>>>>>>> worth it. Not to mention we would need to take care of
>>>> networking
>>>>>>>>>>> each container manually and given that our iptable rules
>>>>>>>>>>> are
>>>> very
>>>>>>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>
>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
>>>>>>>>>>> does
>>>> not
>>>>>>>>>>> provide an api but it does fit the bill quite well
>>> otherwise. I
>>>>>>>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>> model
>>>> and
>>>>>>> you
>>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>> marty@gonsource.com>
>>>>>>>>> wrote:
>>>>>>>>>>>> Licensing aside, I think splitting the various
>>>>>>>>>>>> functions
>>> into
>>>>>>>>>>>> containers is not a good route either. This will force
>>> users
>>>> to
>>>>>>>>>>>> have to maintain
>>>>>>>>>>> and
>>>>>>>>>>>> use containers and adds complexity to the networking
>>> aspects
>>>> of
>>>>>>> ACS.
>>>>>>>>>>>> Complexity decreases stability. Now I understand the
>>> argument
>>>>> that
>>>>>>>>>>>> a monolithic approach also brings its own set of
>>>>>>>>>>>> issues but
>>>> it
>>>>>>> also
>>>>>>>>>>>> simplifies it.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>
>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
>>>>>>>>>>>> of
>>> the
>>>>>>>>>>>> CloudStack project
>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
>>>>>>>>>>>> touch
>>>>> anything
>>>>>>>>>>>> that
>>>>>>>>>>> has
>>>>>>>>>>>> AGPL
>>>>>>>>>>>> - the github repo shows rather infrequent updates.
>>>>>>>>>>>> Quite
>>>>> likely
>>>>>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>> CloudStack
>>>>> community
>>>>>>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>> Split
>>> it
>>>>> into
>>>>>>>>>>>> many docker containers
>>>>>>>>>>>> - password server
>>>>>>>>>>>> - userdata server
>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>> - FW
>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>> - agent
>>>>>>>>>>>> Glue them together with docker compose files (one per
>>>>>>>>>>>> use
>>>>> case -
>>>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>
>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
>>>>>>>>>>>> test
>>> each
>>>> of
>>>>>>> the
>>>>>>>>>>>> components independently and fixing one bug in the
>>>>>>>>>>>> field
>>> (say
>>>>>>> DHCP)
>>>>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>> build per-hypervisor VRs. You could even run on baremetal.
>>>>>>>>>>>>
>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>> - make the traffic traverse the containers that are
>>>>>>>>>>>> needed
>>>> to
>>>>> be
>>>>>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>> - bootstrap the router (how does it find its compose file?
>>>>> where
>>>>>>>>>>>> is the
>>>>>>>>>>>> registry?)
>>>>>>>>>>>> - rethink the command and control of the VR
>>>>>>>>>>>> functions. SSH
>>>>> works,
>>>>>>>>>>>> but something more declarative, idempotent should be
>>>> explored.
>>>>>>>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>> functions
>>> can
>>>>> be
>>>>>>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>> Control
>>>> of
>>>>> the
>>>>>>>>>>> docker
>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> This one does look nice. My biggest concern is the
>>>>>>>>>>>>> lack
>>> of
>>>>>>>>>>>>> VXLANs. It seems that any of the ones we mentioned
>>>>>>>>>>>>> do not
>>>>> have
>>>>>>> an
>>>>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>> save
>>>>> future
>>>>>>>>>>>>> engineering effort for example on ipv6 routing, OSPF etc.
>>>>>>>>>>>>> And the best part is they come with test automation
>>>>> framework.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Instead of replacing the VR in first place we
>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>> as provider. Once it is stable, network offerings
>>>>>>>>>>>>> (on
>>>>> upgrade)
>>>>>>>>>>>>> can be updated to use it and we can drop the VR if
>>>>>>>>>>>>> we
>>> want
>>>> at
>>>>>>>>>>>>> that release
>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>> them
>>>> are
>>>>>>>>>>>>>> running
>>>>>>>>>>>>> without issues. When we replicate the ACS VR
>>>>>>>>>>>>> features in
>>>> new
>>>>>>>>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>> (hidden
>>>>> bugs).
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
>>>>>>>>>>>>>>> keen
>>> on
>>>>> VyOS
>>>>>>>>>>>>>>> (it
>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>>>> @shapeblue
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
>>>>>>>>>>>>>>>> should
>>>> be
>>>>>>>>>>>>>>>> treated as
>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>> Please weigh in with the good and bad of this idea...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
>>>>> potentially
>>>>>>>>>>>>>>>> replacing the ACS VR with the VyOS [1] (Open
>>>>>>>>>>>>>>>> Source
>>>>> Vyatta
>>>>>>>> VM).
>>>>>>>>>>>>>>>> There may be a license issue because I think it
>>>>>>>>>>>>>>>> is
>>>>> licensed
>>>>>>>>>>>>>>>> under GPL, but for the sake of discussion, let's
>>> assume
>>>>> we
>>>>>>>>>>>>>>>> can overcome any
>>>>>>>>>>>>> license issues.
>>>>>>>>>>>>>>>> I have spent some time recently with the VyOS
>>>>>>>>>>>>>>>> and I
>>>> have
>>>>> to
>>>>>>>>>>>>>>>> admit, I was pretty impressed. It is simple and
>>>>> intuitive
>>>>>>>>>>>>>>>> and it gives you a lot more options for auditing
>>>>>>>>>>>>>>>> the
>>>>>>>>> configuration etc...
>>>>>>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
>>> simpler
>>>>> more
>>>>>>>>>>>>>>>> auditable configuration workflow.
>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
>>>>>>>>>>>>>>>> - Handles VPN configuration via the same
>>> configuration
>>>>>>>>> interface.
>>>>>>>>>>>>>>>> - Support for OSPF & BGP.
>>>>>>>>>>>>>>>> - VPN support through OpenVPN & StrongSwan.
>>>>>>>>>>>>>>>> - Easily supports HA (redundant routers) through
>>> VRRP.
>>>>>>>>>>>>>>>> - VXLAN support.
>>>>>>>>>>>>>>>> - Transaction based changes to the VR with
>>>>>>>>>>>>>>>> rollback
>>> on
>>>>>>> error.
>>>>>>>>>>>>>>>> Items that could be difficult to solve:
>>>>>>>>>>>>>>>> - Userdata password reset workflow and
>>> implementation.
>>>>>>>>>>>>>>>> - Upgrade process.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The VyOS is not the only option if we were to
>>> consider
>>>>> this
>>>>>>>>>>> approach.
>>>>>>>>>>>>>>>> Another option, which I don't know as well,
>>>>>>>>>>>>>>>> would be CloudRouter (AGPL
>>>>>>>>>>>>>>>> license) [2] which is purely API driven.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Anyway, would love to hear your thoughts...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1] https://vyos.io/ [2]
>>>>>>>>>>>>>>>> https://cloudrouter.org/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> DISCLAIMER
>>>>>>>>>>>>>> ==========
>>>>>>>>>>>>>> This e-mail may contain privileged and confidential
>>>>> information
>>>>>>>>>>>>>> which is
>>>>>>>>>>>>> the property of Accelerite, a Persistent Systems
>>> business.
>>>>> It is
>>>>>>>>>>>>> intended only for the use of the individual or
>>>>>>>>>>>>> entity to
>>>>> which
>>>>>>> it
>>>>>>>>>>>>> is addressed. If you are not the intended recipient,
>>>>>>>>>>>>> you
>>>> are
>>>>> not
>>>>>>>>>>>>> authorized to read, retain, copy, print, distribute
>>>>>>>>>>>>> or
>>> use
>>>>> this
>>>>>>>>>>>>> message. If you have received this communication in
>>> error,
>>>>>>> please
>>>>>>>>>>>>> notify the sender and delete all copies of this message.
>>>>>>>>>>>>> Accelerite, a Persistent Systems business does not
>>>>>>>>>>>>> accept
>>>> any
>>>>>>>>>>>>> liability for virus
>>>>>>>>>>>> infected mails.
>>>>>>>>>>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
Here is a shortened url for the diagram [1] so it won't wrap and be
confusing for people...
http://bit.ly/2cQaC0X
On Sep 20, 2016 3:51 PM, "Will Stevens" <ws...@cloudops.com> wrote:
> I am going to try to do a bit of a brain dump from me and my team in this
> post, and hopefully I can verbalize the ideas well enough that you can all
> follow along. Please provide feedback, ideas and suggestions...
>
> *Preface:*
> *--------*
> I am currently approaching this problem with the following use cases in
> mind.
>
> - Stabilize/replace the VR for our production cloud deployments as well as
> give the community a free, open source and production grade alternative to
> the current VR.
>
> - I have been scoping the integration of the Fortinet Fortigate (hardware
> and virtual) into ACS to support the following features.
> -- All Isolated Guest Network traffic features (Firewall, PF, NAT,
> Client-to-Site VPN, etc)
> -- All VPC network traffic features (ACLs, PF, NAT, Client-to-Site and
> Site-to-Site VPN, Private Gateway)
> -- Public IPv6 support (NAT64 and NAT46) with IPv4 on the guest side.
>
> *The Setup:*
> *----------*
> I am not going to get into the details of the Fortinet integration details
> in this thread, you can expect a function spec from me soon describing my
> plans for this work. I am currently waiting on hardware to validate my
> implementation strategy.
>
> In this post I will be focusing on the way ACS deals with the VR, network
> plugins and how this could be potentially adapted to give us more
> flexibility going forward.
>
> I have created a diagram [1] to outline the ideas presented in this post.
>
> So if you don't like my ideas here, you can simply use the VR as you
> currently do and you can basically ignore the dotted circles to the right
> of the 'existing VR' in the solid circle.
>
> The basic idea here, is when you create a network service offering and you
> select different Providers to handle different network capabilities, you
> can also specify a VM template for those providers (as well as service
> offering) which will be used for additional VMs which will be created to
> provision those services. In this context, think of templates such as
> VyOS, FortiOS, or a VPX to understand the spirit of this approach.
>
> When a network is provisioned, the current VR will be deployed (ideally
> only offering services like; password server, userdata, DHCP, DNS), and in
> addition, any network provider templates which have been specified will
> also be provisioned. Each 'network VM' will get a leg on both the public
> VLAN and the guest VLAN. In the case of a VPC, the first tier will be a
> little special (I will get into this in a bit) and will not be able to be
> deleted until the VR(s) are all cleaned up and the network is completely
> destroyed. This network is denoted as the 'Client MGMT' network in the
> diagram [1].
>
> You will notice that ONLY the 'existing VR' has network connectivity to
> the ACS MGMT server. It may not necessarily be through a 169.x.x.x
> address, but that illustrates the point for now. The reason for this is
> because we can not guarantee that a Guest User does may not gain access to
> these VRs, and in cases I will get into later, the Guest User may actually
> be given access to those VRs. This does however mean that the ACS MGMT
> server does not have direct access to those VRs in order to configure
> them. To solve this, the 'existing VR' would act as a proxy for the ACS
> Management server over the ACS MGMT network to pass configuration commands
> to the VRs to the right of the 'existing VR' depicted in the graphic.
>
> *The Details:*
> *------------*
> Now that you have a basic idea of the deployment structure I have in mind,
> let's discuss some of the finer details.
>
> - We should be able to relatively easily adapt existing external network
> plugins to be able to be provisioned with this mechanism (assuming they
> have a VM option). The Palo Alto would be an example which I would likely
> use as a proof of concept for this adaptation. It would be important that
> the external plugin could work as an external physical appliance OR as a
> dynamically provisioned VM appliance.
>
> - Enterprise grade network appliances which support VM deployment will no
> longer have to be pre-deployed and shared, but instead will be able to be
> dynamically provisioned on a network by network basis.
>
> - Existing external network appliance plugins targeting physical boxes
> will still work in concert with this approach without any need for
> modification.
>
> - We could add an 'Allow Guest Access' checkbox to the service provider
> section in the network service offering to give the Guest User access to
> their network appliance. This is especially useful for an implementation
> like the FortiGate because it allows for the user to configure UTM features
> which ACS does not orchestrate but the appliance is capable of. This also
> applies to devices like the NetScaler. In the diagram [1] this is depicted
> by the dashed green lines between the devices and the 'Client MGMT'
> network. So in the case of the NetScaler, you would get an NSIP on the
> Client MGMT network which would allow the Guest User to connect to that
> box. I envision a URL pointing to the guest network IP with
> username/password returned for a network which a user can use to connect
> into that network appliance.
>
> - Similar to above, the Guest User will have much more visibility into the
> logging and access on the network. I can't count how many times a customer
> has asked for an easy way to get the IPsec logs from the VR.
>
> I think this is enough to get the idea across. I have worked through a
> bunch of examples on a white board with my team to try to work through a
> bunch of the use cases, edge cases and deployment scenarios, but I am sure
> there are situations we have not thought through yet.
>
> The current VR has been a constant pain for us and our service customers,
> so it is likely that we will be taking some action one way or another
> because this needs to be addressed. I know there is a lot going on here,
> but hopefully I have been able to get our ideas across.
>
> Please ask questions or make suggestions. Are there major pieces we are
> not taking into account? Do you have use cases which would be simplified
> by this approach that I have not called out?
>
> Would love to hear your feedback...
>
> [1] https://objects-east.cloud.ca/v1/5ef827605f884961b94881e
> 928e7a250/swill/vr_network.png
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Tue, Sep 20, 2016 at 5:54 AM, Murali Reddy <mu...@gmail.com>
> wrote:
>
>> Configuration management of network appliances particularly for Cloud and
>> NFV scenarios is still evolving area. Programmability is the not the
>> strength for even the most popular network operating systems like IOS,
>> JunoS etc. So its not surprising why CloudStack integrates in a archaic way
>> with stock linux for the VR.
>>
>> VR was never integral/hardcoded option in CloudStack. Its always been a
>> plugin. CloudStack network orchestration is well abstracted and designed
>> with vision to compose a network with different set of providers for
>> different services. Yes that vision is not fully realised yet, and we don’t
>> have true service function chains. That would be different discussion topic.
>>
>> I tend to agree with Simon, as alternate/interim option we can take hard
>> look whats causing the problems with current VR integration. Personally, I
>> think it would be easier we take a cue from configuration managers and
>> network configuration solutions out there (for e.g promise theory based
>> Cisco ACI) move to more declarative model of expressing desired state of
>> network configuration. Infact current VR from 4.6, actually holds the
>> desired state per service basis, seems to be in that direction.
>>
>> It does make sense to evaluate new appliances which can provide rich
>> semantics (like programmable API, declarative configuration, versioning,
>> commit/rollback etc), but will need significant engineering effort and time
>> to stabilise. We may make same mistakes with integration of other appliance
>> as well, if we fully don’t understand the nature of the current problems
>> with CloudStack core and service provider interaction and current VR
>> integration.
>>
>>
>> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>>
>> >I think our other option is to take a real look at what it would take to
>> fix the VR. In my opinion, a lot of the problems are related to the
>> monolithic python code base and the fact nothing is actually separated.
>> >
>> >Secondly, the python scripts (and bash scripts) don't use any
>> established libraries to complete tasks and instead shell out and run
>> commands that are both hard to track and hard to parse on return.
>> >
>> >
>> >If we daemonized this, used a real api for Agent to VR communication,
>> used common already existing libraries for the system service and network
>> interactions and spent a bit of time separating out code into distinct
>> modules, everything would behave a lot better.
>> >
>> >
>> >The pain and suffering is due to years and years of patches and constant
>> shelling out to complete tasks in my opinion. If we spend time to rethink
>> how we interact with the VR in general and we abstract the systems and
>> networking stuff and use well known and stable libraries to do the work,
>> the VR would be much easier to maintain.
>> >
>> >
>> >- Si
>> >
>> >
>> >
>> >
>> >________________________________
>> >From: Marty Godsey <ma...@gonsource.com>
>> >Sent: Friday, September 16, 2016 12:24 PM
>> >To: dev@cloudstack.apache.org
>> >Subject: RE: [DISCUSS] Replacing the VR
>> >
>> >So based upon this discussion would it be prudent to wait on VyOS 2.0?
>> The current VR is giving us issues but would the time invested in another
>> "solution" be wasted especially if by the time another option is chose,
>> then coded, then tested, then implemented and right as that time happened
>> to be when VyOS 2.0 is released. Of course you said they are just in the
>> scoping range so this could still be a year or more out.
>> >
>> >Thoughts?
>> >
>> >Regards,
>> >Marty Godsey
>> >nSource Solutions
>> >
>> >-----Original Message-----
>> >From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>> Behalf Of Will Stevens
>> >Sent: Friday, September 16, 2016 10:31 AM
>> >To: dev@cloudstack.apache.org
>> >Cc: daniil@baturin.org
>> >Subject: Re: [DISCUSS] Replacing the VR
>> >
>> >I just had a quick chat with a couple of the guys over on the VyOS chat.
>> >I have CC'ed one of them in case we have more licensing questions.
>> >
>> >So here is the status with the license "the code inherited from Vyatta
>> and our modifications from it is GPLv2 (strict, not v2+). The config
>> reading library is GPLv2 too, so anything that links to is is GPLv2.
>> >Some auxiliary components we made after the fork are more permissive,
>> >LGPLv2+ or MIT."
>> >
>> >They are currently in the process of scoping a redesign (VyOS 2.0), "we
>> are planning a clean rewrite that will solve issues of the current config
>> system".
>> >This will include the ability to configure via the API.
>> >
>> >If we have more questions for VyOS, they are very friendly and
>> responsive, so we should be able to get answers.
>> >
>> >*Will STEVENS*
>> >Lead Developer
>> >
>> >*CloudOps* *| *Cloud Solutions Experts
>> >420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>> @CloudOps_
>> >
>> >On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>> >
>> >> I agree with Will Ilya. There are so many problems with the VR right
>> now.
>> >> Most of the outages we've had recently have somehow involved the VR.
>> >> We set custom iptables rules on the VR which can and have easily gone
>> wrong.
>> >> Openswan is broken, Strongswan replacement still needs to be tested.
>> >> VVRP with redundant router still needs work, and not to mention the
>> >> problems we will have when we introduce IPv6 into the whole picture.
>> >>
>> >> I think the spirit of the discussion is to rely on a 3rd party to do
>> >> the networking for us (eg VyOS) and have us handle just the
>> >> orchestration. All the problems that I've described have already been
>> >> solved in VyOS. We also get the advantage of a potential wider
>> >> community to fix and maintain the VR and given our current development
>> >> velocity, it think it totally makes sense to look for a 3rd party
>> option.
>> >>
>> >> -Syed
>> >>
>> >>
>> >> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
>> >> wrote:
>> >>
>> >> > The VR has been biting us far too often recently, which is why we
>> >> > have started looking into alternative implementations.
>> >> >
>> >> > One of the things that is nice about potentially using the VyOS is
>> >> > that
>> >> it
>> >> > is based on Debian, so we should be able to run the other services
>> >> > that
>> >> we
>> >> > currently have like the password server and userdata on the VyOS.
>> >> > This means we would not have to change our architecture initially
>> >> > and could focus on only replacing the networking paths.
>> >> >
>> >> > *Will STEVENS*
>> >> > Lead Developer
>> >> >
>> >> > *CloudOps* *| *Cloud Solutions Experts
>> >> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>> >> > tw @CloudOps_
>> >> >
>> >> > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>> >> >
>> >> > > The more this is discussed the more I think we should stick with
>> >> > > our
>> >> VR.
>> >> > >
>> >> > > All these other options either seem unfinished or with
>> >> > > incompatible license.
>> >> > >
>> >> > > VyOS looks the most promising so far, it's a serious, mature
>> project.
>> >> > > Adopting it though means we'll have to microservice our way out of
>> >> > > it
>> >> > with
>> >> > > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>> >> those
>> >> > > too. Imho this adds complexity we should void.
>> >> > >
>> >> > > --
>> >> > > Sent from the Delta quadrant using Borg technology!
>> >> > >
>> >> > > Nux!
>> >> > > www.nux.ro
>> >> > >
>> >> > > ----- Original Message -----
>> >> > > > From: "Will Stevens" <ws...@cloudops.com>
>> >> > > > To: dev@cloudstack.apache.org
>> >> > > > Sent: Thursday, 15 September, 2016 17:21:28
>> >> > > > Subject: Re: [DISCUSS] Replacing the VR
>> >> > >
>> >> > > > Ya, we would need to add a daemon for VPN as well. Load
>> >> > > > balancing is another aspect which we will need to consider if we
>> went this route.
>> >> > > > Something like https://traefik.io/ could potentially be a good
>> >> > > > fit
>> >> due
>> >> > > to
>> >> > > > its API driven configuration, but it may be more than what we
>> need.
>> >> > > >
>> >> > > > We should probably try define which pieces make sense to be
>> >> > > > solved
>> >> > > together
>> >> > > > and which pieces would be best suited to be broken out.
>> >> > > >
>> >> > > > I think the network connectivity, routing and firewalling should
>> >> > probably
>> >> > > > all stay together since the majority of the tools we would
>> >> potentially
>> >> > > use
>> >> > > > would handle all of that together in a single implementation.
>> >> > > >
>> >> > > > The password server and userdata seems like a good option for
>> >> > > > being
>> >> > > broken
>> >> > > > out and handled independently (and probably rewritten completely
>> >> since
>> >> > > they
>> >> > > > currently have some issues).
>> >> > > >
>> >> > > > Load balancing is another that could warrant splitting out, but
>> >> > > > that depends on what direction we go and how we would be
>> managing it.
>> >> DHCP
>> >> > > and
>> >> > > > DNS are others which could go either way.
>> >> > > >
>> >> > > > If we do split out services, I think we should consolidate as
>> >> > > > much as
>> >> > we
>> >> > > > can into each service we break out. Ideally a network packet
>> >> > > > would
>> >> > never
>> >> > > > hit more than one, maybe two, services. I don't think we should
>> >> > > > be splitting services 'just because', I think we need a valid
>> >> > > > case for splitting any service out because it adds complexity.
>> >> > > > Our project is already complex enough, we need to avoid adding
>> >> > > > complexity unless it
>> >> is
>> >> > > > really needed.
>> >> > > >
>> >> > > > Some more of my thoughts on this anyway...
>> >> > > >
>> >> > > > *Will STEVENS*
>> >> > > > Lead Developer
>> >> > > >
>> >> > > > *CloudOps* *| *Cloud Solutions Experts
>> >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>> >> > > > *|* tw @CloudOps_
>> >> > > >
>> >> > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>> >> > wrote:
>> >> > > >
>> >> > > >> I do agree with you that this probably isn't the right place
>> >> > > >> the
>> >> > > password
>> >> > > >> service and user data.
>> >> > > >>
>> >> > > >>
>> >> > > >> Having said that, after taking a cursory look at the dev docs,
>> >> > > >> it
>> >> > > doesn't
>> >> > > >> seem that difficult to add new daemons:
>> >> https://opensnaproute.github.
>> >> > > >> io/docs/developer.html#creating-new-component
>> >> > > >>
>> >> > > >> <https://opensnaproute.github.io/docs/developer.html#
>> >> > > >> creating-new-component>
>> >> > > >>
>> >> > > >>
>> >> > > >> They've definitely build it with a microservices architecture
>> >> > > >> in
>> >> mind,
>> >> > > so
>> >> > > >> each individual feature is abstracted into it's own small
>> >> > > >> daemon
>> >> > > process.
>> >> > > >> We could just create a daemon for the password server and the
>> >> userdata
>> >> > > >> components if we really had to.
>> >> > > >>
>> >> > > >>
>> >> > > >> - Si
>> >> > > >>
>> >> > > >>
>> >> > > >> ________________________________
>> >> > > >> From: williamstevens@gmail.com <wi...@gmail.com> on
>> >> > > >> behalf
>> >> > of
>> >> > > >> Will Stevens <ws...@cloudops.com>
>> >> > > >> Sent: Thursday, September 15, 2016 9:17 AM
>> >> > > >> To: dev@cloudstack.apache.org
>> >> > > >> Subject: Re: [DISCUSS] Replacing the VR
>> >> > > >>
>> >> > > >> A big part of why I know about it is because it is written in
>> Go.
>> >> :P
>> >> > > >>
>> >> > > >> Yes, it is definitely interesting for the routing and traffic
>> >> handling
>> >> > > >> aspects of the VR. We will likely have to rethink some of the
>> >> pieces
>> >> > a
>> >> > > >> little bit like the password server and userdata if we are to
>> >> > > >> adopt
>> >> a
>> >> > > >> different VR approach. This is where I think some of JohnB and
>> >> > > Chiradeep's
>> >> > > >> ideas make sense. In many ways, it does not make sense for the
>> >> device
>> >> > > >> handling routing and network traffic to also be responsible for
>> >> > > passwords
>> >> > > >> and userdata.
>> >> > > >>
>> >> > > >> *Will STEVENS*
>> >> > > >> Lead Developer
>> >> > > >>
>> >> > > >> *CloudOps* *| *Cloud Solutions Experts
>> >> > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>> >> > > >> *|* tw @CloudOps_
>> >> > > >>
>> >> > > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>> >> > wrote:
>> >> > > >>
>> >> > > >> > I hadn't heard of Flexswitch until you mentioned it. It looks
>> >> pretty
>> >> > > >> cool!
>> >> > > >> > It even supports ONIE install.
>> >> > > >> >
>> >> > > >> > To be honest, the ipsec feature could be added, or we could
>> >> offload
>> >> > > it to
>> >> > > >> > separate vm if we needed to. The fact it is so feature rich
>> >> > > >> > from a
>> >> > > >> routing
>> >> > > >> > perspective (and all API driven) is really nice.
>> >> > > >> >
>> >> > > >> >
>> >> > > >> > Based on the roadmap, it looks like they plan to also support
>> >> > > >> capabilities
>> >> > > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>> >> > > >> > will
>> >> be
>> >> > > huge
>> >> > > >> > for our carrier community that rely on these technologies to
>> >> > > >> > do
>> >> > > private
>> >> > > >> > gateway and inter-VPC interconnections today. We handle this
>> >> > > >> > stuff
>> >> > on
>> >> > > our
>> >> > > >> > ASRs right now with a vlan interconnect into the VR. Being
>> >> > > >> > able to
>> >> > do
>> >> > > >> MPLS
>> >> > > >> > all the way to the VR would be awesome.
>> >> > > >> >
>> >> > > >> >
>> >> > > >> > It also seems to be written in GO (a language here at ENA we
>> >> > > >> > know
>> >> > very
>> >> > > >> > well).
>> >> > > >> >
>> >> > > >> >
>> >> > > >> > - Si
>> >> > > >> >
>> >> > > >> >
>> >> > > >> >
>> >> > > >> > ________________________________
>> >> > > >> > From: Will Stevens <wi...@gmail.com>
>> >> > > >> > Sent: Thursday, September 15, 2016 7:06 AM
>> >> > > >> > To: dev@cloudstack.apache.org
>> >> > > >> > Subject: RE: [DISCUSS] Replacing the VR
>> >> > > >> >
>> >> > > >> > Ya. I don't think it covers our whole use case, but what it
>> >> > > >> > does
>> >> > > cover is
>> >> > > >> > all api driven...
>> >> > > >> >
>> >> > > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>> >> > wrote:
>> >> > > >> >
>> >> > > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it
>> >> > > >> > > was
>> >> not
>> >> > > >> > > intended to do IPSec.
>> >> > > >> > >
>> >> > > >> > > It seems as though VyOS is starting to look like the best
>> >> option.
>> >> > > >> > >
>> >> > > >> > > Regards,
>> >> > > >> > > Marty Godsey
>> >> > > >> > > nSource Solutions
>> >> > > >> > >
>> >> > > >> > > -----Original Message-----
>> >> > > >> > > From: williamstevens@gmail.com
>> >> > > >> > > [mailto:williamstevens@gmail.com
>> >> ]
>> >> > On
>> >> > > >> > > Behalf Of Will Stevens
>> >> > > >> > > Sent: Wednesday, September 14, 2016 11:06 PM
>> >> > > >> > > To: dev@cloudstack.apache.org
>> >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
>> >> > > >> > >
>> >> > > >> > > Or we could go completely crazy and go with something like
>> >> > > FlexSwitch
>> >> > > >> > from
>> >> > > >> > > SnapRoute
>> >> > > >> > > - http://www.snaproute.com/
>> >> > > >> > > - https://opensnaproute.github.io/docs/apis.html
>> >> > > >> > >
>> >> > > >> > > *Will STEVENS*
>> >> > > >> > > Lead Developer
>> >> > > >> > >
>> >> > > >> > > *CloudOps* *| *Cloud Solutions Experts
>> >> > > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>> >> > > >> > > cloudops.com
>> >> > *|*
>> >> > > tw
>> >> > > >> > > @CloudOps_
>> >> > > >> > >
>> >> > > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>> >> > > wstevens@cloudops.com>
>> >> > > >> > > wrote:
>> >> > > >> > >
>> >> > > >> > > > I tend to agree with Syed and Marty. I am not sure what
>> >> > problems
>> >> > > are
>> >> > > >> > > > solved by splitting up the function of the VR into a
>> >> > > >> > > > bunch of
>> >> > > >> separate
>> >> > > >> > > > services. As Syed points out, the complexity added is
>> >> > > non-trivial.
>> >> > > >> > > > We now have to manage all the intercontainer networking
>> >> > > >> > > > as
>> >> well
>> >> > as
>> >> > > >> the
>> >> > > >> > > > orchestrated ACS networking.
>> >> > > >> > > >
>> >> > > >> > > > VyOS is interesting to me because it covers the majority
>> >> > > >> > > > of
>> >> our
>> >> > > use
>> >> > > >> > > > case with a single unified control plane. It also has
>> >> > > >> > > > good
>> >> > > support
>> >> > > >> > > > for extending features we care about, like IPv6, VXLAN,
>> >> > > >> > > > VRRP, transactions, etc...
>> >> > > >> > > >
>> >> > > >> > > > *Will STEVENS*
>> >> > > >> > > > Lead Developer
>> >> > > >> > > >
>> >> > > >> > > > *CloudOps* *| *Cloud Solutions Experts
>> >> > > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>> >> cloudops.com
>> >> > > *|*
>> >> > > >> tw
>> >> > > >> > > > @CloudOps_
>> >> > > >> > > >
>> >> > > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>> >> > sahmed@cloudops.com>
>> >> > > >> > wrote:
>> >> > > >> > > >
>> >> > > >> > > >> Agree with Marty, adding Docker containers to the
>> >> > > >> > > >> picture
>> >> > > although
>> >> > > >> > > >> can make the VR more flexible but the added complexity
>> >> > > >> > > >> is
>> >> just
>> >> > > not
>> >> > > >> > > >> worth it. Not to mention we would need to take care of
>> >> > networking
>> >> > > >> > > >> each container manually and given that our iptable rules
>> >> > > >> > > >> are
>> >> > very
>> >> > > >> > > >> unstable at the moment I don't see a big value add.
>> >> > > >> > > >>
>> >> > > >> > > >> Vyos looks like a better solution to me. I know that it
>> >> > > >> > > >> does
>> >> > not
>> >> > > >> > > >> provide an api but it does fit the bill quite well
>> >> otherwise. I
>> >> > > >> > > >> specially like the fact that it has a transaction based
>> >> > > >> > > >> model
>> >> > and
>> >> > > >> you
>> >> > > >> > > >> can rollback changes if something goes wrong.
>> >> > > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>> >> > > marty@gonsource.com>
>> >> > > >> > > wrote:
>> >> > > >> > > >>
>> >> > > >> > > >> > Licensing aside, I think splitting the various
>> >> > > >> > > >> > functions
>> >> into
>> >> > > >> > > >> > containers is not a good route either. This will force
>> >> users
>> >> > to
>> >> > > >> > > >> > have to maintain
>> >> > > >> > > >> and
>> >> > > >> > > >> > use containers and adds complexity to the networking
>> >> aspects
>> >> > of
>> >> > > >> ACS.
>> >> > > >> > > >> > Complexity decreases stability. Now I understand the
>> >> argument
>> >> > > that
>> >> > > >> > > >> > a monolithic approach also brings its own set of
>> >> > > >> > > >> > issues but
>> >> > it
>> >> > > >> also
>> >> > > >> > > >> > simplifies it.
>> >> > > >> > > >> >
>> >> > > >> > > >> > Regards,
>> >> > > >> > > >> > Marty Godsey
>> >> > > >> > > >> > nSource Solutions
>> >> > > >> > > >> >
>> >> > > >> > > >> > -----Original Message-----
>> >> > > >> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>> >> > > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
>> >> > > >> > > >> > To: dev@cloudstack.apache.org
>> >> > > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR
>> >> > > >> > > >> >
>> >> > > >> > > >> > I rather doubt that the Cloudrouter will fit the needs
>> >> > > >> > > >> > of
>> >> the
>> >> > > >> > > >> > CloudStack project
>> >> > > >> > > >> > - it is AGPL licensed. Many enterprises will not
>> >> > > >> > > >> > touch
>> >> > > anything
>> >> > > >> > > >> > that
>> >> > > >> > > >> has
>> >> > > >> > > >> > AGPL
>> >> > > >> > > >> > - the github repo shows rather infrequent updates.
>> >> > > >> > > >> > Quite
>> >> > > likely
>> >> > > >> > > >> > they aren't considering the use cases of the
>> >> > > >> > > >> > CloudStack
>> >> > > community
>> >> > > >> > > >> >
>> >> > > >> > > >> > I'd back John B's comments on disaggregating the VR.
>> >> > > >> > > >> > Split
>> >> it
>> >> > > into
>> >> > > >> > > >> > many docker containers
>> >> > > >> > > >> > - password server
>> >> > > >> > > >> > - userdata server
>> >> > > >> > > >> > - DHCP / DNS
>> >> > > >> > > >> > - s2s VPN
>> >> > > >> > > >> > - RA VPN
>> >> > > >> > > >> > - intra-VPC routing and ACL
>> >> > > >> > > >> > - Port forwarding + NAT
>> >> > > >> > > >> > - FW
>> >> > > >> > > >> > - LB (public)
>> >> > > >> > > >> > - LB (internal),
>> >> > > >> > > >> > - secondary storage
>> >> > > >> > > >> > - agent
>> >> > > >> > > >> > Glue them together with docker compose files (one per
>> >> > > >> > > >> > use
>> >> > > case -
>> >> > > >> > > >> > basic zone, isolated, VPC, SSVM, etc).
>> >> > > >> > > >> >
>> >> > > >> > > >> > The VR image then becomes a JeOS + docker. You can
>> >> > > >> > > >> > test
>> >> each
>> >> > of
>> >> > > >> the
>> >> > > >> > > >> > components independently and fixing one bug in the
>> >> > > >> > > >> > field
>> >> (say
>> >> > > >> DHCP)
>> >> > > >> > > >> > is hitless to the other components. You don't need to
>> >> > > >> > > >> > build per-hypervisor VRs. You could even run on
>> baremetal.
>> >> > > >> > > >> >
>> >> > > >> > > >> > Along the way you need to figure out how to
>> >> > > >> > > >> > - make the traffic traverse the containers that are
>> >> > > >> > > >> > needed
>> >> > to
>> >> > > be
>> >> > > >> > > >> > traversed (in most cases just 1)
>> >> > > >> > > >> > - bootstrap the router (how does it find its compose
>> file?
>> >> > > where
>> >> > > >> > > >> > is the
>> >> > > >> > > >> > registry?)
>> >> > > >> > > >> > - rethink the command and control of the VR
>> >> > > >> > > >> > functions. SSH
>> >> > > works,
>> >> > > >> > > >> > but something more declarative, idempotent should be
>> >> > explored.
>> >> > > >> > > >> >
>> >> > > >> > > >> > As you do this, it becomes clearer which of the
>> >> > > >> > > >> > functions
>> >> can
>> >> > > be
>> >> > > >> > > >> > substituted by for example CloudRouter. Command and
>> >> > > >> > > >> > Control
>> >> > of
>> >> > > the
>> >> > > >> > > >> docker
>> >> > > >> > > >> > containers can be moved out to another container. Etc.
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>> >> > > >> > > >> > <ma...@gonsource.com>
>> >> > > >> > > >> > wrote:
>> >> > > >> > > >> >
>> >> > > >> > > >> > > This one does look nice. My biggest concern is the
>> >> > > >> > > >> > > lack
>> >> of
>> >> > > >> > > >> > > VXLANs. It seems that any of the ones we mentioned
>> >> > > >> > > >> > > do not
>> >> > > have
>> >> > > >> an
>> >> > > >> > > >> > > API so we may be stuck at the SSH method.
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > Regards,
>> >> > > >> > > >> > > Marty Godsey
>> >> > > >> > > >> > > nSource Solutions
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > -----Original Message-----
>> >> > > >> > > >> > > From: Abhinandan Prateek
>> >> > > >> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
>> >> > > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
>> >> > > >> > > >> > > To: dev@cloudstack.apache.org
>> >> > > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > Cloudrouter looks promising. These have potential to
>> >> > > >> > > >> > > save
>> >> > > future
>> >> > > >> > > >> > > engineering effort for example on ipv6 routing, OSPF
>> etc.
>> >> > > >> > > >> > > And the best part is they come with test automation
>> >> > > framework.
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
>> >> > > >> > > >> > > <ja...@accelerite.com>
>> >> > > >> > > >> > > wrote:
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > >Hi,
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >Instead of replacing the VR in first place we
>> >> > > >> > > >> > > >should add VyOS/cloudrouter
>> >> > > >> > > >> > > as provider. Once it is stable, network offerings
>> >> > > >> > > >> > > (on
>> >> > > upgrade)
>> >> > > >> > > >> > > can be updated to use it and we can drop the VR if
>> >> > > >> > > >> > > we
>> >> want
>> >> > at
>> >> > > >> > > >> > > that release
>> >> > > >> > > >> > onwards.
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >VR is stabilized over a period of time and some of
>> >> > > >> > > >> > > >them
>> >> > are
>> >> > > >> > > >> > > >running
>> >> > > >> > > >> > > without issues. When we replicate the ACS VR
>> >> > > >> > > >> > > features in
>> >> > new
>> >> > > >> > > >> > > solution it takes some to find the missing pieces
>> >> > > >> > > >> > > (hidden
>> >> > > bugs).
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >Thanks,
>> >> > > >> > > >> > > >Jayapal
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >> nux@li.nux.ro> wrote:
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> Hi,
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> I like the idea.
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> Cloudrouter looks really promising, I'm not too
>> >> > > >> > > >> > > >> keen
>> >> on
>> >> > > VyOS
>> >> > > >> > > >> > > >> (it
>> >> > > >> > > >> > > doesn't have a proper http api etc).
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> --
>> >> > > >> > > >> > > >> Sent from the Delta quadrant using Borg
>> technology!
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> Nux!
>> >> > > >> > > >> > > >> www.nux.ro
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > abhinandan.prateek@shapeblue.com
>> >> > > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
>> >> > > >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>> >> > > @shapeblue
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > ----- Original Message -----
>> >> > > >> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
>> >> > > >> > > >> > > >>> To: dev@cloudstack.apache.org
>> >> > > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
>> >> > > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and
>> >> > > >> > > >> > > >>> should
>> >> > be
>> >> > > >> > > >> > > >>> treated as
>> >> > > >> > > >> > > such.
>> >> > > >> > > >> > > >>> Please weigh in with the good and bad of this
>> idea...
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> A couple of us have been discussing the idea of
>> >> > > potentially
>> >> > > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open
>> >> > > >> > > >> > > >>> Source
>> >> > > Vyatta
>> >> > > >> > VM).
>> >> > > >> > > >> > > >>> There may be a license issue because I think it
>> >> > > >> > > >> > > >>> is
>> >> > > licensed
>> >> > > >> > > >> > > >>> under GPL, but for the sake of discussion, let's
>> >> assume
>> >> > > we
>> >> > > >> > > >> > > >>> can overcome any
>> >> > > >> > > >> > > license issues.
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> I have spent some time recently with the VyOS
>> >> > > >> > > >> > > >>> and I
>> >> > have
>> >> > > to
>> >> > > >> > > >> > > >>> admit, I was pretty impressed. It is simple and
>> >> > > intuitive
>> >> > > >> > > >> > > >>> and it gives you a lot more options for auditing
>> >> > > >> > > >> > > >>> the
>> >> > > >> > > configuration etc...
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> Items of potential interest:
>> >> > > >> > > >> > > >>> - Clean up our current VR script spaghetti to a
>> >> simpler
>> >> > > more
>> >> > > >> > > >> > > >>> auditable configuration workflow.
>> >> > > >> > > >> > > >>> - Gives a cleaner path for IPv6 support.
>> >> > > >> > > >> > > >>> - Handles VPN configuration via the same
>> >> configuration
>> >> > > >> > > interface.
>> >> > > >> > > >> > > >>> - Support for OSPF & BGP.
>> >> > > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
>> >> > > >> > > >> > > >>> - Easily supports HA (redundant routers) through
>> >> VRRP.
>> >> > > >> > > >> > > >>> - VXLAN support.
>> >> > > >> > > >> > > >>> - Transaction based changes to the VR with
>> >> > > >> > > >> > > >>> rollback
>> >> on
>> >> > > >> error.
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> Items that could be difficult to solve:
>> >> > > >> > > >> > > >>> - Userdata password reset workflow and
>> >> implementation.
>> >> > > >> > > >> > > >>> - Upgrade process.
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> The VyOS is not the only option if we were to
>> >> consider
>> >> > > this
>> >> > > >> > > >> approach.
>> >> > > >> > > >> > > >>> Another option, which I don't know as well,
>> >> > > >> > > >> > > >>> would be CloudRouter (AGPL
>> >> > > >> > > >> > > >>> license) [2] which is purely API driven.
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> Anyway, would love to hear your thoughts...
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> Will
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> [1] https://vyos.io/ [2]
>> >> > > >> > > >> > > >>> https://cloudrouter.org/
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >DISCLAIMER
>> >> > > >> > > >> > > >==========
>> >> > > >> > > >> > > >This e-mail may contain privileged and confidential
>> >> > > information
>> >> > > >> > > >> > > >which is
>> >> > > >> > > >> > > the property of Accelerite, a Persistent Systems
>> >> business.
>> >> > > It is
>> >> > > >> > > >> > > intended only for the use of the individual or
>> >> > > >> > > >> > > entity to
>> >> > > which
>> >> > > >> it
>> >> > > >> > > >> > > is addressed. If you are not the intended recipient,
>> >> > > >> > > >> > > you
>> >> > are
>> >> > > not
>> >> > > >> > > >> > > authorized to read, retain, copy, print, distribute
>> >> > > >> > > >> > > or
>> >> use
>> >> > > this
>> >> > > >> > > >> > > message. If you have received this communication in
>> >> error,
>> >> > > >> please
>> >> > > >> > > >> > > notify the sender and delete all copies of this
>> message.
>> >> > > >> > > >> > > Accelerite, a Persistent Systems business does not
>> >> > > >> > > >> > > accept
>> >> > any
>> >> > > >> > > >> > > liability for virus
>> >> > > >> > > >> > infected mails.
>> >> > > >> > > >> > >
>> >> > > >> > > >> >
>> >> > > >> > > >>
>> >> > > >> > > >
>> >> > > >> > > >
>> >> > > >> > >
>> >> > > >> >
>> >> > >
>> >> >
>>
>>
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
If the link wrapped, be sure to remove the space that was added by the
wrap. The hash in the URL should be: 5ef827605f884961b94881e928e7a250
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Tue, Sep 20, 2016 at 3:51 PM, Will Stevens <ws...@cloudops.com> wrote:
> I am going to try to do a bit of a brain dump from me and my team in this
> post, and hopefully I can verbalize the ideas well enough that you can all
> follow along. Please provide feedback, ideas and suggestions...
>
> *Preface:*
> *--------*
> I am currently approaching this problem with the following use cases in
> mind.
>
> - Stabilize/replace the VR for our production cloud deployments as well as
> give the community a free, open source and production grade alternative to
> the current VR.
>
> - I have been scoping the integration of the Fortinet Fortigate (hardware
> and virtual) into ACS to support the following features.
> -- All Isolated Guest Network traffic features (Firewall, PF, NAT,
> Client-to-Site VPN, etc)
> -- All VPC network traffic features (ACLs, PF, NAT, Client-to-Site and
> Site-to-Site VPN, Private Gateway)
> -- Public IPv6 support (NAT64 and NAT46) with IPv4 on the guest side.
>
> *The Setup:*
> *----------*
> I am not going to get into the details of the Fortinet integration details
> in this thread, you can expect a function spec from me soon describing my
> plans for this work. I am currently waiting on hardware to validate my
> implementation strategy.
>
> In this post I will be focusing on the way ACS deals with the VR, network
> plugins and how this could be potentially adapted to give us more
> flexibility going forward.
>
> I have created a diagram [1] to outline the ideas presented in this post.
>
> So if you don't like my ideas here, you can simply use the VR as you
> currently do and you can basically ignore the dotted circles to the right
> of the 'existing VR' in the solid circle.
>
> The basic idea here, is when you create a network service offering and you
> select different Providers to handle different network capabilities, you
> can also specify a VM template for those providers (as well as service
> offering) which will be used for additional VMs which will be created to
> provision those services. In this context, think of templates such as
> VyOS, FortiOS, or a VPX to understand the spirit of this approach.
>
> When a network is provisioned, the current VR will be deployed (ideally
> only offering services like; password server, userdata, DHCP, DNS), and in
> addition, any network provider templates which have been specified will
> also be provisioned. Each 'network VM' will get a leg on both the public
> VLAN and the guest VLAN. In the case of a VPC, the first tier will be a
> little special (I will get into this in a bit) and will not be able to be
> deleted until the VR(s) are all cleaned up and the network is completely
> destroyed. This network is denoted as the 'Client MGMT' network in the
> diagram [1].
>
> You will notice that ONLY the 'existing VR' has network connectivity to
> the ACS MGMT server. It may not necessarily be through a 169.x.x.x
> address, but that illustrates the point for now. The reason for this is
> because we can not guarantee that a Guest User does may not gain access to
> these VRs, and in cases I will get into later, the Guest User may actually
> be given access to those VRs. This does however mean that the ACS MGMT
> server does not have direct access to those VRs in order to configure
> them. To solve this, the 'existing VR' would act as a proxy for the ACS
> Management server over the ACS MGMT network to pass configuration commands
> to the VRs to the right of the 'existing VR' depicted in the graphic.
>
> *The Details:*
> *------------*
> Now that you have a basic idea of the deployment structure I have in mind,
> let's discuss some of the finer details.
>
> - We should be able to relatively easily adapt existing external network
> plugins to be able to be provisioned with this mechanism (assuming they
> have a VM option). The Palo Alto would be an example which I would likely
> use as a proof of concept for this adaptation. It would be important that
> the external plugin could work as an external physical appliance OR as a
> dynamically provisioned VM appliance.
>
> - Enterprise grade network appliances which support VM deployment will no
> longer have to be pre-deployed and shared, but instead will be able to be
> dynamically provisioned on a network by network basis.
>
> - Existing external network appliance plugins targeting physical boxes
> will still work in concert with this approach without any need for
> modification.
>
> - We could add an 'Allow Guest Access' checkbox to the service provider
> section in the network service offering to give the Guest User access to
> their network appliance. This is especially useful for an implementation
> like the FortiGate because it allows for the user to configure UTM features
> which ACS does not orchestrate but the appliance is capable of. This also
> applies to devices like the NetScaler. In the diagram [1] this is depicted
> by the dashed green lines between the devices and the 'Client MGMT'
> network. So in the case of the NetScaler, you would get an NSIP on the
> Client MGMT network which would allow the Guest User to connect to that
> box. I envision a URL pointing to the guest network IP with
> username/password returned for a network which a user can use to connect
> into that network appliance.
>
> - Similar to above, the Guest User will have much more visibility into the
> logging and access on the network. I can't count how many times a customer
> has asked for an easy way to get the IPsec logs from the VR.
>
> I think this is enough to get the idea across. I have worked through a
> bunch of examples on a white board with my team to try to work through a
> bunch of the use cases, edge cases and deployment scenarios, but I am sure
> there are situations we have not thought through yet.
>
> The current VR has been a constant pain for us and our service customers,
> so it is likely that we will be taking some action one way or another
> because this needs to be addressed. I know there is a lot going on here,
> but hopefully I have been able to get our ideas across.
>
> Please ask questions or make suggestions. Are there major pieces we are
> not taking into account? Do you have use cases which would be simplified
> by this approach that I have not called out?
>
> Would love to hear your feedback...
>
> [1] https://objects-east.cloud.ca/v1/5ef827605f884961b94881e
> 928e7a250/swill/vr_network.png
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Tue, Sep 20, 2016 at 5:54 AM, Murali Reddy <mu...@gmail.com>
> wrote:
>
>> Configuration management of network appliances particularly for Cloud and
>> NFV scenarios is still evolving area. Programmability is the not the
>> strength for even the most popular network operating systems like IOS,
>> JunoS etc. So its not surprising why CloudStack integrates in a archaic way
>> with stock linux for the VR.
>>
>> VR was never integral/hardcoded option in CloudStack. Its always been a
>> plugin. CloudStack network orchestration is well abstracted and designed
>> with vision to compose a network with different set of providers for
>> different services. Yes that vision is not fully realised yet, and we don’t
>> have true service function chains. That would be different discussion topic.
>>
>> I tend to agree with Simon, as alternate/interim option we can take hard
>> look whats causing the problems with current VR integration. Personally, I
>> think it would be easier we take a cue from configuration managers and
>> network configuration solutions out there (for e.g promise theory based
>> Cisco ACI) move to more declarative model of expressing desired state of
>> network configuration. Infact current VR from 4.6, actually holds the
>> desired state per service basis, seems to be in that direction.
>>
>> It does make sense to evaluate new appliances which can provide rich
>> semantics (like programmable API, declarative configuration, versioning,
>> commit/rollback etc), but will need significant engineering effort and time
>> to stabilise. We may make same mistakes with integration of other appliance
>> as well, if we fully don’t understand the nature of the current problems
>> with CloudStack core and service provider interaction and current VR
>> integration.
>>
>>
>> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>>
>> >I think our other option is to take a real look at what it would take to
>> fix the VR. In my opinion, a lot of the problems are related to the
>> monolithic python code base and the fact nothing is actually separated.
>> >
>> >Secondly, the python scripts (and bash scripts) don't use any
>> established libraries to complete tasks and instead shell out and run
>> commands that are both hard to track and hard to parse on return.
>> >
>> >
>> >If we daemonized this, used a real api for Agent to VR communication,
>> used common already existing libraries for the system service and network
>> interactions and spent a bit of time separating out code into distinct
>> modules, everything would behave a lot better.
>> >
>> >
>> >The pain and suffering is due to years and years of patches and constant
>> shelling out to complete tasks in my opinion. If we spend time to rethink
>> how we interact with the VR in general and we abstract the systems and
>> networking stuff and use well known and stable libraries to do the work,
>> the VR would be much easier to maintain.
>> >
>> >
>> >- Si
>> >
>> >
>> >
>> >
>> >________________________________
>> >From: Marty Godsey <ma...@gonsource.com>
>> >Sent: Friday, September 16, 2016 12:24 PM
>> >To: dev@cloudstack.apache.org
>> >Subject: RE: [DISCUSS] Replacing the VR
>> >
>> >So based upon this discussion would it be prudent to wait on VyOS 2.0?
>> The current VR is giving us issues but would the time invested in another
>> "solution" be wasted especially if by the time another option is chose,
>> then coded, then tested, then implemented and right as that time happened
>> to be when VyOS 2.0 is released. Of course you said they are just in the
>> scoping range so this could still be a year or more out.
>> >
>> >Thoughts?
>> >
>> >Regards,
>> >Marty Godsey
>> >nSource Solutions
>> >
>> >-----Original Message-----
>> >From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>> Behalf Of Will Stevens
>> >Sent: Friday, September 16, 2016 10:31 AM
>> >To: dev@cloudstack.apache.org
>> >Cc: daniil@baturin.org
>> >Subject: Re: [DISCUSS] Replacing the VR
>> >
>> >I just had a quick chat with a couple of the guys over on the VyOS chat.
>> >I have CC'ed one of them in case we have more licensing questions.
>> >
>> >So here is the status with the license "the code inherited from Vyatta
>> and our modifications from it is GPLv2 (strict, not v2+). The config
>> reading library is GPLv2 too, so anything that links to is is GPLv2.
>> >Some auxiliary components we made after the fork are more permissive,
>> >LGPLv2+ or MIT."
>> >
>> >They are currently in the process of scoping a redesign (VyOS 2.0), "we
>> are planning a clean rewrite that will solve issues of the current config
>> system".
>> >This will include the ability to configure via the API.
>> >
>> >If we have more questions for VyOS, they are very friendly and
>> responsive, so we should be able to get answers.
>> >
>> >*Will STEVENS*
>> >Lead Developer
>> >
>> >*CloudOps* *| *Cloud Solutions Experts
>> >420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>> @CloudOps_
>> >
>> >On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>> >
>> >> I agree with Will Ilya. There are so many problems with the VR right
>> now.
>> >> Most of the outages we've had recently have somehow involved the VR.
>> >> We set custom iptables rules on the VR which can and have easily gone
>> wrong.
>> >> Openswan is broken, Strongswan replacement still needs to be tested.
>> >> VVRP with redundant router still needs work, and not to mention the
>> >> problems we will have when we introduce IPv6 into the whole picture.
>> >>
>> >> I think the spirit of the discussion is to rely on a 3rd party to do
>> >> the networking for us (eg VyOS) and have us handle just the
>> >> orchestration. All the problems that I've described have already been
>> >> solved in VyOS. We also get the advantage of a potential wider
>> >> community to fix and maintain the VR and given our current development
>> >> velocity, it think it totally makes sense to look for a 3rd party
>> option.
>> >>
>> >> -Syed
>> >>
>> >>
>> >> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
>> >> wrote:
>> >>
>> >> > The VR has been biting us far too often recently, which is why we
>> >> > have started looking into alternative implementations.
>> >> >
>> >> > One of the things that is nice about potentially using the VyOS is
>> >> > that
>> >> it
>> >> > is based on Debian, so we should be able to run the other services
>> >> > that
>> >> we
>> >> > currently have like the password server and userdata on the VyOS.
>> >> > This means we would not have to change our architecture initially
>> >> > and could focus on only replacing the networking paths.
>> >> >
>> >> > *Will STEVENS*
>> >> > Lead Developer
>> >> >
>> >> > *CloudOps* *| *Cloud Solutions Experts
>> >> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>> >> > tw @CloudOps_
>> >> >
>> >> > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>> >> >
>> >> > > The more this is discussed the more I think we should stick with
>> >> > > our
>> >> VR.
>> >> > >
>> >> > > All these other options either seem unfinished or with
>> >> > > incompatible license.
>> >> > >
>> >> > > VyOS looks the most promising so far, it's a serious, mature
>> project.
>> >> > > Adopting it though means we'll have to microservice our way out of
>> >> > > it
>> >> > with
>> >> > > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>> >> those
>> >> > > too. Imho this adds complexity we should void.
>> >> > >
>> >> > > --
>> >> > > Sent from the Delta quadrant using Borg technology!
>> >> > >
>> >> > > Nux!
>> >> > > www.nux.ro
>> >> > >
>> >> > > ----- Original Message -----
>> >> > > > From: "Will Stevens" <ws...@cloudops.com>
>> >> > > > To: dev@cloudstack.apache.org
>> >> > > > Sent: Thursday, 15 September, 2016 17:21:28
>> >> > > > Subject: Re: [DISCUSS] Replacing the VR
>> >> > >
>> >> > > > Ya, we would need to add a daemon for VPN as well. Load
>> >> > > > balancing is another aspect which we will need to consider if we
>> went this route.
>> >> > > > Something like https://traefik.io/ could potentially be a good
>> >> > > > fit
>> >> due
>> >> > > to
>> >> > > > its API driven configuration, but it may be more than what we
>> need.
>> >> > > >
>> >> > > > We should probably try define which pieces make sense to be
>> >> > > > solved
>> >> > > together
>> >> > > > and which pieces would be best suited to be broken out.
>> >> > > >
>> >> > > > I think the network connectivity, routing and firewalling should
>> >> > probably
>> >> > > > all stay together since the majority of the tools we would
>> >> potentially
>> >> > > use
>> >> > > > would handle all of that together in a single implementation.
>> >> > > >
>> >> > > > The password server and userdata seems like a good option for
>> >> > > > being
>> >> > > broken
>> >> > > > out and handled independently (and probably rewritten completely
>> >> since
>> >> > > they
>> >> > > > currently have some issues).
>> >> > > >
>> >> > > > Load balancing is another that could warrant splitting out, but
>> >> > > > that depends on what direction we go and how we would be
>> managing it.
>> >> DHCP
>> >> > > and
>> >> > > > DNS are others which could go either way.
>> >> > > >
>> >> > > > If we do split out services, I think we should consolidate as
>> >> > > > much as
>> >> > we
>> >> > > > can into each service we break out. Ideally a network packet
>> >> > > > would
>> >> > never
>> >> > > > hit more than one, maybe two, services. I don't think we should
>> >> > > > be splitting services 'just because', I think we need a valid
>> >> > > > case for splitting any service out because it adds complexity.
>> >> > > > Our project is already complex enough, we need to avoid adding
>> >> > > > complexity unless it
>> >> is
>> >> > > > really needed.
>> >> > > >
>> >> > > > Some more of my thoughts on this anyway...
>> >> > > >
>> >> > > > *Will STEVENS*
>> >> > > > Lead Developer
>> >> > > >
>> >> > > > *CloudOps* *| *Cloud Solutions Experts
>> >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>> >> > > > *|* tw @CloudOps_
>> >> > > >
>> >> > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>> >> > wrote:
>> >> > > >
>> >> > > >> I do agree with you that this probably isn't the right place
>> >> > > >> the
>> >> > > password
>> >> > > >> service and user data.
>> >> > > >>
>> >> > > >>
>> >> > > >> Having said that, after taking a cursory look at the dev docs,
>> >> > > >> it
>> >> > > doesn't
>> >> > > >> seem that difficult to add new daemons:
>> >> https://opensnaproute.github.
>> >> > > >> io/docs/developer.html#creating-new-component
>> >> > > >>
>> >> > > >> <https://opensnaproute.github.io/docs/developer.html#
>> >> > > >> creating-new-component>
>> >> > > >>
>> >> > > >>
>> >> > > >> They've definitely build it with a microservices architecture
>> >> > > >> in
>> >> mind,
>> >> > > so
>> >> > > >> each individual feature is abstracted into it's own small
>> >> > > >> daemon
>> >> > > process.
>> >> > > >> We could just create a daemon for the password server and the
>> >> userdata
>> >> > > >> components if we really had to.
>> >> > > >>
>> >> > > >>
>> >> > > >> - Si
>> >> > > >>
>> >> > > >>
>> >> > > >> ________________________________
>> >> > > >> From: williamstevens@gmail.com <wi...@gmail.com> on
>> >> > > >> behalf
>> >> > of
>> >> > > >> Will Stevens <ws...@cloudops.com>
>> >> > > >> Sent: Thursday, September 15, 2016 9:17 AM
>> >> > > >> To: dev@cloudstack.apache.org
>> >> > > >> Subject: Re: [DISCUSS] Replacing the VR
>> >> > > >>
>> >> > > >> A big part of why I know about it is because it is written in
>> Go.
>> >> :P
>> >> > > >>
>> >> > > >> Yes, it is definitely interesting for the routing and traffic
>> >> handling
>> >> > > >> aspects of the VR. We will likely have to rethink some of the
>> >> pieces
>> >> > a
>> >> > > >> little bit like the password server and userdata if we are to
>> >> > > >> adopt
>> >> a
>> >> > > >> different VR approach. This is where I think some of JohnB and
>> >> > > Chiradeep's
>> >> > > >> ideas make sense. In many ways, it does not make sense for the
>> >> device
>> >> > > >> handling routing and network traffic to also be responsible for
>> >> > > passwords
>> >> > > >> and userdata.
>> >> > > >>
>> >> > > >> *Will STEVENS*
>> >> > > >> Lead Developer
>> >> > > >>
>> >> > > >> *CloudOps* *| *Cloud Solutions Experts
>> >> > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>> >> > > >> *|* tw @CloudOps_
>> >> > > >>
>> >> > > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>> >> > wrote:
>> >> > > >>
>> >> > > >> > I hadn't heard of Flexswitch until you mentioned it. It looks
>> >> pretty
>> >> > > >> cool!
>> >> > > >> > It even supports ONIE install.
>> >> > > >> >
>> >> > > >> > To be honest, the ipsec feature could be added, or we could
>> >> offload
>> >> > > it to
>> >> > > >> > separate vm if we needed to. The fact it is so feature rich
>> >> > > >> > from a
>> >> > > >> routing
>> >> > > >> > perspective (and all API driven) is really nice.
>> >> > > >> >
>> >> > > >> >
>> >> > > >> > Based on the roadmap, it looks like they plan to also support
>> >> > > >> capabilities
>> >> > > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>> >> > > >> > will
>> >> be
>> >> > > huge
>> >> > > >> > for our carrier community that rely on these technologies to
>> >> > > >> > do
>> >> > > private
>> >> > > >> > gateway and inter-VPC interconnections today. We handle this
>> >> > > >> > stuff
>> >> > on
>> >> > > our
>> >> > > >> > ASRs right now with a vlan interconnect into the VR. Being
>> >> > > >> > able to
>> >> > do
>> >> > > >> MPLS
>> >> > > >> > all the way to the VR would be awesome.
>> >> > > >> >
>> >> > > >> >
>> >> > > >> > It also seems to be written in GO (a language here at ENA we
>> >> > > >> > know
>> >> > very
>> >> > > >> > well).
>> >> > > >> >
>> >> > > >> >
>> >> > > >> > - Si
>> >> > > >> >
>> >> > > >> >
>> >> > > >> >
>> >> > > >> > ________________________________
>> >> > > >> > From: Will Stevens <wi...@gmail.com>
>> >> > > >> > Sent: Thursday, September 15, 2016 7:06 AM
>> >> > > >> > To: dev@cloudstack.apache.org
>> >> > > >> > Subject: RE: [DISCUSS] Replacing the VR
>> >> > > >> >
>> >> > > >> > Ya. I don't think it covers our whole use case, but what it
>> >> > > >> > does
>> >> > > cover is
>> >> > > >> > all api driven...
>> >> > > >> >
>> >> > > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>> >> > wrote:
>> >> > > >> >
>> >> > > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it
>> >> > > >> > > was
>> >> not
>> >> > > >> > > intended to do IPSec.
>> >> > > >> > >
>> >> > > >> > > It seems as though VyOS is starting to look like the best
>> >> option.
>> >> > > >> > >
>> >> > > >> > > Regards,
>> >> > > >> > > Marty Godsey
>> >> > > >> > > nSource Solutions
>> >> > > >> > >
>> >> > > >> > > -----Original Message-----
>> >> > > >> > > From: williamstevens@gmail.com
>> >> > > >> > > [mailto:williamstevens@gmail.com
>> >> ]
>> >> > On
>> >> > > >> > > Behalf Of Will Stevens
>> >> > > >> > > Sent: Wednesday, September 14, 2016 11:06 PM
>> >> > > >> > > To: dev@cloudstack.apache.org
>> >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
>> >> > > >> > >
>> >> > > >> > > Or we could go completely crazy and go with something like
>> >> > > FlexSwitch
>> >> > > >> > from
>> >> > > >> > > SnapRoute
>> >> > > >> > > - http://www.snaproute.com/
>> >> > > >> > > - https://opensnaproute.github.io/docs/apis.html
>> >> > > >> > >
>> >> > > >> > > *Will STEVENS*
>> >> > > >> > > Lead Developer
>> >> > > >> > >
>> >> > > >> > > *CloudOps* *| *Cloud Solutions Experts
>> >> > > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>> >> > > >> > > cloudops.com
>> >> > *|*
>> >> > > tw
>> >> > > >> > > @CloudOps_
>> >> > > >> > >
>> >> > > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>> >> > > wstevens@cloudops.com>
>> >> > > >> > > wrote:
>> >> > > >> > >
>> >> > > >> > > > I tend to agree with Syed and Marty. I am not sure what
>> >> > problems
>> >> > > are
>> >> > > >> > > > solved by splitting up the function of the VR into a
>> >> > > >> > > > bunch of
>> >> > > >> separate
>> >> > > >> > > > services. As Syed points out, the complexity added is
>> >> > > non-trivial.
>> >> > > >> > > > We now have to manage all the intercontainer networking
>> >> > > >> > > > as
>> >> well
>> >> > as
>> >> > > >> the
>> >> > > >> > > > orchestrated ACS networking.
>> >> > > >> > > >
>> >> > > >> > > > VyOS is interesting to me because it covers the majority
>> >> > > >> > > > of
>> >> our
>> >> > > use
>> >> > > >> > > > case with a single unified control plane. It also has
>> >> > > >> > > > good
>> >> > > support
>> >> > > >> > > > for extending features we care about, like IPv6, VXLAN,
>> >> > > >> > > > VRRP, transactions, etc...
>> >> > > >> > > >
>> >> > > >> > > > *Will STEVENS*
>> >> > > >> > > > Lead Developer
>> >> > > >> > > >
>> >> > > >> > > > *CloudOps* *| *Cloud Solutions Experts
>> >> > > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>> >> cloudops.com
>> >> > > *|*
>> >> > > >> tw
>> >> > > >> > > > @CloudOps_
>> >> > > >> > > >
>> >> > > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>> >> > sahmed@cloudops.com>
>> >> > > >> > wrote:
>> >> > > >> > > >
>> >> > > >> > > >> Agree with Marty, adding Docker containers to the
>> >> > > >> > > >> picture
>> >> > > although
>> >> > > >> > > >> can make the VR more flexible but the added complexity
>> >> > > >> > > >> is
>> >> just
>> >> > > not
>> >> > > >> > > >> worth it. Not to mention we would need to take care of
>> >> > networking
>> >> > > >> > > >> each container manually and given that our iptable rules
>> >> > > >> > > >> are
>> >> > very
>> >> > > >> > > >> unstable at the moment I don't see a big value add.
>> >> > > >> > > >>
>> >> > > >> > > >> Vyos looks like a better solution to me. I know that it
>> >> > > >> > > >> does
>> >> > not
>> >> > > >> > > >> provide an api but it does fit the bill quite well
>> >> otherwise. I
>> >> > > >> > > >> specially like the fact that it has a transaction based
>> >> > > >> > > >> model
>> >> > and
>> >> > > >> you
>> >> > > >> > > >> can rollback changes if something goes wrong.
>> >> > > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>> >> > > marty@gonsource.com>
>> >> > > >> > > wrote:
>> >> > > >> > > >>
>> >> > > >> > > >> > Licensing aside, I think splitting the various
>> >> > > >> > > >> > functions
>> >> into
>> >> > > >> > > >> > containers is not a good route either. This will force
>> >> users
>> >> > to
>> >> > > >> > > >> > have to maintain
>> >> > > >> > > >> and
>> >> > > >> > > >> > use containers and adds complexity to the networking
>> >> aspects
>> >> > of
>> >> > > >> ACS.
>> >> > > >> > > >> > Complexity decreases stability. Now I understand the
>> >> argument
>> >> > > that
>> >> > > >> > > >> > a monolithic approach also brings its own set of
>> >> > > >> > > >> > issues but
>> >> > it
>> >> > > >> also
>> >> > > >> > > >> > simplifies it.
>> >> > > >> > > >> >
>> >> > > >> > > >> > Regards,
>> >> > > >> > > >> > Marty Godsey
>> >> > > >> > > >> > nSource Solutions
>> >> > > >> > > >> >
>> >> > > >> > > >> > -----Original Message-----
>> >> > > >> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>> >> > > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
>> >> > > >> > > >> > To: dev@cloudstack.apache.org
>> >> > > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR
>> >> > > >> > > >> >
>> >> > > >> > > >> > I rather doubt that the Cloudrouter will fit the needs
>> >> > > >> > > >> > of
>> >> the
>> >> > > >> > > >> > CloudStack project
>> >> > > >> > > >> > - it is AGPL licensed. Many enterprises will not
>> >> > > >> > > >> > touch
>> >> > > anything
>> >> > > >> > > >> > that
>> >> > > >> > > >> has
>> >> > > >> > > >> > AGPL
>> >> > > >> > > >> > - the github repo shows rather infrequent updates.
>> >> > > >> > > >> > Quite
>> >> > > likely
>> >> > > >> > > >> > they aren't considering the use cases of the
>> >> > > >> > > >> > CloudStack
>> >> > > community
>> >> > > >> > > >> >
>> >> > > >> > > >> > I'd back John B's comments on disaggregating the VR.
>> >> > > >> > > >> > Split
>> >> it
>> >> > > into
>> >> > > >> > > >> > many docker containers
>> >> > > >> > > >> > - password server
>> >> > > >> > > >> > - userdata server
>> >> > > >> > > >> > - DHCP / DNS
>> >> > > >> > > >> > - s2s VPN
>> >> > > >> > > >> > - RA VPN
>> >> > > >> > > >> > - intra-VPC routing and ACL
>> >> > > >> > > >> > - Port forwarding + NAT
>> >> > > >> > > >> > - FW
>> >> > > >> > > >> > - LB (public)
>> >> > > >> > > >> > - LB (internal),
>> >> > > >> > > >> > - secondary storage
>> >> > > >> > > >> > - agent
>> >> > > >> > > >> > Glue them together with docker compose files (one per
>> >> > > >> > > >> > use
>> >> > > case -
>> >> > > >> > > >> > basic zone, isolated, VPC, SSVM, etc).
>> >> > > >> > > >> >
>> >> > > >> > > >> > The VR image then becomes a JeOS + docker. You can
>> >> > > >> > > >> > test
>> >> each
>> >> > of
>> >> > > >> the
>> >> > > >> > > >> > components independently and fixing one bug in the
>> >> > > >> > > >> > field
>> >> (say
>> >> > > >> DHCP)
>> >> > > >> > > >> > is hitless to the other components. You don't need to
>> >> > > >> > > >> > build per-hypervisor VRs. You could even run on
>> baremetal.
>> >> > > >> > > >> >
>> >> > > >> > > >> > Along the way you need to figure out how to
>> >> > > >> > > >> > - make the traffic traverse the containers that are
>> >> > > >> > > >> > needed
>> >> > to
>> >> > > be
>> >> > > >> > > >> > traversed (in most cases just 1)
>> >> > > >> > > >> > - bootstrap the router (how does it find its compose
>> file?
>> >> > > where
>> >> > > >> > > >> > is the
>> >> > > >> > > >> > registry?)
>> >> > > >> > > >> > - rethink the command and control of the VR
>> >> > > >> > > >> > functions. SSH
>> >> > > works,
>> >> > > >> > > >> > but something more declarative, idempotent should be
>> >> > explored.
>> >> > > >> > > >> >
>> >> > > >> > > >> > As you do this, it becomes clearer which of the
>> >> > > >> > > >> > functions
>> >> can
>> >> > > be
>> >> > > >> > > >> > substituted by for example CloudRouter. Command and
>> >> > > >> > > >> > Control
>> >> > of
>> >> > > the
>> >> > > >> > > >> docker
>> >> > > >> > > >> > containers can be moved out to another container. Etc.
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> >
>> >> > > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>> >> > > >> > > >> > <ma...@gonsource.com>
>> >> > > >> > > >> > wrote:
>> >> > > >> > > >> >
>> >> > > >> > > >> > > This one does look nice. My biggest concern is the
>> >> > > >> > > >> > > lack
>> >> of
>> >> > > >> > > >> > > VXLANs. It seems that any of the ones we mentioned
>> >> > > >> > > >> > > do not
>> >> > > have
>> >> > > >> an
>> >> > > >> > > >> > > API so we may be stuck at the SSH method.
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > Regards,
>> >> > > >> > > >> > > Marty Godsey
>> >> > > >> > > >> > > nSource Solutions
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > -----Original Message-----
>> >> > > >> > > >> > > From: Abhinandan Prateek
>> >> > > >> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
>> >> > > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
>> >> > > >> > > >> > > To: dev@cloudstack.apache.org
>> >> > > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > Cloudrouter looks promising. These have potential to
>> >> > > >> > > >> > > save
>> >> > > future
>> >> > > >> > > >> > > engineering effort for example on ipv6 routing, OSPF
>> etc.
>> >> > > >> > > >> > > And the best part is they come with test automation
>> >> > > framework.
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
>> >> > > >> > > >> > > <ja...@accelerite.com>
>> >> > > >> > > >> > > wrote:
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > >Hi,
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >Instead of replacing the VR in first place we
>> >> > > >> > > >> > > >should add VyOS/cloudrouter
>> >> > > >> > > >> > > as provider. Once it is stable, network offerings
>> >> > > >> > > >> > > (on
>> >> > > upgrade)
>> >> > > >> > > >> > > can be updated to use it and we can drop the VR if
>> >> > > >> > > >> > > we
>> >> want
>> >> > at
>> >> > > >> > > >> > > that release
>> >> > > >> > > >> > onwards.
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >VR is stabilized over a period of time and some of
>> >> > > >> > > >> > > >them
>> >> > are
>> >> > > >> > > >> > > >running
>> >> > > >> > > >> > > without issues. When we replicate the ACS VR
>> >> > > >> > > >> > > features in
>> >> > new
>> >> > > >> > > >> > > solution it takes some to find the missing pieces
>> >> > > >> > > >> > > (hidden
>> >> > > bugs).
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >Thanks,
>> >> > > >> > > >> > > >Jayapal
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >> nux@li.nux.ro> wrote:
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> Hi,
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> I like the idea.
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> Cloudrouter looks really promising, I'm not too
>> >> > > >> > > >> > > >> keen
>> >> on
>> >> > > VyOS
>> >> > > >> > > >> > > >> (it
>> >> > > >> > > >> > > doesn't have a proper http api etc).
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> --
>> >> > > >> > > >> > > >> Sent from the Delta quadrant using Borg
>> technology!
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >> Nux!
>> >> > > >> > > >> > > >> www.nux.ro
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > abhinandan.prateek@shapeblue.com
>> >> > > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
>> >> > > >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>> >> > > @shapeblue
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > >
>> >> > > >> > > >> > > ----- Original Message -----
>> >> > > >> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
>> >> > > >> > > >> > > >>> To: dev@cloudstack.apache.org
>> >> > > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
>> >> > > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
>> >> > > >> > > >> > > >>
>> >> > > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and
>> >> > > >> > > >> > > >>> should
>> >> > be
>> >> > > >> > > >> > > >>> treated as
>> >> > > >> > > >> > > such.
>> >> > > >> > > >> > > >>> Please weigh in with the good and bad of this
>> idea...
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> A couple of us have been discussing the idea of
>> >> > > potentially
>> >> > > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open
>> >> > > >> > > >> > > >>> Source
>> >> > > Vyatta
>> >> > > >> > VM).
>> >> > > >> > > >> > > >>> There may be a license issue because I think it
>> >> > > >> > > >> > > >>> is
>> >> > > licensed
>> >> > > >> > > >> > > >>> under GPL, but for the sake of discussion, let's
>> >> assume
>> >> > > we
>> >> > > >> > > >> > > >>> can overcome any
>> >> > > >> > > >> > > license issues.
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> I have spent some time recently with the VyOS
>> >> > > >> > > >> > > >>> and I
>> >> > have
>> >> > > to
>> >> > > >> > > >> > > >>> admit, I was pretty impressed. It is simple and
>> >> > > intuitive
>> >> > > >> > > >> > > >>> and it gives you a lot more options for auditing
>> >> > > >> > > >> > > >>> the
>> >> > > >> > > configuration etc...
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> Items of potential interest:
>> >> > > >> > > >> > > >>> - Clean up our current VR script spaghetti to a
>> >> simpler
>> >> > > more
>> >> > > >> > > >> > > >>> auditable configuration workflow.
>> >> > > >> > > >> > > >>> - Gives a cleaner path for IPv6 support.
>> >> > > >> > > >> > > >>> - Handles VPN configuration via the same
>> >> configuration
>> >> > > >> > > interface.
>> >> > > >> > > >> > > >>> - Support for OSPF & BGP.
>> >> > > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
>> >> > > >> > > >> > > >>> - Easily supports HA (redundant routers) through
>> >> VRRP.
>> >> > > >> > > >> > > >>> - VXLAN support.
>> >> > > >> > > >> > > >>> - Transaction based changes to the VR with
>> >> > > >> > > >> > > >>> rollback
>> >> on
>> >> > > >> error.
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> Items that could be difficult to solve:
>> >> > > >> > > >> > > >>> - Userdata password reset workflow and
>> >> implementation.
>> >> > > >> > > >> > > >>> - Upgrade process.
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> The VyOS is not the only option if we were to
>> >> consider
>> >> > > this
>> >> > > >> > > >> approach.
>> >> > > >> > > >> > > >>> Another option, which I don't know as well,
>> >> > > >> > > >> > > >>> would be CloudRouter (AGPL
>> >> > > >> > > >> > > >>> license) [2] which is purely API driven.
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> Anyway, would love to hear your thoughts...
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> Will
>> >> > > >> > > >> > > >>>
>> >> > > >> > > >> > > >>> [1] https://vyos.io/ [2]
>> >> > > >> > > >> > > >>> https://cloudrouter.org/
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >
>> >> > > >> > > >> > > >DISCLAIMER
>> >> > > >> > > >> > > >==========
>> >> > > >> > > >> > > >This e-mail may contain privileged and confidential
>> >> > > information
>> >> > > >> > > >> > > >which is
>> >> > > >> > > >> > > the property of Accelerite, a Persistent Systems
>> >> business.
>> >> > > It is
>> >> > > >> > > >> > > intended only for the use of the individual or
>> >> > > >> > > >> > > entity to
>> >> > > which
>> >> > > >> it
>> >> > > >> > > >> > > is addressed. If you are not the intended recipient,
>> >> > > >> > > >> > > you
>> >> > are
>> >> > > not
>> >> > > >> > > >> > > authorized to read, retain, copy, print, distribute
>> >> > > >> > > >> > > or
>> >> use
>> >> > > this
>> >> > > >> > > >> > > message. If you have received this communication in
>> >> error,
>> >> > > >> please
>> >> > > >> > > >> > > notify the sender and delete all copies of this
>> message.
>> >> > > >> > > >> > > Accelerite, a Persistent Systems business does not
>> >> > > >> > > >> > > accept
>> >> > any
>> >> > > >> > > >> > > liability for virus
>> >> > > >> > > >> > infected mails.
>> >> > > >> > > >> > >
>> >> > > >> > > >> >
>> >> > > >> > > >>
>> >> > > >> > > >
>> >> > > >> > > >
>> >> > > >> > >
>> >> > > >> >
>> >> > >
>> >> >
>>
>>
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
I am going to try to do a bit of a brain dump from me and my team in this
post, and hopefully I can verbalize the ideas well enough that you can all
follow along. Please provide feedback, ideas and suggestions...
*Preface:*
*--------*
I am currently approaching this problem with the following use cases in
mind.
- Stabilize/replace the VR for our production cloud deployments as well as
give the community a free, open source and production grade alternative to
the current VR.
- I have been scoping the integration of the Fortinet Fortigate (hardware
and virtual) into ACS to support the following features.
-- All Isolated Guest Network traffic features (Firewall, PF, NAT,
Client-to-Site VPN, etc)
-- All VPC network traffic features (ACLs, PF, NAT, Client-to-Site and
Site-to-Site VPN, Private Gateway)
-- Public IPv6 support (NAT64 and NAT46) with IPv4 on the guest side.
*The Setup:*
*----------*
I am not going to get into the details of the Fortinet integration details
in this thread, you can expect a function spec from me soon describing my
plans for this work. I am currently waiting on hardware to validate my
implementation strategy.
In this post I will be focusing on the way ACS deals with the VR, network
plugins and how this could be potentially adapted to give us more
flexibility going forward.
I have created a diagram [1] to outline the ideas presented in this post.
So if you don't like my ideas here, you can simply use the VR as you
currently do and you can basically ignore the dotted circles to the right
of the 'existing VR' in the solid circle.
The basic idea here, is when you create a network service offering and you
select different Providers to handle different network capabilities, you
can also specify a VM template for those providers (as well as service
offering) which will be used for additional VMs which will be created to
provision those services. In this context, think of templates such as
VyOS, FortiOS, or a VPX to understand the spirit of this approach.
When a network is provisioned, the current VR will be deployed (ideally
only offering services like; password server, userdata, DHCP, DNS), and in
addition, any network provider templates which have been specified will
also be provisioned. Each 'network VM' will get a leg on both the public
VLAN and the guest VLAN. In the case of a VPC, the first tier will be a
little special (I will get into this in a bit) and will not be able to be
deleted until the VR(s) are all cleaned up and the network is completely
destroyed. This network is denoted as the 'Client MGMT' network in the
diagram [1].
You will notice that ONLY the 'existing VR' has network connectivity to the
ACS MGMT server. It may not necessarily be through a 169.x.x.x address,
but that illustrates the point for now. The reason for this is because we
can not guarantee that a Guest User does may not gain access to these VRs,
and in cases I will get into later, the Guest User may actually be given
access to those VRs. This does however mean that the ACS MGMT server does
not have direct access to those VRs in order to configure them. To solve
this, the 'existing VR' would act as a proxy for the ACS Management server
over the ACS MGMT network to pass configuration commands to the VRs to the
right of the 'existing VR' depicted in the graphic.
*The Details:*
*------------*
Now that you have a basic idea of the deployment structure I have in mind,
let's discuss some of the finer details.
- We should be able to relatively easily adapt existing external network
plugins to be able to be provisioned with this mechanism (assuming they
have a VM option). The Palo Alto would be an example which I would likely
use as a proof of concept for this adaptation. It would be important that
the external plugin could work as an external physical appliance OR as a
dynamically provisioned VM appliance.
- Enterprise grade network appliances which support VM deployment will no
longer have to be pre-deployed and shared, but instead will be able to be
dynamically provisioned on a network by network basis.
- Existing external network appliance plugins targeting physical boxes will
still work in concert with this approach without any need for modification.
- We could add an 'Allow Guest Access' checkbox to the service provider
section in the network service offering to give the Guest User access to
their network appliance. This is especially useful for an implementation
like the FortiGate because it allows for the user to configure UTM features
which ACS does not orchestrate but the appliance is capable of. This also
applies to devices like the NetScaler. In the diagram [1] this is depicted
by the dashed green lines between the devices and the 'Client MGMT'
network. So in the case of the NetScaler, you would get an NSIP on the
Client MGMT network which would allow the Guest User to connect to that
box. I envision a URL pointing to the guest network IP with
username/password returned for a network which a user can use to connect
into that network appliance.
- Similar to above, the Guest User will have much more visibility into the
logging and access on the network. I can't count how many times a customer
has asked for an easy way to get the IPsec logs from the VR.
I think this is enough to get the idea across. I have worked through a
bunch of examples on a white board with my team to try to work through a
bunch of the use cases, edge cases and deployment scenarios, but I am sure
there are situations we have not thought through yet.
The current VR has been a constant pain for us and our service customers,
so it is likely that we will be taking some action one way or another
because this needs to be addressed. I know there is a lot going on here,
but hopefully I have been able to get our ideas across.
Please ask questions or make suggestions. Are there major pieces we are
not taking into account? Do you have use cases which would be simplified
by this approach that I have not called out?
Would love to hear your feedback...
[1] https://objects-east.cloud.ca/v1/5ef827605f884961b94881e928e7a2
50/swill/vr_network.png
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Tue, Sep 20, 2016 at 5:54 AM, Murali Reddy <mu...@gmail.com>
wrote:
> Configuration management of network appliances particularly for Cloud and
> NFV scenarios is still evolving area. Programmability is the not the
> strength for even the most popular network operating systems like IOS,
> JunoS etc. So its not surprising why CloudStack integrates in a archaic way
> with stock linux for the VR.
>
> VR was never integral/hardcoded option in CloudStack. Its always been a
> plugin. CloudStack network orchestration is well abstracted and designed
> with vision to compose a network with different set of providers for
> different services. Yes that vision is not fully realised yet, and we don’t
> have true service function chains. That would be different discussion topic.
>
> I tend to agree with Simon, as alternate/interim option we can take hard
> look whats causing the problems with current VR integration. Personally, I
> think it would be easier we take a cue from configuration managers and
> network configuration solutions out there (for e.g promise theory based
> Cisco ACI) move to more declarative model of expressing desired state of
> network configuration. Infact current VR from 4.6, actually holds the
> desired state per service basis, seems to be in that direction.
>
> It does make sense to evaluate new appliances which can provide rich
> semantics (like programmable API, declarative configuration, versioning,
> commit/rollback etc), but will need significant engineering effort and time
> to stabilise. We may make same mistakes with integration of other appliance
> as well, if we fully don’t understand the nature of the current problems
> with CloudStack core and service provider interaction and current VR
> integration.
>
>
> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>
> >I think our other option is to take a real look at what it would take to
> fix the VR. In my opinion, a lot of the problems are related to the
> monolithic python code base and the fact nothing is actually separated.
> >
> >Secondly, the python scripts (and bash scripts) don't use any established
> libraries to complete tasks and instead shell out and run commands that are
> both hard to track and hard to parse on return.
> >
> >
> >If we daemonized this, used a real api for Agent to VR communication,
> used common already existing libraries for the system service and network
> interactions and spent a bit of time separating out code into distinct
> modules, everything would behave a lot better.
> >
> >
> >The pain and suffering is due to years and years of patches and constant
> shelling out to complete tasks in my opinion. If we spend time to rethink
> how we interact with the VR in general and we abstract the systems and
> networking stuff and use well known and stable libraries to do the work,
> the VR would be much easier to maintain.
> >
> >
> >- Si
> >
> >
> >
> >
> >________________________________
> >From: Marty Godsey <ma...@gonsource.com>
> >Sent: Friday, September 16, 2016 12:24 PM
> >To: dev@cloudstack.apache.org
> >Subject: RE: [DISCUSS] Replacing the VR
> >
> >So based upon this discussion would it be prudent to wait on VyOS 2.0?
> The current VR is giving us issues but would the time invested in another
> "solution" be wasted especially if by the time another option is chose,
> then coded, then tested, then implemented and right as that time happened
> to be when VyOS 2.0 is released. Of course you said they are just in the
> scoping range so this could still be a year or more out.
> >
> >Thoughts?
> >
> >Regards,
> >Marty Godsey
> >nSource Solutions
> >
> >-----Original Message-----
> >From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> >Sent: Friday, September 16, 2016 10:31 AM
> >To: dev@cloudstack.apache.org
> >Cc: daniil@baturin.org
> >Subject: Re: [DISCUSS] Replacing the VR
> >
> >I just had a quick chat with a couple of the guys over on the VyOS chat.
> >I have CC'ed one of them in case we have more licensing questions.
> >
> >So here is the status with the license "the code inherited from Vyatta
> and our modifications from it is GPLv2 (strict, not v2+). The config
> reading library is GPLv2 too, so anything that links to is is GPLv2.
> >Some auxiliary components we made after the fork are more permissive,
> >LGPLv2+ or MIT."
> >
> >They are currently in the process of scoping a redesign (VyOS 2.0), "we
> are planning a clean rewrite that will solve issues of the current config
> system".
> >This will include the ability to configure via the API.
> >
> >If we have more questions for VyOS, they are very friendly and
> responsive, so we should be able to get answers.
> >
> >*Will STEVENS*
> >Lead Developer
> >
> >*CloudOps* *| *Cloud Solutions Experts
> >420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
> >
> >On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
> >
> >> I agree with Will Ilya. There are so many problems with the VR right
> now.
> >> Most of the outages we've had recently have somehow involved the VR.
> >> We set custom iptables rules on the VR which can and have easily gone
> wrong.
> >> Openswan is broken, Strongswan replacement still needs to be tested.
> >> VVRP with redundant router still needs work, and not to mention the
> >> problems we will have when we introduce IPv6 into the whole picture.
> >>
> >> I think the spirit of the discussion is to rely on a 3rd party to do
> >> the networking for us (eg VyOS) and have us handle just the
> >> orchestration. All the problems that I've described have already been
> >> solved in VyOS. We also get the advantage of a potential wider
> >> community to fix and maintain the VR and given our current development
> >> velocity, it think it totally makes sense to look for a 3rd party
> option.
> >>
> >> -Syed
> >>
> >>
> >> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
> >> wrote:
> >>
> >> > The VR has been biting us far too often recently, which is why we
> >> > have started looking into alternative implementations.
> >> >
> >> > One of the things that is nice about potentially using the VyOS is
> >> > that
> >> it
> >> > is based on Debian, so we should be able to run the other services
> >> > that
> >> we
> >> > currently have like the password server and userdata on the VyOS.
> >> > This means we would not have to change our architecture initially
> >> > and could focus on only replacing the networking paths.
> >> >
> >> > *Will STEVENS*
> >> > Lead Developer
> >> >
> >> > *CloudOps* *| *Cloud Solutions Experts
> >> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> >> > tw @CloudOps_
> >> >
> >> > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> >> >
> >> > > The more this is discussed the more I think we should stick with
> >> > > our
> >> VR.
> >> > >
> >> > > All these other options either seem unfinished or with
> >> > > incompatible license.
> >> > >
> >> > > VyOS looks the most promising so far, it's a serious, mature
> project.
> >> > > Adopting it though means we'll have to microservice our way out of
> >> > > it
> >> > with
> >> > > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
> >> those
> >> > > too. Imho this adds complexity we should void.
> >> > >
> >> > > --
> >> > > Sent from the Delta quadrant using Borg technology!
> >> > >
> >> > > Nux!
> >> > > www.nux.ro
> >> > >
> >> > > ----- Original Message -----
> >> > > > From: "Will Stevens" <ws...@cloudops.com>
> >> > > > To: dev@cloudstack.apache.org
> >> > > > Sent: Thursday, 15 September, 2016 17:21:28
> >> > > > Subject: Re: [DISCUSS] Replacing the VR
> >> > >
> >> > > > Ya, we would need to add a daemon for VPN as well. Load
> >> > > > balancing is another aspect which we will need to consider if we
> went this route.
> >> > > > Something like https://traefik.io/ could potentially be a good
> >> > > > fit
> >> due
> >> > > to
> >> > > > its API driven configuration, but it may be more than what we
> need.
> >> > > >
> >> > > > We should probably try define which pieces make sense to be
> >> > > > solved
> >> > > together
> >> > > > and which pieces would be best suited to be broken out.
> >> > > >
> >> > > > I think the network connectivity, routing and firewalling should
> >> > probably
> >> > > > all stay together since the majority of the tools we would
> >> potentially
> >> > > use
> >> > > > would handle all of that together in a single implementation.
> >> > > >
> >> > > > The password server and userdata seems like a good option for
> >> > > > being
> >> > > broken
> >> > > > out and handled independently (and probably rewritten completely
> >> since
> >> > > they
> >> > > > currently have some issues).
> >> > > >
> >> > > > Load balancing is another that could warrant splitting out, but
> >> > > > that depends on what direction we go and how we would be managing
> it.
> >> DHCP
> >> > > and
> >> > > > DNS are others which could go either way.
> >> > > >
> >> > > > If we do split out services, I think we should consolidate as
> >> > > > much as
> >> > we
> >> > > > can into each service we break out. Ideally a network packet
> >> > > > would
> >> > never
> >> > > > hit more than one, maybe two, services. I don't think we should
> >> > > > be splitting services 'just because', I think we need a valid
> >> > > > case for splitting any service out because it adds complexity.
> >> > > > Our project is already complex enough, we need to avoid adding
> >> > > > complexity unless it
> >> is
> >> > > > really needed.
> >> > > >
> >> > > > Some more of my thoughts on this anyway...
> >> > > >
> >> > > > *Will STEVENS*
> >> > > > Lead Developer
> >> > > >
> >> > > > *CloudOps* *| *Cloud Solutions Experts
> >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> >> > > > *|* tw @CloudOps_
> >> > > >
> >> > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
> >> > wrote:
> >> > > >
> >> > > >> I do agree with you that this probably isn't the right place
> >> > > >> the
> >> > > password
> >> > > >> service and user data.
> >> > > >>
> >> > > >>
> >> > > >> Having said that, after taking a cursory look at the dev docs,
> >> > > >> it
> >> > > doesn't
> >> > > >> seem that difficult to add new daemons:
> >> https://opensnaproute.github.
> >> > > >> io/docs/developer.html#creating-new-component
> >> > > >>
> >> > > >> <https://opensnaproute.github.io/docs/developer.html#
> >> > > >> creating-new-component>
> >> > > >>
> >> > > >>
> >> > > >> They've definitely build it with a microservices architecture
> >> > > >> in
> >> mind,
> >> > > so
> >> > > >> each individual feature is abstracted into it's own small
> >> > > >> daemon
> >> > > process.
> >> > > >> We could just create a daemon for the password server and the
> >> userdata
> >> > > >> components if we really had to.
> >> > > >>
> >> > > >>
> >> > > >> - Si
> >> > > >>
> >> > > >>
> >> > > >> ________________________________
> >> > > >> From: williamstevens@gmail.com <wi...@gmail.com> on
> >> > > >> behalf
> >> > of
> >> > > >> Will Stevens <ws...@cloudops.com>
> >> > > >> Sent: Thursday, September 15, 2016 9:17 AM
> >> > > >> To: dev@cloudstack.apache.org
> >> > > >> Subject: Re: [DISCUSS] Replacing the VR
> >> > > >>
> >> > > >> A big part of why I know about it is because it is written in Go.
> >> :P
> >> > > >>
> >> > > >> Yes, it is definitely interesting for the routing and traffic
> >> handling
> >> > > >> aspects of the VR. We will likely have to rethink some of the
> >> pieces
> >> > a
> >> > > >> little bit like the password server and userdata if we are to
> >> > > >> adopt
> >> a
> >> > > >> different VR approach. This is where I think some of JohnB and
> >> > > Chiradeep's
> >> > > >> ideas make sense. In many ways, it does not make sense for the
> >> device
> >> > > >> handling routing and network traffic to also be responsible for
> >> > > passwords
> >> > > >> and userdata.
> >> > > >>
> >> > > >> *Will STEVENS*
> >> > > >> Lead Developer
> >> > > >>
> >> > > >> *CloudOps* *| *Cloud Solutions Experts
> >> > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> >> > > >> *|* tw @CloudOps_
> >> > > >>
> >> > > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
> >> > wrote:
> >> > > >>
> >> > > >> > I hadn't heard of Flexswitch until you mentioned it. It looks
> >> pretty
> >> > > >> cool!
> >> > > >> > It even supports ONIE install.
> >> > > >> >
> >> > > >> > To be honest, the ipsec feature could be added, or we could
> >> offload
> >> > > it to
> >> > > >> > separate vm if we needed to. The fact it is so feature rich
> >> > > >> > from a
> >> > > >> routing
> >> > > >> > perspective (and all API driven) is really nice.
> >> > > >> >
> >> > > >> >
> >> > > >> > Based on the roadmap, it looks like they plan to also support
> >> > > >> capabilities
> >> > > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
> >> > > >> > will
> >> be
> >> > > huge
> >> > > >> > for our carrier community that rely on these technologies to
> >> > > >> > do
> >> > > private
> >> > > >> > gateway and inter-VPC interconnections today. We handle this
> >> > > >> > stuff
> >> > on
> >> > > our
> >> > > >> > ASRs right now with a vlan interconnect into the VR. Being
> >> > > >> > able to
> >> > do
> >> > > >> MPLS
> >> > > >> > all the way to the VR would be awesome.
> >> > > >> >
> >> > > >> >
> >> > > >> > It also seems to be written in GO (a language here at ENA we
> >> > > >> > know
> >> > very
> >> > > >> > well).
> >> > > >> >
> >> > > >> >
> >> > > >> > - Si
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> > ________________________________
> >> > > >> > From: Will Stevens <wi...@gmail.com>
> >> > > >> > Sent: Thursday, September 15, 2016 7:06 AM
> >> > > >> > To: dev@cloudstack.apache.org
> >> > > >> > Subject: RE: [DISCUSS] Replacing the VR
> >> > > >> >
> >> > > >> > Ya. I don't think it covers our whole use case, but what it
> >> > > >> > does
> >> > > cover is
> >> > > >> > all api driven...
> >> > > >> >
> >> > > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
> >> > wrote:
> >> > > >> >
> >> > > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it
> >> > > >> > > was
> >> not
> >> > > >> > > intended to do IPSec.
> >> > > >> > >
> >> > > >> > > It seems as though VyOS is starting to look like the best
> >> option.
> >> > > >> > >
> >> > > >> > > Regards,
> >> > > >> > > Marty Godsey
> >> > > >> > > nSource Solutions
> >> > > >> > >
> >> > > >> > > -----Original Message-----
> >> > > >> > > From: williamstevens@gmail.com
> >> > > >> > > [mailto:williamstevens@gmail.com
> >> ]
> >> > On
> >> > > >> > > Behalf Of Will Stevens
> >> > > >> > > Sent: Wednesday, September 14, 2016 11:06 PM
> >> > > >> > > To: dev@cloudstack.apache.org
> >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> >> > > >> > >
> >> > > >> > > Or we could go completely crazy and go with something like
> >> > > FlexSwitch
> >> > > >> > from
> >> > > >> > > SnapRoute
> >> > > >> > > - http://www.snaproute.com/
> >> > > >> > > - https://opensnaproute.github.io/docs/apis.html
> >> > > >> > >
> >> > > >> > > *Will STEVENS*
> >> > > >> > > Lead Developer
> >> > > >> > >
> >> > > >> > > *CloudOps* *| *Cloud Solutions Experts
> >> > > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> >> > > >> > > cloudops.com
> >> > *|*
> >> > > tw
> >> > > >> > > @CloudOps_
> >> > > >> > >
> >> > > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> >> > > wstevens@cloudops.com>
> >> > > >> > > wrote:
> >> > > >> > >
> >> > > >> > > > I tend to agree with Syed and Marty. I am not sure what
> >> > problems
> >> > > are
> >> > > >> > > > solved by splitting up the function of the VR into a
> >> > > >> > > > bunch of
> >> > > >> separate
> >> > > >> > > > services. As Syed points out, the complexity added is
> >> > > non-trivial.
> >> > > >> > > > We now have to manage all the intercontainer networking
> >> > > >> > > > as
> >> well
> >> > as
> >> > > >> the
> >> > > >> > > > orchestrated ACS networking.
> >> > > >> > > >
> >> > > >> > > > VyOS is interesting to me because it covers the majority
> >> > > >> > > > of
> >> our
> >> > > use
> >> > > >> > > > case with a single unified control plane. It also has
> >> > > >> > > > good
> >> > > support
> >> > > >> > > > for extending features we care about, like IPv6, VXLAN,
> >> > > >> > > > VRRP, transactions, etc...
> >> > > >> > > >
> >> > > >> > > > *Will STEVENS*
> >> > > >> > > > Lead Developer
> >> > > >> > > >
> >> > > >> > > > *CloudOps* *| *Cloud Solutions Experts
> >> > > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> >> cloudops.com
> >> > > *|*
> >> > > >> tw
> >> > > >> > > > @CloudOps_
> >> > > >> > > >
> >> > > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> >> > sahmed@cloudops.com>
> >> > > >> > wrote:
> >> > > >> > > >
> >> > > >> > > >> Agree with Marty, adding Docker containers to the
> >> > > >> > > >> picture
> >> > > although
> >> > > >> > > >> can make the VR more flexible but the added complexity
> >> > > >> > > >> is
> >> just
> >> > > not
> >> > > >> > > >> worth it. Not to mention we would need to take care of
> >> > networking
> >> > > >> > > >> each container manually and given that our iptable rules
> >> > > >> > > >> are
> >> > very
> >> > > >> > > >> unstable at the moment I don't see a big value add.
> >> > > >> > > >>
> >> > > >> > > >> Vyos looks like a better solution to me. I know that it
> >> > > >> > > >> does
> >> > not
> >> > > >> > > >> provide an api but it does fit the bill quite well
> >> otherwise. I
> >> > > >> > > >> specially like the fact that it has a transaction based
> >> > > >> > > >> model
> >> > and
> >> > > >> you
> >> > > >> > > >> can rollback changes if something goes wrong.
> >> > > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> >> > > marty@gonsource.com>
> >> > > >> > > wrote:
> >> > > >> > > >>
> >> > > >> > > >> > Licensing aside, I think splitting the various
> >> > > >> > > >> > functions
> >> into
> >> > > >> > > >> > containers is not a good route either. This will force
> >> users
> >> > to
> >> > > >> > > >> > have to maintain
> >> > > >> > > >> and
> >> > > >> > > >> > use containers and adds complexity to the networking
> >> aspects
> >> > of
> >> > > >> ACS.
> >> > > >> > > >> > Complexity decreases stability. Now I understand the
> >> argument
> >> > > that
> >> > > >> > > >> > a monolithic approach also brings its own set of
> >> > > >> > > >> > issues but
> >> > it
> >> > > >> also
> >> > > >> > > >> > simplifies it.
> >> > > >> > > >> >
> >> > > >> > > >> > Regards,
> >> > > >> > > >> > Marty Godsey
> >> > > >> > > >> > nSource Solutions
> >> > > >> > > >> >
> >> > > >> > > >> > -----Original Message-----
> >> > > >> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> >> > > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
> >> > > >> > > >> > To: dev@cloudstack.apache.org
> >> > > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR
> >> > > >> > > >> >
> >> > > >> > > >> > I rather doubt that the Cloudrouter will fit the needs
> >> > > >> > > >> > of
> >> the
> >> > > >> > > >> > CloudStack project
> >> > > >> > > >> > - it is AGPL licensed. Many enterprises will not
> >> > > >> > > >> > touch
> >> > > anything
> >> > > >> > > >> > that
> >> > > >> > > >> has
> >> > > >> > > >> > AGPL
> >> > > >> > > >> > - the github repo shows rather infrequent updates.
> >> > > >> > > >> > Quite
> >> > > likely
> >> > > >> > > >> > they aren't considering the use cases of the
> >> > > >> > > >> > CloudStack
> >> > > community
> >> > > >> > > >> >
> >> > > >> > > >> > I'd back John B's comments on disaggregating the VR.
> >> > > >> > > >> > Split
> >> it
> >> > > into
> >> > > >> > > >> > many docker containers
> >> > > >> > > >> > - password server
> >> > > >> > > >> > - userdata server
> >> > > >> > > >> > - DHCP / DNS
> >> > > >> > > >> > - s2s VPN
> >> > > >> > > >> > - RA VPN
> >> > > >> > > >> > - intra-VPC routing and ACL
> >> > > >> > > >> > - Port forwarding + NAT
> >> > > >> > > >> > - FW
> >> > > >> > > >> > - LB (public)
> >> > > >> > > >> > - LB (internal),
> >> > > >> > > >> > - secondary storage
> >> > > >> > > >> > - agent
> >> > > >> > > >> > Glue them together with docker compose files (one per
> >> > > >> > > >> > use
> >> > > case -
> >> > > >> > > >> > basic zone, isolated, VPC, SSVM, etc).
> >> > > >> > > >> >
> >> > > >> > > >> > The VR image then becomes a JeOS + docker. You can
> >> > > >> > > >> > test
> >> each
> >> > of
> >> > > >> the
> >> > > >> > > >> > components independently and fixing one bug in the
> >> > > >> > > >> > field
> >> (say
> >> > > >> DHCP)
> >> > > >> > > >> > is hitless to the other components. You don't need to
> >> > > >> > > >> > build per-hypervisor VRs. You could even run on
> baremetal.
> >> > > >> > > >> >
> >> > > >> > > >> > Along the way you need to figure out how to
> >> > > >> > > >> > - make the traffic traverse the containers that are
> >> > > >> > > >> > needed
> >> > to
> >> > > be
> >> > > >> > > >> > traversed (in most cases just 1)
> >> > > >> > > >> > - bootstrap the router (how does it find its compose
> file?
> >> > > where
> >> > > >> > > >> > is the
> >> > > >> > > >> > registry?)
> >> > > >> > > >> > - rethink the command and control of the VR
> >> > > >> > > >> > functions. SSH
> >> > > works,
> >> > > >> > > >> > but something more declarative, idempotent should be
> >> > explored.
> >> > > >> > > >> >
> >> > > >> > > >> > As you do this, it becomes clearer which of the
> >> > > >> > > >> > functions
> >> can
> >> > > be
> >> > > >> > > >> > substituted by for example CloudRouter. Command and
> >> > > >> > > >> > Control
> >> > of
> >> > > the
> >> > > >> > > >> docker
> >> > > >> > > >> > containers can be moved out to another container. Etc.
> >> > > >> > > >> >
> >> > > >> > > >> >
> >> > > >> > > >> >
> >> > > >> > > >> >
> >> > > >> > > >> >
> >> > > >> > > >> >
> >> > > >> > > >> >
> >> > > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> >> > > >> > > >> > <ma...@gonsource.com>
> >> > > >> > > >> > wrote:
> >> > > >> > > >> >
> >> > > >> > > >> > > This one does look nice. My biggest concern is the
> >> > > >> > > >> > > lack
> >> of
> >> > > >> > > >> > > VXLANs. It seems that any of the ones we mentioned
> >> > > >> > > >> > > do not
> >> > > have
> >> > > >> an
> >> > > >> > > >> > > API so we may be stuck at the SSH method.
> >> > > >> > > >> > >
> >> > > >> > > >> > > Regards,
> >> > > >> > > >> > > Marty Godsey
> >> > > >> > > >> > > nSource Solutions
> >> > > >> > > >> > >
> >> > > >> > > >> > > -----Original Message-----
> >> > > >> > > >> > > From: Abhinandan Prateek
> >> > > >> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
> >> > > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> >> > > >> > > >> > > To: dev@cloudstack.apache.org
> >> > > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> >> > > >> > > >> > >
> >> > > >> > > >> > > Cloudrouter looks promising. These have potential to
> >> > > >> > > >> > > save
> >> > > future
> >> > > >> > > >> > > engineering effort for example on ipv6 routing, OSPF
> etc.
> >> > > >> > > >> > > And the best part is they come with test automation
> >> > > framework.
> >> > > >> > > >> > >
> >> > > >> > > >> > >
> >> > > >> > > >> > >
> >> > > >> > > >> > >
> >> > > >> > > >> > >
> >> > > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> >> > > >> > > >> > > <ja...@accelerite.com>
> >> > > >> > > >> > > wrote:
> >> > > >> > > >> > >
> >> > > >> > > >> > > >Hi,
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >Instead of replacing the VR in first place we
> >> > > >> > > >> > > >should add VyOS/cloudrouter
> >> > > >> > > >> > > as provider. Once it is stable, network offerings
> >> > > >> > > >> > > (on
> >> > > upgrade)
> >> > > >> > > >> > > can be updated to use it and we can drop the VR if
> >> > > >> > > >> > > we
> >> want
> >> > at
> >> > > >> > > >> > > that release
> >> > > >> > > >> > onwards.
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >VR is stabilized over a period of time and some of
> >> > > >> > > >> > > >them
> >> > are
> >> > > >> > > >> > > >running
> >> > > >> > > >> > > without issues. When we replicate the ACS VR
> >> > > >> > > >> > > features in
> >> > new
> >> > > >> > > >> > > solution it takes some to find the missing pieces
> >> > > >> > > >> > > (hidden
> >> > > bugs).
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >Thanks,
> >> > > >> > > >> > > >Jayapal
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >> nux@li.nux.ro> wrote:
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >> Hi,
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >> I like the idea.
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >> Cloudrouter looks really promising, I'm not too
> >> > > >> > > >> > > >> keen
> >> on
> >> > > VyOS
> >> > > >> > > >> > > >> (it
> >> > > >> > > >> > > doesn't have a proper http api etc).
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >> --
> >> > > >> > > >> > > >> Sent from the Delta quadrant using Borg technology!
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >> Nux!
> >> > > >> > > >> > > >> www.nux.ro
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > abhinandan.prateek@shapeblue.com
> >> > > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
> >> > > >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> >> > > @shapeblue
> >> > > >> > > >> > >
> >> > > >> > > >> > >
> >> > > >> > > >> > >
> >> > > >> > > >> > > ----- Original Message -----
> >> > > >> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> >> > > >> > > >> > > >>> To: dev@cloudstack.apache.org
> >> > > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> >> > > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and
> >> > > >> > > >> > > >>> should
> >> > be
> >> > > >> > > >> > > >>> treated as
> >> > > >> > > >> > > such.
> >> > > >> > > >> > > >>> Please weigh in with the good and bad of this
> idea...
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> A couple of us have been discussing the idea of
> >> > > potentially
> >> > > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open
> >> > > >> > > >> > > >>> Source
> >> > > Vyatta
> >> > > >> > VM).
> >> > > >> > > >> > > >>> There may be a license issue because I think it
> >> > > >> > > >> > > >>> is
> >> > > licensed
> >> > > >> > > >> > > >>> under GPL, but for the sake of discussion, let's
> >> assume
> >> > > we
> >> > > >> > > >> > > >>> can overcome any
> >> > > >> > > >> > > license issues.
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> I have spent some time recently with the VyOS
> >> > > >> > > >> > > >>> and I
> >> > have
> >> > > to
> >> > > >> > > >> > > >>> admit, I was pretty impressed. It is simple and
> >> > > intuitive
> >> > > >> > > >> > > >>> and it gives you a lot more options for auditing
> >> > > >> > > >> > > >>> the
> >> > > >> > > configuration etc...
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> Items of potential interest:
> >> > > >> > > >> > > >>> - Clean up our current VR script spaghetti to a
> >> simpler
> >> > > more
> >> > > >> > > >> > > >>> auditable configuration workflow.
> >> > > >> > > >> > > >>> - Gives a cleaner path for IPv6 support.
> >> > > >> > > >> > > >>> - Handles VPN configuration via the same
> >> configuration
> >> > > >> > > interface.
> >> > > >> > > >> > > >>> - Support for OSPF & BGP.
> >> > > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
> >> > > >> > > >> > > >>> - Easily supports HA (redundant routers) through
> >> VRRP.
> >> > > >> > > >> > > >>> - VXLAN support.
> >> > > >> > > >> > > >>> - Transaction based changes to the VR with
> >> > > >> > > >> > > >>> rollback
> >> on
> >> > > >> error.
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> Items that could be difficult to solve:
> >> > > >> > > >> > > >>> - Userdata password reset workflow and
> >> implementation.
> >> > > >> > > >> > > >>> - Upgrade process.
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> The VyOS is not the only option if we were to
> >> consider
> >> > > this
> >> > > >> > > >> approach.
> >> > > >> > > >> > > >>> Another option, which I don't know as well,
> >> > > >> > > >> > > >>> would be CloudRouter (AGPL
> >> > > >> > > >> > > >>> license) [2] which is purely API driven.
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> Anyway, would love to hear your thoughts...
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> Will
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> [1] https://vyos.io/ [2]
> >> > > >> > > >> > > >>> https://cloudrouter.org/
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >DISCLAIMER
> >> > > >> > > >> > > >==========
> >> > > >> > > >> > > >This e-mail may contain privileged and confidential
> >> > > information
> >> > > >> > > >> > > >which is
> >> > > >> > > >> > > the property of Accelerite, a Persistent Systems
> >> business.
> >> > > It is
> >> > > >> > > >> > > intended only for the use of the individual or
> >> > > >> > > >> > > entity to
> >> > > which
> >> > > >> it
> >> > > >> > > >> > > is addressed. If you are not the intended recipient,
> >> > > >> > > >> > > you
> >> > are
> >> > > not
> >> > > >> > > >> > > authorized to read, retain, copy, print, distribute
> >> > > >> > > >> > > or
> >> use
> >> > > this
> >> > > >> > > >> > > message. If you have received this communication in
> >> error,
> >> > > >> please
> >> > > >> > > >> > > notify the sender and delete all copies of this
> message.
> >> > > >> > > >> > > Accelerite, a Persistent Systems business does not
> >> > > >> > > >> > > accept
> >> > any
> >> > > >> > > >> > > liability for virus
> >> > > >> > > >> > infected mails.
> >> > > >> > > >> > >
> >> > > >> > > >> >
> >> > > >> > > >>
> >> > > >> > > >
> >> > > >> > > >
> >> > > >> > >
> >> > > >> >
> >> > >
> >> >
>
>
Re: [DISCUSS] Replacing the VR
Posted by Pierre-Luc Dion <pd...@apache.org>.
I'm not sure how it would help to containerized VR services, but it would
certainly help on maintaining them.
I really like the idea of using a configuration management tool on the VR
that would be responsible to maintain the VR at the expected state by
CloudStack.
At the moment on our side we are struggling with old libraries and
configuration issues on the current VR, I am still wondering where the
effort worth between a new default VR based on vyos or container based or
fix what we have and make it better :-S
But I find the thread very healthy, thank you :)
On Tue, Sep 20, 2016 at 5:54 AM, Murali Reddy <mu...@gmail.com>
wrote:
> Configuration management of network appliances particularly for Cloud and
> NFV scenarios is still evolving area. Programmability is the not the
> strength for even the most popular network operating systems like IOS,
> JunoS etc. So its not surprising why CloudStack integrates in a archaic way
> with stock linux for the VR.
>
> VR was never integral/hardcoded option in CloudStack. Its always been a
> plugin. CloudStack network orchestration is well abstracted and designed
> with vision to compose a network with different set of providers for
> different services. Yes that vision is not fully realised yet, and we don’t
> have true service function chains. That would be different discussion topic.
>
> I tend to agree with Simon, as alternate/interim option we can take hard
> look whats causing the problems with current VR integration. Personally, I
> think it would be easier we take a cue from configuration managers and
> network configuration solutions out there (for e.g promise theory based
> Cisco ACI) move to more declarative model of expressing desired state of
> network configuration. Infact current VR from 4.6, actually holds the
> desired state per service basis, seems to be in that direction.
>
> It does make sense to evaluate new appliances which can provide rich
> semantics (like programmable API, declarative configuration, versioning,
> commit/rollback etc), but will need significant engineering effort and time
> to stabilise. We may make same mistakes with integration of other appliance
> as well, if we fully don’t understand the nature of the current problems
> with CloudStack core and service provider interaction and current VR
> integration.
>
>
> On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>
> >I think our other option is to take a real look at what it would take to
> fix the VR. In my opinion, a lot of the problems are related to the
> monolithic python code base and the fact nothing is actually separated.
> >
> >Secondly, the python scripts (and bash scripts) don't use any established
> libraries to complete tasks and instead shell out and run commands that are
> both hard to track and hard to parse on return.
> >
> >
> >If we daemonized this, used a real api for Agent to VR communication,
> used common already existing libraries for the system service and network
> interactions and spent a bit of time separating out code into distinct
> modules, everything would behave a lot better.
> >
> >
> >The pain and suffering is due to years and years of patches and constant
> shelling out to complete tasks in my opinion. If we spend time to rethink
> how we interact with the VR in general and we abstract the systems and
> networking stuff and use well known and stable libraries to do the work,
> the VR would be much easier to maintain.
> >
> >
> >- Si
> >
> >
> >
> >
> >________________________________
> >From: Marty Godsey <ma...@gonsource.com>
> >Sent: Friday, September 16, 2016 12:24 PM
> >To: dev@cloudstack.apache.org
> >Subject: RE: [DISCUSS] Replacing the VR
> >
> >So based upon this discussion would it be prudent to wait on VyOS 2.0?
> The current VR is giving us issues but would the time invested in another
> "solution" be wasted especially if by the time another option is chose,
> then coded, then tested, then implemented and right as that time happened
> to be when VyOS 2.0 is released. Of course you said they are just in the
> scoping range so this could still be a year or more out.
> >
> >Thoughts?
> >
> >Regards,
> >Marty Godsey
> >nSource Solutions
> >
> >-----Original Message-----
> >From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> >Sent: Friday, September 16, 2016 10:31 AM
> >To: dev@cloudstack.apache.org
> >Cc: daniil@baturin.org
> >Subject: Re: [DISCUSS] Replacing the VR
> >
> >I just had a quick chat with a couple of the guys over on the VyOS chat.
> >I have CC'ed one of them in case we have more licensing questions.
> >
> >So here is the status with the license "the code inherited from Vyatta
> and our modifications from it is GPLv2 (strict, not v2+). The config
> reading library is GPLv2 too, so anything that links to is is GPLv2.
> >Some auxiliary components we made after the fork are more permissive,
> >LGPLv2+ or MIT."
> >
> >They are currently in the process of scoping a redesign (VyOS 2.0), "we
> are planning a clean rewrite that will solve issues of the current config
> system".
> >This will include the ability to configure via the API.
> >
> >If we have more questions for VyOS, they are very friendly and
> responsive, so we should be able to get answers.
> >
> >*Will STEVENS*
> >Lead Developer
> >
> >*CloudOps* *| *Cloud Solutions Experts
> >420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
> >
> >On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
> >
> >> I agree with Will Ilya. There are so many problems with the VR right
> now.
> >> Most of the outages we've had recently have somehow involved the VR.
> >> We set custom iptables rules on the VR which can and have easily gone
> wrong.
> >> Openswan is broken, Strongswan replacement still needs to be tested.
> >> VVRP with redundant router still needs work, and not to mention the
> >> problems we will have when we introduce IPv6 into the whole picture.
> >>
> >> I think the spirit of the discussion is to rely on a 3rd party to do
> >> the networking for us (eg VyOS) and have us handle just the
> >> orchestration. All the problems that I've described have already been
> >> solved in VyOS. We also get the advantage of a potential wider
> >> community to fix and maintain the VR and given our current development
> >> velocity, it think it totally makes sense to look for a 3rd party
> option.
> >>
> >> -Syed
> >>
> >>
> >> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
> >> wrote:
> >>
> >> > The VR has been biting us far too often recently, which is why we
> >> > have started looking into alternative implementations.
> >> >
> >> > One of the things that is nice about potentially using the VyOS is
> >> > that
> >> it
> >> > is based on Debian, so we should be able to run the other services
> >> > that
> >> we
> >> > currently have like the password server and userdata on the VyOS.
> >> > This means we would not have to change our architecture initially
> >> > and could focus on only replacing the networking paths.
> >> >
> >> > *Will STEVENS*
> >> > Lead Developer
> >> >
> >> > *CloudOps* *| *Cloud Solutions Experts
> >> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> >> > tw @CloudOps_
> >> >
> >> > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> >> >
> >> > > The more this is discussed the more I think we should stick with
> >> > > our
> >> VR.
> >> > >
> >> > > All these other options either seem unfinished or with
> >> > > incompatible license.
> >> > >
> >> > > VyOS looks the most promising so far, it's a serious, mature
> project.
> >> > > Adopting it though means we'll have to microservice our way out of
> >> > > it
> >> > with
> >> > > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
> >> those
> >> > > too. Imho this adds complexity we should void.
> >> > >
> >> > > --
> >> > > Sent from the Delta quadrant using Borg technology!
> >> > >
> >> > > Nux!
> >> > > www.nux.ro
> >> > >
> >> > > ----- Original Message -----
> >> > > > From: "Will Stevens" <ws...@cloudops.com>
> >> > > > To: dev@cloudstack.apache.org
> >> > > > Sent: Thursday, 15 September, 2016 17:21:28
> >> > > > Subject: Re: [DISCUSS] Replacing the VR
> >> > >
> >> > > > Ya, we would need to add a daemon for VPN as well. Load
> >> > > > balancing is another aspect which we will need to consider if we
> went this route.
> >> > > > Something like https://traefik.io/ could potentially be a good
> >> > > > fit
> >> due
> >> > > to
> >> > > > its API driven configuration, but it may be more than what we
> need.
> >> > > >
> >> > > > We should probably try define which pieces make sense to be
> >> > > > solved
> >> > > together
> >> > > > and which pieces would be best suited to be broken out.
> >> > > >
> >> > > > I think the network connectivity, routing and firewalling should
> >> > probably
> >> > > > all stay together since the majority of the tools we would
> >> potentially
> >> > > use
> >> > > > would handle all of that together in a single implementation.
> >> > > >
> >> > > > The password server and userdata seems like a good option for
> >> > > > being
> >> > > broken
> >> > > > out and handled independently (and probably rewritten completely
> >> since
> >> > > they
> >> > > > currently have some issues).
> >> > > >
> >> > > > Load balancing is another that could warrant splitting out, but
> >> > > > that depends on what direction we go and how we would be managing
> it.
> >> DHCP
> >> > > and
> >> > > > DNS are others which could go either way.
> >> > > >
> >> > > > If we do split out services, I think we should consolidate as
> >> > > > much as
> >> > we
> >> > > > can into each service we break out. Ideally a network packet
> >> > > > would
> >> > never
> >> > > > hit more than one, maybe two, services. I don't think we should
> >> > > > be splitting services 'just because', I think we need a valid
> >> > > > case for splitting any service out because it adds complexity.
> >> > > > Our project is already complex enough, we need to avoid adding
> >> > > > complexity unless it
> >> is
> >> > > > really needed.
> >> > > >
> >> > > > Some more of my thoughts on this anyway...
> >> > > >
> >> > > > *Will STEVENS*
> >> > > > Lead Developer
> >> > > >
> >> > > > *CloudOps* *| *Cloud Solutions Experts
> >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> >> > > > *|* tw @CloudOps_
> >> > > >
> >> > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
> >> > wrote:
> >> > > >
> >> > > >> I do agree with you that this probably isn't the right place
> >> > > >> the
> >> > > password
> >> > > >> service and user data.
> >> > > >>
> >> > > >>
> >> > > >> Having said that, after taking a cursory look at the dev docs,
> >> > > >> it
> >> > > doesn't
> >> > > >> seem that difficult to add new daemons:
> >> https://opensnaproute.github.
> >> > > >> io/docs/developer.html#creating-new-component
> >> > > >>
> >> > > >> <https://opensnaproute.github.io/docs/developer.html#
> >> > > >> creating-new-component>
> >> > > >>
> >> > > >>
> >> > > >> They've definitely build it with a microservices architecture
> >> > > >> in
> >> mind,
> >> > > so
> >> > > >> each individual feature is abstracted into it's own small
> >> > > >> daemon
> >> > > process.
> >> > > >> We could just create a daemon for the password server and the
> >> userdata
> >> > > >> components if we really had to.
> >> > > >>
> >> > > >>
> >> > > >> - Si
> >> > > >>
> >> > > >>
> >> > > >> ________________________________
> >> > > >> From: williamstevens@gmail.com <wi...@gmail.com> on
> >> > > >> behalf
> >> > of
> >> > > >> Will Stevens <ws...@cloudops.com>
> >> > > >> Sent: Thursday, September 15, 2016 9:17 AM
> >> > > >> To: dev@cloudstack.apache.org
> >> > > >> Subject: Re: [DISCUSS] Replacing the VR
> >> > > >>
> >> > > >> A big part of why I know about it is because it is written in Go.
> >> :P
> >> > > >>
> >> > > >> Yes, it is definitely interesting for the routing and traffic
> >> handling
> >> > > >> aspects of the VR. We will likely have to rethink some of the
> >> pieces
> >> > a
> >> > > >> little bit like the password server and userdata if we are to
> >> > > >> adopt
> >> a
> >> > > >> different VR approach. This is where I think some of JohnB and
> >> > > Chiradeep's
> >> > > >> ideas make sense. In many ways, it does not make sense for the
> >> device
> >> > > >> handling routing and network traffic to also be responsible for
> >> > > passwords
> >> > > >> and userdata.
> >> > > >>
> >> > > >> *Will STEVENS*
> >> > > >> Lead Developer
> >> > > >>
> >> > > >> *CloudOps* *| *Cloud Solutions Experts
> >> > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> >> > > >> *|* tw @CloudOps_
> >> > > >>
> >> > > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
> >> > wrote:
> >> > > >>
> >> > > >> > I hadn't heard of Flexswitch until you mentioned it. It looks
> >> pretty
> >> > > >> cool!
> >> > > >> > It even supports ONIE install.
> >> > > >> >
> >> > > >> > To be honest, the ipsec feature could be added, or we could
> >> offload
> >> > > it to
> >> > > >> > separate vm if we needed to. The fact it is so feature rich
> >> > > >> > from a
> >> > > >> routing
> >> > > >> > perspective (and all API driven) is really nice.
> >> > > >> >
> >> > > >> >
> >> > > >> > Based on the roadmap, it looks like they plan to also support
> >> > > >> capabilities
> >> > > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
> >> > > >> > will
> >> be
> >> > > huge
> >> > > >> > for our carrier community that rely on these technologies to
> >> > > >> > do
> >> > > private
> >> > > >> > gateway and inter-VPC interconnections today. We handle this
> >> > > >> > stuff
> >> > on
> >> > > our
> >> > > >> > ASRs right now with a vlan interconnect into the VR. Being
> >> > > >> > able to
> >> > do
> >> > > >> MPLS
> >> > > >> > all the way to the VR would be awesome.
> >> > > >> >
> >> > > >> >
> >> > > >> > It also seems to be written in GO (a language here at ENA we
> >> > > >> > know
> >> > very
> >> > > >> > well).
> >> > > >> >
> >> > > >> >
> >> > > >> > - Si
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> > ________________________________
> >> > > >> > From: Will Stevens <wi...@gmail.com>
> >> > > >> > Sent: Thursday, September 15, 2016 7:06 AM
> >> > > >> > To: dev@cloudstack.apache.org
> >> > > >> > Subject: RE: [DISCUSS] Replacing the VR
> >> > > >> >
> >> > > >> > Ya. I don't think it covers our whole use case, but what it
> >> > > >> > does
> >> > > cover is
> >> > > >> > all api driven...
> >> > > >> >
> >> > > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
> >> > wrote:
> >> > > >> >
> >> > > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it
> >> > > >> > > was
> >> not
> >> > > >> > > intended to do IPSec.
> >> > > >> > >
> >> > > >> > > It seems as though VyOS is starting to look like the best
> >> option.
> >> > > >> > >
> >> > > >> > > Regards,
> >> > > >> > > Marty Godsey
> >> > > >> > > nSource Solutions
> >> > > >> > >
> >> > > >> > > -----Original Message-----
> >> > > >> > > From: williamstevens@gmail.com
> >> > > >> > > [mailto:williamstevens@gmail.com
> >> ]
> >> > On
> >> > > >> > > Behalf Of Will Stevens
> >> > > >> > > Sent: Wednesday, September 14, 2016 11:06 PM
> >> > > >> > > To: dev@cloudstack.apache.org
> >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> >> > > >> > >
> >> > > >> > > Or we could go completely crazy and go with something like
> >> > > FlexSwitch
> >> > > >> > from
> >> > > >> > > SnapRoute
> >> > > >> > > - http://www.snaproute.com/
> >> > > >> > > - https://opensnaproute.github.io/docs/apis.html
> >> > > >> > >
> >> > > >> > > *Will STEVENS*
> >> > > >> > > Lead Developer
> >> > > >> > >
> >> > > >> > > *CloudOps* *| *Cloud Solutions Experts
> >> > > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> >> > > >> > > cloudops.com
> >> > *|*
> >> > > tw
> >> > > >> > > @CloudOps_
> >> > > >> > >
> >> > > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> >> > > wstevens@cloudops.com>
> >> > > >> > > wrote:
> >> > > >> > >
> >> > > >> > > > I tend to agree with Syed and Marty. I am not sure what
> >> > problems
> >> > > are
> >> > > >> > > > solved by splitting up the function of the VR into a
> >> > > >> > > > bunch of
> >> > > >> separate
> >> > > >> > > > services. As Syed points out, the complexity added is
> >> > > non-trivial.
> >> > > >> > > > We now have to manage all the intercontainer networking
> >> > > >> > > > as
> >> well
> >> > as
> >> > > >> the
> >> > > >> > > > orchestrated ACS networking.
> >> > > >> > > >
> >> > > >> > > > VyOS is interesting to me because it covers the majority
> >> > > >> > > > of
> >> our
> >> > > use
> >> > > >> > > > case with a single unified control plane. It also has
> >> > > >> > > > good
> >> > > support
> >> > > >> > > > for extending features we care about, like IPv6, VXLAN,
> >> > > >> > > > VRRP, transactions, etc...
> >> > > >> > > >
> >> > > >> > > > *Will STEVENS*
> >> > > >> > > > Lead Developer
> >> > > >> > > >
> >> > > >> > > > *CloudOps* *| *Cloud Solutions Experts
> >> > > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> >> cloudops.com
> >> > > *|*
> >> > > >> tw
> >> > > >> > > > @CloudOps_
> >> > > >> > > >
> >> > > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> >> > sahmed@cloudops.com>
> >> > > >> > wrote:
> >> > > >> > > >
> >> > > >> > > >> Agree with Marty, adding Docker containers to the
> >> > > >> > > >> picture
> >> > > although
> >> > > >> > > >> can make the VR more flexible but the added complexity
> >> > > >> > > >> is
> >> just
> >> > > not
> >> > > >> > > >> worth it. Not to mention we would need to take care of
> >> > networking
> >> > > >> > > >> each container manually and given that our iptable rules
> >> > > >> > > >> are
> >> > very
> >> > > >> > > >> unstable at the moment I don't see a big value add.
> >> > > >> > > >>
> >> > > >> > > >> Vyos looks like a better solution to me. I know that it
> >> > > >> > > >> does
> >> > not
> >> > > >> > > >> provide an api but it does fit the bill quite well
> >> otherwise. I
> >> > > >> > > >> specially like the fact that it has a transaction based
> >> > > >> > > >> model
> >> > and
> >> > > >> you
> >> > > >> > > >> can rollback changes if something goes wrong.
> >> > > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> >> > > marty@gonsource.com>
> >> > > >> > > wrote:
> >> > > >> > > >>
> >> > > >> > > >> > Licensing aside, I think splitting the various
> >> > > >> > > >> > functions
> >> into
> >> > > >> > > >> > containers is not a good route either. This will force
> >> users
> >> > to
> >> > > >> > > >> > have to maintain
> >> > > >> > > >> and
> >> > > >> > > >> > use containers and adds complexity to the networking
> >> aspects
> >> > of
> >> > > >> ACS.
> >> > > >> > > >> > Complexity decreases stability. Now I understand the
> >> argument
> >> > > that
> >> > > >> > > >> > a monolithic approach also brings its own set of
> >> > > >> > > >> > issues but
> >> > it
> >> > > >> also
> >> > > >> > > >> > simplifies it.
> >> > > >> > > >> >
> >> > > >> > > >> > Regards,
> >> > > >> > > >> > Marty Godsey
> >> > > >> > > >> > nSource Solutions
> >> > > >> > > >> >
> >> > > >> > > >> > -----Original Message-----
> >> > > >> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> >> > > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
> >> > > >> > > >> > To: dev@cloudstack.apache.org
> >> > > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR
> >> > > >> > > >> >
> >> > > >> > > >> > I rather doubt that the Cloudrouter will fit the needs
> >> > > >> > > >> > of
> >> the
> >> > > >> > > >> > CloudStack project
> >> > > >> > > >> > - it is AGPL licensed. Many enterprises will not
> >> > > >> > > >> > touch
> >> > > anything
> >> > > >> > > >> > that
> >> > > >> > > >> has
> >> > > >> > > >> > AGPL
> >> > > >> > > >> > - the github repo shows rather infrequent updates.
> >> > > >> > > >> > Quite
> >> > > likely
> >> > > >> > > >> > they aren't considering the use cases of the
> >> > > >> > > >> > CloudStack
> >> > > community
> >> > > >> > > >> >
> >> > > >> > > >> > I'd back John B's comments on disaggregating the VR.
> >> > > >> > > >> > Split
> >> it
> >> > > into
> >> > > >> > > >> > many docker containers
> >> > > >> > > >> > - password server
> >> > > >> > > >> > - userdata server
> >> > > >> > > >> > - DHCP / DNS
> >> > > >> > > >> > - s2s VPN
> >> > > >> > > >> > - RA VPN
> >> > > >> > > >> > - intra-VPC routing and ACL
> >> > > >> > > >> > - Port forwarding + NAT
> >> > > >> > > >> > - FW
> >> > > >> > > >> > - LB (public)
> >> > > >> > > >> > - LB (internal),
> >> > > >> > > >> > - secondary storage
> >> > > >> > > >> > - agent
> >> > > >> > > >> > Glue them together with docker compose files (one per
> >> > > >> > > >> > use
> >> > > case -
> >> > > >> > > >> > basic zone, isolated, VPC, SSVM, etc).
> >> > > >> > > >> >
> >> > > >> > > >> > The VR image then becomes a JeOS + docker. You can
> >> > > >> > > >> > test
> >> each
> >> > of
> >> > > >> the
> >> > > >> > > >> > components independently and fixing one bug in the
> >> > > >> > > >> > field
> >> (say
> >> > > >> DHCP)
> >> > > >> > > >> > is hitless to the other components. You don't need to
> >> > > >> > > >> > build per-hypervisor VRs. You could even run on
> baremetal.
> >> > > >> > > >> >
> >> > > >> > > >> > Along the way you need to figure out how to
> >> > > >> > > >> > - make the traffic traverse the containers that are
> >> > > >> > > >> > needed
> >> > to
> >> > > be
> >> > > >> > > >> > traversed (in most cases just 1)
> >> > > >> > > >> > - bootstrap the router (how does it find its compose
> file?
> >> > > where
> >> > > >> > > >> > is the
> >> > > >> > > >> > registry?)
> >> > > >> > > >> > - rethink the command and control of the VR
> >> > > >> > > >> > functions. SSH
> >> > > works,
> >> > > >> > > >> > but something more declarative, idempotent should be
> >> > explored.
> >> > > >> > > >> >
> >> > > >> > > >> > As you do this, it becomes clearer which of the
> >> > > >> > > >> > functions
> >> can
> >> > > be
> >> > > >> > > >> > substituted by for example CloudRouter. Command and
> >> > > >> > > >> > Control
> >> > of
> >> > > the
> >> > > >> > > >> docker
> >> > > >> > > >> > containers can be moved out to another container. Etc.
> >> > > >> > > >> >
> >> > > >> > > >> >
> >> > > >> > > >> >
> >> > > >> > > >> >
> >> > > >> > > >> >
> >> > > >> > > >> >
> >> > > >> > > >> >
> >> > > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> >> > > >> > > >> > <ma...@gonsource.com>
> >> > > >> > > >> > wrote:
> >> > > >> > > >> >
> >> > > >> > > >> > > This one does look nice. My biggest concern is the
> >> > > >> > > >> > > lack
> >> of
> >> > > >> > > >> > > VXLANs. It seems that any of the ones we mentioned
> >> > > >> > > >> > > do not
> >> > > have
> >> > > >> an
> >> > > >> > > >> > > API so we may be stuck at the SSH method.
> >> > > >> > > >> > >
> >> > > >> > > >> > > Regards,
> >> > > >> > > >> > > Marty Godsey
> >> > > >> > > >> > > nSource Solutions
> >> > > >> > > >> > >
> >> > > >> > > >> > > -----Original Message-----
> >> > > >> > > >> > > From: Abhinandan Prateek
> >> > > >> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
> >> > > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> >> > > >> > > >> > > To: dev@cloudstack.apache.org
> >> > > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> >> > > >> > > >> > >
> >> > > >> > > >> > > Cloudrouter looks promising. These have potential to
> >> > > >> > > >> > > save
> >> > > future
> >> > > >> > > >> > > engineering effort for example on ipv6 routing, OSPF
> etc.
> >> > > >> > > >> > > And the best part is they come with test automation
> >> > > framework.
> >> > > >> > > >> > >
> >> > > >> > > >> > >
> >> > > >> > > >> > >
> >> > > >> > > >> > >
> >> > > >> > > >> > >
> >> > > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> >> > > >> > > >> > > <ja...@accelerite.com>
> >> > > >> > > >> > > wrote:
> >> > > >> > > >> > >
> >> > > >> > > >> > > >Hi,
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >Instead of replacing the VR in first place we
> >> > > >> > > >> > > >should add VyOS/cloudrouter
> >> > > >> > > >> > > as provider. Once it is stable, network offerings
> >> > > >> > > >> > > (on
> >> > > upgrade)
> >> > > >> > > >> > > can be updated to use it and we can drop the VR if
> >> > > >> > > >> > > we
> >> want
> >> > at
> >> > > >> > > >> > > that release
> >> > > >> > > >> > onwards.
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >VR is stabilized over a period of time and some of
> >> > > >> > > >> > > >them
> >> > are
> >> > > >> > > >> > > >running
> >> > > >> > > >> > > without issues. When we replicate the ACS VR
> >> > > >> > > >> > > features in
> >> > new
> >> > > >> > > >> > > solution it takes some to find the missing pieces
> >> > > >> > > >> > > (hidden
> >> > > bugs).
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >Thanks,
> >> > > >> > > >> > > >Jayapal
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >> nux@li.nux.ro> wrote:
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >> Hi,
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >> I like the idea.
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >> Cloudrouter looks really promising, I'm not too
> >> > > >> > > >> > > >> keen
> >> on
> >> > > VyOS
> >> > > >> > > >> > > >> (it
> >> > > >> > > >> > > doesn't have a proper http api etc).
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >> --
> >> > > >> > > >> > > >> Sent from the Delta quadrant using Borg technology!
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >> Nux!
> >> > > >> > > >> > > >> www.nux.ro
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > abhinandan.prateek@shapeblue.com
> >> > > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
> >> > > >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> >> > > @shapeblue
> >> > > >> > > >> > >
> >> > > >> > > >> > >
> >> > > >> > > >> > >
> >> > > >> > > >> > > ----- Original Message -----
> >> > > >> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> >> > > >> > > >> > > >>> To: dev@cloudstack.apache.org
> >> > > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> >> > > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
> >> > > >> > > >> > > >>
> >> > > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and
> >> > > >> > > >> > > >>> should
> >> > be
> >> > > >> > > >> > > >>> treated as
> >> > > >> > > >> > > such.
> >> > > >> > > >> > > >>> Please weigh in with the good and bad of this
> idea...
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> A couple of us have been discussing the idea of
> >> > > potentially
> >> > > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open
> >> > > >> > > >> > > >>> Source
> >> > > Vyatta
> >> > > >> > VM).
> >> > > >> > > >> > > >>> There may be a license issue because I think it
> >> > > >> > > >> > > >>> is
> >> > > licensed
> >> > > >> > > >> > > >>> under GPL, but for the sake of discussion, let's
> >> assume
> >> > > we
> >> > > >> > > >> > > >>> can overcome any
> >> > > >> > > >> > > license issues.
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> I have spent some time recently with the VyOS
> >> > > >> > > >> > > >>> and I
> >> > have
> >> > > to
> >> > > >> > > >> > > >>> admit, I was pretty impressed. It is simple and
> >> > > intuitive
> >> > > >> > > >> > > >>> and it gives you a lot more options for auditing
> >> > > >> > > >> > > >>> the
> >> > > >> > > configuration etc...
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> Items of potential interest:
> >> > > >> > > >> > > >>> - Clean up our current VR script spaghetti to a
> >> simpler
> >> > > more
> >> > > >> > > >> > > >>> auditable configuration workflow.
> >> > > >> > > >> > > >>> - Gives a cleaner path for IPv6 support.
> >> > > >> > > >> > > >>> - Handles VPN configuration via the same
> >> configuration
> >> > > >> > > interface.
> >> > > >> > > >> > > >>> - Support for OSPF & BGP.
> >> > > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
> >> > > >> > > >> > > >>> - Easily supports HA (redundant routers) through
> >> VRRP.
> >> > > >> > > >> > > >>> - VXLAN support.
> >> > > >> > > >> > > >>> - Transaction based changes to the VR with
> >> > > >> > > >> > > >>> rollback
> >> on
> >> > > >> error.
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> Items that could be difficult to solve:
> >> > > >> > > >> > > >>> - Userdata password reset workflow and
> >> implementation.
> >> > > >> > > >> > > >>> - Upgrade process.
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> The VyOS is not the only option if we were to
> >> consider
> >> > > this
> >> > > >> > > >> approach.
> >> > > >> > > >> > > >>> Another option, which I don't know as well,
> >> > > >> > > >> > > >>> would be CloudRouter (AGPL
> >> > > >> > > >> > > >>> license) [2] which is purely API driven.
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> Anyway, would love to hear your thoughts...
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> Will
> >> > > >> > > >> > > >>>
> >> > > >> > > >> > > >>> [1] https://vyos.io/ [2]
> >> > > >> > > >> > > >>> https://cloudrouter.org/
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >
> >> > > >> > > >> > > >DISCLAIMER
> >> > > >> > > >> > > >==========
> >> > > >> > > >> > > >This e-mail may contain privileged and confidential
> >> > > information
> >> > > >> > > >> > > >which is
> >> > > >> > > >> > > the property of Accelerite, a Persistent Systems
> >> business.
> >> > > It is
> >> > > >> > > >> > > intended only for the use of the individual or
> >> > > >> > > >> > > entity to
> >> > > which
> >> > > >> it
> >> > > >> > > >> > > is addressed. If you are not the intended recipient,
> >> > > >> > > >> > > you
> >> > are
> >> > > not
> >> > > >> > > >> > > authorized to read, retain, copy, print, distribute
> >> > > >> > > >> > > or
> >> use
> >> > > this
> >> > > >> > > >> > > message. If you have received this communication in
> >> error,
> >> > > >> please
> >> > > >> > > >> > > notify the sender and delete all copies of this
> message.
> >> > > >> > > >> > > Accelerite, a Persistent Systems business does not
> >> > > >> > > >> > > accept
> >> > any
> >> > > >> > > >> > > liability for virus
> >> > > >> > > >> > infected mails.
> >> > > >> > > >> > >
> >> > > >> > > >> >
> >> > > >> > > >>
> >> > > >> > > >
> >> > > >> > > >
> >> > > >> > >
> >> > > >> >
> >> > >
> >> >
>
>
Re: [DISCUSS] Replacing the VR
Posted by Murali Reddy <mu...@gmail.com>.
Configuration management of network appliances particularly for Cloud and NFV scenarios is still evolving area. Programmability is the not the strength for even the most popular network operating systems like IOS, JunoS etc. So its not surprising why CloudStack integrates in a archaic way with stock linux for the VR.
VR was never integral/hardcoded option in CloudStack. Its always been a plugin. CloudStack network orchestration is well abstracted and designed with vision to compose a network with different set of providers for different services. Yes that vision is not fully realised yet, and we don’t have true service function chains. That would be different discussion topic.
I tend to agree with Simon, as alternate/interim option we can take hard look whats causing the problems with current VR integration. Personally, I think it would be easier we take a cue from configuration managers and network configuration solutions out there (for e.g promise theory based Cisco ACI) move to more declarative model of expressing desired state of network configuration. Infact current VR from 4.6, actually holds the desired state per service basis, seems to be in that direction.
It does make sense to evaluate new appliances which can provide rich semantics (like programmable API, declarative configuration, versioning, commit/rollback etc), but will need significant engineering effort and time to stabilise. We may make same mistakes with integration of other appliance as well, if we fully don’t understand the nature of the current problems with CloudStack core and service provider interaction and current VR integration.
On 16/09/16, 11:59 PM, "Simon Weller" <sw...@ena.com> wrote:
>I think our other option is to take a real look at what it would take to fix the VR. In my opinion, a lot of the problems are related to the monolithic python code base and the fact nothing is actually separated.
>
>Secondly, the python scripts (and bash scripts) don't use any established libraries to complete tasks and instead shell out and run commands that are both hard to track and hard to parse on return.
>
>
>If we daemonized this, used a real api for Agent to VR communication, used common already existing libraries for the system service and network interactions and spent a bit of time separating out code into distinct modules, everything would behave a lot better.
>
>
>The pain and suffering is due to years and years of patches and constant shelling out to complete tasks in my opinion. If we spend time to rethink how we interact with the VR in general and we abstract the systems and networking stuff and use well known and stable libraries to do the work, the VR would be much easier to maintain.
>
>
>- Si
>
>
>
>
>________________________________
>From: Marty Godsey <ma...@gonsource.com>
>Sent: Friday, September 16, 2016 12:24 PM
>To: dev@cloudstack.apache.org
>Subject: RE: [DISCUSS] Replacing the VR
>
>So based upon this discussion would it be prudent to wait on VyOS 2.0? The current VR is giving us issues but would the time invested in another "solution" be wasted especially if by the time another option is chose, then coded, then tested, then implemented and right as that time happened to be when VyOS 2.0 is released. Of course you said they are just in the scoping range so this could still be a year or more out.
>
>Thoughts?
>
>Regards,
>Marty Godsey
>nSource Solutions
>
>-----Original Message-----
>From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
>Sent: Friday, September 16, 2016 10:31 AM
>To: dev@cloudstack.apache.org
>Cc: daniil@baturin.org
>Subject: Re: [DISCUSS] Replacing the VR
>
>I just had a quick chat with a couple of the guys over on the VyOS chat.
>I have CC'ed one of them in case we have more licensing questions.
>
>So here is the status with the license "the code inherited from Vyatta and our modifications from it is GPLv2 (strict, not v2+). The config reading library is GPLv2 too, so anything that links to is is GPLv2.
>Some auxiliary components we made after the fork are more permissive,
>LGPLv2+ or MIT."
>
>They are currently in the process of scoping a redesign (VyOS 2.0), "we are planning a clean rewrite that will solve issues of the current config system".
>This will include the ability to configure via the API.
>
>If we have more questions for VyOS, they are very friendly and responsive, so we should be able to get answers.
>
>*Will STEVENS*
>Lead Developer
>
>*CloudOps* *| *Cloud Solutions Experts
>420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_
>
>On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>
>> I agree with Will Ilya. There are so many problems with the VR right now.
>> Most of the outages we've had recently have somehow involved the VR.
>> We set custom iptables rules on the VR which can and have easily gone wrong.
>> Openswan is broken, Strongswan replacement still needs to be tested.
>> VVRP with redundant router still needs work, and not to mention the
>> problems we will have when we introduce IPv6 into the whole picture.
>>
>> I think the spirit of the discussion is to rely on a 3rd party to do
>> the networking for us (eg VyOS) and have us handle just the
>> orchestration. All the problems that I've described have already been
>> solved in VyOS. We also get the advantage of a potential wider
>> community to fix and maintain the VR and given our current development
>> velocity, it think it totally makes sense to look for a 3rd party option.
>>
>> -Syed
>>
>>
>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
>> wrote:
>>
>> > The VR has been biting us far too often recently, which is why we
>> > have started looking into alternative implementations.
>> >
>> > One of the things that is nice about potentially using the VyOS is
>> > that
>> it
>> > is based on Debian, so we should be able to run the other services
>> > that
>> we
>> > currently have like the password server and userdata on the VyOS.
>> > This means we would not have to change our architecture initially
>> > and could focus on only replacing the networking paths.
>> >
>> > *Will STEVENS*
>> > Lead Developer
>> >
>> > *CloudOps* *| *Cloud Solutions Experts
>> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>> > tw @CloudOps_
>> >
>> > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>> >
>> > > The more this is discussed the more I think we should stick with
>> > > our
>> VR.
>> > >
>> > > All these other options either seem unfinished or with
>> > > incompatible license.
>> > >
>> > > VyOS looks the most promising so far, it's a serious, mature project.
>> > > Adopting it though means we'll have to microservice our way out of
>> > > it
>> > with
>> > > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>> those
>> > > too. Imho this adds complexity we should void.
>> > >
>> > > --
>> > > Sent from the Delta quadrant using Borg technology!
>> > >
>> > > Nux!
>> > > www.nux.ro
>> > >
>> > > ----- Original Message -----
>> > > > From: "Will Stevens" <ws...@cloudops.com>
>> > > > To: dev@cloudstack.apache.org
>> > > > Sent: Thursday, 15 September, 2016 17:21:28
>> > > > Subject: Re: [DISCUSS] Replacing the VR
>> > >
>> > > > Ya, we would need to add a daemon for VPN as well. Load
>> > > > balancing is another aspect which we will need to consider if we went this route.
>> > > > Something like https://traefik.io/ could potentially be a good
>> > > > fit
>> due
>> > > to
>> > > > its API driven configuration, but it may be more than what we need.
>> > > >
>> > > > We should probably try define which pieces make sense to be
>> > > > solved
>> > > together
>> > > > and which pieces would be best suited to be broken out.
>> > > >
>> > > > I think the network connectivity, routing and firewalling should
>> > probably
>> > > > all stay together since the majority of the tools we would
>> potentially
>> > > use
>> > > > would handle all of that together in a single implementation.
>> > > >
>> > > > The password server and userdata seems like a good option for
>> > > > being
>> > > broken
>> > > > out and handled independently (and probably rewritten completely
>> since
>> > > they
>> > > > currently have some issues).
>> > > >
>> > > > Load balancing is another that could warrant splitting out, but
>> > > > that depends on what direction we go and how we would be managing it.
>> DHCP
>> > > and
>> > > > DNS are others which could go either way.
>> > > >
>> > > > If we do split out services, I think we should consolidate as
>> > > > much as
>> > we
>> > > > can into each service we break out. Ideally a network packet
>> > > > would
>> > never
>> > > > hit more than one, maybe two, services. I don't think we should
>> > > > be splitting services 'just because', I think we need a valid
>> > > > case for splitting any service out because it adds complexity.
>> > > > Our project is already complex enough, we need to avoid adding
>> > > > complexity unless it
>> is
>> > > > really needed.
>> > > >
>> > > > Some more of my thoughts on this anyway...
>> > > >
>> > > > *Will STEVENS*
>> > > > Lead Developer
>> > > >
>> > > > *CloudOps* *| *Cloud Solutions Experts
>> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>> > > > *|* tw @CloudOps_
>> > > >
>> > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>> > wrote:
>> > > >
>> > > >> I do agree with you that this probably isn't the right place
>> > > >> the
>> > > password
>> > > >> service and user data.
>> > > >>
>> > > >>
>> > > >> Having said that, after taking a cursory look at the dev docs,
>> > > >> it
>> > > doesn't
>> > > >> seem that difficult to add new daemons:
>> https://opensnaproute.github.
>> > > >> io/docs/developer.html#creating-new-component
>> > > >>
>> > > >> <https://opensnaproute.github.io/docs/developer.html#
>> > > >> creating-new-component>
>> > > >>
>> > > >>
>> > > >> They've definitely build it with a microservices architecture
>> > > >> in
>> mind,
>> > > so
>> > > >> each individual feature is abstracted into it's own small
>> > > >> daemon
>> > > process.
>> > > >> We could just create a daemon for the password server and the
>> userdata
>> > > >> components if we really had to.
>> > > >>
>> > > >>
>> > > >> - Si
>> > > >>
>> > > >>
>> > > >> ________________________________
>> > > >> From: williamstevens@gmail.com <wi...@gmail.com> on
>> > > >> behalf
>> > of
>> > > >> Will Stevens <ws...@cloudops.com>
>> > > >> Sent: Thursday, September 15, 2016 9:17 AM
>> > > >> To: dev@cloudstack.apache.org
>> > > >> Subject: Re: [DISCUSS] Replacing the VR
>> > > >>
>> > > >> A big part of why I know about it is because it is written in Go.
>> :P
>> > > >>
>> > > >> Yes, it is definitely interesting for the routing and traffic
>> handling
>> > > >> aspects of the VR. We will likely have to rethink some of the
>> pieces
>> > a
>> > > >> little bit like the password server and userdata if we are to
>> > > >> adopt
>> a
>> > > >> different VR approach. This is where I think some of JohnB and
>> > > Chiradeep's
>> > > >> ideas make sense. In many ways, it does not make sense for the
>> device
>> > > >> handling routing and network traffic to also be responsible for
>> > > passwords
>> > > >> and userdata.
>> > > >>
>> > > >> *Will STEVENS*
>> > > >> Lead Developer
>> > > >>
>> > > >> *CloudOps* *| *Cloud Solutions Experts
>> > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>> > > >> *|* tw @CloudOps_
>> > > >>
>> > > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>> > wrote:
>> > > >>
>> > > >> > I hadn't heard of Flexswitch until you mentioned it. It looks
>> pretty
>> > > >> cool!
>> > > >> > It even supports ONIE install.
>> > > >> >
>> > > >> > To be honest, the ipsec feature could be added, or we could
>> offload
>> > > it to
>> > > >> > separate vm if we needed to. The fact it is so feature rich
>> > > >> > from a
>> > > >> routing
>> > > >> > perspective (and all API driven) is really nice.
>> > > >> >
>> > > >> >
>> > > >> > Based on the roadmap, it looks like they plan to also support
>> > > >> capabilities
>> > > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>> > > >> > will
>> be
>> > > huge
>> > > >> > for our carrier community that rely on these technologies to
>> > > >> > do
>> > > private
>> > > >> > gateway and inter-VPC interconnections today. We handle this
>> > > >> > stuff
>> > on
>> > > our
>> > > >> > ASRs right now with a vlan interconnect into the VR. Being
>> > > >> > able to
>> > do
>> > > >> MPLS
>> > > >> > all the way to the VR would be awesome.
>> > > >> >
>> > > >> >
>> > > >> > It also seems to be written in GO (a language here at ENA we
>> > > >> > know
>> > very
>> > > >> > well).
>> > > >> >
>> > > >> >
>> > > >> > - Si
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > ________________________________
>> > > >> > From: Will Stevens <wi...@gmail.com>
>> > > >> > Sent: Thursday, September 15, 2016 7:06 AM
>> > > >> > To: dev@cloudstack.apache.org
>> > > >> > Subject: RE: [DISCUSS] Replacing the VR
>> > > >> >
>> > > >> > Ya. I don't think it covers our whole use case, but what it
>> > > >> > does
>> > > cover is
>> > > >> > all api driven...
>> > > >> >
>> > > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>> > wrote:
>> > > >> >
>> > > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it
>> > > >> > > was
>> not
>> > > >> > > intended to do IPSec.
>> > > >> > >
>> > > >> > > It seems as though VyOS is starting to look like the best
>> option.
>> > > >> > >
>> > > >> > > Regards,
>> > > >> > > Marty Godsey
>> > > >> > > nSource Solutions
>> > > >> > >
>> > > >> > > -----Original Message-----
>> > > >> > > From: williamstevens@gmail.com
>> > > >> > > [mailto:williamstevens@gmail.com
>> ]
>> > On
>> > > >> > > Behalf Of Will Stevens
>> > > >> > > Sent: Wednesday, September 14, 2016 11:06 PM
>> > > >> > > To: dev@cloudstack.apache.org
>> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
>> > > >> > >
>> > > >> > > Or we could go completely crazy and go with something like
>> > > FlexSwitch
>> > > >> > from
>> > > >> > > SnapRoute
>> > > >> > > - http://www.snaproute.com/
>> > > >> > > - https://opensnaproute.github.io/docs/apis.html
>> > > >> > >
>> > > >> > > *Will STEVENS*
>> > > >> > > Lead Developer
>> > > >> > >
>> > > >> > > *CloudOps* *| *Cloud Solutions Experts
>> > > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>> > > >> > > cloudops.com
>> > *|*
>> > > tw
>> > > >> > > @CloudOps_
>> > > >> > >
>> > > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>> > > wstevens@cloudops.com>
>> > > >> > > wrote:
>> > > >> > >
>> > > >> > > > I tend to agree with Syed and Marty. I am not sure what
>> > problems
>> > > are
>> > > >> > > > solved by splitting up the function of the VR into a
>> > > >> > > > bunch of
>> > > >> separate
>> > > >> > > > services. As Syed points out, the complexity added is
>> > > non-trivial.
>> > > >> > > > We now have to manage all the intercontainer networking
>> > > >> > > > as
>> well
>> > as
>> > > >> the
>> > > >> > > > orchestrated ACS networking.
>> > > >> > > >
>> > > >> > > > VyOS is interesting to me because it covers the majority
>> > > >> > > > of
>> our
>> > > use
>> > > >> > > > case with a single unified control plane. It also has
>> > > >> > > > good
>> > > support
>> > > >> > > > for extending features we care about, like IPv6, VXLAN,
>> > > >> > > > VRRP, transactions, etc...
>> > > >> > > >
>> > > >> > > > *Will STEVENS*
>> > > >> > > > Lead Developer
>> > > >> > > >
>> > > >> > > > *CloudOps* *| *Cloud Solutions Experts
>> > > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>> cloudops.com
>> > > *|*
>> > > >> tw
>> > > >> > > > @CloudOps_
>> > > >> > > >
>> > > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>> > sahmed@cloudops.com>
>> > > >> > wrote:
>> > > >> > > >
>> > > >> > > >> Agree with Marty, adding Docker containers to the
>> > > >> > > >> picture
>> > > although
>> > > >> > > >> can make the VR more flexible but the added complexity
>> > > >> > > >> is
>> just
>> > > not
>> > > >> > > >> worth it. Not to mention we would need to take care of
>> > networking
>> > > >> > > >> each container manually and given that our iptable rules
>> > > >> > > >> are
>> > very
>> > > >> > > >> unstable at the moment I don't see a big value add.
>> > > >> > > >>
>> > > >> > > >> Vyos looks like a better solution to me. I know that it
>> > > >> > > >> does
>> > not
>> > > >> > > >> provide an api but it does fit the bill quite well
>> otherwise. I
>> > > >> > > >> specially like the fact that it has a transaction based
>> > > >> > > >> model
>> > and
>> > > >> you
>> > > >> > > >> can rollback changes if something goes wrong.
>> > > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>> > > marty@gonsource.com>
>> > > >> > > wrote:
>> > > >> > > >>
>> > > >> > > >> > Licensing aside, I think splitting the various
>> > > >> > > >> > functions
>> into
>> > > >> > > >> > containers is not a good route either. This will force
>> users
>> > to
>> > > >> > > >> > have to maintain
>> > > >> > > >> and
>> > > >> > > >> > use containers and adds complexity to the networking
>> aspects
>> > of
>> > > >> ACS.
>> > > >> > > >> > Complexity decreases stability. Now I understand the
>> argument
>> > > that
>> > > >> > > >> > a monolithic approach also brings its own set of
>> > > >> > > >> > issues but
>> > it
>> > > >> also
>> > > >> > > >> > simplifies it.
>> > > >> > > >> >
>> > > >> > > >> > Regards,
>> > > >> > > >> > Marty Godsey
>> > > >> > > >> > nSource Solutions
>> > > >> > > >> >
>> > > >> > > >> > -----Original Message-----
>> > > >> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>> > > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
>> > > >> > > >> > To: dev@cloudstack.apache.org
>> > > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR
>> > > >> > > >> >
>> > > >> > > >> > I rather doubt that the Cloudrouter will fit the needs
>> > > >> > > >> > of
>> the
>> > > >> > > >> > CloudStack project
>> > > >> > > >> > - it is AGPL licensed. Many enterprises will not
>> > > >> > > >> > touch
>> > > anything
>> > > >> > > >> > that
>> > > >> > > >> has
>> > > >> > > >> > AGPL
>> > > >> > > >> > - the github repo shows rather infrequent updates.
>> > > >> > > >> > Quite
>> > > likely
>> > > >> > > >> > they aren't considering the use cases of the
>> > > >> > > >> > CloudStack
>> > > community
>> > > >> > > >> >
>> > > >> > > >> > I'd back John B's comments on disaggregating the VR.
>> > > >> > > >> > Split
>> it
>> > > into
>> > > >> > > >> > many docker containers
>> > > >> > > >> > - password server
>> > > >> > > >> > - userdata server
>> > > >> > > >> > - DHCP / DNS
>> > > >> > > >> > - s2s VPN
>> > > >> > > >> > - RA VPN
>> > > >> > > >> > - intra-VPC routing and ACL
>> > > >> > > >> > - Port forwarding + NAT
>> > > >> > > >> > - FW
>> > > >> > > >> > - LB (public)
>> > > >> > > >> > - LB (internal),
>> > > >> > > >> > - secondary storage
>> > > >> > > >> > - agent
>> > > >> > > >> > Glue them together with docker compose files (one per
>> > > >> > > >> > use
>> > > case -
>> > > >> > > >> > basic zone, isolated, VPC, SSVM, etc).
>> > > >> > > >> >
>> > > >> > > >> > The VR image then becomes a JeOS + docker. You can
>> > > >> > > >> > test
>> each
>> > of
>> > > >> the
>> > > >> > > >> > components independently and fixing one bug in the
>> > > >> > > >> > field
>> (say
>> > > >> DHCP)
>> > > >> > > >> > is hitless to the other components. You don't need to
>> > > >> > > >> > build per-hypervisor VRs. You could even run on baremetal.
>> > > >> > > >> >
>> > > >> > > >> > Along the way you need to figure out how to
>> > > >> > > >> > - make the traffic traverse the containers that are
>> > > >> > > >> > needed
>> > to
>> > > be
>> > > >> > > >> > traversed (in most cases just 1)
>> > > >> > > >> > - bootstrap the router (how does it find its compose file?
>> > > where
>> > > >> > > >> > is the
>> > > >> > > >> > registry?)
>> > > >> > > >> > - rethink the command and control of the VR
>> > > >> > > >> > functions. SSH
>> > > works,
>> > > >> > > >> > but something more declarative, idempotent should be
>> > explored.
>> > > >> > > >> >
>> > > >> > > >> > As you do this, it becomes clearer which of the
>> > > >> > > >> > functions
>> can
>> > > be
>> > > >> > > >> > substituted by for example CloudRouter. Command and
>> > > >> > > >> > Control
>> > of
>> > > the
>> > > >> > > >> docker
>> > > >> > > >> > containers can be moved out to another container. Etc.
>> > > >> > > >> >
>> > > >> > > >> >
>> > > >> > > >> >
>> > > >> > > >> >
>> > > >> > > >> >
>> > > >> > > >> >
>> > > >> > > >> >
>> > > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>> > > >> > > >> > <ma...@gonsource.com>
>> > > >> > > >> > wrote:
>> > > >> > > >> >
>> > > >> > > >> > > This one does look nice. My biggest concern is the
>> > > >> > > >> > > lack
>> of
>> > > >> > > >> > > VXLANs. It seems that any of the ones we mentioned
>> > > >> > > >> > > do not
>> > > have
>> > > >> an
>> > > >> > > >> > > API so we may be stuck at the SSH method.
>> > > >> > > >> > >
>> > > >> > > >> > > Regards,
>> > > >> > > >> > > Marty Godsey
>> > > >> > > >> > > nSource Solutions
>> > > >> > > >> > >
>> > > >> > > >> > > -----Original Message-----
>> > > >> > > >> > > From: Abhinandan Prateek
>> > > >> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
>> > > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
>> > > >> > > >> > > To: dev@cloudstack.apache.org
>> > > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
>> > > >> > > >> > >
>> > > >> > > >> > > Cloudrouter looks promising. These have potential to
>> > > >> > > >> > > save
>> > > future
>> > > >> > > >> > > engineering effort for example on ipv6 routing, OSPF etc.
>> > > >> > > >> > > And the best part is they come with test automation
>> > > framework.
>> > > >> > > >> > >
>> > > >> > > >> > >
>> > > >> > > >> > >
>> > > >> > > >> > >
>> > > >> > > >> > >
>> > > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
>> > > >> > > >> > > <ja...@accelerite.com>
>> > > >> > > >> > > wrote:
>> > > >> > > >> > >
>> > > >> > > >> > > >Hi,
>> > > >> > > >> > > >
>> > > >> > > >> > > >Instead of replacing the VR in first place we
>> > > >> > > >> > > >should add VyOS/cloudrouter
>> > > >> > > >> > > as provider. Once it is stable, network offerings
>> > > >> > > >> > > (on
>> > > upgrade)
>> > > >> > > >> > > can be updated to use it and we can drop the VR if
>> > > >> > > >> > > we
>> want
>> > at
>> > > >> > > >> > > that release
>> > > >> > > >> > onwards.
>> > > >> > > >> > > >
>> > > >> > > >> > > >VR is stabilized over a period of time and some of
>> > > >> > > >> > > >them
>> > are
>> > > >> > > >> > > >running
>> > > >> > > >> > > without issues. When we replicate the ACS VR
>> > > >> > > >> > > features in
>> > new
>> > > >> > > >> > > solution it takes some to find the missing pieces
>> > > >> > > >> > > (hidden
>> > > bugs).
>> > > >> > > >> > > >
>> > > >> > > >> > > >Thanks,
>> > > >> > > >> > > >Jayapal
>> > > >> > > >> > > >
>> > > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
>> > > >> > > >> > > >
>> > > >> > > >> > > >> nux@li.nux.ro> wrote:
>> > > >> > > >> > > >>
>> > > >> > > >> > > >> Hi,
>> > > >> > > >> > > >>
>> > > >> > > >> > > >> I like the idea.
>> > > >> > > >> > > >>
>> > > >> > > >> > > >> Cloudrouter looks really promising, I'm not too
>> > > >> > > >> > > >> keen
>> on
>> > > VyOS
>> > > >> > > >> > > >> (it
>> > > >> > > >> > > doesn't have a proper http api etc).
>> > > >> > > >> > > >>
>> > > >> > > >> > > >> --
>> > > >> > > >> > > >> Sent from the Delta quadrant using Borg technology!
>> > > >> > > >> > > >>
>> > > >> > > >> > > >> Nux!
>> > > >> > > >> > > >> www.nux.ro
>> > > >> > > >> > > >>
>> > > >> > > >> > > >>
>> > > >> > > >> > > abhinandan.prateek@shapeblue.com
>> > > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
>> > > >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>> > > @shapeblue
>> > > >> > > >> > >
>> > > >> > > >> > >
>> > > >> > > >> > >
>> > > >> > > >> > > ----- Original Message -----
>> > > >> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
>> > > >> > > >> > > >>> To: dev@cloudstack.apache.org
>> > > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
>> > > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
>> > > >> > > >> > > >>
>> > > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and
>> > > >> > > >> > > >>> should
>> > be
>> > > >> > > >> > > >>> treated as
>> > > >> > > >> > > such.
>> > > >> > > >> > > >>> Please weigh in with the good and bad of this idea...
>> > > >> > > >> > > >>>
>> > > >> > > >> > > >>> A couple of us have been discussing the idea of
>> > > potentially
>> > > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open
>> > > >> > > >> > > >>> Source
>> > > Vyatta
>> > > >> > VM).
>> > > >> > > >> > > >>> There may be a license issue because I think it
>> > > >> > > >> > > >>> is
>> > > licensed
>> > > >> > > >> > > >>> under GPL, but for the sake of discussion, let's
>> assume
>> > > we
>> > > >> > > >> > > >>> can overcome any
>> > > >> > > >> > > license issues.
>> > > >> > > >> > > >>>
>> > > >> > > >> > > >>> I have spent some time recently with the VyOS
>> > > >> > > >> > > >>> and I
>> > have
>> > > to
>> > > >> > > >> > > >>> admit, I was pretty impressed. It is simple and
>> > > intuitive
>> > > >> > > >> > > >>> and it gives you a lot more options for auditing
>> > > >> > > >> > > >>> the
>> > > >> > > configuration etc...
>> > > >> > > >> > > >>>
>> > > >> > > >> > > >>> Items of potential interest:
>> > > >> > > >> > > >>> - Clean up our current VR script spaghetti to a
>> simpler
>> > > more
>> > > >> > > >> > > >>> auditable configuration workflow.
>> > > >> > > >> > > >>> - Gives a cleaner path for IPv6 support.
>> > > >> > > >> > > >>> - Handles VPN configuration via the same
>> configuration
>> > > >> > > interface.
>> > > >> > > >> > > >>> - Support for OSPF & BGP.
>> > > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
>> > > >> > > >> > > >>> - Easily supports HA (redundant routers) through
>> VRRP.
>> > > >> > > >> > > >>> - VXLAN support.
>> > > >> > > >> > > >>> - Transaction based changes to the VR with
>> > > >> > > >> > > >>> rollback
>> on
>> > > >> error.
>> > > >> > > >> > > >>>
>> > > >> > > >> > > >>> Items that could be difficult to solve:
>> > > >> > > >> > > >>> - Userdata password reset workflow and
>> implementation.
>> > > >> > > >> > > >>> - Upgrade process.
>> > > >> > > >> > > >>>
>> > > >> > > >> > > >>> The VyOS is not the only option if we were to
>> consider
>> > > this
>> > > >> > > >> approach.
>> > > >> > > >> > > >>> Another option, which I don't know as well,
>> > > >> > > >> > > >>> would be CloudRouter (AGPL
>> > > >> > > >> > > >>> license) [2] which is purely API driven.
>> > > >> > > >> > > >>>
>> > > >> > > >> > > >>> Anyway, would love to hear your thoughts...
>> > > >> > > >> > > >>>
>> > > >> > > >> > > >>> Will
>> > > >> > > >> > > >>>
>> > > >> > > >> > > >>> [1] https://vyos.io/ [2]
>> > > >> > > >> > > >>> https://cloudrouter.org/
>> > > >> > > >> > > >
>> > > >> > > >> > > >
>> > > >> > > >> > > >
>> > > >> > > >> > > >
>> > > >> > > >> > > >DISCLAIMER
>> > > >> > > >> > > >==========
>> > > >> > > >> > > >This e-mail may contain privileged and confidential
>> > > information
>> > > >> > > >> > > >which is
>> > > >> > > >> > > the property of Accelerite, a Persistent Systems
>> business.
>> > > It is
>> > > >> > > >> > > intended only for the use of the individual or
>> > > >> > > >> > > entity to
>> > > which
>> > > >> it
>> > > >> > > >> > > is addressed. If you are not the intended recipient,
>> > > >> > > >> > > you
>> > are
>> > > not
>> > > >> > > >> > > authorized to read, retain, copy, print, distribute
>> > > >> > > >> > > or
>> use
>> > > this
>> > > >> > > >> > > message. If you have received this communication in
>> error,
>> > > >> please
>> > > >> > > >> > > notify the sender and delete all copies of this message.
>> > > >> > > >> > > Accelerite, a Persistent Systems business does not
>> > > >> > > >> > > accept
>> > any
>> > > >> > > >> > > liability for virus
>> > > >> > > >> > infected mails.
>> > > >> > > >> > >
>> > > >> > > >> >
>> > > >> > > >>
>> > > >> > > >
>> > > >> > > >
>> > > >> > >
>> > > >> >
>> > >
>> >
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
Thank you for the feedback Matthew. The Debian 6 EOL thing is something
that VyOS has taken very seriously. Everything has been ported to Debian 8
now, but it is still in beta and is not considered 'stable' yet because
they have not been able to regression test the entire feature set yet. In
talking to them on Friday, the Debian 8 version should be the stable
release within the next 6 months.
Our team discussed some of the implementation details regarding VyOS this
morning and Syed had some really good ideas. I will see if I can get him
to post his ideas to the list.
Cheers,
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Mon, Sep 19, 2016 at 11:32 AM, Matthew Smart <msmart@smartsoftwareinc.com
> wrote:
> We have been using Vyatta (and then Vyos) after the fork for all of our
> production routers for over four years now. It is rock solid and performs
> better than the Juniper routers we replaced them with. It is debian based
> underneath its custom shell whose syntax is very much like Juniper. A sudo
> su gets you a standard bash shell just like in any other debian distro and
> I have installed additional packages in the past. It is really easy to
> configure in that the entire router config is held in a single file with a
> dead simple file structure. You can use the commands in their custom shell
> to alter the config but I prefer to just upload a mod to that file... I
> know, I like to live dangerously! lol. Seriously though, they do have
> versioning/rollback capability but last I checked a rollback forced a
> reboot of the router.
>
> The major downside, and one we are struggling with now is that it is
> running an EOL version of debian and that is a HUGE liability for something
> as security critical as an edge router. They have a lot of work ahead of
> them to get out 2.0 but aside from the EOL Debian, Vyos "just works" and
> out of the box would exceed the current VR in pretty much every way.
> Personally, I find the API issue to be less important. #1 priority has to
> be getting a VR that functions at an enterprise level. API integration is
> important but pales compared to that.
>
> Conversely, we are still not in production with Cloudstack specifically
> because of the current state of the VR. IOW, the current VR implementation
> is directly affecting Cloudstack adoption at least anecdotally.
>
> My team is very new to contributing to CS and our time is limited but,
> given my many years of experience administering networks with Vyos at the
> core, if you go that route I will gladly lend my knowledge in helping you
> map out an implementation.
>
> I have evaluated Cloud Router in the past and found it lacking in basic
> features (and especially documentation) that made it a no go for even a
> test build. It is promising and I hope it continues to mature but Vyos is a
> battle hardened and proven technology. Given the option, I always go with
> the known good solution over the shiny one.
>
> I have also used PFsense. Really great project but I have not found it to
> be as stable for me as Vyos, though that might be my lack of BSD experience.
>
> Thanks,
>
>
> Matthew Smart
> President
> Smart Software Solutions Inc.
> 108 S Pierre St.
> Pierre, SD 57501
>
> Phone: (605) 280-0383
> Skype: msmart13
> Email: msmart@smartsoftwareinc.com
>
> On 09/18/2016 11:09 AM, Will Stevens wrote:
>
>> At this point in the discussion, I don't think we should rule anything
>> out.
>> I think it makes sense to explore all the options and then isolate some
>> front runners in terms of software and architecture.
>>
>> On Sep 18, 2016 1:08 AM, "ilya" <il...@gmail.com> wrote:
>>
>> Our options become much better if we consider BSD based routers.
>>>
>>> Would that be on the table?
>>>
>>> https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributions
>>>
>>>
>>> On 9/16/16 12:04 PM, Will Stevens wrote:
>>>
>>>> Ya, your points are all valid Simon. The lack of standard libraries to
>>>> handle a lot of the details is a problem. I don't think it is an
>>>> unsolvable problem, but if we spend the time to do that, will we have
>>>> something that will work for us for the next 5 years? This may be the
>>>> shortest path to getting us where we need to be for the time being.
>>>>
>>>> What is the best case scenario for the VR going forward which will last
>>>>
>>> us
>>>
>>>> the next 5 years? Maybe we just clean up what we have to do a major
>>>> restructuring of the pieces and how they are implemented. We need to
>>>>
>>> keep
>>>
>>>> in mind how maintainable this implementation is because that is going to
>>>>
>>> be
>>>
>>>> key going forward IMO.
>>>>
>>>>
>>>>
>>>> *Will STEVENS*
>>>> Lead Developer
>>>>
>>>> *CloudOps* *| *Cloud Solutions Experts
>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>>> w cloudops.com *|* tw @CloudOps_
>>>>
>>>> On Fri, Sep 16, 2016 at 2:29 PM, Simon Weller <sw...@ena.com> wrote:
>>>>
>>>> I think our other option is to take a real look at what it would take to
>>>>> fix the VR. In my opinion, a lot of the problems are related to the
>>>>> monolithic python code base and the fact nothing is actually separated.
>>>>>
>>>>> Secondly, the python scripts (and bash scripts) don't use any
>>>>>
>>>> established
>>>
>>>> libraries to complete tasks and instead shell out and run commands that
>>>>>
>>>> are
>>>
>>>> both hard to track and hard to parse on return.
>>>>>
>>>>>
>>>>> If we daemonized this, used a real api for Agent to VR communication,
>>>>>
>>>> used
>>>
>>>> common already existing libraries for the system service and network
>>>>> interactions and spent a bit of time separating out code into distinct
>>>>> modules, everything would behave a lot better.
>>>>>
>>>>>
>>>>> The pain and suffering is due to years and years of patches and
>>>>> constant
>>>>> shelling out to complete tasks in my opinion. If we spend time to
>>>>>
>>>> rethink
>>>
>>>> how we interact with the VR in general and we abstract the systems and
>>>>> networking stuff and use well known and stable libraries to do the
>>>>> work,
>>>>> the VR would be much easier to maintain.
>>>>>
>>>>>
>>>>> - Si
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ________________________________
>>>>> From: Marty Godsey <ma...@gonsource.com>
>>>>> Sent: Friday, September 16, 2016 12:24 PM
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>
>>>>> So based upon this discussion would it be prudent to wait on VyOS 2.0?
>>>>>
>>>> The
>>>
>>>> current VR is giving us issues but would the time invested in another
>>>>> "solution" be wasted especially if by the time another option is chose,
>>>>> then coded, then tested, then implemented and right as that time
>>>>>
>>>> happened
>>>
>>>> to be when VyOS 2.0 is released. Of course you said they are just in
>>>>>
>>>> the
>>>
>>>> scoping range so this could still be a year or more out.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> Regards,
>>>>> Marty Godsey
>>>>> nSource Solutions
>>>>>
>>>>> -----Original Message-----
>>>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>>>>> Behalf Of Will Stevens
>>>>> Sent: Friday, September 16, 2016 10:31 AM
>>>>> To: dev@cloudstack.apache.org
>>>>> Cc: daniil@baturin.org
>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>
>>>>> I just had a quick chat with a couple of the guys over on the VyOS
>>>>> chat.
>>>>> I have CC'ed one of them in case we have more licensing questions.
>>>>>
>>>>> So here is the status with the license "the code inherited from Vyatta
>>>>>
>>>> and
>>>
>>>> our modifications from it is GPLv2 (strict, not v2+). The config reading
>>>>> library is GPLv2 too, so anything that links to is is GPLv2.
>>>>> Some auxiliary components we made after the fork are more permissive,
>>>>> LGPLv2+ or MIT."
>>>>>
>>>>> They are currently in the process of scoping a redesign (VyOS 2.0), "we
>>>>> are planning a clean rewrite that will solve issues of the current
>>>>>
>>>> config
>>>
>>>> system".
>>>>> This will include the ability to configure via the API.
>>>>>
>>>>> If we have more questions for VyOS, they are very friendly and
>>>>>
>>>> responsive,
>>>
>>>> so we should be able to get answers.
>>>>>
>>>>> *Will STEVENS*
>>>>> Lead Developer
>>>>>
>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>>>>> @CloudOps_
>>>>>
>>>>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com>
>>>>>
>>>> wrote:
>>>
>>>> I agree with Will Ilya. There are so many problems with the VR right
>>>>>>
>>>>> now.
>>>
>>>> Most of the outages we've had recently have somehow involved the VR.
>>>>>> We set custom iptables rules on the VR which can and have easily gone
>>>>>>
>>>>> wrong.
>>>>>
>>>>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>>>>> VVRP with redundant router still needs work, and not to mention the
>>>>>> problems we will have when we introduce IPv6 into the whole picture.
>>>>>>
>>>>>> I think the spirit of the discussion is to rely on a 3rd party to do
>>>>>> the networking for us (eg VyOS) and have us handle just the
>>>>>> orchestration. All the problems that I've described have already been
>>>>>> solved in VyOS. We also get the advantage of a potential wider
>>>>>> community to fix and maintain the VR and given our current development
>>>>>> velocity, it think it totally makes sense to look for a 3rd party
>>>>>>
>>>>> option.
>>>
>>>> -Syed
>>>>>>
>>>>>>
>>>>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
>>>>>> wrote:
>>>>>>
>>>>>> The VR has been biting us far too often recently, which is why we
>>>>>>> have started looking into alternative implementations.
>>>>>>>
>>>>>>> One of the things that is nice about potentially using the VyOS is
>>>>>>> that
>>>>>>>
>>>>>> it
>>>>>>
>>>>>>> is based on Debian, so we should be able to run the other services
>>>>>>> that
>>>>>>>
>>>>>> we
>>>>>>
>>>>>>> currently have like the password server and userdata on the VyOS.
>>>>>>> This means we would not have to change our architecture initially
>>>>>>> and could focus on only replacing the networking paths.
>>>>>>>
>>>>>>> *Will STEVENS*
>>>>>>> Lead Developer
>>>>>>>
>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>>>>> tw @CloudOps_
>>>>>>>
>>>>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>>>>
>>>>>>> The more this is discussed the more I think we should stick with
>>>>>>>> our
>>>>>>>>
>>>>>>> VR.
>>>>>>
>>>>>>> All these other options either seem unfinished or with
>>>>>>>> incompatible license.
>>>>>>>>
>>>>>>>> VyOS looks the most promising so far, it's a serious, mature
>>>>>>>> project.
>>>>>>>> Adopting it though means we'll have to microservice our way out of
>>>>>>>> it
>>>>>>>>
>>>>>>> with
>>>>>>>
>>>>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>>>>>>>>
>>>>>>> those
>>>>>>
>>>>>>> too. Imho this adds complexity we should void.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>
>>>>>>>> Nux!
>>>>>>>> www.nux.ro
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>>
>>>>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>> Ya, we would need to add a daemon for VPN as well. Load
>>>>>>>>> balancing is another aspect which we will need to consider if we
>>>>>>>>>
>>>>>>>> went this route.
>>>>>
>>>>>> Something like https://traefik.io/ could potentially be a good
>>>>>>>>> fit
>>>>>>>>>
>>>>>>>> due
>>>>>>
>>>>>>> to
>>>>>>>>
>>>>>>>>> its API driven configuration, but it may be more than what we need.
>>>>>>>>>
>>>>>>>>> We should probably try define which pieces make sense to be
>>>>>>>>> solved
>>>>>>>>>
>>>>>>>> together
>>>>>>>>
>>>>>>>>> and which pieces would be best suited to be broken out.
>>>>>>>>>
>>>>>>>>> I think the network connectivity, routing and firewalling should
>>>>>>>>>
>>>>>>>> probably
>>>>>>>
>>>>>>>> all stay together since the majority of the tools we would
>>>>>>>>>
>>>>>>>> potentially
>>>>>>
>>>>>>> use
>>>>>>>>
>>>>>>>>> would handle all of that together in a single implementation.
>>>>>>>>>
>>>>>>>>> The password server and userdata seems like a good option for
>>>>>>>>> being
>>>>>>>>>
>>>>>>>> broken
>>>>>>>>
>>>>>>>>> out and handled independently (and probably rewritten completely
>>>>>>>>>
>>>>>>>> since
>>>>>>
>>>>>>> they
>>>>>>>>
>>>>>>>>> currently have some issues).
>>>>>>>>>
>>>>>>>>> Load balancing is another that could warrant splitting out, but
>>>>>>>>> that depends on what direction we go and how we would be managing
>>>>>>>>>
>>>>>>>> it.
>>>>>
>>>>>> DHCP
>>>>>>
>>>>>>> and
>>>>>>>>
>>>>>>>>> DNS are others which could go either way.
>>>>>>>>>
>>>>>>>>> If we do split out services, I think we should consolidate as
>>>>>>>>> much as
>>>>>>>>>
>>>>>>>> we
>>>>>>>
>>>>>>>> can into each service we break out. Ideally a network packet
>>>>>>>>> would
>>>>>>>>>
>>>>>>>> never
>>>>>>>
>>>>>>>> hit more than one, maybe two, services. I don't think we should
>>>>>>>>> be splitting services 'just because', I think we need a valid
>>>>>>>>> case for splitting any service out because it adds complexity.
>>>>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>>>>> complexity unless it
>>>>>>>>>
>>>>>>>> is
>>>>>>
>>>>>>> really needed.
>>>>>>>>>
>>>>>>>>> Some more of my thoughts on this anyway...
>>>>>>>>>
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>
>>>>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>>>>>>>>>
>>>>>>>> wrote:
>>>>>>>
>>>>>>>> I do agree with you that this probably isn't the right place
>>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>> password
>>>>>>>>
>>>>>>>>> service and user data.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Having said that, after taking a cursory look at the dev docs,
>>>>>>>>>> it
>>>>>>>>>>
>>>>>>>>> doesn't
>>>>>>>>
>>>>>>>>> seem that difficult to add new daemons:
>>>>>>>>>>
>>>>>>>>> https://opensnaproute.github.
>>>>>>
>>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>>>>
>>>>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>>>>> creating-new-component>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> They've definitely build it with a microservices architecture
>>>>>>>>>> in
>>>>>>>>>>
>>>>>>>>> mind,
>>>>>>
>>>>>>> so
>>>>>>>>
>>>>>>>>> each individual feature is abstracted into it's own small
>>>>>>>>>> daemon
>>>>>>>>>>
>>>>>>>>> process.
>>>>>>>>
>>>>>>>>> We could just create a daemon for the password server and the
>>>>>>>>>>
>>>>>>>>> userdata
>>>>>>
>>>>>>> components if we really had to.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Si
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ________________________________
>>>>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
>>>>>>>>>> behalf
>>>>>>>>>>
>>>>>>>>> of
>>>>>>>
>>>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>
>>>>>>>>>> A big part of why I know about it is because it is written in Go.
>>>>>>>>>>
>>>>>>>>> :P
>>>>>>
>>>>>>> Yes, it is definitely interesting for the routing and traffic
>>>>>>>>>>
>>>>>>>>> handling
>>>>>>
>>>>>>> aspects of the VR. We will likely have to rethink some of the
>>>>>>>>>>
>>>>>>>>> pieces
>>>>>>
>>>>>>> a
>>>>>>>
>>>>>>>> little bit like the password server and userdata if we are to
>>>>>>>>>> adopt
>>>>>>>>>>
>>>>>>>>> a
>>>>>>
>>>>>>> different VR approach. This is where I think some of JohnB and
>>>>>>>>>>
>>>>>>>>> Chiradeep's
>>>>>>>>
>>>>>>>>> ideas make sense. In many ways, it does not make sense for the
>>>>>>>>>>
>>>>>>>>> device
>>>>>>
>>>>>>> handling routing and network traffic to also be responsible for
>>>>>>>>>>
>>>>>>>>> passwords
>>>>>>>>
>>>>>>>>> and userdata.
>>>>>>>>>>
>>>>>>>>>> *Will STEVENS*
>>>>>>>>>> Lead Developer
>>>>>>>>>>
>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>
>>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
>>>>>>>>>>>
>>>>>>>>>> pretty
>>>>>>
>>>>>>> cool!
>>>>>>>>>>
>>>>>>>>>>> It even supports ONIE install.
>>>>>>>>>>>
>>>>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>>>>>>>>>>
>>>>>>>>>> offload
>>>>>>
>>>>>>> it to
>>>>>>>>
>>>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>>>>> from a
>>>>>>>>>>>
>>>>>>>>>> routing
>>>>>>>>>>
>>>>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Based on the roadmap, it looks like they plan to also support
>>>>>>>>>>>
>>>>>>>>>> capabilities
>>>>>>>>>>
>>>>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>>>>> will
>>>>>>>>>>>
>>>>>>>>>> be
>>>>>>
>>>>>>> huge
>>>>>>>>
>>>>>>>>> for our carrier community that rely on these technologies to
>>>>>>>>>>> do
>>>>>>>>>>>
>>>>>>>>>> private
>>>>>>>>
>>>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>>>>> stuff
>>>>>>>>>>>
>>>>>>>>>> on
>>>>>>>
>>>>>>>> our
>>>>>>>>
>>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
>>>>>>>>>>> able to
>>>>>>>>>>>
>>>>>>>>>> do
>>>>>>>
>>>>>>>> MPLS
>>>>>>>>>>
>>>>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>>>>> know
>>>>>>>>>>>
>>>>>>>>>> very
>>>>>>>
>>>>>>>> well).
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> - Si
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ________________________________
>>>>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>>>>
>>>>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>>>>> does
>>>>>>>>>>>
>>>>>>>>>> cover is
>>>>>>>>
>>>>>>>>> all api driven...
>>>>>>>>>>>
>>>>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>
>>>>>>>> Though I don’t see VPN in Snaproute.. Makes sense since it
>>>>>>>>>>>> was
>>>>>>>>>>>>
>>>>>>>>>>> not
>>>>>>
>>>>>>> intended to do IPSec.
>>>>>>>>>>>>
>>>>>>>>>>>> It seems as though VyOS is starting to look like the best
>>>>>>>>>>>>
>>>>>>>>>>> option.
>>>>>>
>>>>>>> Regards,
>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>>>>> [mailto:williamstevens@gmail.com
>>>>>>>>>>>>
>>>>>>>>>>> ]
>>>>>>
>>>>>>> On
>>>>>>>
>>>>>>>> Behalf Of Will Stevens
>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>
>>>>>>>>>>>> Or we could go completely crazy and go with something like
>>>>>>>>>>>>
>>>>>>>>>>> FlexSwitch
>>>>>>>>
>>>>>>>>> from
>>>>>>>>>>>
>>>>>>>>>>>> SnapRoute
>>>>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>>>>
>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>
>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>>> cloudops.com
>>>>>>>>>>>>
>>>>>>>>>>> *|*
>>>>>>>
>>>>>>>> tw
>>>>>>>>
>>>>>>>>> @CloudOps_
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>>>>>>>>>
>>>>>>>>>>> wstevens@cloudops.com>
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
>>>>>>>>>>>>>
>>>>>>>>>>>> problems
>>>>>>>
>>>>>>>> are
>>>>>>>>
>>>>>>>>> solved by splitting up the function of the VR into a
>>>>>>>>>>>>> bunch of
>>>>>>>>>>>>>
>>>>>>>>>>>> separate
>>>>>>>>>>
>>>>>>>>>>> services. As Syed points out, the complexity added is
>>>>>>>>>>>>>
>>>>>>>>>>>> non-trivial.
>>>>>>>>
>>>>>>>>> We now have to manage all the intercontainer networking
>>>>>>>>>>>>> as
>>>>>>>>>>>>>
>>>>>>>>>>>> well
>>>>>>
>>>>>>> as
>>>>>>>
>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>>>>
>>>>>>>>>>>>> VyOS is interesting to me because it covers the majority
>>>>>>>>>>>>> of
>>>>>>>>>>>>>
>>>>>>>>>>>> our
>>>>>>
>>>>>>> use
>>>>>>>>
>>>>>>>>> case with a single unified control plane. It also has
>>>>>>>>>>>>> good
>>>>>>>>>>>>>
>>>>>>>>>>>> support
>>>>>>>>
>>>>>>>>> for extending features we care about, like IPv6, VXLAN,
>>>>>>>>>>>>> VRRP, transactions, etc...
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>>
>>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>>>>
>>>>>>>>>>>> cloudops.com
>>>>>>
>>>>>>> *|*
>>>>>>>>
>>>>>>>>> tw
>>>>>>>>>>
>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>>>>>>>>>>>
>>>>>>>>>>>> sahmed@cloudops.com>
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Agree with Marty, adding Docker containers to the
>>>>>>>>>>>>>> picture
>>>>>>>>>>>>>>
>>>>>>>>>>>>> although
>>>>>>>>
>>>>>>>>> can make the VR more flexible but the added complexity
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>
>>>>>>>>>>>>> just
>>>>>>
>>>>>>> not
>>>>>>>>
>>>>>>>>> worth it. Not to mention we would need to take care of
>>>>>>>>>>>>>>
>>>>>>>>>>>>> networking
>>>>>>>
>>>>>>>> each container manually and given that our iptable rules
>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>
>>>>>>>>>>>>> very
>>>>>>>
>>>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>
>>>>>>>>>>>>> not
>>>>>>>
>>>>>>>> provide an api but it does fit the bill quite well
>>>>>>>>>>>>>>
>>>>>>>>>>>>> otherwise. I
>>>>>>
>>>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>>>>> model
>>>>>>>>>>>>>>
>>>>>>>>>>>>> and
>>>>>>>
>>>>>>>> you
>>>>>>>>>>
>>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>>>>>>>>>>>
>>>>>>>>>>>>> marty@gonsource.com>
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Licensing aside, I think splitting the various
>>>>>>>>>>>>>>> functions
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> into
>>>>>>
>>>>>>> containers is not a good route either. This will force
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> users
>>>>>>
>>>>>>> to
>>>>>>>
>>>>>>>> have to maintain
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> use containers and adds complexity to the networking
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> aspects
>>>>>>
>>>>>>> of
>>>>>>>
>>>>>>>> ACS.
>>>>>>>>>>
>>>>>>>>>>> Complexity decreases stability. Now I understand the
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> argument
>>>>>>
>>>>>>> that
>>>>>>>>
>>>>>>>>> a monolithic approach also brings its own set of
>>>>>>>>>>>>>>> issues but
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> it
>>>>>>>
>>>>>>>> also
>>>>>>>>>>
>>>>>>>>>>> simplifies it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> the
>>>>>>
>>>>>>> CloudStack project
>>>>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
>>>>>>>>>>>>>>> touch
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> anything
>>>>>>>>
>>>>>>>>> that
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> AGPL
>>>>>>>>>>>>>>> - the github repo shows rather infrequent updates.
>>>>>>>>>>>>>>> Quite
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> likely
>>>>>>>>
>>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>>>>> CloudStack
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> community
>>>>>>>>
>>>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>>>>> Split
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> it
>>>>>>
>>>>>>> into
>>>>>>>>
>>>>>>>>> many docker containers
>>>>>>>>>>>>>>> - password server
>>>>>>>>>>>>>>> - userdata server
>>>>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>>>>> - FW
>>>>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>>>>> - agent
>>>>>>>>>>>>>>> Glue them together with docker compose files (one per
>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> case -
>>>>>>>>
>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
>>>>>>>>>>>>>>> test
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> each
>>>>>>
>>>>>>> of
>>>>>>>
>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>> components independently and fixing one bug in the
>>>>>>>>>>>>>>> field
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (say
>>>>>>
>>>>>>> DHCP)
>>>>>>>>>>
>>>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>>>>> build per-hypervisor VRs. You could even run on
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> baremetal.
>>>>>
>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>>>>> - make the traffic traverse the containers that are
>>>>>>>>>>>>>>> needed
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> to
>>>>>>>
>>>>>>>> be
>>>>>>>>
>>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>>>>> - bootstrap the router (how does it find its compose
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> file?
>>>>>
>>>>>> where
>>>>>>>>
>>>>>>>>> is the
>>>>>>>>>>>>>>> registry?)
>>>>>>>>>>>>>>> - rethink the command and control of the VR
>>>>>>>>>>>>>>> functions. SSH
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> works,
>>>>>>>>
>>>>>>>>> but something more declarative, idempotent should be
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> explored.
>>>>>>>
>>>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>>>>> functions
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> can
>>>>>>
>>>>>>> be
>>>>>>>>
>>>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>>>>> Control
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> of
>>>>>>>
>>>>>>>> the
>>>>>>>>
>>>>>>>>> docker
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This one does look nice. My biggest concern is the
>>>>>>>>>>>>>>>> lack
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> of
>>>>>>
>>>>>>> VXLANs. It seems that any of the ones we mentioned
>>>>>>>>>>>>>>>> do not
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> have
>>>>>>>>
>>>>>>>>> an
>>>>>>>>>>
>>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>>>>> save
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> future
>>>>>>>>
>>>>>>>>> engineering effort for example on ipv6 routing, OSPF
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> etc.
>>>>>
>>>>>> And the best part is they come with test automation
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> framework.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Instead of replacing the VR in first place we
>>>>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> as provider. Once it is stable, network offerings
>>>>>>>>>>>>>>>> (on
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> upgrade)
>>>>>>>>
>>>>>>>>> can be updated to use it and we can drop the VR if
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> want
>>>>>>
>>>>>>> at
>>>>>>>
>>>>>>>> that release
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>>>>> them
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> are
>>>>>>>
>>>>>>>> running
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> without issues. When we replicate the ACS VR
>>>>>>>>>>>>>>>> features in
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> new
>>>>>>>
>>>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>>>>> (hidden
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> bugs).
>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
>>>>>>>>>>>>>>>>>> keen
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> on
>>>>>>
>>>>>>> VyOS
>>>>>>>>
>>>>>>>>> (it
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> @shapeblue
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> be
>>>>>>>
>>>>>>>> treated as
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Please weigh in with the good and bad of this
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> idea...
>>>>>
>>>>>> A couple of us have been discussing the idea of
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> potentially
>>>>>>>>
>>>>>>>>> replacing the ACS VR with the VyOS [1] (Open
>>>>>>>>>>>>>>>>>>> Source
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Vyatta
>>>>>>>>
>>>>>>>>> VM).
>>>>>>>>>>
>>>>>>>>>>
Re: [DISCUSS] Replacing the VR
Posted by Matthew Smart <ms...@smartsoftwareinc.com>.
We have been using Vyatta (and then Vyos) after the fork for all of our
production routers for over four years now. It is rock solid and
performs better than the Juniper routers we replaced them with. It is
debian based underneath its custom shell whose syntax is very much like
Juniper. A sudo su gets you a standard bash shell just like in any other
debian distro and I have installed additional packages in the past. It
is really easy to configure in that the entire router config is held in
a single file with a dead simple file structure. You can use the
commands in their custom shell to alter the config but I prefer to just
upload a mod to that file... I know, I like to live dangerously! lol.
Seriously though, they do have versioning/rollback capability but last I
checked a rollback forced a reboot of the router.
The major downside, and one we are struggling with now is that it is
running an EOL version of debian and that is a HUGE liability for
something as security critical as an edge router. They have a lot of
work ahead of them to get out 2.0 but aside from the EOL Debian, Vyos
"just works" and out of the box would exceed the current VR in pretty
much every way. Personally, I find the API issue to be less important.
#1 priority has to be getting a VR that functions at an enterprise
level. API integration is important but pales compared to that.
Conversely, we are still not in production with Cloudstack specifically
because of the current state of the VR. IOW, the current VR
implementation is directly affecting Cloudstack adoption at least
anecdotally.
My team is very new to contributing to CS and our time is limited but,
given my many years of experience administering networks with Vyos at
the core, if you go that route I will gladly lend my knowledge in
helping you map out an implementation.
I have evaluated Cloud Router in the past and found it lacking in basic
features (and especially documentation) that made it a no go for even a
test build. It is promising and I hope it continues to mature but Vyos
is a battle hardened and proven technology. Given the option, I always
go with the known good solution over the shiny one.
I have also used PFsense. Really great project but I have not found it
to be as stable for me as Vyos, though that might be my lack of BSD
experience.
Thanks,
Matthew Smart
President
Smart Software Solutions Inc.
108 S Pierre St.
Pierre, SD 57501
Phone: (605) 280-0383
Skype: msmart13
Email: msmart@smartsoftwareinc.com
On 09/18/2016 11:09 AM, Will Stevens wrote:
> At this point in the discussion, I don't think we should rule anything out.
> I think it makes sense to explore all the options and then isolate some
> front runners in terms of software and architecture.
>
> On Sep 18, 2016 1:08 AM, "ilya" <il...@gmail.com> wrote:
>
>> Our options become much better if we consider BSD based routers.
>>
>> Would that be on the table?
>>
>> https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributions
>>
>>
>> On 9/16/16 12:04 PM, Will Stevens wrote:
>>> Ya, your points are all valid Simon. The lack of standard libraries to
>>> handle a lot of the details is a problem. I don't think it is an
>>> unsolvable problem, but if we spend the time to do that, will we have
>>> something that will work for us for the next 5 years? This may be the
>>> shortest path to getting us where we need to be for the time being.
>>>
>>> What is the best case scenario for the VR going forward which will last
>> us
>>> the next 5 years? Maybe we just clean up what we have to do a major
>>> restructuring of the pieces and how they are implemented. We need to
>> keep
>>> in mind how maintainable this implementation is because that is going to
>> be
>>> key going forward IMO.
>>>
>>>
>>>
>>> *Will STEVENS*
>>> Lead Developer
>>>
>>> *CloudOps* *| *Cloud Solutions Experts
>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>> w cloudops.com *|* tw @CloudOps_
>>>
>>> On Fri, Sep 16, 2016 at 2:29 PM, Simon Weller <sw...@ena.com> wrote:
>>>
>>>> I think our other option is to take a real look at what it would take to
>>>> fix the VR. In my opinion, a lot of the problems are related to the
>>>> monolithic python code base and the fact nothing is actually separated.
>>>>
>>>> Secondly, the python scripts (and bash scripts) don't use any
>> established
>>>> libraries to complete tasks and instead shell out and run commands that
>> are
>>>> both hard to track and hard to parse on return.
>>>>
>>>>
>>>> If we daemonized this, used a real api for Agent to VR communication,
>> used
>>>> common already existing libraries for the system service and network
>>>> interactions and spent a bit of time separating out code into distinct
>>>> modules, everything would behave a lot better.
>>>>
>>>>
>>>> The pain and suffering is due to years and years of patches and constant
>>>> shelling out to complete tasks in my opinion. If we spend time to
>> rethink
>>>> how we interact with the VR in general and we abstract the systems and
>>>> networking stuff and use well known and stable libraries to do the work,
>>>> the VR would be much easier to maintain.
>>>>
>>>>
>>>> - Si
>>>>
>>>>
>>>>
>>>>
>>>> ________________________________
>>>> From: Marty Godsey <ma...@gonsource.com>
>>>> Sent: Friday, September 16, 2016 12:24 PM
>>>> To: dev@cloudstack.apache.org
>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>
>>>> So based upon this discussion would it be prudent to wait on VyOS 2.0?
>> The
>>>> current VR is giving us issues but would the time invested in another
>>>> "solution" be wasted especially if by the time another option is chose,
>>>> then coded, then tested, then implemented and right as that time
>> happened
>>>> to be when VyOS 2.0 is released. Of course you said they are just in
>> the
>>>> scoping range so this could still be a year or more out.
>>>>
>>>> Thoughts?
>>>>
>>>> Regards,
>>>> Marty Godsey
>>>> nSource Solutions
>>>>
>>>> -----Original Message-----
>>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>>>> Behalf Of Will Stevens
>>>> Sent: Friday, September 16, 2016 10:31 AM
>>>> To: dev@cloudstack.apache.org
>>>> Cc: daniil@baturin.org
>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>
>>>> I just had a quick chat with a couple of the guys over on the VyOS chat.
>>>> I have CC'ed one of them in case we have more licensing questions.
>>>>
>>>> So here is the status with the license "the code inherited from Vyatta
>> and
>>>> our modifications from it is GPLv2 (strict, not v2+). The config reading
>>>> library is GPLv2 too, so anything that links to is is GPLv2.
>>>> Some auxiliary components we made after the fork are more permissive,
>>>> LGPLv2+ or MIT."
>>>>
>>>> They are currently in the process of scoping a redesign (VyOS 2.0), "we
>>>> are planning a clean rewrite that will solve issues of the current
>> config
>>>> system".
>>>> This will include the ability to configure via the API.
>>>>
>>>> If we have more questions for VyOS, they are very friendly and
>> responsive,
>>>> so we should be able to get answers.
>>>>
>>>> *Will STEVENS*
>>>> Lead Developer
>>>>
>>>> *CloudOps* *| *Cloud Solutions Experts
>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>>>> @CloudOps_
>>>>
>>>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com>
>> wrote:
>>>>> I agree with Will Ilya. There are so many problems with the VR right
>> now.
>>>>> Most of the outages we've had recently have somehow involved the VR.
>>>>> We set custom iptables rules on the VR which can and have easily gone
>>>> wrong.
>>>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>>>> VVRP with redundant router still needs work, and not to mention the
>>>>> problems we will have when we introduce IPv6 into the whole picture.
>>>>>
>>>>> I think the spirit of the discussion is to rely on a 3rd party to do
>>>>> the networking for us (eg VyOS) and have us handle just the
>>>>> orchestration. All the problems that I've described have already been
>>>>> solved in VyOS. We also get the advantage of a potential wider
>>>>> community to fix and maintain the VR and given our current development
>>>>> velocity, it think it totally makes sense to look for a 3rd party
>> option.
>>>>> -Syed
>>>>>
>>>>>
>>>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
>>>>> wrote:
>>>>>
>>>>>> The VR has been biting us far too often recently, which is why we
>>>>>> have started looking into alternative implementations.
>>>>>>
>>>>>> One of the things that is nice about potentially using the VyOS is
>>>>>> that
>>>>> it
>>>>>> is based on Debian, so we should be able to run the other services
>>>>>> that
>>>>> we
>>>>>> currently have like the password server and userdata on the VyOS.
>>>>>> This means we would not have to change our architecture initially
>>>>>> and could focus on only replacing the networking paths.
>>>>>>
>>>>>> *Will STEVENS*
>>>>>> Lead Developer
>>>>>>
>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>>>> tw @CloudOps_
>>>>>>
>>>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>>>
>>>>>>> The more this is discussed the more I think we should stick with
>>>>>>> our
>>>>> VR.
>>>>>>> All these other options either seem unfinished or with
>>>>>>> incompatible license.
>>>>>>>
>>>>>>> VyOS looks the most promising so far, it's a serious, mature project.
>>>>>>> Adopting it though means we'll have to microservice our way out of
>>>>>>> it
>>>>>> with
>>>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>>>>> those
>>>>>>> too. Imho this adds complexity we should void.
>>>>>>>
>>>>>>> --
>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>
>>>>>>> Nux!
>>>>>>> www.nux.ro
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>> Ya, we would need to add a daemon for VPN as well. Load
>>>>>>>> balancing is another aspect which we will need to consider if we
>>>> went this route.
>>>>>>>> Something like https://traefik.io/ could potentially be a good
>>>>>>>> fit
>>>>> due
>>>>>>> to
>>>>>>>> its API driven configuration, but it may be more than what we need.
>>>>>>>>
>>>>>>>> We should probably try define which pieces make sense to be
>>>>>>>> solved
>>>>>>> together
>>>>>>>> and which pieces would be best suited to be broken out.
>>>>>>>>
>>>>>>>> I think the network connectivity, routing and firewalling should
>>>>>> probably
>>>>>>>> all stay together since the majority of the tools we would
>>>>> potentially
>>>>>>> use
>>>>>>>> would handle all of that together in a single implementation.
>>>>>>>>
>>>>>>>> The password server and userdata seems like a good option for
>>>>>>>> being
>>>>>>> broken
>>>>>>>> out and handled independently (and probably rewritten completely
>>>>> since
>>>>>>> they
>>>>>>>> currently have some issues).
>>>>>>>>
>>>>>>>> Load balancing is another that could warrant splitting out, but
>>>>>>>> that depends on what direction we go and how we would be managing
>>>> it.
>>>>> DHCP
>>>>>>> and
>>>>>>>> DNS are others which could go either way.
>>>>>>>>
>>>>>>>> If we do split out services, I think we should consolidate as
>>>>>>>> much as
>>>>>> we
>>>>>>>> can into each service we break out. Ideally a network packet
>>>>>>>> would
>>>>>> never
>>>>>>>> hit more than one, maybe two, services. I don't think we should
>>>>>>>> be splitting services 'just because', I think we need a valid
>>>>>>>> case for splitting any service out because it adds complexity.
>>>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>>>> complexity unless it
>>>>> is
>>>>>>>> really needed.
>>>>>>>>
>>>>>>>> Some more of my thoughts on this anyway...
>>>>>>>>
>>>>>>>> *Will STEVENS*
>>>>>>>> Lead Developer
>>>>>>>>
>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>> *|* tw @CloudOps_
>>>>>>>>
>>>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>>>>>> wrote:
>>>>>>>>> I do agree with you that this probably isn't the right place
>>>>>>>>> the
>>>>>>> password
>>>>>>>>> service and user data.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Having said that, after taking a cursory look at the dev docs,
>>>>>>>>> it
>>>>>>> doesn't
>>>>>>>>> seem that difficult to add new daemons:
>>>>> https://opensnaproute.github.
>>>>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>>>
>>>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>>>> creating-new-component>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> They've definitely build it with a microservices architecture
>>>>>>>>> in
>>>>> mind,
>>>>>>> so
>>>>>>>>> each individual feature is abstracted into it's own small
>>>>>>>>> daemon
>>>>>>> process.
>>>>>>>>> We could just create a daemon for the password server and the
>>>>> userdata
>>>>>>>>> components if we really had to.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> - Si
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ________________________________
>>>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
>>>>>>>>> behalf
>>>>>> of
>>>>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>
>>>>>>>>> A big part of why I know about it is because it is written in Go.
>>>>> :P
>>>>>>>>> Yes, it is definitely interesting for the routing and traffic
>>>>> handling
>>>>>>>>> aspects of the VR. We will likely have to rethink some of the
>>>>> pieces
>>>>>> a
>>>>>>>>> little bit like the password server and userdata if we are to
>>>>>>>>> adopt
>>>>> a
>>>>>>>>> different VR approach. This is where I think some of JohnB and
>>>>>>> Chiradeep's
>>>>>>>>> ideas make sense. In many ways, it does not make sense for the
>>>>> device
>>>>>>>>> handling routing and network traffic to also be responsible for
>>>>>>> passwords
>>>>>>>>> and userdata.
>>>>>>>>>
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>
>>>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>>>>>> wrote:
>>>>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
>>>>> pretty
>>>>>>>>> cool!
>>>>>>>>>> It even supports ONIE install.
>>>>>>>>>>
>>>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>>>> offload
>>>>>>> it to
>>>>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>>>> from a
>>>>>>>>> routing
>>>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Based on the roadmap, it looks like they plan to also support
>>>>>>>>> capabilities
>>>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>>>> will
>>>>> be
>>>>>>> huge
>>>>>>>>>> for our carrier community that rely on these technologies to
>>>>>>>>>> do
>>>>>>> private
>>>>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>>>> stuff
>>>>>> on
>>>>>>> our
>>>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
>>>>>>>>>> able to
>>>>>> do
>>>>>>>>> MPLS
>>>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>>>> know
>>>>>> very
>>>>>>>>>> well).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Si
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ________________________________
>>>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>>>
>>>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>>>> does
>>>>>>> cover is
>>>>>>>>>> all api driven...
>>>>>>>>>>
>>>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>>>>>> wrote:
>>>>>>>>>>> Though I don\u2019t see VPN in Snaproute.. Makes sense since it
>>>>>>>>>>> was
>>>>> not
>>>>>>>>>>> intended to do IPSec.
>>>>>>>>>>>
>>>>>>>>>>> It seems as though VyOS is starting to look like the best
>>>>> option.
>>>>>>>>>>> Regards,
>>>>>>>>>>> Marty Godsey
>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>>>> [mailto:williamstevens@gmail.com
>>>>> ]
>>>>>> On
>>>>>>>>>>> Behalf Of Will Stevens
>>>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>
>>>>>>>>>>> Or we could go completely crazy and go with something like
>>>>>>> FlexSwitch
>>>>>>>>>> from
>>>>>>>>>>> SnapRoute
>>>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>>>
>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>> Lead Developer
>>>>>>>>>>>
>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>> cloudops.com
>>>>>> *|*
>>>>>>> tw
>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>>>> wstevens@cloudops.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
>>>>>> problems
>>>>>>> are
>>>>>>>>>>>> solved by splitting up the function of the VR into a
>>>>>>>>>>>> bunch of
>>>>>>>>> separate
>>>>>>>>>>>> services. As Syed points out, the complexity added is
>>>>>>> non-trivial.
>>>>>>>>>>>> We now have to manage all the intercontainer networking
>>>>>>>>>>>> as
>>>>> well
>>>>>> as
>>>>>>>>> the
>>>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>>>
>>>>>>>>>>>> VyOS is interesting to me because it covers the majority
>>>>>>>>>>>> of
>>>>> our
>>>>>>> use
>>>>>>>>>>>> case with a single unified control plane. It also has
>>>>>>>>>>>> good
>>>>>>> support
>>>>>>>>>>>> for extending features we care about, like IPv6, VXLAN,
>>>>>>>>>>>> VRRP, transactions, etc...
>>>>>>>>>>>>
>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>
>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>> cloudops.com
>>>>>>> *|*
>>>>>>>>> tw
>>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>>>> sahmed@cloudops.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>> Agree with Marty, adding Docker containers to the
>>>>>>>>>>>>> picture
>>>>>>> although
>>>>>>>>>>>>> can make the VR more flexible but the added complexity
>>>>>>>>>>>>> is
>>>>> just
>>>>>>> not
>>>>>>>>>>>>> worth it. Not to mention we would need to take care of
>>>>>> networking
>>>>>>>>>>>>> each container manually and given that our iptable rules
>>>>>>>>>>>>> are
>>>>>> very
>>>>>>>>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
>>>>>>>>>>>>> does
>>>>>> not
>>>>>>>>>>>>> provide an api but it does fit the bill quite well
>>>>> otherwise. I
>>>>>>>>>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>>>> model
>>>>>> and
>>>>>>>>> you
>>>>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>>>> marty@gonsource.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> Licensing aside, I think splitting the various
>>>>>>>>>>>>>> functions
>>>>> into
>>>>>>>>>>>>>> containers is not a good route either. This will force
>>>>> users
>>>>>> to
>>>>>>>>>>>>>> have to maintain
>>>>>>>>>>>>> and
>>>>>>>>>>>>>> use containers and adds complexity to the networking
>>>>> aspects
>>>>>> of
>>>>>>>>> ACS.
>>>>>>>>>>>>>> Complexity decreases stability. Now I understand the
>>>>> argument
>>>>>>> that
>>>>>>>>>>>>>> a monolithic approach also brings its own set of
>>>>>>>>>>>>>> issues but
>>>>>> it
>>>>>>>>> also
>>>>>>>>>>>>>> simplifies it.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
>>>>>>>>>>>>>> of
>>>>> the
>>>>>>>>>>>>>> CloudStack project
>>>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
>>>>>>>>>>>>>> touch
>>>>>>> anything
>>>>>>>>>>>>>> that
>>>>>>>>>>>>> has
>>>>>>>>>>>>>> AGPL
>>>>>>>>>>>>>> - the github repo shows rather infrequent updates.
>>>>>>>>>>>>>> Quite
>>>>>>> likely
>>>>>>>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>>>> CloudStack
>>>>>>> community
>>>>>>>>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>>>> Split
>>>>> it
>>>>>>> into
>>>>>>>>>>>>>> many docker containers
>>>>>>>>>>>>>> - password server
>>>>>>>>>>>>>> - userdata server
>>>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>>>> - FW
>>>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>>>> - agent
>>>>>>>>>>>>>> Glue them together with docker compose files (one per
>>>>>>>>>>>>>> use
>>>>>>> case -
>>>>>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
>>>>>>>>>>>>>> test
>>>>> each
>>>>>> of
>>>>>>>>> the
>>>>>>>>>>>>>> components independently and fixing one bug in the
>>>>>>>>>>>>>> field
>>>>> (say
>>>>>>>>> DHCP)
>>>>>>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>>>> build per-hypervisor VRs. You could even run on
>>>> baremetal.
>>>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>>>> - make the traffic traverse the containers that are
>>>>>>>>>>>>>> needed
>>>>>> to
>>>>>>> be
>>>>>>>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>>>> - bootstrap the router (how does it find its compose
>>>> file?
>>>>>>> where
>>>>>>>>>>>>>> is the
>>>>>>>>>>>>>> registry?)
>>>>>>>>>>>>>> - rethink the command and control of the VR
>>>>>>>>>>>>>> functions. SSH
>>>>>>> works,
>>>>>>>>>>>>>> but something more declarative, idempotent should be
>>>>>> explored.
>>>>>>>>>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>>>> functions
>>>>> can
>>>>>>> be
>>>>>>>>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>>>> Control
>>>>>> of
>>>>>>> the
>>>>>>>>>>>>> docker
>>>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This one does look nice. My biggest concern is the
>>>>>>>>>>>>>>> lack
>>>>> of
>>>>>>>>>>>>>>> VXLANs. It seems that any of the ones we mentioned
>>>>>>>>>>>>>>> do not
>>>>>>> have
>>>>>>>>> an
>>>>>>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>>>> save
>>>>>>> future
>>>>>>>>>>>>>>> engineering effort for example on ipv6 routing, OSPF
>>>> etc.
>>>>>>>>>>>>>>> And the best part is they come with test automation
>>>>>>> framework.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Instead of replacing the VR in first place we
>>>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>>>> as provider. Once it is stable, network offerings
>>>>>>>>>>>>>>> (on
>>>>>>> upgrade)
>>>>>>>>>>>>>>> can be updated to use it and we can drop the VR if
>>>>>>>>>>>>>>> we
>>>>> want
>>>>>> at
>>>>>>>>>>>>>>> that release
>>>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>>>> them
>>>>>> are
>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>> without issues. When we replicate the ACS VR
>>>>>>>>>>>>>>> features in
>>>>>> new
>>>>>>>>>>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>>>> (hidden
>>>>>>> bugs).
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
>>>>>>>>>>>>>>>>> keen
>>>>> on
>>>>>>> VyOS
>>>>>>>>>>>>>>>>> (it
>>>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>>>>>> @shapeblue
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
>>>>>>>>>>>>>>>>>> should
>>>>>> be
>>>>>>>>>>>>>>>>>> treated as
>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>> Please weigh in with the good and bad of this
>>>> idea...
>>>>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
>>>>>>> potentially
>>>>>>>>>>>>>>>>>> replacing the ACS VR with the VyOS [1] (Open
>>>>>>>>>>>>>>>>>> Source
>>>>>>> Vyatta
>>>>>>>>>> VM).
>>>>>>>>>>>>>>>>>> There may be a license issue because I think it
>>>>>>>>>>>>>>>>>> is
>>>>>>> licensed
>>>>>>>>>>>>>>>>>> under GPL, but for the sake of discussion, let's
>>>>> assume
>>>>>>> we
>>>>>>>>>>>>>>>>>> can overcome any
>>>>>>>>>>>>>>> license issues.
>>>>>>>>>>>>>>>>>> I have spent some time recently with the VyOS
>>>>>>>>>>>>>>>>>> and I
>>>>>> have
>>>>>>> to
>>>>>>>>>>>>>>>>>> admit, I was pretty impressed. It is simple and
>>>>>>> intuitive
>>>>>>>>>>>>>>>>>> and it gives you a lot more options for auditing
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>> configuration etc...
>>>>>>>>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
>>>>> simpler
>>>>>>> more
>>>>>>>>>>>>>>>>>> auditable configuration workflow.
>>>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
>>>>>>>>>>>>>>>>>> - Handles VPN configuration via the same
>>>>> configuration
>>>>>>>>>>> interface.
>>>>>>>>>>>>>>>>>> - Support for OSPF & BGP.
>>>>>>>>>>>>>>>>>> - VPN support through OpenVPN & StrongSwan.
>>>>>>>>>>>>>>>>>> - Easily supports HA (redundant routers) through
>>>>> VRRP.
>>>>>>>>>>>>>>>>>> - VXLAN support.
>>>>>>>>>>>>>>>>>> - Transaction based changes to the VR with
>>>>>>>>>>>>>>>>>> rollback
>>>>> on
>>>>>>>>> error.
>>>>>>>>>>>>>>>>>> Items that could be difficult to solve:
>>>>>>>>>>>>>>>>>> - Userdata password reset workflow and
>>>>> implementation.
>>>>>>>>>>>>>>>>>> - Upgrade process.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The VyOS is not the only option if we were to
>>>>> consider
>>>>>>> this
>>>>>>>>>>>>> approach.
>>>>>>>>>>>>>>>>>> Another option, which I don't know as well,
>>>>>>>>>>>>>>>>>> would be CloudRouter (AGPL
>>>>>>>>>>>>>>>>>> license) [2] which is purely API driven.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Anyway, would love to hear your thoughts...
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> [1] https://vyos.io/ [2]
>>>>>>>>>>>>>>>>>> https://cloudrouter.org/
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> DISCLAIMER
>>>>>>>>>>>>>>>> ==========
>>>>>>>>>>>>>>>> This e-mail may contain privileged and confidential
>>>>>>> information
>>>>>>>>>>>>>>>> which is
>>>>>>>>>>>>>>> the property of Accelerite, a Persistent Systems
>>>>> business.
>>>>>>> It is
>>>>>>>>>>>>>>> intended only for the use of the individual or
>>>>>>>>>>>>>>> entity to
>>>>>>> which
>>>>>>>>> it
>>>>>>>>>>>>>>> is addressed. If you are not the intended recipient,
>>>>>>>>>>>>>>> you
>>>>>> are
>>>>>>> not
>>>>>>>>>>>>>>> authorized to read, retain, copy, print, distribute
>>>>>>>>>>>>>>> or
>>>>> use
>>>>>>> this
>>>>>>>>>>>>>>> message. If you have received this communication in
>>>>> error,
>>>>>>>>> please
>>>>>>>>>>>>>>> notify the sender and delete all copies of this
>>>> message.
>>>>>>>>>>>>>>> Accelerite, a Persistent Systems business does not
>>>>>>>>>>>>>>> accept
>>>>>> any
>>>>>>>>>>>>>>> liability for virus
>>>>>>>>>>>>>> infected mails.
>>>>>>>>>>>>
Re: [DISCUSS] Replacing the VR
Posted by Matthew Smart <ms...@smartsoftwareinc.com>.
Abstracting that interaction would also make future implementations MUCH
less painful and allows third parties who want their networking products
to be CS compatible to write their own implementation of a published API
that CS defines. Let them do the legwork for their products, expand CS
compatibility, and ease future refactors of VR all in one package. I
like it.
Caveat: I am clueless about how VR configuration happens under the hood
currently or if abstracting it is a monumental task.
Matthew Smart
President
Smart Software Solutions Inc.
108 S Pierre St.
Pierre, SD 57501
Phone: (605) 280-0383
Skype: msmart13
Email: msmart@smartsoftwareinc.com
On 09/19/2016 11:50 AM, Paul Angus wrote:
> Hi All,
>
> From the looks of this thread and the different preferences that parties have, the way forward looks to be a model that allows multiple different VRs to be used is the answer.
>
> This is something that I have had in mind for some time now.
>
> By abstracting the roles and configuration of the VR, drivers can be written which transform requests such as 'add these firewall rules' into something that a VyOs, Fortigate, pfSense, Cisco ASAv, a Juniper vSRX or a homebrew VR can understand and implement.
>
> Ideally I'd like to see the features separated and service chaining introduced such that someone might use a stock VR with a VPN appliance or a Cisco ASAv in front of a VR doing load balancing.
>
> This moves us forward into a much more NFV-like world.
>
>
>
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
>
>
>
> -----Original Message-----
> From: Syed Ahmed [mailto:sahmed@cloudops.com]
> Sent: 19 September 2016 17:07
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Hey Guys,
>
> Will and I had a discussion in the morning on around VyOS and I have an
> idea that could work, here me out.
>
> The way Cloudstack currently communicates with VR is via the Hypervisor
> (let's assume KVM and XenServer for now). The management server sends a
> command as a JSON to the hypervisor which proxies them to the VR and they
> get executed there. Now, what we could do is add a proxy on the hypervisor
> which translates the management server JSON to VyOS commands (and
> vice-versa). There is a VyOS API lib in python which we clould use. Now
> this would require 0 changes on the Cloudstack core networking side. There
> may be a few minor changes required to push this proxy on the hypervisor
> but other than that there is nothing major. Now in the mean time, the VyOS
> guys can work on their v2.0 and when they have a stable enough API we can
> make it a first class citizen in Cloudstack.
>
> This approach would not work for VMWare as in VMWare the management server
> directly talks to the VR. However, this problem could be fixed by adding a
> middle VM which does the proxying to-and-from the VR. This would also
> improve the overall security of the system in VMWare as well.
>
> I am not too concerned about VMWare at the moment as most of us use either
> KVM or XenServer. What do you guys think of this approach?
>
> -Syed
>
> On Mon, Sep 19, 2016 at 11:42 AM, Stephan Seitz <
> s.seitz@secretresearchfacility.com> wrote:
>
>> Hi!
>>
>> Just to add my 2 cents to that thread:
>>
>> I'ld really like to see something like vyatta or pfsense integrated as
>> "standard" VR.
>>
>> We'd also talked internally about replacing the VR with some more
>> mature "appliance"-like router distro.
>>
>> pfsense e.g. comes AFAIK with no defined API but instead has a very
>> nice GUI.
>> How would this fit into the concept of configuring the VR via ACS?
>> Would parts of the GUI - like IP configuration and basic firewall rules
>> - hidden or greyed?
>> Where would one save the configuration, VPN certificates and so on?
>>
>>
>> - Stephan
>>
>>
>> Am Sonntag, den 18.09.2016, 15:19 +0000 schrieb Marty Godsey:
>>> On this note I also mentioned pfsense earlier.
>>>
>>> www.pfsense.org
>>>
>>>
>>> Regards,
>>> Marty Godsey
>>>
>>> -----Original Message-----
>>> From: ilya [mailto:ilya.mailing.lists@gmail.com]
>>> Sent: Sunday, September 18, 2016 1:09 AM
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: [DISCUSS] Replacing the VR
>>>
>>> Our options become much better if we consider BSD based routers.
>>>
>>> Would that be on the table?
>>>
>>> https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributio
>>> ns
>>>
>>>
>>> On 9/16/16 12:04 PM, Will Stevens wrote:
>>>> Ya, your points are all valid Simon. The lack of standard
>>>> libraries
>>>> to handle a lot of the details is a problem. I don't think it is
>>>> an
>>>> unsolvable problem, but if we spend the time to do that, will we
>>>> have
>>>> something that will work for us for the next 5 years? This may be
>>>> the
>>>> shortest path to getting us where we need to be for the time being.
>>>>
>>>> What is the best case scenario for the VR going forward which will
>>>> last us the next 5 years? Maybe we just clean up what we have to
>>>> do a
>>>> major restructuring of the pieces and how they are
>>>> implemented. We
>>>> need to keep in mind how maintainable this implementation is
>>>> because
>>>> that is going to be key going forward IMO.
>>>>
>>>>
>>>>
>>>> *Will STEVENS*
>>>> Lead Developer
>>>>
>>>> *CloudOps* *| *Cloud Solutions Experts
>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>> tw
>>>> @CloudOps_
>>>>
>>>> On Fri, Sep 16, 2016 at 2:29 PM, Simon Weller <sw...@ena.com>
>>>> wrote:
>>>>
>>>>> I think our other option is to take a real look at what it would
>>>>> take
>>>>> to fix the VR. In my opinion, a lot of the problems are related
>>>>> to
>>>>> the monolithic python code base and the fact nothing is actually
>>>>> separated.
>>>>>
>>>>> Secondly, the python scripts (and bash scripts) don't use any
>>>>> established libraries to complete tasks and instead shell out and
>>>>> run
>>>>> commands that are both hard to track and hard to parse on return.
>>>>>
>>>>>
>>>>> If we daemonized this, used a real api for Agent to VR
>>>>> communication,
>>>>> used common already existing libraries for the system service
>>>>> and
>>>>> network interactions and spent a bit of time separating out code
>>>>> into
>>>>> distinct modules, everything would behave a lot better.
>>>>>
>>>>>
>>>>> The pain and suffering is due to years and years of patches and
>>>>> constant shelling out to complete tasks in my opinion. If we
>>>>> spend
>>>>> time to rethink how we interact with the VR in general and we
>>>>> abstract the systems and networking stuff and use well known and
>>>>> stable libraries to do the work, the VR would be much easier to
>>>>> maintain.
>>>>>
>>>>>
>>>>> - Si
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ________________________________
>>>>> From: Marty Godsey <ma...@gonsource.com>
>>>>> Sent: Friday, September 16, 2016 12:24 PM
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>
>>>>> So based upon this discussion would it be prudent to wait on
>>>>> VyOS
>>>>> 2.0? The current VR is giving us issues but would the time
>>>>> invested
>>>>> in another "solution" be wasted especially if by the time
>>>>> another
>>>>> option is chose, then coded, then tested, then implemented and
>>>>> right
>>>>> as that time happened to be when VyOS 2.0 is released. Of course
>>>>> you
>>>>> said they are just in the scoping range so this could still be a
>>>>> year or more out.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> Regards,
>>>>> Marty Godsey
>>>>> nSource Solutions
>>>>>
>>>>> -----Original Message-----
>>>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com]
>>>>> On
>>>>> Behalf Of Will Stevens
>>>>> Sent: Friday, September 16, 2016 10:31 AM
>>>>> To: dev@cloudstack.apache.org
>>>>> Cc: daniil@baturin.org
>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>
>>>>> I just had a quick chat with a couple of the guys over on the
>>>>> VyOS chat.
>>>>> I have CC'ed one of them in case we have more licensing
>>>>> questions.
>>>>>
>>>>> So here is the status with the license "the code inherited from
>>>>> Vyatta and our modifications from it is GPLv2 (strict, not v2+).
>>>>> The
>>>>> config reading library is GPLv2 too, so anything that links to is
>>>>> is GPLv2.
>>>>> Some auxiliary components we made after the fork are more
>>>>> permissive,
>>>>> LGPLv2+ or MIT."
>>>>>
>>>>> They are currently in the process of scoping a redesign (VyOS
>>>>> 2.0),
>>>>> "we are planning a clean rewrite that will solve issues of the
>>>>> current config system".
>>>>> This will include the ability to configure via the API.
>>>>>
>>>>> If we have more questions for VyOS, they are very friendly and
>>>>> responsive, so we should be able to get answers.
>>>>>
>>>>> *Will STEVENS*
>>>>> Lead Developer
>>>>>
>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>> *|* tw
>>>>> @CloudOps_
>>>>>
>>>>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com>
>>>>> wrote:
>>>>>
>>>>>> I agree with Will Ilya. There are so many problems with the VR
>>>>>> right now.
>>>>>> Most of the outages we've had recently have somehow involved
>>>>>> the VR.
>>>>>> We set custom iptables rules on the VR which can and have
>>>>>> easily
>>>>>> gone
>>>>> wrong.
>>>>>> Openswan is broken, Strongswan replacement still needs to be
>>>>>> tested.
>>>>>> VVRP with redundant router still needs work, and not to mention
>>>>>> the
>>>>>> problems we will have when we introduce IPv6 into the whole
>>>>>> picture.
>>>>>>
>>>>>> I think the spirit of the discussion is to rely on a 3rd party
>>>>>> to do
>>>>>> the networking for us (eg VyOS) and have us handle just the
>>>>>> orchestration. All the problems that I've described have
>>>>>> already
>>>>>> been solved in VyOS. We also get the advantage of a potential
>>>>>> wider
>>>>>> community to fix and maintain the VR and given our current
>>>>>> development velocity, it think it totally makes sense to look
>>>>>> for a 3rd party option.
>>>>>>
>>>>>> -Syed
>>>>>>
>>>>>>
>>>>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens
>>>>>> <ws...@cloudops.com>
>>>>>> wrote:
>>>>>>
>>>>>>> The VR has been biting us far too often recently, which is
>>>>>>> why we
>>>>>>> have started looking into alternative implementations.
>>>>>>>
>>>>>>> One of the things that is nice about potentially using the
>>>>>>> VyOS is
>>>>>>> that
>>>>>> it
>>>>>>> is based on Debian, so we should be able to run the other
>>>>>>> services
>>>>>>> that
>>>>>> we
>>>>>>> currently have like the password server and userdata on the
>>>>>>> VyOS.
>>>>>>> This means we would not have to change our architecture
>>>>>>> initially
>>>>>>> and could focus on only replacing the networking paths.
>>>>>>>
>>>>>>> *Will STEVENS*
>>>>>>> Lead Developer
>>>>>>>
>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>> cloudops.com *|*
>>>>>>> tw @CloudOps_
>>>>>>>
>>>>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>>>>
>>>>>>>> The more this is discussed the more I think we should stick
>>>>>>>> with
>>>>>>>> our
>>>>>> VR.
>>>>>>>>
>>>>>>>> All these other options either seem unfinished or with
>>>>>>>> incompatible license.
>>>>>>>>
>>>>>>>> VyOS looks the most promising so far, it's a serious,
>>>>>>>> mature project.
>>>>>>>> Adopting it though means we'll have to microservice our way
>>>>>>>> out of
>>>>>>>> it
>>>>>>> with
>>>>>>>> extra machines for DNS/USERDATA/etc, unless we can make
>>>>>>>> VyOS serve
>>>>>> those
>>>>>>>> too. Imho this adds complexity we should void.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>
>>>>>>>> Nux!
>>>>>>>> www.nux.ro
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>
>>>>>>>>> Ya, we would need to add a daemon for VPN as well. Load
>>>>>>>>> balancing is another aspect which we will need to
>>>>>>>>> consider if we
>>>>> went this route.
>>>>>>>>> Something like https://traefik.io/ could potentially be a
>>>>>>>>> good
>>>>>>>>> fit
>>>>>> due
>>>>>>>> to
>>>>>>>>> its API driven configuration, but it may be more than
>>>>>>>>> what we need.
>>>>>>>>>
>>>>>>>>> We should probably try define which pieces make sense to
>>>>>>>>> be
>>>>>>>>> solved
>>>>>>>> together
>>>>>>>>> and which pieces would be best suited to be broken out.
>>>>>>>>>
>>>>>>>>> I think the network connectivity, routing and firewalling
>>>>>>>>> should
>>>>>>> probably
>>>>>>>>> all stay together since the majority of the tools we
>>>>>>>>> would
>>>>>> potentially
>>>>>>>> use
>>>>>>>>> would handle all of that together in a single
>>>>>>>>> implementation.
>>>>>>>>>
>>>>>>>>> The password server and userdata seems like a good option
>>>>>>>>> for
>>>>>>>>> being
>>>>>>>> broken
>>>>>>>>> out and handled independently (and probably rewritten
>>>>>>>>> completely
>>>>>> since
>>>>>>>> they
>>>>>>>>> currently have some issues).
>>>>>>>>>
>>>>>>>>> Load balancing is another that could warrant splitting
>>>>>>>>> out, but
>>>>>>>>> that depends on what direction we go and how we would be
>>>>>>>>> managing
>>>>> it.
>>>>>> DHCP
>>>>>>>> and
>>>>>>>>> DNS are others which could go either way.
>>>>>>>>>
>>>>>>>>> If we do split out services, I think we should
>>>>>>>>> consolidate as
>>>>>>>>> much as
>>>>>>> we
>>>>>>>>> can into each service we break out. Ideally a network
>>>>>>>>> packet
>>>>>>>>> would
>>>>>>> never
>>>>>>>>> hit more than one, maybe two, services. I don't think we
>>>>>>>>> should
>>>>>>>>> be splitting services 'just because', I think we need a
>>>>>>>>> valid
>>>>>>>>> case for splitting any service out because it adds
>>>>>>>>> complexity.
>>>>>>>>> Our project is already complex enough, we need to avoid
>>>>>>>>> adding
>>>>>>>>> complexity unless it
>>>>>> is
>>>>>>>>> really needed.
>>>>>>>>>
>>>>>>>>> Some more of my thoughts on this anyway...
>>>>>>>>>
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>> cloudops.com
>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>
>>>>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sweller@e
>>>>>>>>> na.com>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I do agree with you that this probably isn't the right
>>>>>>>>>> place the
>>>>>>>> password
>>>>>>>>>> service and user data.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Having said that, after taking a cursory look at the
>>>>>>>>>> dev docs,
>>>>>>>>>> it
>>>>>>>> doesn't
>>>>>>>>>> seem that difficult to add new daemons:
>>>>>> https://opensnaproute.github.
>>>>>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>>>>
>>>>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>>>>> creating-new-component>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> They've definitely build it with a microservices
>>>>>>>>>> architecture in
>>>>>> mind,
>>>>>>>> so
>>>>>>>>>> each individual feature is abstracted into it's own
>>>>>>>>>> small daemon
>>>>>>>> process.
>>>>>>>>>> We could just create a daemon for the password server
>>>>>>>>>> and the
>>>>>> userdata
>>>>>>>>>> components if we really had to.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Si
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ________________________________
>>>>>>>>>> From: williamstevens@gmail.com <williamstevens@gmail.co
>>>>>>>>>> m> on
>>>>>>>>>> behalf
>>>>>>> of
>>>>>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>
>>>>>>>>>> A big part of why I know about it is because it is
>>>>>>>>>> written in Go.
>>>>>> :P
>>>>>>>>>>
>>>>>>>>>> Yes, it is definitely interesting for the routing and
>>>>>>>>>> traffic
>>>>>> handling
>>>>>>>>>> aspects of the VR. We will likely have to rethink some
>>>>>>>>>> of the
>>>>>> pieces
>>>>>>> a
>>>>>>>>>> little bit like the password server and userdata if we
>>>>>>>>>> are to
>>>>>>>>>> adopt
>>>>>> a
>>>>>>>>>> different VR approach. This is where I think some of
>>>>>>>>>> JohnB and
>>>>>>>> Chiradeep's
>>>>>>>>>> ideas make sense. In many ways, it does not make sense
>>>>>>>>>> for the
>>>>>> device
>>>>>>>>>> handling routing and network traffic to also be
>>>>>>>>>> responsible for
>>>>>>>> passwords
>>>>>>>>>> and userdata.
>>>>>>>>>>
>>>>>>>>>> *Will STEVENS*
>>>>>>>>>> Lead Developer
>>>>>>>>>>
>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>> cloudops.com
>>>>>>>>>> *|* tw @CloudOps_
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sweller@
>>>>>>>>>> ena.com>
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I hadn't heard of Flexswitch until you mentioned it.
>>>>>>>>>>> It looks
>>>>>> pretty
>>>>>>>>>> cool!
>>>>>>>>>>> It even supports ONIE install.
>>>>>>>>>>>
>>>>>>>>>>> To be honest, the ipsec feature could be added, or we
>>>>>>>>>>> could
>>>>>> offload
>>>>>>>> it to
>>>>>>>>>>> separate vm if we needed to. The fact it is so
>>>>>>>>>>> feature rich
>>>>>>>>>>> from a
>>>>>>>>>> routing
>>>>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Based on the roadmap, it looks like they plan to also
>>>>>>>>>>> support
>>>>>>>>>> capabilities
>>>>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the
>>>>>>>>>>> future. This
>>>>>>>>>>> will
>>>>>> be
>>>>>>>> huge
>>>>>>>>>>> for our carrier community that rely on these
>>>>>>>>>>> technologies to do
>>>>>>>> private
>>>>>>>>>>> gateway and inter-VPC interconnections today. We
>>>>>>>>>>> handle this
>>>>>>>>>>> stuff
>>>>>>> on
>>>>>>>> our
>>>>>>>>>>> ASRs right now with a vlan interconnect into the VR.
>>>>>>>>>>> Being able
>>>>>>>>>>> to
>>>>>>> do
>>>>>>>>>> MPLS
>>>>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> It also seems to be written in GO (a language here at
>>>>>>>>>>> ENA we
>>>>>>>>>>> know
>>>>>>> very
>>>>>>>>>>> well).
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> - Si
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ________________________________
>>>>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>>>>
>>>>>>>>>>> Ya. I don't think it covers our whole use case, but
>>>>>>>>>>> what it
>>>>>>>>>>> does
>>>>>>>> cover is
>>>>>>>>>>> all api driven...
>>>>>>>>>>>
>>>>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <marty@gonsou
>>>>>>>>>>> rce.com>
>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Though I don\u2019t see VPN in Snaproute.. Makes sense
>>>>>>>>>>>> since it was
>>>>>> not
>>>>>>>>>>>> intended to do IPSec.
>>>>>>>>>>>>
>>>>>>>>>>>> It seems as though VyOS is starting to look like
>>>>>>>>>>>> the best
>>>>>> option.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>>>>> [mailto:williamstevens@gmail.com
>>>>>> ]
>>>>>>> On
>>>>>>>>>>>> Behalf Of Will Stevens
>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>
>>>>>>>>>>>> Or we could go completely crazy and go with
>>>>>>>>>>>> something like
>>>>>>>> FlexSwitch
>>>>>>>>>>> from
>>>>>>>>>>>> SnapRoute
>>>>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>>>>
>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>
>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>>>>> cloudops.com
>>>>>>> *|*
>>>>>>>> tw
>>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>>>>> wstevens@cloudops.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I tend to agree with Syed and Marty. I am not
>>>>>>>>>>>>> sure what
>>>>>>> problems
>>>>>>>> are
>>>>>>>>>>>>> solved by splitting up the function of the VR
>>>>>>>>>>>>> into a bunch of
>>>>>>>>>> separate
>>>>>>>>>>>>> services. As Syed points out, the complexity
>>>>>>>>>>>>> added is
>>>>>>>> non-trivial.
>>>>>>>>>>>>> We now have to manage all the intercontainer
>>>>>>>>>>>>> networking as
>>>>>> well
>>>>>>> as
>>>>>>>>>> the
>>>>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>>>>
>>>>>>>>>>>>> VyOS is interesting to me because it covers the
>>>>>>>>>>>>> majority of
>>>>>> our
>>>>>>>> use
>>>>>>>>>>>>> case with a single unified control plane. It
>>>>>>>>>>>>> also has good
>>>>>>>> support
>>>>>>>>>>>>> for extending features we care about, like IPv6,
>>>>>>>>>>>>> VXLAN, VRRP,
>>>>>>>>>>>>> transactions, etc...
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Will STEVENS*
>>>>>>>>>>>>> Lead Developer
>>>>>>>>>>>>>
>>>>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>> cloudops.com
>>>>>>>> *|*
>>>>>>>>>> tw
>>>>>>>>>>>>> @CloudOps_
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>>>>> sahmed@cloudops.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Agree with Marty, adding Docker containers to
>>>>>>>>>>>>>> the picture
>>>>>>>> although
>>>>>>>>>>>>>> can make the VR more flexible but the added
>>>>>>>>>>>>>> complexity is
>>>>>> just
>>>>>>>> not
>>>>>>>>>>>>>> worth it. Not to mention we would need to take
>>>>>>>>>>>>>> care of
>>>>>>> networking
>>>>>>>>>>>>>> each container manually and given that our
>>>>>>>>>>>>>> iptable rules are
>>>>>>> very
>>>>>>>>>>>>>> unstable at the moment I don't see a big value
>>>>>>>>>>>>>> add.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Vyos looks like a better solution to me. I know
>>>>>>>>>>>>>> that it does
>>>>>>> not
>>>>>>>>>>>>>> provide an api but it does fit the bill quite
>>>>>>>>>>>>>> well
>>>>>> otherwise. I
>>>>>>>>>>>>>> specially like the fact that it has a
>>>>>>>>>>>>>> transaction based
>>>>>>>>>>>>>> model
>>>>>>> and
>>>>>>>>>> you
>>>>>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>>>>> marty@gonsource.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Licensing aside, I think splitting the
>>>>>>>>>>>>>>> various functions
>>>>>> into
>>>>>>>>>>>>>>> containers is not a good route either. This
>>>>>>>>>>>>>>> will force
>>>>>> users
>>>>>>> to
>>>>>>>>>>>>>>> have to maintain
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> use containers and adds complexity to the
>>>>>>>>>>>>>>> networking
>>>>>> aspects
>>>>>>> of
>>>>>>>>>> ACS.
>>>>>>>>>>>>>>> Complexity decreases stability. Now I
>>>>>>>>>>>>>>> understand the
>>>>>> argument
>>>>>>>> that
>>>>>>>>>>>>>>> a monolithic approach also brings its own set
>>>>>>>>>>>>>>> of issues but
>>>>>>> it
>>>>>>>>>> also
>>>>>>>>>>>>>>> simplifies it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gma
>>>>>>>>>>>>>>> il.com]
>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit
>>>>>>>>>>>>>>> the needs
>>>>>>>>>>>>>>> of
>>>>>> the
>>>>>>>>>>>>>>> CloudStack project
>>>>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will
>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>> touch
>>>>>>>> anything
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>> AGPL
>>>>>>>>>>>>>>> - the github repo shows rather infrequent
>>>>>>>>>>>>>>> updates.
>>>>>>>>>>>>>>> Quite
>>>>>>>> likely
>>>>>>>>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>>>>> CloudStack
>>>>>>>> community
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'd back John B's comments on disaggregating
>>>>>>>>>>>>>>> the VR.
>>>>>>>>>>>>>>> Split
>>>>>> it
>>>>>>>> into
>>>>>>>>>>>>>>> many docker containers
>>>>>>>>>>>>>>> - password server
>>>>>>>>>>>>>>> - userdata server
>>>>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>>>>> - FW
>>>>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>>>>> - agent
>>>>>>>>>>>>>>> Glue them together with docker compose files
>>>>>>>>>>>>>>> (one per
>>>>>>>>>>>>>>> use
>>>>>>>> case -
>>>>>>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The VR image then becomes a JeOS + docker.
>>>>>>>>>>>>>>> You can
>>>>>>>>>>>>>>> test
>>>>>> each
>>>>>>> of
>>>>>>>>>> the
>>>>>>>>>>>>>>> components independently and fixing one bug
>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>> field
>>>>>> (say
>>>>>>>>>> DHCP)
>>>>>>>>>>>>>>> is hitless to the other components. You don't
>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>> build per-hypervisor VRs. You could even run
>>>>>>>>>>>>>>> on
>>>>> baremetal.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>>>>> - make the traffic traverse the containers
>>>>>>>>>>>>>>> that are
>>>>>>>>>>>>>>> needed
>>>>>>> to
>>>>>>>> be
>>>>>>>>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>>>>> - bootstrap the router (how does it find its
>>>>>>>>>>>>>>> compose
>>>>> file?
>>>>>>>> where
>>>>>>>>>>>>>>> is the
>>>>>>>>>>>>>>> registry?)
>>>>>>>>>>>>>>> - rethink the command and control of the VR
>>>>>>>>>>>>>>> functions. SSH
>>>>>>>> works,
>>>>>>>>>>>>>>> but something more declarative, idempotent
>>>>>>>>>>>>>>> should be
>>>>>>> explored.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As you do this, it becomes clearer which of
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> functions
>>>>>> can
>>>>>>>> be
>>>>>>>>>>>>>>> substituted by for example CloudRouter.
>>>>>>>>>>>>>>> Command and
>>>>>>>>>>>>>>> Control
>>>>>>> of
>>>>>>>> the
>>>>>>>>>>>>>> docker
>>>>>>>>>>>>>>> containers can be moved out to another
>>>>>>>>>>>>>>> container. Etc.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty
>>>>>>>>>>>>>>> Godsey
>>>>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This one does look nice. My biggest concern
>>>>>>>>>>>>>>>> is the
>>>>>>>>>>>>>>>> lack
>>>>>> of
>>>>>>>>>>>>>>>> VXLANs. It seems that any of the ones we
>>>>>>>>>>>>>>>> mentioned
>>>>>>>>>>>>>>>> do not
>>>>>>>> have
>>>>>>>>>> an
>>>>>>>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cloudrouter looks promising. These have
>>>>>>>>>>>>>>>> potential to
>>>>>>>>>>>>>>>> save
>>>>>>>> future
>>>>>>>>>>>>>>>> engineering effort for example on ipv6
>>>>>>>>>>>>>>>> routing, OSPF
>>>>> etc.
>>>>>>>>>>>>>>>> And the best part is they come with test
>>>>>>>>>>>>>>>> automation
>>>>>>>> framework.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Instead of replacing the VR in first
>>>>>>>>>>>>>>>>> place we
>>>>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>>>>> as provider. Once it is stable, network
>>>>>>>>>>>>>>>> offerings
>>>>>>>>>>>>>>>> (on
>>>>>>>> upgrade)
>>>>>>>>>>>>>>>> can be updated to use it and we can drop
>>>>>>>>>>>>>>>> the VR if
>>>>>>>>>>>>>>>> we
>>>>>> want
>>>>>>> at
>>>>>>>>>>>>>>>> that release
>>>>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> VR is stabilized over a period of time
>>>>>>>>>>>>>>>>> and some of
>>>>>>>>>>>>>>>>> them
>>>>>>> are
>>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>> without issues. When we replicate the ACS
>>>>>>>>>>>>>>>> VR
>>>>>>>>>>>>>>>> features in
>>>>>>> new
>>>>>>>>>>>>>>>> solution it takes some to find the missing
>>>>>>>>>>>>>>>> pieces
>>>>>>>>>>>>>>>> (hidden
>>>>>>>> bugs).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm
>>>>>>>>>>>>>>>>>> not too
>>>>>>>>>>>>>>>>>> keen
>>>>>> on
>>>>>>>> VyOS
>>>>>>>>>>>>>>>>>> (it
>>>>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg
>>>>>>>>>>>>>>>>>> technology!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>>>>> ;
>>>>>>>>>>>>>>>> 53 Chandos Place, Covent Garden,
>>>>>>>>>>>>>>>> London WC2N 4HSUK
>>>>>>>> @shapeblue
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>>>>> From: "Will Stevens" <williamstevens@
>>>>>>>>>>>>>>>>>>> gmail.com>
>>>>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016
>>>>>>>>>>>>>>>>>>> 21:20:11
>>>>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought
>>>>>>>>>>>>>>>>>>> experiment and
>>>>>>>>>>>>>>>>>>> should
>>>>>>> be
>>>>>>>>>>>>>>>>>>> treated as
>>>>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>>>>> Please weigh in with the good and bad
>>>>>>>>>>>>>>>>>>> of this
>>>>> idea...
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> A couple of us have been discussing
>>>>>>>>>>>>>>>>>>> the idea of
>>>>>>>> potentially
>>>>>>>>>>>>>>>>>>> replacing the ACS VR with the VyOS
>>>>>>>>>>>>>>>>>>> [1] (Open
>>>>>>>>>>>>>>>>>>> Source
>>>>>>>> Vyatta
>>>>>>>>>>> VM).
>>>>>>>>>>>>>>>>>>> There may be a license issue because
>>>>>>>>>>>>>>>>>>> I think it
>>>>>>>>>>>>>>>>>>> is
>>>>>>>> licensed
>>>>>>>>>>>>>>>>>>> under GPL, but for the sake of
>>>>>>>>>>>>>>>>>>> discussion, let's
>>>>>> assume
>>>>>>>> we
>>>>>>>>>>>>>>>>>>> can overcome any
>>>>>>>>>>>>>>>> license issues.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I have spent some time recently with
>>>>>>>>>>>>>>>>>>> the VyOS
>>>>>>>>>>>>>>>>>>> and I
>>>>>>> have
>>>>>>>> to
>>>>>>>>>>>>>>>>>>> admit, I was pretty impressed. It is
>>>>>>>>>>>>>>>>>>> simple and
>>>>>>>> intuitive
>>>>>>>>>>>>>>>>>>> and it gives you a lot more options
>>>>>>>>>>>>>>>>>>> for auditing
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>> configuration etc...
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>>>>>>>>>> - Clean up our current VR script
>>>>>>>>>>>>>>>>>>> spaghetti to a
>>>>>> simpler
>>>>>>>> more
>>>>>>>>>>>>>>>>>>> auditable configuration workflow.
>>>>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6
>>>>>>>>>>>>>>>>>>> support.
>>>>>>>>>>>>>>>>>>> - Handles VPN configuration via the
>>>>>>>>>>>>>>>>>>> same
>>>>>> configuration
>>>>>>>>>>>> interface.
>>>>>>>>>>>>>>>>>>> - Support for OSPF & BGP.
>>>>>>>>>>>>>>>>>>> - VPN support through OpenVPN &
>>>>>>>>>>>>>>>>>>> StrongSwan.
>>>>>>>>>>>>>>>>>>> - Easily supports HA (redundant
>>>>>>>>>>>>>>>>>>> routers) through
>>>>>> VRRP.
>>>>>>>>>>>>>>>>>>> - VXLAN support.
>>>>>>>>>>>>>>>>>>> - Transaction based changes to the VR
>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>> rollback
>>>>>> on
>>>>>>>>>> error.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Items that could be difficult to
>>>>>>>>>>>>>>>>>>> solve:
>>>>>>>>>>>>>>>>>>> - Userdata password reset workflow
>>>>>>>>>>>>>>>>>>> and
>>>>>> implementation.
>>>>>>>>>>>>>>>>>>> - Upgrade process.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> The VyOS is not the only option if we
>>>>>>>>>>>>>>>>>>> were to
>>>>>> consider
>>>>>>>> this
>>>>>>>>>>>>>> approach.
>>>>>>>>>>>>>>>>>>> Another option, which I don't know as
>>>>>>>>>>>>>>>>>>> well,
>>>>>>>>>>>>>>>>>>> would be CloudRouter (AGPL
>>>>>>>>>>>>>>>>>>> license) [2] which is purely API
>>>>>>>>>>>>>>>>>>> driven.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Anyway, would love to hear your
>>>>>>>>>>>>>>>>>>> thoughts...
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [1] https://vyos.io/ [2]
>>>>>>>>>>>>>>>>>>> https://cloudrouter.org/
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> DISCLAIMER
>>>>>>>>>>>>>>>>> ==========
>>>>>>>>>>>>>>>>> This e-mail may contain privileged and
>>>>>>>>>>>>>>>>> confidential
>>>>>>>> information
>>>>>>>>>>>>>>>>> which is
>>>>>>>>>>>>>>>> the property of Accelerite, a Persistent
>>>>>>>>>>>>>>>> Systems
>>>>>> business.
>>>>>>>> It is
>>>>>>>>>>>>>>>> intended only for the use of the individual
>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>> entity to
>>>>>>>> which
>>>>>>>>>> it
>>>>>>>>>>>>>>>> is addressed. If you are not the intended
>>>>>>>>>>>>>>>> recipient,
>>>>>>>>>>>>>>>> you
>>>>>>> are
>>>>>>>> not
>>>>>>>>>>>>>>>> authorized to read, retain, copy, print,
>>>>>>>>>>>>>>>> distribute
>>>>>>>>>>>>>>>> or
>>>>>> use
>>>>>>>> this
>>>>>>>>>>>>>>>> message. If you have received this
>>>>>>>>>>>>>>>> communication in
>>>>>> error,
>>>>>>>>>> please
>>>>>>>>>>>>>>>> notify the sender and delete all copies of
>>>>>>>>>>>>>>>> this
>>>>> message.
>>>>>>>>>>>>>>>> Accelerite, a Persistent Systems business
>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>> accept
>>>>>>> any
>>>>>>>>>>>>>>>> liability for virus
>>>>>>>>>>>>>>> infected mails.
>>>>>>>>>>>>>>>>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <wi...@gmail.com>.
At this point in the discussion, I don't think we should rule anything out.
I think it makes sense to explore all the options and then isolate some
front runners in terms of software and architecture.
On Sep 18, 2016 1:08 AM, "ilya" <il...@gmail.com> wrote:
> Our options become much better if we consider BSD based routers.
>
> Would that be on the table?
>
> https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributions
>
>
> On 9/16/16 12:04 PM, Will Stevens wrote:
> > Ya, your points are all valid Simon. The lack of standard libraries to
> > handle a lot of the details is a problem. I don't think it is an
> > unsolvable problem, but if we spend the time to do that, will we have
> > something that will work for us for the next 5 years? This may be the
> > shortest path to getting us where we need to be for the time being.
> >
> > What is the best case scenario for the VR going forward which will last
> us
> > the next 5 years? Maybe we just clean up what we have to do a major
> > restructuring of the pieces and how they are implemented. We need to
> keep
> > in mind how maintainable this implementation is because that is going to
> be
> > key going forward IMO.
> >
> >
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > w cloudops.com *|* tw @CloudOps_
> >
> > On Fri, Sep 16, 2016 at 2:29 PM, Simon Weller <sw...@ena.com> wrote:
> >
> >> I think our other option is to take a real look at what it would take to
> >> fix the VR. In my opinion, a lot of the problems are related to the
> >> monolithic python code base and the fact nothing is actually separated.
> >>
> >> Secondly, the python scripts (and bash scripts) don't use any
> established
> >> libraries to complete tasks and instead shell out and run commands that
> are
> >> both hard to track and hard to parse on return.
> >>
> >>
> >> If we daemonized this, used a real api for Agent to VR communication,
> used
> >> common already existing libraries for the system service and network
> >> interactions and spent a bit of time separating out code into distinct
> >> modules, everything would behave a lot better.
> >>
> >>
> >> The pain and suffering is due to years and years of patches and constant
> >> shelling out to complete tasks in my opinion. If we spend time to
> rethink
> >> how we interact with the VR in general and we abstract the systems and
> >> networking stuff and use well known and stable libraries to do the work,
> >> the VR would be much easier to maintain.
> >>
> >>
> >> - Si
> >>
> >>
> >>
> >>
> >> ________________________________
> >> From: Marty Godsey <ma...@gonsource.com>
> >> Sent: Friday, September 16, 2016 12:24 PM
> >> To: dev@cloudstack.apache.org
> >> Subject: RE: [DISCUSS] Replacing the VR
> >>
> >> So based upon this discussion would it be prudent to wait on VyOS 2.0?
> The
> >> current VR is giving us issues but would the time invested in another
> >> "solution" be wasted especially if by the time another option is chose,
> >> then coded, then tested, then implemented and right as that time
> happened
> >> to be when VyOS 2.0 is released. Of course you said they are just in
> the
> >> scoping range so this could still be a year or more out.
> >>
> >> Thoughts?
> >>
> >> Regards,
> >> Marty Godsey
> >> nSource Solutions
> >>
> >> -----Original Message-----
> >> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> >> Behalf Of Will Stevens
> >> Sent: Friday, September 16, 2016 10:31 AM
> >> To: dev@cloudstack.apache.org
> >> Cc: daniil@baturin.org
> >> Subject: Re: [DISCUSS] Replacing the VR
> >>
> >> I just had a quick chat with a couple of the guys over on the VyOS chat.
> >> I have CC'ed one of them in case we have more licensing questions.
> >>
> >> So here is the status with the license "the code inherited from Vyatta
> and
> >> our modifications from it is GPLv2 (strict, not v2+). The config reading
> >> library is GPLv2 too, so anything that links to is is GPLv2.
> >> Some auxiliary components we made after the fork are more permissive,
> >> LGPLv2+ or MIT."
> >>
> >> They are currently in the process of scoping a redesign (VyOS 2.0), "we
> >> are planning a clean rewrite that will solve issues of the current
> config
> >> system".
> >> This will include the ability to configure via the API.
> >>
> >> If we have more questions for VyOS, they are very friendly and
> responsive,
> >> so we should be able to get answers.
> >>
> >> *Will STEVENS*
> >> Lead Developer
> >>
> >> *CloudOps* *| *Cloud Solutions Experts
> >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> >> @CloudOps_
> >>
> >> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com>
> wrote:
> >>
> >>> I agree with Will Ilya. There are so many problems with the VR right
> now.
> >>> Most of the outages we've had recently have somehow involved the VR.
> >>> We set custom iptables rules on the VR which can and have easily gone
> >> wrong.
> >>> Openswan is broken, Strongswan replacement still needs to be tested.
> >>> VVRP with redundant router still needs work, and not to mention the
> >>> problems we will have when we introduce IPv6 into the whole picture.
> >>>
> >>> I think the spirit of the discussion is to rely on a 3rd party to do
> >>> the networking for us (eg VyOS) and have us handle just the
> >>> orchestration. All the problems that I've described have already been
> >>> solved in VyOS. We also get the advantage of a potential wider
> >>> community to fix and maintain the VR and given our current development
> >>> velocity, it think it totally makes sense to look for a 3rd party
> option.
> >>>
> >>> -Syed
> >>>
> >>>
> >>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
> >>> wrote:
> >>>
> >>>> The VR has been biting us far too often recently, which is why we
> >>>> have started looking into alternative implementations.
> >>>>
> >>>> One of the things that is nice about potentially using the VyOS is
> >>>> that
> >>> it
> >>>> is based on Debian, so we should be able to run the other services
> >>>> that
> >>> we
> >>>> currently have like the password server and userdata on the VyOS.
> >>>> This means we would not have to change our architecture initially
> >>>> and could focus on only replacing the networking paths.
> >>>>
> >>>> *Will STEVENS*
> >>>> Lead Developer
> >>>>
> >>>> *CloudOps* *| *Cloud Solutions Experts
> >>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> >>>> tw @CloudOps_
> >>>>
> >>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> >>>>
> >>>>> The more this is discussed the more I think we should stick with
> >>>>> our
> >>> VR.
> >>>>>
> >>>>> All these other options either seem unfinished or with
> >>>>> incompatible license.
> >>>>>
> >>>>> VyOS looks the most promising so far, it's a serious, mature project.
> >>>>> Adopting it though means we'll have to microservice our way out of
> >>>>> it
> >>>> with
> >>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
> >>> those
> >>>>> too. Imho this adds complexity we should void.
> >>>>>
> >>>>> --
> >>>>> Sent from the Delta quadrant using Borg technology!
> >>>>>
> >>>>> Nux!
> >>>>> www.nux.ro
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>> From: "Will Stevens" <ws...@cloudops.com>
> >>>>>> To: dev@cloudstack.apache.org
> >>>>>> Sent: Thursday, 15 September, 2016 17:21:28
> >>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>
> >>>>>> Ya, we would need to add a daemon for VPN as well. Load
> >>>>>> balancing is another aspect which we will need to consider if we
> >> went this route.
> >>>>>> Something like https://traefik.io/ could potentially be a good
> >>>>>> fit
> >>> due
> >>>>> to
> >>>>>> its API driven configuration, but it may be more than what we need.
> >>>>>>
> >>>>>> We should probably try define which pieces make sense to be
> >>>>>> solved
> >>>>> together
> >>>>>> and which pieces would be best suited to be broken out.
> >>>>>>
> >>>>>> I think the network connectivity, routing and firewalling should
> >>>> probably
> >>>>>> all stay together since the majority of the tools we would
> >>> potentially
> >>>>> use
> >>>>>> would handle all of that together in a single implementation.
> >>>>>>
> >>>>>> The password server and userdata seems like a good option for
> >>>>>> being
> >>>>> broken
> >>>>>> out and handled independently (and probably rewritten completely
> >>> since
> >>>>> they
> >>>>>> currently have some issues).
> >>>>>>
> >>>>>> Load balancing is another that could warrant splitting out, but
> >>>>>> that depends on what direction we go and how we would be managing
> >> it.
> >>> DHCP
> >>>>> and
> >>>>>> DNS are others which could go either way.
> >>>>>>
> >>>>>> If we do split out services, I think we should consolidate as
> >>>>>> much as
> >>>> we
> >>>>>> can into each service we break out. Ideally a network packet
> >>>>>> would
> >>>> never
> >>>>>> hit more than one, maybe two, services. I don't think we should
> >>>>>> be splitting services 'just because', I think we need a valid
> >>>>>> case for splitting any service out because it adds complexity.
> >>>>>> Our project is already complex enough, we need to avoid adding
> >>>>>> complexity unless it
> >>> is
> >>>>>> really needed.
> >>>>>>
> >>>>>> Some more of my thoughts on this anyway...
> >>>>>>
> >>>>>> *Will STEVENS*
> >>>>>> Lead Developer
> >>>>>>
> >>>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> >>>>>> *|* tw @CloudOps_
> >>>>>>
> >>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> I do agree with you that this probably isn't the right place
> >>>>>>> the
> >>>>> password
> >>>>>>> service and user data.
> >>>>>>>
> >>>>>>>
> >>>>>>> Having said that, after taking a cursory look at the dev docs,
> >>>>>>> it
> >>>>> doesn't
> >>>>>>> seem that difficult to add new daemons:
> >>> https://opensnaproute.github.
> >>>>>>> io/docs/developer.html#creating-new-component
> >>>>>>>
> >>>>>>> <https://opensnaproute.github.io/docs/developer.html#
> >>>>>>> creating-new-component>
> >>>>>>>
> >>>>>>>
> >>>>>>> They've definitely build it with a microservices architecture
> >>>>>>> in
> >>> mind,
> >>>>> so
> >>>>>>> each individual feature is abstracted into it's own small
> >>>>>>> daemon
> >>>>> process.
> >>>>>>> We could just create a daemon for the password server and the
> >>> userdata
> >>>>>>> components if we really had to.
> >>>>>>>
> >>>>>>>
> >>>>>>> - Si
> >>>>>>>
> >>>>>>>
> >>>>>>> ________________________________
> >>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
> >>>>>>> behalf
> >>>> of
> >>>>>>> Will Stevens <ws...@cloudops.com>
> >>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
> >>>>>>> To: dev@cloudstack.apache.org
> >>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>
> >>>>>>> A big part of why I know about it is because it is written in Go.
> >>> :P
> >>>>>>>
> >>>>>>> Yes, it is definitely interesting for the routing and traffic
> >>> handling
> >>>>>>> aspects of the VR. We will likely have to rethink some of the
> >>> pieces
> >>>> a
> >>>>>>> little bit like the password server and userdata if we are to
> >>>>>>> adopt
> >>> a
> >>>>>>> different VR approach. This is where I think some of JohnB and
> >>>>> Chiradeep's
> >>>>>>> ideas make sense. In many ways, it does not make sense for the
> >>> device
> >>>>>>> handling routing and network traffic to also be responsible for
> >>>>> passwords
> >>>>>>> and userdata.
> >>>>>>>
> >>>>>>> *Will STEVENS*
> >>>>>>> Lead Developer
> >>>>>>>
> >>>>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> >>>>>>> *|* tw @CloudOps_
> >>>>>>>
> >>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
> >>> pretty
> >>>>>>> cool!
> >>>>>>>> It even supports ONIE install.
> >>>>>>>>
> >>>>>>>> To be honest, the ipsec feature could be added, or we could
> >>> offload
> >>>>> it to
> >>>>>>>> separate vm if we needed to. The fact it is so feature rich
> >>>>>>>> from a
> >>>>>>> routing
> >>>>>>>> perspective (and all API driven) is really nice.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Based on the roadmap, it looks like they plan to also support
> >>>>>>> capabilities
> >>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
> >>>>>>>> will
> >>> be
> >>>>> huge
> >>>>>>>> for our carrier community that rely on these technologies to
> >>>>>>>> do
> >>>>> private
> >>>>>>>> gateway and inter-VPC interconnections today. We handle this
> >>>>>>>> stuff
> >>>> on
> >>>>> our
> >>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
> >>>>>>>> able to
> >>>> do
> >>>>>>> MPLS
> >>>>>>>> all the way to the VR would be awesome.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> It also seems to be written in GO (a language here at ENA we
> >>>>>>>> know
> >>>> very
> >>>>>>>> well).
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> - Si
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> ________________________________
> >>>>>>>> From: Will Stevens <wi...@gmail.com>
> >>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
> >>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
> >>>>>>>>
> >>>>>>>> Ya. I don't think it covers our whole use case, but what it
> >>>>>>>> does
> >>>>> cover is
> >>>>>>>> all api driven...
> >>>>>>>>
> >>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
> >>>> wrote:
> >>>>>>>>
> >>>>>>>>> Though I don’t see VPN in Snaproute.. Makes sense since it
> >>>>>>>>> was
> >>> not
> >>>>>>>>> intended to do IPSec.
> >>>>>>>>>
> >>>>>>>>> It seems as though VyOS is starting to look like the best
> >>> option.
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>> Marty Godsey
> >>>>>>>>> nSource Solutions
> >>>>>>>>>
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: williamstevens@gmail.com
> >>>>>>>>> [mailto:williamstevens@gmail.com
> >>> ]
> >>>> On
> >>>>>>>>> Behalf Of Will Stevens
> >>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
> >>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>>>
> >>>>>>>>> Or we could go completely crazy and go with something like
> >>>>> FlexSwitch
> >>>>>>>> from
> >>>>>>>>> SnapRoute
> >>>>>>>>> - http://www.snaproute.com/
> >>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
> >>>>>>>>>
> >>>>>>>>> *Will STEVENS*
> >>>>>>>>> Lead Developer
> >>>>>>>>>
> >>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> >>>>>>>>> cloudops.com
> >>>> *|*
> >>>>> tw
> >>>>>>>>> @CloudOps_
> >>>>>>>>>
> >>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> >>>>> wstevens@cloudops.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
> >>>> problems
> >>>>> are
> >>>>>>>>>> solved by splitting up the function of the VR into a
> >>>>>>>>>> bunch of
> >>>>>>> separate
> >>>>>>>>>> services. As Syed points out, the complexity added is
> >>>>> non-trivial.
> >>>>>>>>>> We now have to manage all the intercontainer networking
> >>>>>>>>>> as
> >>> well
> >>>> as
> >>>>>>> the
> >>>>>>>>>> orchestrated ACS networking.
> >>>>>>>>>>
> >>>>>>>>>> VyOS is interesting to me because it covers the majority
> >>>>>>>>>> of
> >>> our
> >>>>> use
> >>>>>>>>>> case with a single unified control plane. It also has
> >>>>>>>>>> good
> >>>>> support
> >>>>>>>>>> for extending features we care about, like IPv6, VXLAN,
> >>>>>>>>>> VRRP, transactions, etc...
> >>>>>>>>>>
> >>>>>>>>>> *Will STEVENS*
> >>>>>>>>>> Lead Developer
> >>>>>>>>>>
> >>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> >>> cloudops.com
> >>>>> *|*
> >>>>>>> tw
> >>>>>>>>>> @CloudOps_
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> >>>> sahmed@cloudops.com>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Agree with Marty, adding Docker containers to the
> >>>>>>>>>>> picture
> >>>>> although
> >>>>>>>>>>> can make the VR more flexible but the added complexity
> >>>>>>>>>>> is
> >>> just
> >>>>> not
> >>>>>>>>>>> worth it. Not to mention we would need to take care of
> >>>> networking
> >>>>>>>>>>> each container manually and given that our iptable rules
> >>>>>>>>>>> are
> >>>> very
> >>>>>>>>>>> unstable at the moment I don't see a big value add.
> >>>>>>>>>>>
> >>>>>>>>>>> Vyos looks like a better solution to me. I know that it
> >>>>>>>>>>> does
> >>>> not
> >>>>>>>>>>> provide an api but it does fit the bill quite well
> >>> otherwise. I
> >>>>>>>>>>> specially like the fact that it has a transaction based
> >>>>>>>>>>> model
> >>>> and
> >>>>>>> you
> >>>>>>>>>>> can rollback changes if something goes wrong.
> >>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> >>>>> marty@gonsource.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Licensing aside, I think splitting the various
> >>>>>>>>>>>> functions
> >>> into
> >>>>>>>>>>>> containers is not a good route either. This will force
> >>> users
> >>>> to
> >>>>>>>>>>>> have to maintain
> >>>>>>>>>>> and
> >>>>>>>>>>>> use containers and adds complexity to the networking
> >>> aspects
> >>>> of
> >>>>>>> ACS.
> >>>>>>>>>>>> Complexity decreases stability. Now I understand the
> >>> argument
> >>>>> that
> >>>>>>>>>>>> a monolithic approach also brings its own set of
> >>>>>>>>>>>> issues but
> >>>> it
> >>>>>>> also
> >>>>>>>>>>>> simplifies it.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Regards,
> >>>>>>>>>>>> Marty Godsey
> >>>>>>>>>>>> nSource Solutions
> >>>>>>>>>>>>
> >>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> >>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
> >>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>>>>>>
> >>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
> >>>>>>>>>>>> of
> >>> the
> >>>>>>>>>>>> CloudStack project
> >>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
> >>>>>>>>>>>> touch
> >>>>> anything
> >>>>>>>>>>>> that
> >>>>>>>>>>> has
> >>>>>>>>>>>> AGPL
> >>>>>>>>>>>> - the github repo shows rather infrequent updates.
> >>>>>>>>>>>> Quite
> >>>>> likely
> >>>>>>>>>>>> they aren't considering the use cases of the
> >>>>>>>>>>>> CloudStack
> >>>>> community
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'd back John B's comments on disaggregating the VR.
> >>>>>>>>>>>> Split
> >>> it
> >>>>> into
> >>>>>>>>>>>> many docker containers
> >>>>>>>>>>>> - password server
> >>>>>>>>>>>> - userdata server
> >>>>>>>>>>>> - DHCP / DNS
> >>>>>>>>>>>> - s2s VPN
> >>>>>>>>>>>> - RA VPN
> >>>>>>>>>>>> - intra-VPC routing and ACL
> >>>>>>>>>>>> - Port forwarding + NAT
> >>>>>>>>>>>> - FW
> >>>>>>>>>>>> - LB (public)
> >>>>>>>>>>>> - LB (internal),
> >>>>>>>>>>>> - secondary storage
> >>>>>>>>>>>> - agent
> >>>>>>>>>>>> Glue them together with docker compose files (one per
> >>>>>>>>>>>> use
> >>>>> case -
> >>>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
> >>>>>>>>>>>>
> >>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
> >>>>>>>>>>>> test
> >>> each
> >>>> of
> >>>>>>> the
> >>>>>>>>>>>> components independently and fixing one bug in the
> >>>>>>>>>>>> field
> >>> (say
> >>>>>>> DHCP)
> >>>>>>>>>>>> is hitless to the other components. You don't need to
> >>>>>>>>>>>> build per-hypervisor VRs. You could even run on
> >> baremetal.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Along the way you need to figure out how to
> >>>>>>>>>>>> - make the traffic traverse the containers that are
> >>>>>>>>>>>> needed
> >>>> to
> >>>>> be
> >>>>>>>>>>>> traversed (in most cases just 1)
> >>>>>>>>>>>> - bootstrap the router (how does it find its compose
> >> file?
> >>>>> where
> >>>>>>>>>>>> is the
> >>>>>>>>>>>> registry?)
> >>>>>>>>>>>> - rethink the command and control of the VR
> >>>>>>>>>>>> functions. SSH
> >>>>> works,
> >>>>>>>>>>>> but something more declarative, idempotent should be
> >>>> explored.
> >>>>>>>>>>>>
> >>>>>>>>>>>> As you do this, it becomes clearer which of the
> >>>>>>>>>>>> functions
> >>> can
> >>>>> be
> >>>>>>>>>>>> substituted by for example CloudRouter. Command and
> >>>>>>>>>>>> Control
> >>>> of
> >>>>> the
> >>>>>>>>>>> docker
> >>>>>>>>>>>> containers can be moved out to another container. Etc.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> >>>>>>>>>>>> <ma...@gonsource.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> This one does look nice. My biggest concern is the
> >>>>>>>>>>>>> lack
> >>> of
> >>>>>>>>>>>>> VXLANs. It seems that any of the ones we mentioned
> >>>>>>>>>>>>> do not
> >>>>> have
> >>>>>>> an
> >>>>>>>>>>>>> API so we may be stuck at the SSH method.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>> Marty Godsey
> >>>>>>>>>>>>> nSource Solutions
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>>> From: Abhinandan Prateek
> >>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
> >>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
> >>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
> >>>>>>>>>>>>> save
> >>>>> future
> >>>>>>>>>>>>> engineering effort for example on ipv6 routing, OSPF
> >> etc.
> >>>>>>>>>>>>> And the best part is they come with test automation
> >>>>> framework.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
> >>>>>>>>>>>>> <ja...@accelerite.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Instead of replacing the VR in first place we
> >>>>>>>>>>>>>> should add VyOS/cloudrouter
> >>>>>>>>>>>>> as provider. Once it is stable, network offerings
> >>>>>>>>>>>>> (on
> >>>>> upgrade)
> >>>>>>>>>>>>> can be updated to use it and we can drop the VR if
> >>>>>>>>>>>>> we
> >>> want
> >>>> at
> >>>>>>>>>>>>> that release
> >>>>>>>>>>>> onwards.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> VR is stabilized over a period of time and some of
> >>>>>>>>>>>>>> them
> >>>> are
> >>>>>>>>>>>>>> running
> >>>>>>>>>>>>> without issues. When we replicate the ACS VR
> >>>>>>>>>>>>> features in
> >>>> new
> >>>>>>>>>>>>> solution it takes some to find the missing pieces
> >>>>>>>>>>>>> (hidden
> >>>>> bugs).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> Jayapal
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I like the idea.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
> >>>>>>>>>>>>>>> keen
> >>> on
> >>>>> VyOS
> >>>>>>>>>>>>>>> (it
> >>>>>>>>>>>>> doesn't have a proper http api etc).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Nux!
> >>>>>>>>>>>>>>> www.nux.ro
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
> >>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
> >>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> >>>>> @shapeblue
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
> >>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
> >>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
> >>>>>>>>>>>>>>>> should
> >>>> be
> >>>>>>>>>>>>>>>> treated as
> >>>>>>>>>>>>> such.
> >>>>>>>>>>>>>>>> Please weigh in with the good and bad of this
> >> idea...
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
> >>>>> potentially
> >>>>>>>>>>>>>>>> replacing the ACS VR with the VyOS [1] (Open
> >>>>>>>>>>>>>>>> Source
> >>>>> Vyatta
> >>>>>>>> VM).
> >>>>>>>>>>>>>>>> There may be a license issue because I think it
> >>>>>>>>>>>>>>>> is
> >>>>> licensed
> >>>>>>>>>>>>>>>> under GPL, but for the sake of discussion, let's
> >>> assume
> >>>>> we
> >>>>>>>>>>>>>>>> can overcome any
> >>>>>>>>>>>>> license issues.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I have spent some time recently with the VyOS
> >>>>>>>>>>>>>>>> and I
> >>>> have
> >>>>> to
> >>>>>>>>>>>>>>>> admit, I was pretty impressed. It is simple and
> >>>>> intuitive
> >>>>>>>>>>>>>>>> and it gives you a lot more options for auditing
> >>>>>>>>>>>>>>>> the
> >>>>>>>>> configuration etc...
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Items of potential interest:
> >>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
> >>> simpler
> >>>>> more
> >>>>>>>>>>>>>>>> auditable configuration workflow.
> >>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
> >>>>>>>>>>>>>>>> - Handles VPN configuration via the same
> >>> configuration
> >>>>>>>>> interface.
> >>>>>>>>>>>>>>>> - Support for OSPF & BGP.
> >>>>>>>>>>>>>>>> - VPN support through OpenVPN & StrongSwan.
> >>>>>>>>>>>>>>>> - Easily supports HA (redundant routers) through
> >>> VRRP.
> >>>>>>>>>>>>>>>> - VXLAN support.
> >>>>>>>>>>>>>>>> - Transaction based changes to the VR with
> >>>>>>>>>>>>>>>> rollback
> >>> on
> >>>>>>> error.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Items that could be difficult to solve:
> >>>>>>>>>>>>>>>> - Userdata password reset workflow and
> >>> implementation.
> >>>>>>>>>>>>>>>> - Upgrade process.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The VyOS is not the only option if we were to
> >>> consider
> >>>>> this
> >>>>>>>>>>> approach.
> >>>>>>>>>>>>>>>> Another option, which I don't know as well,
> >>>>>>>>>>>>>>>> would be CloudRouter (AGPL
> >>>>>>>>>>>>>>>> license) [2] which is purely API driven.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Anyway, would love to hear your thoughts...
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Will
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> [1] https://vyos.io/ [2]
> >>>>>>>>>>>>>>>> https://cloudrouter.org/
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> DISCLAIMER
> >>>>>>>>>>>>>> ==========
> >>>>>>>>>>>>>> This e-mail may contain privileged and confidential
> >>>>> information
> >>>>>>>>>>>>>> which is
> >>>>>>>>>>>>> the property of Accelerite, a Persistent Systems
> >>> business.
> >>>>> It is
> >>>>>>>>>>>>> intended only for the use of the individual or
> >>>>>>>>>>>>> entity to
> >>>>> which
> >>>>>>> it
> >>>>>>>>>>>>> is addressed. If you are not the intended recipient,
> >>>>>>>>>>>>> you
> >>>> are
> >>>>> not
> >>>>>>>>>>>>> authorized to read, retain, copy, print, distribute
> >>>>>>>>>>>>> or
> >>> use
> >>>>> this
> >>>>>>>>>>>>> message. If you have received this communication in
> >>> error,
> >>>>>>> please
> >>>>>>>>>>>>> notify the sender and delete all copies of this
> >> message.
> >>>>>>>>>>>>> Accelerite, a Persistent Systems business does not
> >>>>>>>>>>>>> accept
> >>>> any
> >>>>>>>>>>>>> liability for virus
> >>>>>>>>>>>> infected mails.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
RE: [DISCUSS] Replacing the VR
Posted by Paul Angus <pa...@shapeblue.com>.
Hi All,
From the looks of this thread and the different preferences that parties have, the way forward looks to be a model that allows multiple different VRs to be used is the answer.
This is something that I have had in mind for some time now.
By abstracting the roles and configuration of the VR, drivers can be written which transform requests such as 'add these firewall rules' into something that a VyOs, Fortigate, pfSense, Cisco ASAv, a Juniper vSRX or a homebrew VR can understand and implement.
Ideally I'd like to see the features separated and service chaining introduced such that someone might use a stock VR with a VPN appliance or a Cisco ASAv in front of a VR doing load balancing.
This moves us forward into a much more NFV-like world.
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: Syed Ahmed [mailto:sahmed@cloudops.com]
Sent: 19 September 2016 17:07
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
Hey Guys,
Will and I had a discussion in the morning on around VyOS and I have an
idea that could work, here me out.
The way Cloudstack currently communicates with VR is via the Hypervisor
(let's assume KVM and XenServer for now). The management server sends a
command as a JSON to the hypervisor which proxies them to the VR and they
get executed there. Now, what we could do is add a proxy on the hypervisor
which translates the management server JSON to VyOS commands (and
vice-versa). There is a VyOS API lib in python which we clould use. Now
this would require 0 changes on the Cloudstack core networking side. There
may be a few minor changes required to push this proxy on the hypervisor
but other than that there is nothing major. Now in the mean time, the VyOS
guys can work on their v2.0 and when they have a stable enough API we can
make it a first class citizen in Cloudstack.
This approach would not work for VMWare as in VMWare the management server
directly talks to the VR. However, this problem could be fixed by adding a
middle VM which does the proxying to-and-from the VR. This would also
improve the overall security of the system in VMWare as well.
I am not too concerned about VMWare at the moment as most of us use either
KVM or XenServer. What do you guys think of this approach?
-Syed
On Mon, Sep 19, 2016 at 11:42 AM, Stephan Seitz <
s.seitz@secretresearchfacility.com> wrote:
> Hi!
>
> Just to add my 2 cents to that thread:
>
> I'ld really like to see something like vyatta or pfsense integrated as
> "standard" VR.
>
> We'd also talked internally about replacing the VR with some more
> mature "appliance"-like router distro.
>
> pfsense e.g. comes AFAIK with no defined API but instead has a very
> nice GUI.
> How would this fit into the concept of configuring the VR via ACS?
> Would parts of the GUI - like IP configuration and basic firewall rules
> - hidden or greyed?
> Where would one save the configuration, VPN certificates and so on?
>
>
> - Stephan
>
>
> Am Sonntag, den 18.09.2016, 15:19 +0000 schrieb Marty Godsey:
> > On this note I also mentioned pfsense earlier.
> >
> > www.pfsense.org
> >
> >
> > Regards,
> > Marty Godsey
> >
> > -----Original Message-----
> > From: ilya [mailto:ilya.mailing.lists@gmail.com]
> > Sent: Sunday, September 18, 2016 1:09 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] Replacing the VR
> >
> > Our options become much better if we consider BSD based routers.
> >
> > Would that be on the table?
> >
> > https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributio
> > ns
> >
> >
> > On 9/16/16 12:04 PM, Will Stevens wrote:
> > >
> > > Ya, your points are all valid Simon. The lack of standard
> > > libraries
> > > to handle a lot of the details is a problem. I don't think it is
> > > an
> > > unsolvable problem, but if we spend the time to do that, will we
> > > have
> > > something that will work for us for the next 5 years? This may be
> > > the
> > > shortest path to getting us where we need to be for the time being.
> > >
> > > What is the best case scenario for the VR going forward which will
> > > last us the next 5 years? Maybe we just clean up what we have to
> > > do a
> > > major restructuring of the pieces and how they are
> > > implemented. We
> > > need to keep in mind how maintainable this implementation is
> > > because
> > > that is going to be key going forward IMO.
> > >
> > >
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > > tw
> > > @CloudOps_
> > >
> > > On Fri, Sep 16, 2016 at 2:29 PM, Simon Weller <sw...@ena.com>
> > > wrote:
> > >
> > > >
> > > > I think our other option is to take a real look at what it would
> > > > take
> > > > to fix the VR. In my opinion, a lot of the problems are related
> > > > to
> > > > the monolithic python code base and the fact nothing is actually
> > > > separated.
> > > >
> > > > Secondly, the python scripts (and bash scripts) don't use any
> > > > established libraries to complete tasks and instead shell out and
> > > > run
> > > > commands that are both hard to track and hard to parse on return.
> > > >
> > > >
> > > > If we daemonized this, used a real api for Agent to VR
> > > > communication,
> > > > used common already existing libraries for the system service
> > > > and
> > > > network interactions and spent a bit of time separating out code
> > > > into
> > > > distinct modules, everything would behave a lot better.
> > > >
> > > >
> > > > The pain and suffering is due to years and years of patches and
> > > > constant shelling out to complete tasks in my opinion. If we
> > > > spend
> > > > time to rethink how we interact with the VR in general and we
> > > > abstract the systems and networking stuff and use well known and
> > > > stable libraries to do the work, the VR would be much easier to
> > > > maintain.
> > > >
> > > >
> > > > - Si
> > > >
> > > >
> > > >
> > > >
> > > > ________________________________
> > > > From: Marty Godsey <ma...@gonsource.com>
> > > > Sent: Friday, September 16, 2016 12:24 PM
> > > > To: dev@cloudstack.apache.org
> > > > Subject: RE: [DISCUSS] Replacing the VR
> > > >
> > > > So based upon this discussion would it be prudent to wait on
> > > > VyOS
> > > > 2.0? The current VR is giving us issues but would the time
> > > > invested
> > > > in another "solution" be wasted especially if by the time
> > > > another
> > > > option is chose, then coded, then tested, then implemented and
> > > > right
> > > > as that time happened to be when VyOS 2.0 is released. Of course
> > > > you
> > > > said they are just in the scoping range so this could still be a
> > > > year or more out.
> > > >
> > > > Thoughts?
> > > >
> > > > Regards,
> > > > Marty Godsey
> > > > nSource Solutions
> > > >
> > > > -----Original Message-----
> > > > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com]
> > > > On
> > > > Behalf Of Will Stevens
> > > > Sent: Friday, September 16, 2016 10:31 AM
> > > > To: dev@cloudstack.apache.org
> > > > Cc: daniil@baturin.org
> > > > Subject: Re: [DISCUSS] Replacing the VR
> > > >
> > > > I just had a quick chat with a couple of the guys over on the
> > > > VyOS chat.
> > > > I have CC'ed one of them in case we have more licensing
> > > > questions.
> > > >
> > > > So here is the status with the license "the code inherited from
> > > > Vyatta and our modifications from it is GPLv2 (strict, not v2+).
> > > > The
> > > > config reading library is GPLv2 too, so anything that links to is
> > > > is GPLv2.
> > > > Some auxiliary components we made after the fork are more
> > > > permissive,
> > > > LGPLv2+ or MIT."
> > > >
> > > > They are currently in the process of scoping a redesign (VyOS
> > > > 2.0),
> > > > "we are planning a clean rewrite that will solve issues of the
> > > > current config system".
> > > > This will include the ability to configure via the API.
> > > >
> > > > If we have more questions for VyOS, they are very friendly and
> > > > responsive, so we should be able to get answers.
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > *CloudOps* *| *Cloud Solutions Experts
> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> > > > *|* tw
> > > > @CloudOps_
> > > >
> > > > On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com>
> > > > wrote:
> > > >
> > > > >
> > > > > I agree with Will Ilya. There are so many problems with the VR
> > > > > right now.
> > > > > Most of the outages we've had recently have somehow involved
> > > > > the VR.
> > > > > We set custom iptables rules on the VR which can and have
> > > > > easily
> > > > > gone
> > > > wrong.
> > > > >
> > > > > Openswan is broken, Strongswan replacement still needs to be
> > > > > tested.
> > > > > VVRP with redundant router still needs work, and not to mention
> > > > > the
> > > > > problems we will have when we introduce IPv6 into the whole
> > > > > picture.
> > > > >
> > > > > I think the spirit of the discussion is to rely on a 3rd party
> > > > > to do
> > > > > the networking for us (eg VyOS) and have us handle just the
> > > > > orchestration. All the problems that I've described have
> > > > > already
> > > > > been solved in VyOS. We also get the advantage of a potential
> > > > > wider
> > > > > community to fix and maintain the VR and given our current
> > > > > development velocity, it think it totally makes sense to look
> > > > > for a 3rd party option.
> > > > >
> > > > > -Syed
> > > > >
> > > > >
> > > > > On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens
> > > > > <ws...@cloudops.com>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > The VR has been biting us far too often recently, which is
> > > > > > why we
> > > > > > have started looking into alternative implementations.
> > > > > >
> > > > > > One of the things that is nice about potentially using the
> > > > > > VyOS is
> > > > > > that
> > > > > it
> > > > > >
> > > > > > is based on Debian, so we should be able to run the other
> > > > > > services
> > > > > > that
> > > > > we
> > > > > >
> > > > > > currently have like the password server and userdata on the
> > > > > > VyOS.
> > > > > > This means we would not have to change our architecture
> > > > > > initially
> > > > > > and could focus on only replacing the networking paths.
> > > > > >
> > > > > > *Will STEVENS*
> > > > > > Lead Developer
> > > > > >
> > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > > cloudops.com *|*
> > > > > > tw @CloudOps_
> > > > > >
> > > > > > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> > > > > >
> > > > > > >
> > > > > > > The more this is discussed the more I think we should stick
> > > > > > > with
> > > > > > > our
> > > > > VR.
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > All these other options either seem unfinished or with
> > > > > > > incompatible license.
> > > > > > >
> > > > > > > VyOS looks the most promising so far, it's a serious,
> > > > > > > mature project.
> > > > > > > Adopting it though means we'll have to microservice our way
> > > > > > > out of
> > > > > > > it
> > > > > > with
> > > > > > >
> > > > > > > extra machines for DNS/USERDATA/etc, unless we can make
> > > > > > > VyOS serve
> > > > > those
> > > > > >
> > > > > > >
> > > > > > > too. Imho this adds complexity we should void.
> > > > > > >
> > > > > > > --
> > > > > > > Sent from the Delta quadrant using Borg technology!
> > > > > > >
> > > > > > > Nux!
> > > > > > > www.nux.ro
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > > >
> > > > > > > > From: "Will Stevens" <ws...@cloudops.com>
> > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > Sent: Thursday, 15 September, 2016 17:21:28
> > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > >
> > > > > > > > Ya, we would need to add a daemon for VPN as well. Load
> > > > > > > > balancing is another aspect which we will need to
> > > > > > > > consider if we
> > > > went this route.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Something like https://traefik.io/ could potentially be a
> > > > > > > > good
> > > > > > > > fit
> > > > > due
> > > > > >
> > > > > > >
> > > > > > > to
> > > > > > > >
> > > > > > > > its API driven configuration, but it may be more than
> > > > > > > > what we need.
> > > > > > > >
> > > > > > > > We should probably try define which pieces make sense to
> > > > > > > > be
> > > > > > > > solved
> > > > > > > together
> > > > > > > >
> > > > > > > > and which pieces would be best suited to be broken out.
> > > > > > > >
> > > > > > > > I think the network connectivity, routing and firewalling
> > > > > > > > should
> > > > > > probably
> > > > > > >
> > > > > > > >
> > > > > > > > all stay together since the majority of the tools we
> > > > > > > > would
> > > > > potentially
> > > > > >
> > > > > > >
> > > > > > > use
> > > > > > > >
> > > > > > > > would handle all of that together in a single
> > > > > > > > implementation.
> > > > > > > >
> > > > > > > > The password server and userdata seems like a good option
> > > > > > > > for
> > > > > > > > being
> > > > > > > broken
> > > > > > > >
> > > > > > > > out and handled independently (and probably rewritten
> > > > > > > > completely
> > > > > since
> > > > > >
> > > > > > >
> > > > > > > they
> > > > > > > >
> > > > > > > > currently have some issues).
> > > > > > > >
> > > > > > > > Load balancing is another that could warrant splitting
> > > > > > > > out, but
> > > > > > > > that depends on what direction we go and how we would be
> > > > > > > > managing
> > > > it.
> > > > >
> > > > > DHCP
> > > > > >
> > > > > > >
> > > > > > > and
> > > > > > > >
> > > > > > > > DNS are others which could go either way.
> > > > > > > >
> > > > > > > > If we do split out services, I think we should
> > > > > > > > consolidate as
> > > > > > > > much as
> > > > > > we
> > > > > > >
> > > > > > > >
> > > > > > > > can into each service we break out. Ideally a network
> > > > > > > > packet
> > > > > > > > would
> > > > > > never
> > > > > > >
> > > > > > > >
> > > > > > > > hit more than one, maybe two, services. I don't think we
> > > > > > > > should
> > > > > > > > be splitting services 'just because', I think we need a
> > > > > > > > valid
> > > > > > > > case for splitting any service out because it adds
> > > > > > > > complexity.
> > > > > > > > Our project is already complex enough, we need to avoid
> > > > > > > > adding
> > > > > > > > complexity unless it
> > > > > is
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > really needed.
> > > > > > > >
> > > > > > > > Some more of my thoughts on this anyway...
> > > > > > > >
> > > > > > > > *Will STEVENS*
> > > > > > > > Lead Developer
> > > > > > > >
> > > > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > > > > cloudops.com
> > > > > > > > *|* tw @CloudOps_
> > > > > > > >
> > > > > > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sweller@e
> > > > > > > > na.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > I do agree with you that this probably isn't the right
> > > > > > > > > place the
> > > > > > > password
> > > > > > > >
> > > > > > > > >
> > > > > > > > > service and user data.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Having said that, after taking a cursory look at the
> > > > > > > > > dev docs,
> > > > > > > > > it
> > > > > > > doesn't
> > > > > > > >
> > > > > > > > >
> > > > > > > > > seem that difficult to add new daemons:
> > > > > https://opensnaproute.github.
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > io/docs/developer.html#creating-new-component
> > > > > > > > >
> > > > > > > > > <https://opensnaproute.github.io/docs/developer.html#
> > > > > > > > > creating-new-component>
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > They've definitely build it with a microservices
> > > > > > > > > architecture in
> > > > > mind,
> > > > > >
> > > > > > >
> > > > > > > so
> > > > > > > >
> > > > > > > > >
> > > > > > > > > each individual feature is abstracted into it's own
> > > > > > > > > small daemon
> > > > > > > process.
> > > > > > > >
> > > > > > > > >
> > > > > > > > > We could just create a daemon for the password server
> > > > > > > > > and the
> > > > > userdata
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > components if we really had to.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > - Si
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > ________________________________
> > > > > > > > > From: williamstevens@gmail.com <williamstevens@gmail.co
> > > > > > > > > m> on
> > > > > > > > > behalf
> > > > > > of
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Will Stevens <ws...@cloudops.com>
> > > > > > > > > Sent: Thursday, September 15, 2016 9:17 AM
> > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > > >
> > > > > > > > > A big part of why I know about it is because it is
> > > > > > > > > written in Go.
> > > > > :P
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Yes, it is definitely interesting for the routing and
> > > > > > > > > traffic
> > > > > handling
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > aspects of the VR. We will likely have to rethink some
> > > > > > > > > of the
> > > > > pieces
> > > > > >
> > > > > > a
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > little bit like the password server and userdata if we
> > > > > > > > > are to
> > > > > > > > > adopt
> > > > > a
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > different VR approach. This is where I think some of
> > > > > > > > > JohnB and
> > > > > > > Chiradeep's
> > > > > > > >
> > > > > > > > >
> > > > > > > > > ideas make sense. In many ways, it does not make sense
> > > > > > > > > for the
> > > > > device
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > handling routing and network traffic to also be
> > > > > > > > > responsible for
> > > > > > > passwords
> > > > > > > >
> > > > > > > > >
> > > > > > > > > and userdata.
> > > > > > > > >
> > > > > > > > > *Will STEVENS*
> > > > > > > > > Lead Developer
> > > > > > > > >
> > > > > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > > > > > cloudops.com
> > > > > > > > > *|* tw @CloudOps_
> > > > > > > > >
> > > > > > > > > On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sweller@
> > > > > > > > > ena.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I hadn't heard of Flexswitch until you mentioned it.
> > > > > > > > > > It looks
> > > > > pretty
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > cool!
> > > > > > > > > >
> > > > > > > > > > It even supports ONIE install.
> > > > > > > > > >
> > > > > > > > > > To be honest, the ipsec feature could be added, or we
> > > > > > > > > > could
> > > > > offload
> > > > > >
> > > > > > >
> > > > > > > it to
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > separate vm if we needed to. The fact it is so
> > > > > > > > > > feature rich
> > > > > > > > > > from a
> > > > > > > > > routing
> > > > > > > > > >
> > > > > > > > > > perspective (and all API driven) is really nice.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Based on the roadmap, it looks like they plan to also
> > > > > > > > > > support
> > > > > > > > > capabilities
> > > > > > > > > >
> > > > > > > > > > such as BGP-MPLS based L3VPN, EVPN, VPLS in the
> > > > > > > > > > future. This
> > > > > > > > > > will
> > > > > be
> > > > > >
> > > > > > >
> > > > > > > huge
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > for our carrier community that rely on these
> > > > > > > > > > technologies to do
> > > > > > > private
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > gateway and inter-VPC interconnections today. We
> > > > > > > > > > handle this
> > > > > > > > > > stuff
> > > > > > on
> > > > > > >
> > > > > > > our
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ASRs right now with a vlan interconnect into the VR.
> > > > > > > > > > Being able
> > > > > > > > > > to
> > > > > > do
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > MPLS
> > > > > > > > > >
> > > > > > > > > > all the way to the VR would be awesome.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > It also seems to be written in GO (a language here at
> > > > > > > > > > ENA we
> > > > > > > > > > know
> > > > > > very
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > well).
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > - Si
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ________________________________
> > > > > > > > > > From: Will Stevens <wi...@gmail.com>
> > > > > > > > > > Sent: Thursday, September 15, 2016 7:06 AM
> > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > Subject: RE: [DISCUSS] Replacing the VR
> > > > > > > > > >
> > > > > > > > > > Ya. I don't think it covers our whole use case, but
> > > > > > > > > > what it
> > > > > > > > > > does
> > > > > > > cover is
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > all api driven...
> > > > > > > > > >
> > > > > > > > > > On Sep 15, 2016 1:48 AM, "Marty Godsey" <marty@gonsou
> > > > > > > > > > rce.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Though I don’t see VPN in Snaproute.. Makes sense
> > > > > > > > > > > since it was
> > > > > not
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > intended to do IPSec.
> > > > > > > > > > >
> > > > > > > > > > > It seems as though VyOS is starting to look like
> > > > > > > > > > > the best
> > > > > option.
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > > Marty Godsey
> > > > > > > > > > > nSource Solutions
> > > > > > > > > > >
> > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > From: williamstevens@gmail.com
> > > > > > > > > > > [mailto:williamstevens@gmail.com
> > > > > ]
> > > > > >
> > > > > > On
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Behalf Of Will Stevens
> > > > > > > > > > > Sent: Wednesday, September 14, 2016 11:06 PM
> > > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > > > > >
> > > > > > > > > > > Or we could go completely crazy and go with
> > > > > > > > > > > something like
> > > > > > > FlexSwitch
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > from
> > > > > > > > > > >
> > > > > > > > > > > SnapRoute
> > > > > > > > > > > - http://www.snaproute.com/
> > > > > > > > > > > - https://opensnaproute.github.io/docs/apis.html
> > > > > > > > > > >
> > > > > > > > > > > *Will STEVENS*
> > > > > > > > > > > Lead Developer
> > > > > > > > > > >
> > > > > > > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > > > > > > > cloudops.com
> > > > > > *|*
> > > > > > >
> > > > > > > tw
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > @CloudOps_
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> > > > > > > wstevens@cloudops.com>
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > I tend to agree with Syed and Marty. I am not
> > > > > > > > > > > > sure what
> > > > > > problems
> > > > > > >
> > > > > > > are
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > solved by splitting up the function of the VR
> > > > > > > > > > > > into a bunch of
> > > > > > > > > separate
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > services. As Syed points out, the complexity
> > > > > > > > > > > > added is
> > > > > > > non-trivial.
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > We now have to manage all the intercontainer
> > > > > > > > > > > > networking as
> > > > > well
> > > > > >
> > > > > > as
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > orchestrated ACS networking.
> > > > > > > > > > > >
> > > > > > > > > > > > VyOS is interesting to me because it covers the
> > > > > > > > > > > > majority of
> > > > > our
> > > > > >
> > > > > > >
> > > > > > > use
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > case with a single unified control plane. It
> > > > > > > > > > > > also has good
> > > > > > > support
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > for extending features we care about, like IPv6,
> > > > > > > > > > > > VXLAN, VRRP,
> > > > > > > > > > > > transactions, etc...
> > > > > > > > > > > >
> > > > > > > > > > > > *Will STEVENS*
> > > > > > > > > > > > Lead Developer
> > > > > > > > > > > >
> > > > > > > > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > > > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > cloudops.com
> > > > > >
> > > > > > >
> > > > > > > *|*
> > > > > > > >
> > > > > > > > >
> > > > > > > > > tw
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > @CloudOps_
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> > > > > > sahmed@cloudops.com>
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Agree with Marty, adding Docker containers to
> > > > > > > > > > > > > the picture
> > > > > > > although
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > can make the VR more flexible but the added
> > > > > > > > > > > > > complexity is
> > > > > just
> > > > > >
> > > > > > >
> > > > > > > not
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > worth it. Not to mention we would need to take
> > > > > > > > > > > > > care of
> > > > > > networking
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > each container manually and given that our
> > > > > > > > > > > > > iptable rules are
> > > > > > very
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > unstable at the moment I don't see a big value
> > > > > > > > > > > > > add.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Vyos looks like a better solution to me. I know
> > > > > > > > > > > > > that it does
> > > > > > not
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > provide an api but it does fit the bill quite
> > > > > > > > > > > > > well
> > > > > otherwise. I
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > specially like the fact that it has a
> > > > > > > > > > > > > transaction based
> > > > > > > > > > > > > model
> > > > > > and
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > you
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > can rollback changes if something goes wrong.
> > > > > > > > > > > > > On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> > > > > > > marty@gonsource.com>
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Licensing aside, I think splitting the
> > > > > > > > > > > > > > various functions
> > > > > into
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > containers is not a good route either. This
> > > > > > > > > > > > > > will force
> > > > > users
> > > > > >
> > > > > > to
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > have to maintain
> > > > > > > > > > > > > and
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > use containers and adds complexity to the
> > > > > > > > > > > > > > networking
> > > > > aspects
> > > > > >
> > > > > > of
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > ACS.
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Complexity decreases stability. Now I
> > > > > > > > > > > > > > understand the
> > > > > argument
> > > > > >
> > > > > > >
> > > > > > > that
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > a monolithic approach also brings its own set
> > > > > > > > > > > > > > of issues but
> > > > > > it
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > also
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > simplifies it.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > > Marty Godsey
> > > > > > > > > > > > > > nSource Solutions
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > > > From: Chiradeep Vittal [mailto:chiradeepv@gma
> > > > > > > > > > > > > > il.com]
> > > > > > > > > > > > > > Sent: Wednesday, September 14, 2016 5:37 PM
> > > > > > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I rather doubt that the Cloudrouter will fit
> > > > > > > > > > > > > > the needs
> > > > > > > > > > > > > > of
> > > > > the
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > CloudStack project
> > > > > > > > > > > > > > - it is AGPL licensed. Many enterprises will
> > > > > > > > > > > > > > not
> > > > > > > > > > > > > > touch
> > > > > > > anything
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > has
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > AGPL
> > > > > > > > > > > > > > - the github repo shows rather infrequent
> > > > > > > > > > > > > > updates.
> > > > > > > > > > > > > > Quite
> > > > > > > likely
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > they aren't considering the use cases of the
> > > > > > > > > > > > > > CloudStack
> > > > > > > community
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I'd back John B's comments on disaggregating
> > > > > > > > > > > > > > the VR.
> > > > > > > > > > > > > > Split
> > > > > it
> > > > > >
> > > > > > >
> > > > > > > into
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > many docker containers
> > > > > > > > > > > > > > - password server
> > > > > > > > > > > > > > - userdata server
> > > > > > > > > > > > > > - DHCP / DNS
> > > > > > > > > > > > > > - s2s VPN
> > > > > > > > > > > > > > - RA VPN
> > > > > > > > > > > > > > - intra-VPC routing and ACL
> > > > > > > > > > > > > > - Port forwarding + NAT
> > > > > > > > > > > > > > - FW
> > > > > > > > > > > > > > - LB (public)
> > > > > > > > > > > > > > - LB (internal),
> > > > > > > > > > > > > > - secondary storage
> > > > > > > > > > > > > > - agent
> > > > > > > > > > > > > > Glue them together with docker compose files
> > > > > > > > > > > > > > (one per
> > > > > > > > > > > > > > use
> > > > > > > case -
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > basic zone, isolated, VPC, SSVM, etc).
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The VR image then becomes a JeOS + docker.
> > > > > > > > > > > > > > You can
> > > > > > > > > > > > > > test
> > > > > each
> > > > > >
> > > > > > of
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > components independently and fixing one bug
> > > > > > > > > > > > > > in the
> > > > > > > > > > > > > > field
> > > > > (say
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > DHCP)
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > is hitless to the other components. You don't
> > > > > > > > > > > > > > need to
> > > > > > > > > > > > > > build per-hypervisor VRs. You could even run
> > > > > > > > > > > > > > on
> > > > baremetal.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Along the way you need to figure out how to
> > > > > > > > > > > > > > - make the traffic traverse the containers
> > > > > > > > > > > > > > that are
> > > > > > > > > > > > > > needed
> > > > > > to
> > > > > > >
> > > > > > > be
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > traversed (in most cases just 1)
> > > > > > > > > > > > > > - bootstrap the router (how does it find its
> > > > > > > > > > > > > > compose
> > > > file?
> > > > >
> > > > > >
> > > > > > >
> > > > > > > where
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > is the
> > > > > > > > > > > > > > registry?)
> > > > > > > > > > > > > > - rethink the command and control of the VR
> > > > > > > > > > > > > > functions. SSH
> > > > > > > works,
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > but something more declarative, idempotent
> > > > > > > > > > > > > > should be
> > > > > > explored.
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > As you do this, it becomes clearer which of
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > functions
> > > > > can
> > > > > >
> > > > > > >
> > > > > > > be
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > substituted by for example CloudRouter.
> > > > > > > > > > > > > > Command and
> > > > > > > > > > > > > > Control
> > > > > > of
> > > > > > >
> > > > > > > the
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > docker
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > containers can be moved out to another
> > > > > > > > > > > > > > container. Etc.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, Sep 14, 2016 at 12:59 AM, Marty
> > > > > > > > > > > > > > Godsey
> > > > > > > > > > > > > > <ma...@gonsource.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > This one does look nice. My biggest concern
> > > > > > > > > > > > > > > is the
> > > > > > > > > > > > > > > lack
> > > > > of
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > VXLANs. It seems that any of the ones we
> > > > > > > > > > > > > > > mentioned
> > > > > > > > > > > > > > > do not
> > > > > > > have
> > > > > > > >
> > > > > > > > >
> > > > > > > > > an
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > API so we may be stuck at the SSH method.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > > > Marty Godsey
> > > > > > > > > > > > > > > nSource Solutions
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > > > > From: Abhinandan Prateek
> > > > > > > > > > > > > > > [mailto:abhinandan.prateek@shapeblue.com]
> > > > > > > > > > > > > > > Sent: Wednesday, September 14, 2016 2:26 AM
> > > > > > > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Cloudrouter looks promising. These have
> > > > > > > > > > > > > > > potential to
> > > > > > > > > > > > > > > save
> > > > > > > future
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > engineering effort for example on ipv6
> > > > > > > > > > > > > > > routing, OSPF
> > > > etc.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > And the best part is they come with test
> > > > > > > > > > > > > > > automation
> > > > > > > framework.
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> > > > > > > > > > > > > > > <ja...@accelerite.com>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Instead of replacing the VR in first
> > > > > > > > > > > > > > > > place we
> > > > > > > > > > > > > > > > should add VyOS/cloudrouter
> > > > > > > > > > > > > > > as provider. Once it is stable, network
> > > > > > > > > > > > > > > offerings
> > > > > > > > > > > > > > > (on
> > > > > > > upgrade)
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > can be updated to use it and we can drop
> > > > > > > > > > > > > > > the VR if
> > > > > > > > > > > > > > > we
> > > > > want
> > > > > >
> > > > > > at
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > that release
> > > > > > > > > > > > > > onwards.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > VR is stabilized over a period of time
> > > > > > > > > > > > > > > > and some of
> > > > > > > > > > > > > > > > them
> > > > > > are
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > running
> > > > > > > > > > > > > > > without issues. When we replicate the ACS
> > > > > > > > > > > > > > > VR
> > > > > > > > > > > > > > > features in
> > > > > > new
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > solution it takes some to find the missing
> > > > > > > > > > > > > > > pieces
> > > > > > > > > > > > > > > (hidden
> > > > > > > bugs).
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > Jayapal
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > On Sep 13, 2016, at 2:52 PM, Nux! <
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > nux@li.nux.ro> wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I like the idea.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Cloudrouter looks really promising, I'm
> > > > > > > > > > > > > > > > > not too
> > > > > > > > > > > > > > > > > keen
> > > > > on
> > > > > >
> > > > > > >
> > > > > > > VyOS
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > (it
> > > > > > > > > > > > > > > doesn't have a proper http api etc).
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > Sent from the Delta quadrant using Borg
> > > > > > > > > > > > > > > > > technology!
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Nux!
> > > > > > > > > > > > > > > > > www.nux.ro
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > abhinandan.prateek@shapeblue.com
> > > > > > > > > > > > > > > www.shapeblue.com<http://www.shapeblue.com>
> > > > > > > > > > > > > > > ;
> > > > > > > > > > > > > > > 53 Chandos Place, Covent Garden,
> > > > > > > > > > > > > > > London WC2N 4HSUK
> > > > > > > @shapeblue
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > ----- Original Message -----
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > From: "Will Stevens" <williamstevens@
> > > > > > > > > > > > > > > > > > gmail.com>
> > > > > > > > > > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > > > > > > > > > Sent: Monday, 12 September, 2016
> > > > > > > > > > > > > > > > > > 21:20:11
> > > > > > > > > > > > > > > > > > Subject: [DISCUSS] Replacing the VR
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > *Disclaimer:* This is a thought
> > > > > > > > > > > > > > > > > > experiment and
> > > > > > > > > > > > > > > > > > should
> > > > > > be
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > treated as
> > > > > > > > > > > > > > > such.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Please weigh in with the good and bad
> > > > > > > > > > > > > > > > > > of this
> > > > idea...
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > A couple of us have been discussing
> > > > > > > > > > > > > > > > > > the idea of
> > > > > > > potentially
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > replacing the ACS VR with the VyOS
> > > > > > > > > > > > > > > > > > [1] (Open
> > > > > > > > > > > > > > > > > > Source
> > > > > > > Vyatta
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > VM).
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > There may be a license issue because
> > > > > > > > > > > > > > > > > > I think it
> > > > > > > > > > > > > > > > > > is
> > > > > > > licensed
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > under GPL, but for the sake of
> > > > > > > > > > > > > > > > > > discussion, let's
> > > > > assume
> > > > > >
> > > > > > >
> > > > > > > we
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > can overcome any
> > > > > > > > > > > > > > > license issues.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > I have spent some time recently with
> > > > > > > > > > > > > > > > > > the VyOS
> > > > > > > > > > > > > > > > > > and I
> > > > > > have
> > > > > > >
> > > > > > > to
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > admit, I was pretty impressed. It is
> > > > > > > > > > > > > > > > > > simple and
> > > > > > > intuitive
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > and it gives you a lot more options
> > > > > > > > > > > > > > > > > > for auditing
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > configuration etc...
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Items of potential interest:
> > > > > > > > > > > > > > > > > > - Clean up our current VR script
> > > > > > > > > > > > > > > > > > spaghetti to a
> > > > > simpler
> > > > > >
> > > > > > >
> > > > > > > more
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > auditable configuration workflow.
> > > > > > > > > > > > > > > > > > - Gives a cleaner path for IPv6
> > > > > > > > > > > > > > > > > > support.
> > > > > > > > > > > > > > > > > > - Handles VPN configuration via the
> > > > > > > > > > > > > > > > > > same
> > > > > configuration
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > interface.
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > - Support for OSPF & BGP.
> > > > > > > > > > > > > > > > > > - VPN support through OpenVPN &
> > > > > > > > > > > > > > > > > > StrongSwan.
> > > > > > > > > > > > > > > > > > - Easily supports HA (redundant
> > > > > > > > > > > > > > > > > > routers) through
> > > > > VRRP.
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > - VXLAN support.
> > > > > > > > > > > > > > > > > > - Transaction based changes to the VR
> > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > rollback
> > > > > on
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > error.
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Items that could be difficult to
> > > > > > > > > > > > > > > > > > solve:
> > > > > > > > > > > > > > > > > > - Userdata password reset workflow
> > > > > > > > > > > > > > > > > > and
> > > > > implementation.
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > - Upgrade process.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > The VyOS is not the only option if we
> > > > > > > > > > > > > > > > > > were to
> > > > > consider
> > > > > >
> > > > > > >
> > > > > > > this
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > approach.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Another option, which I don't know as
> > > > > > > > > > > > > > > > > > well,
> > > > > > > > > > > > > > > > > > would be CloudRouter (AGPL
> > > > > > > > > > > > > > > > > > license) [2] which is purely API
> > > > > > > > > > > > > > > > > > driven.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Anyway, would love to hear your
> > > > > > > > > > > > > > > > > > thoughts...
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Will
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > [1] https://vyos.io/ [2]
> > > > > > > > > > > > > > > > > > https://cloudrouter.org/
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > DISCLAIMER
> > > > > > > > > > > > > > > > ==========
> > > > > > > > > > > > > > > > This e-mail may contain privileged and
> > > > > > > > > > > > > > > > confidential
> > > > > > > information
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > which is
> > > > > > > > > > > > > > > the property of Accelerite, a Persistent
> > > > > > > > > > > > > > > Systems
> > > > > business.
> > > > > >
> > > > > > >
> > > > > > > It is
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > intended only for the use of the individual
> > > > > > > > > > > > > > > or
> > > > > > > > > > > > > > > entity to
> > > > > > > which
> > > > > > > >
> > > > > > > > >
> > > > > > > > > it
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > is addressed. If you are not the intended
> > > > > > > > > > > > > > > recipient,
> > > > > > > > > > > > > > > you
> > > > > > are
> > > > > > >
> > > > > > > not
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > authorized to read, retain, copy, print,
> > > > > > > > > > > > > > > distribute
> > > > > > > > > > > > > > > or
> > > > > use
> > > > > >
> > > > > > >
> > > > > > > this
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > message. If you have received this
> > > > > > > > > > > > > > > communication in
> > > > > error,
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > please
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > notify the sender and delete all copies of
> > > > > > > > > > > > > > > this
> > > > message.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Accelerite, a Persistent Systems business
> > > > > > > > > > > > > > > does not
> > > > > > > > > > > > > > > accept
> > > > > > any
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > liability for virus
> > > > > > > > > > > > > > infected mails.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > >
>
Re: [DISCUSS] Replacing the VR
Posted by Syed Ahmed <sa...@cloudops.com>.
Hey Guys,
Will and I had a discussion in the morning on around VyOS and I have an
idea that could work, here me out.
The way Cloudstack currently communicates with VR is via the Hypervisor
(let's assume KVM and XenServer for now). The management server sends a
command as a JSON to the hypervisor which proxies them to the VR and they
get executed there. Now, what we could do is add a proxy on the hypervisor
which translates the management server JSON to VyOS commands (and
vice-versa). There is a VyOS API lib in python which we clould use. Now
this would require 0 changes on the Cloudstack core networking side. There
may be a few minor changes required to push this proxy on the hypervisor
but other than that there is nothing major. Now in the mean time, the VyOS
guys can work on their v2.0 and when they have a stable enough API we can
make it a first class citizen in Cloudstack.
This approach would not work for VMWare as in VMWare the management server
directly talks to the VR. However, this problem could be fixed by adding a
middle VM which does the proxying to-and-from the VR. This would also
improve the overall security of the system in VMWare as well.
I am not too concerned about VMWare at the moment as most of us use either
KVM or XenServer. What do you guys think of this approach?
-Syed
On Mon, Sep 19, 2016 at 11:42 AM, Stephan Seitz <
s.seitz@secretresearchfacility.com> wrote:
> Hi!
>
> Just to add my 2 cents to that thread:
>
> I'ld really like to see something like vyatta or pfsense integrated as
> "standard" VR.
>
> We'd also talked internally about replacing the VR with some more
> mature "appliance"-like router distro.
>
> pfsense e.g. comes AFAIK with no defined API but instead has a very
> nice GUI.
> How would this fit into the concept of configuring the VR via ACS?
> Would parts of the GUI - like IP configuration and basic firewall rules
> - hidden or greyed?
> Where would one save the configuration, VPN certificates and so on?
>
>
> - Stephan
>
>
> Am Sonntag, den 18.09.2016, 15:19 +0000 schrieb Marty Godsey:
> > On this note I also mentioned pfsense earlier.
> >
> > www.pfsense.org
> >
> >
> > Regards,
> > Marty Godsey
> >
> > -----Original Message-----
> > From: ilya [mailto:ilya.mailing.lists@gmail.com]
> > Sent: Sunday, September 18, 2016 1:09 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] Replacing the VR
> >
> > Our options become much better if we consider BSD based routers.
> >
> > Would that be on the table?
> >
> > https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributio
> > ns
> >
> >
> > On 9/16/16 12:04 PM, Will Stevens wrote:
> > >
> > > Ya, your points are all valid Simon. The lack of standard
> > > libraries
> > > to handle a lot of the details is a problem. I don't think it is
> > > an
> > > unsolvable problem, but if we spend the time to do that, will we
> > > have
> > > something that will work for us for the next 5 years? This may be
> > > the
> > > shortest path to getting us where we need to be for the time being.
> > >
> > > What is the best case scenario for the VR going forward which will
> > > last us the next 5 years? Maybe we just clean up what we have to
> > > do a
> > > major restructuring of the pieces and how they are
> > > implemented. We
> > > need to keep in mind how maintainable this implementation is
> > > because
> > > that is going to be key going forward IMO.
> > >
> > >
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > > tw
> > > @CloudOps_
> > >
> > > On Fri, Sep 16, 2016 at 2:29 PM, Simon Weller <sw...@ena.com>
> > > wrote:
> > >
> > > >
> > > > I think our other option is to take a real look at what it would
> > > > take
> > > > to fix the VR. In my opinion, a lot of the problems are related
> > > > to
> > > > the monolithic python code base and the fact nothing is actually
> > > > separated.
> > > >
> > > > Secondly, the python scripts (and bash scripts) don't use any
> > > > established libraries to complete tasks and instead shell out and
> > > > run
> > > > commands that are both hard to track and hard to parse on return.
> > > >
> > > >
> > > > If we daemonized this, used a real api for Agent to VR
> > > > communication,
> > > > used common already existing libraries for the system service
> > > > and
> > > > network interactions and spent a bit of time separating out code
> > > > into
> > > > distinct modules, everything would behave a lot better.
> > > >
> > > >
> > > > The pain and suffering is due to years and years of patches and
> > > > constant shelling out to complete tasks in my opinion. If we
> > > > spend
> > > > time to rethink how we interact with the VR in general and we
> > > > abstract the systems and networking stuff and use well known and
> > > > stable libraries to do the work, the VR would be much easier to
> > > > maintain.
> > > >
> > > >
> > > > - Si
> > > >
> > > >
> > > >
> > > >
> > > > ________________________________
> > > > From: Marty Godsey <ma...@gonsource.com>
> > > > Sent: Friday, September 16, 2016 12:24 PM
> > > > To: dev@cloudstack.apache.org
> > > > Subject: RE: [DISCUSS] Replacing the VR
> > > >
> > > > So based upon this discussion would it be prudent to wait on
> > > > VyOS
> > > > 2.0? The current VR is giving us issues but would the time
> > > > invested
> > > > in another "solution" be wasted especially if by the time
> > > > another
> > > > option is chose, then coded, then tested, then implemented and
> > > > right
> > > > as that time happened to be when VyOS 2.0 is released. Of course
> > > > you
> > > > said they are just in the scoping range so this could still be a
> > > > year or more out.
> > > >
> > > > Thoughts?
> > > >
> > > > Regards,
> > > > Marty Godsey
> > > > nSource Solutions
> > > >
> > > > -----Original Message-----
> > > > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com]
> > > > On
> > > > Behalf Of Will Stevens
> > > > Sent: Friday, September 16, 2016 10:31 AM
> > > > To: dev@cloudstack.apache.org
> > > > Cc: daniil@baturin.org
> > > > Subject: Re: [DISCUSS] Replacing the VR
> > > >
> > > > I just had a quick chat with a couple of the guys over on the
> > > > VyOS chat.
> > > > I have CC'ed one of them in case we have more licensing
> > > > questions.
> > > >
> > > > So here is the status with the license "the code inherited from
> > > > Vyatta and our modifications from it is GPLv2 (strict, not v2+).
> > > > The
> > > > config reading library is GPLv2 too, so anything that links to is
> > > > is GPLv2.
> > > > Some auxiliary components we made after the fork are more
> > > > permissive,
> > > > LGPLv2+ or MIT."
> > > >
> > > > They are currently in the process of scoping a redesign (VyOS
> > > > 2.0),
> > > > "we are planning a clean rewrite that will solve issues of the
> > > > current config system".
> > > > This will include the ability to configure via the API.
> > > >
> > > > If we have more questions for VyOS, they are very friendly and
> > > > responsive, so we should be able to get answers.
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > *CloudOps* *| *Cloud Solutions Experts
> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> > > > *|* tw
> > > > @CloudOps_
> > > >
> > > > On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com>
> > > > wrote:
> > > >
> > > > >
> > > > > I agree with Will Ilya. There are so many problems with the VR
> > > > > right now.
> > > > > Most of the outages we've had recently have somehow involved
> > > > > the VR.
> > > > > We set custom iptables rules on the VR which can and have
> > > > > easily
> > > > > gone
> > > > wrong.
> > > > >
> > > > > Openswan is broken, Strongswan replacement still needs to be
> > > > > tested.
> > > > > VVRP with redundant router still needs work, and not to mention
> > > > > the
> > > > > problems we will have when we introduce IPv6 into the whole
> > > > > picture.
> > > > >
> > > > > I think the spirit of the discussion is to rely on a 3rd party
> > > > > to do
> > > > > the networking for us (eg VyOS) and have us handle just the
> > > > > orchestration. All the problems that I've described have
> > > > > already
> > > > > been solved in VyOS. We also get the advantage of a potential
> > > > > wider
> > > > > community to fix and maintain the VR and given our current
> > > > > development velocity, it think it totally makes sense to look
> > > > > for a 3rd party option.
> > > > >
> > > > > -Syed
> > > > >
> > > > >
> > > > > On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens
> > > > > <ws...@cloudops.com>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > The VR has been biting us far too often recently, which is
> > > > > > why we
> > > > > > have started looking into alternative implementations.
> > > > > >
> > > > > > One of the things that is nice about potentially using the
> > > > > > VyOS is
> > > > > > that
> > > > > it
> > > > > >
> > > > > > is based on Debian, so we should be able to run the other
> > > > > > services
> > > > > > that
> > > > > we
> > > > > >
> > > > > > currently have like the password server and userdata on the
> > > > > > VyOS.
> > > > > > This means we would not have to change our architecture
> > > > > > initially
> > > > > > and could focus on only replacing the networking paths.
> > > > > >
> > > > > > *Will STEVENS*
> > > > > > Lead Developer
> > > > > >
> > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > > cloudops.com *|*
> > > > > > tw @CloudOps_
> > > > > >
> > > > > > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> > > > > >
> > > > > > >
> > > > > > > The more this is discussed the more I think we should stick
> > > > > > > with
> > > > > > > our
> > > > > VR.
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > All these other options either seem unfinished or with
> > > > > > > incompatible license.
> > > > > > >
> > > > > > > VyOS looks the most promising so far, it's a serious,
> > > > > > > mature project.
> > > > > > > Adopting it though means we'll have to microservice our way
> > > > > > > out of
> > > > > > > it
> > > > > > with
> > > > > > >
> > > > > > > extra machines for DNS/USERDATA/etc, unless we can make
> > > > > > > VyOS serve
> > > > > those
> > > > > >
> > > > > > >
> > > > > > > too. Imho this adds complexity we should void.
> > > > > > >
> > > > > > > --
> > > > > > > Sent from the Delta quadrant using Borg technology!
> > > > > > >
> > > > > > > Nux!
> > > > > > > www.nux.ro
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > > >
> > > > > > > > From: "Will Stevens" <ws...@cloudops.com>
> > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > Sent: Thursday, 15 September, 2016 17:21:28
> > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > >
> > > > > > > > Ya, we would need to add a daemon for VPN as well. Load
> > > > > > > > balancing is another aspect which we will need to
> > > > > > > > consider if we
> > > > went this route.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Something like https://traefik.io/ could potentially be a
> > > > > > > > good
> > > > > > > > fit
> > > > > due
> > > > > >
> > > > > > >
> > > > > > > to
> > > > > > > >
> > > > > > > > its API driven configuration, but it may be more than
> > > > > > > > what we need.
> > > > > > > >
> > > > > > > > We should probably try define which pieces make sense to
> > > > > > > > be
> > > > > > > > solved
> > > > > > > together
> > > > > > > >
> > > > > > > > and which pieces would be best suited to be broken out.
> > > > > > > >
> > > > > > > > I think the network connectivity, routing and firewalling
> > > > > > > > should
> > > > > > probably
> > > > > > >
> > > > > > > >
> > > > > > > > all stay together since the majority of the tools we
> > > > > > > > would
> > > > > potentially
> > > > > >
> > > > > > >
> > > > > > > use
> > > > > > > >
> > > > > > > > would handle all of that together in a single
> > > > > > > > implementation.
> > > > > > > >
> > > > > > > > The password server and userdata seems like a good option
> > > > > > > > for
> > > > > > > > being
> > > > > > > broken
> > > > > > > >
> > > > > > > > out and handled independently (and probably rewritten
> > > > > > > > completely
> > > > > since
> > > > > >
> > > > > > >
> > > > > > > they
> > > > > > > >
> > > > > > > > currently have some issues).
> > > > > > > >
> > > > > > > > Load balancing is another that could warrant splitting
> > > > > > > > out, but
> > > > > > > > that depends on what direction we go and how we would be
> > > > > > > > managing
> > > > it.
> > > > >
> > > > > DHCP
> > > > > >
> > > > > > >
> > > > > > > and
> > > > > > > >
> > > > > > > > DNS are others which could go either way.
> > > > > > > >
> > > > > > > > If we do split out services, I think we should
> > > > > > > > consolidate as
> > > > > > > > much as
> > > > > > we
> > > > > > >
> > > > > > > >
> > > > > > > > can into each service we break out. Ideally a network
> > > > > > > > packet
> > > > > > > > would
> > > > > > never
> > > > > > >
> > > > > > > >
> > > > > > > > hit more than one, maybe two, services. I don't think we
> > > > > > > > should
> > > > > > > > be splitting services 'just because', I think we need a
> > > > > > > > valid
> > > > > > > > case for splitting any service out because it adds
> > > > > > > > complexity.
> > > > > > > > Our project is already complex enough, we need to avoid
> > > > > > > > adding
> > > > > > > > complexity unless it
> > > > > is
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > really needed.
> > > > > > > >
> > > > > > > > Some more of my thoughts on this anyway...
> > > > > > > >
> > > > > > > > *Will STEVENS*
> > > > > > > > Lead Developer
> > > > > > > >
> > > > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > > > > cloudops.com
> > > > > > > > *|* tw @CloudOps_
> > > > > > > >
> > > > > > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sweller@e
> > > > > > > > na.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > I do agree with you that this probably isn't the right
> > > > > > > > > place the
> > > > > > > password
> > > > > > > >
> > > > > > > > >
> > > > > > > > > service and user data.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Having said that, after taking a cursory look at the
> > > > > > > > > dev docs,
> > > > > > > > > it
> > > > > > > doesn't
> > > > > > > >
> > > > > > > > >
> > > > > > > > > seem that difficult to add new daemons:
> > > > > https://opensnaproute.github.
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > io/docs/developer.html#creating-new-component
> > > > > > > > >
> > > > > > > > > <https://opensnaproute.github.io/docs/developer.html#
> > > > > > > > > creating-new-component>
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > They've definitely build it with a microservices
> > > > > > > > > architecture in
> > > > > mind,
> > > > > >
> > > > > > >
> > > > > > > so
> > > > > > > >
> > > > > > > > >
> > > > > > > > > each individual feature is abstracted into it's own
> > > > > > > > > small daemon
> > > > > > > process.
> > > > > > > >
> > > > > > > > >
> > > > > > > > > We could just create a daemon for the password server
> > > > > > > > > and the
> > > > > userdata
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > components if we really had to.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > - Si
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > ________________________________
> > > > > > > > > From: williamstevens@gmail.com <williamstevens@gmail.co
> > > > > > > > > m> on
> > > > > > > > > behalf
> > > > > > of
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Will Stevens <ws...@cloudops.com>
> > > > > > > > > Sent: Thursday, September 15, 2016 9:17 AM
> > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > > >
> > > > > > > > > A big part of why I know about it is because it is
> > > > > > > > > written in Go.
> > > > > :P
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Yes, it is definitely interesting for the routing and
> > > > > > > > > traffic
> > > > > handling
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > aspects of the VR. We will likely have to rethink some
> > > > > > > > > of the
> > > > > pieces
> > > > > >
> > > > > > a
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > little bit like the password server and userdata if we
> > > > > > > > > are to
> > > > > > > > > adopt
> > > > > a
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > different VR approach. This is where I think some of
> > > > > > > > > JohnB and
> > > > > > > Chiradeep's
> > > > > > > >
> > > > > > > > >
> > > > > > > > > ideas make sense. In many ways, it does not make sense
> > > > > > > > > for the
> > > > > device
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > handling routing and network traffic to also be
> > > > > > > > > responsible for
> > > > > > > passwords
> > > > > > > >
> > > > > > > > >
> > > > > > > > > and userdata.
> > > > > > > > >
> > > > > > > > > *Will STEVENS*
> > > > > > > > > Lead Developer
> > > > > > > > >
> > > > > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > > > > > cloudops.com
> > > > > > > > > *|* tw @CloudOps_
> > > > > > > > >
> > > > > > > > > On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sweller@
> > > > > > > > > ena.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I hadn't heard of Flexswitch until you mentioned it.
> > > > > > > > > > It looks
> > > > > pretty
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > cool!
> > > > > > > > > >
> > > > > > > > > > It even supports ONIE install.
> > > > > > > > > >
> > > > > > > > > > To be honest, the ipsec feature could be added, or we
> > > > > > > > > > could
> > > > > offload
> > > > > >
> > > > > > >
> > > > > > > it to
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > separate vm if we needed to. The fact it is so
> > > > > > > > > > feature rich
> > > > > > > > > > from a
> > > > > > > > > routing
> > > > > > > > > >
> > > > > > > > > > perspective (and all API driven) is really nice.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Based on the roadmap, it looks like they plan to also
> > > > > > > > > > support
> > > > > > > > > capabilities
> > > > > > > > > >
> > > > > > > > > > such as BGP-MPLS based L3VPN, EVPN, VPLS in the
> > > > > > > > > > future. This
> > > > > > > > > > will
> > > > > be
> > > > > >
> > > > > > >
> > > > > > > huge
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > for our carrier community that rely on these
> > > > > > > > > > technologies to do
> > > > > > > private
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > gateway and inter-VPC interconnections today. We
> > > > > > > > > > handle this
> > > > > > > > > > stuff
> > > > > > on
> > > > > > >
> > > > > > > our
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ASRs right now with a vlan interconnect into the VR.
> > > > > > > > > > Being able
> > > > > > > > > > to
> > > > > > do
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > MPLS
> > > > > > > > > >
> > > > > > > > > > all the way to the VR would be awesome.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > It also seems to be written in GO (a language here at
> > > > > > > > > > ENA we
> > > > > > > > > > know
> > > > > > very
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > well).
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > - Si
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ________________________________
> > > > > > > > > > From: Will Stevens <wi...@gmail.com>
> > > > > > > > > > Sent: Thursday, September 15, 2016 7:06 AM
> > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > Subject: RE: [DISCUSS] Replacing the VR
> > > > > > > > > >
> > > > > > > > > > Ya. I don't think it covers our whole use case, but
> > > > > > > > > > what it
> > > > > > > > > > does
> > > > > > > cover is
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > all api driven...
> > > > > > > > > >
> > > > > > > > > > On Sep 15, 2016 1:48 AM, "Marty Godsey" <marty@gonsou
> > > > > > > > > > rce.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Though I don’t see VPN in Snaproute.. Makes sense
> > > > > > > > > > > since it was
> > > > > not
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > intended to do IPSec.
> > > > > > > > > > >
> > > > > > > > > > > It seems as though VyOS is starting to look like
> > > > > > > > > > > the best
> > > > > option.
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > > Marty Godsey
> > > > > > > > > > > nSource Solutions
> > > > > > > > > > >
> > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > From: williamstevens@gmail.com
> > > > > > > > > > > [mailto:williamstevens@gmail.com
> > > > > ]
> > > > > >
> > > > > > On
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Behalf Of Will Stevens
> > > > > > > > > > > Sent: Wednesday, September 14, 2016 11:06 PM
> > > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > > > > >
> > > > > > > > > > > Or we could go completely crazy and go with
> > > > > > > > > > > something like
> > > > > > > FlexSwitch
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > from
> > > > > > > > > > >
> > > > > > > > > > > SnapRoute
> > > > > > > > > > > - http://www.snaproute.com/
> > > > > > > > > > > - https://opensnaproute.github.io/docs/apis.html
> > > > > > > > > > >
> > > > > > > > > > > *Will STEVENS*
> > > > > > > > > > > Lead Developer
> > > > > > > > > > >
> > > > > > > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > > > > > > > cloudops.com
> > > > > > *|*
> > > > > > >
> > > > > > > tw
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > @CloudOps_
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> > > > > > > wstevens@cloudops.com>
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > I tend to agree with Syed and Marty. I am not
> > > > > > > > > > > > sure what
> > > > > > problems
> > > > > > >
> > > > > > > are
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > solved by splitting up the function of the VR
> > > > > > > > > > > > into a bunch of
> > > > > > > > > separate
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > services. As Syed points out, the complexity
> > > > > > > > > > > > added is
> > > > > > > non-trivial.
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > We now have to manage all the intercontainer
> > > > > > > > > > > > networking as
> > > > > well
> > > > > >
> > > > > > as
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > orchestrated ACS networking.
> > > > > > > > > > > >
> > > > > > > > > > > > VyOS is interesting to me because it covers the
> > > > > > > > > > > > majority of
> > > > > our
> > > > > >
> > > > > > >
> > > > > > > use
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > case with a single unified control plane. It
> > > > > > > > > > > > also has good
> > > > > > > support
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > for extending features we care about, like IPv6,
> > > > > > > > > > > > VXLAN, VRRP,
> > > > > > > > > > > > transactions, etc...
> > > > > > > > > > > >
> > > > > > > > > > > > *Will STEVENS*
> > > > > > > > > > > > Lead Developer
> > > > > > > > > > > >
> > > > > > > > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > > > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > cloudops.com
> > > > > >
> > > > > > >
> > > > > > > *|*
> > > > > > > >
> > > > > > > > >
> > > > > > > > > tw
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > @CloudOps_
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> > > > > > sahmed@cloudops.com>
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Agree with Marty, adding Docker containers to
> > > > > > > > > > > > > the picture
> > > > > > > although
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > can make the VR more flexible but the added
> > > > > > > > > > > > > complexity is
> > > > > just
> > > > > >
> > > > > > >
> > > > > > > not
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > worth it. Not to mention we would need to take
> > > > > > > > > > > > > care of
> > > > > > networking
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > each container manually and given that our
> > > > > > > > > > > > > iptable rules are
> > > > > > very
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > unstable at the moment I don't see a big value
> > > > > > > > > > > > > add.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Vyos looks like a better solution to me. I know
> > > > > > > > > > > > > that it does
> > > > > > not
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > provide an api but it does fit the bill quite
> > > > > > > > > > > > > well
> > > > > otherwise. I
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > specially like the fact that it has a
> > > > > > > > > > > > > transaction based
> > > > > > > > > > > > > model
> > > > > > and
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > you
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > can rollback changes if something goes wrong.
> > > > > > > > > > > > > On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> > > > > > > marty@gonsource.com>
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Licensing aside, I think splitting the
> > > > > > > > > > > > > > various functions
> > > > > into
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > containers is not a good route either. This
> > > > > > > > > > > > > > will force
> > > > > users
> > > > > >
> > > > > > to
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > have to maintain
> > > > > > > > > > > > > and
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > use containers and adds complexity to the
> > > > > > > > > > > > > > networking
> > > > > aspects
> > > > > >
> > > > > > of
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > ACS.
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Complexity decreases stability. Now I
> > > > > > > > > > > > > > understand the
> > > > > argument
> > > > > >
> > > > > > >
> > > > > > > that
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > a monolithic approach also brings its own set
> > > > > > > > > > > > > > of issues but
> > > > > > it
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > also
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > simplifies it.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > > Marty Godsey
> > > > > > > > > > > > > > nSource Solutions
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > > > From: Chiradeep Vittal [mailto:chiradeepv@gma
> > > > > > > > > > > > > > il.com]
> > > > > > > > > > > > > > Sent: Wednesday, September 14, 2016 5:37 PM
> > > > > > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I rather doubt that the Cloudrouter will fit
> > > > > > > > > > > > > > the needs
> > > > > > > > > > > > > > of
> > > > > the
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > CloudStack project
> > > > > > > > > > > > > > - it is AGPL licensed. Many enterprises will
> > > > > > > > > > > > > > not
> > > > > > > > > > > > > > touch
> > > > > > > anything
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > has
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > AGPL
> > > > > > > > > > > > > > - the github repo shows rather infrequent
> > > > > > > > > > > > > > updates.
> > > > > > > > > > > > > > Quite
> > > > > > > likely
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > they aren't considering the use cases of the
> > > > > > > > > > > > > > CloudStack
> > > > > > > community
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I'd back John B's comments on disaggregating
> > > > > > > > > > > > > > the VR.
> > > > > > > > > > > > > > Split
> > > > > it
> > > > > >
> > > > > > >
> > > > > > > into
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > many docker containers
> > > > > > > > > > > > > > - password server
> > > > > > > > > > > > > > - userdata server
> > > > > > > > > > > > > > - DHCP / DNS
> > > > > > > > > > > > > > - s2s VPN
> > > > > > > > > > > > > > - RA VPN
> > > > > > > > > > > > > > - intra-VPC routing and ACL
> > > > > > > > > > > > > > - Port forwarding + NAT
> > > > > > > > > > > > > > - FW
> > > > > > > > > > > > > > - LB (public)
> > > > > > > > > > > > > > - LB (internal),
> > > > > > > > > > > > > > - secondary storage
> > > > > > > > > > > > > > - agent
> > > > > > > > > > > > > > Glue them together with docker compose files
> > > > > > > > > > > > > > (one per
> > > > > > > > > > > > > > use
> > > > > > > case -
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > basic zone, isolated, VPC, SSVM, etc).
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The VR image then becomes a JeOS + docker.
> > > > > > > > > > > > > > You can
> > > > > > > > > > > > > > test
> > > > > each
> > > > > >
> > > > > > of
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > components independently and fixing one bug
> > > > > > > > > > > > > > in the
> > > > > > > > > > > > > > field
> > > > > (say
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > DHCP)
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > is hitless to the other components. You don't
> > > > > > > > > > > > > > need to
> > > > > > > > > > > > > > build per-hypervisor VRs. You could even run
> > > > > > > > > > > > > > on
> > > > baremetal.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Along the way you need to figure out how to
> > > > > > > > > > > > > > - make the traffic traverse the containers
> > > > > > > > > > > > > > that are
> > > > > > > > > > > > > > needed
> > > > > > to
> > > > > > >
> > > > > > > be
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > traversed (in most cases just 1)
> > > > > > > > > > > > > > - bootstrap the router (how does it find its
> > > > > > > > > > > > > > compose
> > > > file?
> > > > >
> > > > > >
> > > > > > >
> > > > > > > where
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > is the
> > > > > > > > > > > > > > registry?)
> > > > > > > > > > > > > > - rethink the command and control of the VR
> > > > > > > > > > > > > > functions. SSH
> > > > > > > works,
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > but something more declarative, idempotent
> > > > > > > > > > > > > > should be
> > > > > > explored.
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > As you do this, it becomes clearer which of
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > functions
> > > > > can
> > > > > >
> > > > > > >
> > > > > > > be
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > substituted by for example CloudRouter.
> > > > > > > > > > > > > > Command and
> > > > > > > > > > > > > > Control
> > > > > > of
> > > > > > >
> > > > > > > the
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > docker
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > containers can be moved out to another
> > > > > > > > > > > > > > container. Etc.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, Sep 14, 2016 at 12:59 AM, Marty
> > > > > > > > > > > > > > Godsey
> > > > > > > > > > > > > > <ma...@gonsource.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > This one does look nice. My biggest concern
> > > > > > > > > > > > > > > is the
> > > > > > > > > > > > > > > lack
> > > > > of
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > VXLANs. It seems that any of the ones we
> > > > > > > > > > > > > > > mentioned
> > > > > > > > > > > > > > > do not
> > > > > > > have
> > > > > > > >
> > > > > > > > >
> > > > > > > > > an
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > API so we may be stuck at the SSH method.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > > > Marty Godsey
> > > > > > > > > > > > > > > nSource Solutions
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > > > > From: Abhinandan Prateek
> > > > > > > > > > > > > > > [mailto:abhinandan.prateek@shapeblue.com]
> > > > > > > > > > > > > > > Sent: Wednesday, September 14, 2016 2:26 AM
> > > > > > > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Cloudrouter looks promising. These have
> > > > > > > > > > > > > > > potential to
> > > > > > > > > > > > > > > save
> > > > > > > future
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > engineering effort for example on ipv6
> > > > > > > > > > > > > > > routing, OSPF
> > > > etc.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > And the best part is they come with test
> > > > > > > > > > > > > > > automation
> > > > > > > framework.
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> > > > > > > > > > > > > > > <ja...@accelerite.com>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Instead of replacing the VR in first
> > > > > > > > > > > > > > > > place we
> > > > > > > > > > > > > > > > should add VyOS/cloudrouter
> > > > > > > > > > > > > > > as provider. Once it is stable, network
> > > > > > > > > > > > > > > offerings
> > > > > > > > > > > > > > > (on
> > > > > > > upgrade)
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > can be updated to use it and we can drop
> > > > > > > > > > > > > > > the VR if
> > > > > > > > > > > > > > > we
> > > > > want
> > > > > >
> > > > > > at
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > that release
> > > > > > > > > > > > > > onwards.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > VR is stabilized over a period of time
> > > > > > > > > > > > > > > > and some of
> > > > > > > > > > > > > > > > them
> > > > > > are
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > running
> > > > > > > > > > > > > > > without issues. When we replicate the ACS
> > > > > > > > > > > > > > > VR
> > > > > > > > > > > > > > > features in
> > > > > > new
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > solution it takes some to find the missing
> > > > > > > > > > > > > > > pieces
> > > > > > > > > > > > > > > (hidden
> > > > > > > bugs).
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > Jayapal
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > On Sep 13, 2016, at 2:52 PM, Nux! <
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > nux@li.nux.ro> wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I like the idea.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Cloudrouter looks really promising, I'm
> > > > > > > > > > > > > > > > > not too
> > > > > > > > > > > > > > > > > keen
> > > > > on
> > > > > >
> > > > > > >
> > > > > > > VyOS
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > (it
> > > > > > > > > > > > > > > doesn't have a proper http api etc).
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > Sent from the Delta quadrant using Borg
> > > > > > > > > > > > > > > > > technology!
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Nux!
> > > > > > > > > > > > > > > > > www.nux.ro
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > abhinandan.prateek@shapeblue.com
> > > > > > > > > > > > > > > www.shapeblue.com<http://www.shapeblue.com>
> > > > > > > > > > > > > > > ;
> > > > > > > > > > > > > > > 53 Chandos Place, Covent Garden,
> > > > > > > > > > > > > > > London WC2N 4HSUK
> > > > > > > @shapeblue
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > ----- Original Message -----
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > From: "Will Stevens" <williamstevens@
> > > > > > > > > > > > > > > > > > gmail.com>
> > > > > > > > > > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > > > > > > > > > Sent: Monday, 12 September, 2016
> > > > > > > > > > > > > > > > > > 21:20:11
> > > > > > > > > > > > > > > > > > Subject: [DISCUSS] Replacing the VR
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > *Disclaimer:* This is a thought
> > > > > > > > > > > > > > > > > > experiment and
> > > > > > > > > > > > > > > > > > should
> > > > > > be
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > treated as
> > > > > > > > > > > > > > > such.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Please weigh in with the good and bad
> > > > > > > > > > > > > > > > > > of this
> > > > idea...
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > A couple of us have been discussing
> > > > > > > > > > > > > > > > > > the idea of
> > > > > > > potentially
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > replacing the ACS VR with the VyOS
> > > > > > > > > > > > > > > > > > [1] (Open
> > > > > > > > > > > > > > > > > > Source
> > > > > > > Vyatta
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > VM).
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > There may be a license issue because
> > > > > > > > > > > > > > > > > > I think it
> > > > > > > > > > > > > > > > > > is
> > > > > > > licensed
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > under GPL, but for the sake of
> > > > > > > > > > > > > > > > > > discussion, let's
> > > > > assume
> > > > > >
> > > > > > >
> > > > > > > we
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > can overcome any
> > > > > > > > > > > > > > > license issues.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > I have spent some time recently with
> > > > > > > > > > > > > > > > > > the VyOS
> > > > > > > > > > > > > > > > > > and I
> > > > > > have
> > > > > > >
> > > > > > > to
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > admit, I was pretty impressed. It is
> > > > > > > > > > > > > > > > > > simple and
> > > > > > > intuitive
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > and it gives you a lot more options
> > > > > > > > > > > > > > > > > > for auditing
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > configuration etc...
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Items of potential interest:
> > > > > > > > > > > > > > > > > > - Clean up our current VR script
> > > > > > > > > > > > > > > > > > spaghetti to a
> > > > > simpler
> > > > > >
> > > > > > >
> > > > > > > more
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > auditable configuration workflow.
> > > > > > > > > > > > > > > > > > - Gives a cleaner path for IPv6
> > > > > > > > > > > > > > > > > > support.
> > > > > > > > > > > > > > > > > > - Handles VPN configuration via the
> > > > > > > > > > > > > > > > > > same
> > > > > configuration
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > interface.
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > - Support for OSPF & BGP.
> > > > > > > > > > > > > > > > > > - VPN support through OpenVPN &
> > > > > > > > > > > > > > > > > > StrongSwan.
> > > > > > > > > > > > > > > > > > - Easily supports HA (redundant
> > > > > > > > > > > > > > > > > > routers) through
> > > > > VRRP.
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > - VXLAN support.
> > > > > > > > > > > > > > > > > > - Transaction based changes to the VR
> > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > rollback
> > > > > on
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > error.
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Items that could be difficult to
> > > > > > > > > > > > > > > > > > solve:
> > > > > > > > > > > > > > > > > > - Userdata password reset workflow
> > > > > > > > > > > > > > > > > > and
> > > > > implementation.
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > - Upgrade process.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > The VyOS is not the only option if we
> > > > > > > > > > > > > > > > > > were to
> > > > > consider
> > > > > >
> > > > > > >
> > > > > > > this
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > approach.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Another option, which I don't know as
> > > > > > > > > > > > > > > > > > well,
> > > > > > > > > > > > > > > > > > would be CloudRouter (AGPL
> > > > > > > > > > > > > > > > > > license) [2] which is purely API
> > > > > > > > > > > > > > > > > > driven.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Anyway, would love to hear your
> > > > > > > > > > > > > > > > > > thoughts...
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Will
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > [1] https://vyos.io/ [2]
> > > > > > > > > > > > > > > > > > https://cloudrouter.org/
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > DISCLAIMER
> > > > > > > > > > > > > > > > ==========
> > > > > > > > > > > > > > > > This e-mail may contain privileged and
> > > > > > > > > > > > > > > > confidential
> > > > > > > information
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > which is
> > > > > > > > > > > > > > > the property of Accelerite, a Persistent
> > > > > > > > > > > > > > > Systems
> > > > > business.
> > > > > >
> > > > > > >
> > > > > > > It is
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > intended only for the use of the individual
> > > > > > > > > > > > > > > or
> > > > > > > > > > > > > > > entity to
> > > > > > > which
> > > > > > > >
> > > > > > > > >
> > > > > > > > > it
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > is addressed. If you are not the intended
> > > > > > > > > > > > > > > recipient,
> > > > > > > > > > > > > > > you
> > > > > > are
> > > > > > >
> > > > > > > not
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > authorized to read, retain, copy, print,
> > > > > > > > > > > > > > > distribute
> > > > > > > > > > > > > > > or
> > > > > use
> > > > > >
> > > > > > >
> > > > > > > this
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > message. If you have received this
> > > > > > > > > > > > > > > communication in
> > > > > error,
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > please
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > notify the sender and delete all copies of
> > > > > > > > > > > > > > > this
> > > > message.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Accelerite, a Persistent Systems business
> > > > > > > > > > > > > > > does not
> > > > > > > > > > > > > > > accept
> > > > > > any
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > liability for virus
> > > > > > > > > > > > > > infected mails.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > >
>
Re: [DISCUSS] Replacing the VR
Posted by Stephan Seitz <s....@secretresearchfacility.com>.
Hi!
Just to add my 2 cents to that thread:
I'ld really like to see something like vyatta or pfsense integrated as
"standard" VR.
We'd also talked internally about replacing the VR with some more
mature "appliance"-like router distro.
pfsense e.g. comes AFAIK with no defined API but instead has a very
nice GUI.
How would this fit into the concept of configuring the VR via ACS?
Would parts of the GUI - like IP configuration and basic firewall rules
- hidden or greyed?
Where would one save the configuration, VPN certificates and so on?
- Stephan
Am Sonntag, den 18.09.2016, 15:19 +0000 schrieb Marty Godsey:
> On this note I also mentioned pfsense earlier.
>
> www.pfsense.org
>
>
> Regards,
> Marty Godsey
>
> -----Original Message-----
> From: ilya [mailto:ilya.mailing.lists@gmail.com]
> Sent: Sunday, September 18, 2016 1:09 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Our options become much better if we consider BSD based routers.
>
> Would that be on the table?
>
> https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributio
> ns
>
>
> On 9/16/16 12:04 PM, Will Stevens wrote:
> >
> > Ya, your points are all valid Simon. The lack of standard
> > libraries
> > to handle a lot of the details is a problem. I don't think it is
> > an
> > unsolvable problem, but if we spend the time to do that, will we
> > have
> > something that will work for us for the next 5 years? This may be
> > the
> > shortest path to getting us where we need to be for the time being.
> >
> > What is the best case scenario for the VR going forward which will
> > last us the next 5 years? Maybe we just clean up what we have to
> > do a
> > major restructuring of the pieces and how they are
> > implemented. We
> > need to keep in mind how maintainable this implementation is
> > because
> > that is going to be key going forward IMO.
> >
> >
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > tw
> > @CloudOps_
> >
> > On Fri, Sep 16, 2016 at 2:29 PM, Simon Weller <sw...@ena.com>
> > wrote:
> >
> > >
> > > I think our other option is to take a real look at what it would
> > > take
> > > to fix the VR. In my opinion, a lot of the problems are related
> > > to
> > > the monolithic python code base and the fact nothing is actually
> > > separated.
> > >
> > > Secondly, the python scripts (and bash scripts) don't use any
> > > established libraries to complete tasks and instead shell out and
> > > run
> > > commands that are both hard to track and hard to parse on return.
> > >
> > >
> > > If we daemonized this, used a real api for Agent to VR
> > > communication,
> > > used common already existing libraries for the system service
> > > and
> > > network interactions and spent a bit of time separating out code
> > > into
> > > distinct modules, everything would behave a lot better.
> > >
> > >
> > > The pain and suffering is due to years and years of patches and
> > > constant shelling out to complete tasks in my opinion. If we
> > > spend
> > > time to rethink how we interact with the VR in general and we
> > > abstract the systems and networking stuff and use well known and
> > > stable libraries to do the work, the VR would be much easier to
> > > maintain.
> > >
> > >
> > > - Si
> > >
> > >
> > >
> > >
> > > ________________________________
> > > From: Marty Godsey <ma...@gonsource.com>
> > > Sent: Friday, September 16, 2016 12:24 PM
> > > To: dev@cloudstack.apache.org
> > > Subject: RE: [DISCUSS] Replacing the VR
> > >
> > > So based upon this discussion would it be prudent to wait on
> > > VyOS
> > > 2.0? The current VR is giving us issues but would the time
> > > invested
> > > in another "solution" be wasted especially if by the time
> > > another
> > > option is chose, then coded, then tested, then implemented and
> > > right
> > > as that time happened to be when VyOS 2.0 is released. Of course
> > > you
> > > said they are just in the scoping range so this could still be a
> > > year or more out.
> > >
> > > Thoughts?
> > >
> > > Regards,
> > > Marty Godsey
> > > nSource Solutions
> > >
> > > -----Original Message-----
> > > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com]
> > > On
> > > Behalf Of Will Stevens
> > > Sent: Friday, September 16, 2016 10:31 AM
> > > To: dev@cloudstack.apache.org
> > > Cc: daniil@baturin.org
> > > Subject: Re: [DISCUSS] Replacing the VR
> > >
> > > I just had a quick chat with a couple of the guys over on the
> > > VyOS chat.
> > > I have CC'ed one of them in case we have more licensing
> > > questions.
> > >
> > > So here is the status with the license "the code inherited from
> > > Vyatta and our modifications from it is GPLv2 (strict, not v2+).
> > > The
> > > config reading library is GPLv2 too, so anything that links to is
> > > is GPLv2.
> > > Some auxiliary components we made after the fork are more
> > > permissive,
> > > LGPLv2+ or MIT."
> > >
> > > They are currently in the process of scoping a redesign (VyOS
> > > 2.0),
> > > "we are planning a clean rewrite that will solve issues of the
> > > current config system".
> > > This will include the ability to configure via the API.
> > >
> > > If we have more questions for VyOS, they are very friendly and
> > > responsive, so we should be able to get answers.
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> > > *|* tw
> > > @CloudOps_
> > >
> > > On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com>
> > > wrote:
> > >
> > > >
> > > > I agree with Will Ilya. There are so many problems with the VR
> > > > right now.
> > > > Most of the outages we've had recently have somehow involved
> > > > the VR.
> > > > We set custom iptables rules on the VR which can and have
> > > > easily
> > > > gone
> > > wrong.
> > > >
> > > > Openswan is broken, Strongswan replacement still needs to be
> > > > tested.
> > > > VVRP with redundant router still needs work, and not to mention
> > > > the
> > > > problems we will have when we introduce IPv6 into the whole
> > > > picture.
> > > >
> > > > I think the spirit of the discussion is to rely on a 3rd party
> > > > to do
> > > > the networking for us (eg VyOS) and have us handle just the
> > > > orchestration. All the problems that I've described have
> > > > already
> > > > been solved in VyOS. We also get the advantage of a potential
> > > > wider
> > > > community to fix and maintain the VR and given our current
> > > > development velocity, it think it totally makes sense to look
> > > > for a 3rd party option.
> > > >
> > > > -Syed
> > > >
> > > >
> > > > On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens
> > > > <ws...@cloudops.com>
> > > > wrote:
> > > >
> > > > >
> > > > > The VR has been biting us far too often recently, which is
> > > > > why we
> > > > > have started looking into alternative implementations.
> > > > >
> > > > > One of the things that is nice about potentially using the
> > > > > VyOS is
> > > > > that
> > > > it
> > > > >
> > > > > is based on Debian, so we should be able to run the other
> > > > > services
> > > > > that
> > > > we
> > > > >
> > > > > currently have like the password server and userdata on the
> > > > > VyOS.
> > > > > This means we would not have to change our architecture
> > > > > initially
> > > > > and could focus on only replacing the networking paths.
> > > > >
> > > > > *Will STEVENS*
> > > > > Lead Developer
> > > > >
> > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > cloudops.com *|*
> > > > > tw @CloudOps_
> > > > >
> > > > > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> > > > >
> > > > > >
> > > > > > The more this is discussed the more I think we should stick
> > > > > > with
> > > > > > our
> > > > VR.
> > > > >
> > > > > >
> > > > > >
> > > > > > All these other options either seem unfinished or with
> > > > > > incompatible license.
> > > > > >
> > > > > > VyOS looks the most promising so far, it's a serious,
> > > > > > mature project.
> > > > > > Adopting it though means we'll have to microservice our way
> > > > > > out of
> > > > > > it
> > > > > with
> > > > > >
> > > > > > extra machines for DNS/USERDATA/etc, unless we can make
> > > > > > VyOS serve
> > > > those
> > > > >
> > > > > >
> > > > > > too. Imho this adds complexity we should void.
> > > > > >
> > > > > > --
> > > > > > Sent from the Delta quadrant using Borg technology!
> > > > > >
> > > > > > Nux!
> > > > > > www.nux.ro
> > > > > >
> > > > > > ----- Original Message -----
> > > > > > >
> > > > > > > From: "Will Stevens" <ws...@cloudops.com>
> > > > > > > To: dev@cloudstack.apache.org
> > > > > > > Sent: Thursday, 15 September, 2016 17:21:28
> > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > >
> > > > > > > Ya, we would need to add a daemon for VPN as well. Load
> > > > > > > balancing is another aspect which we will need to
> > > > > > > consider if we
> > > went this route.
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > Something like https://traefik.io/ could potentially be a
> > > > > > > good
> > > > > > > fit
> > > > due
> > > > >
> > > > > >
> > > > > > to
> > > > > > >
> > > > > > > its API driven configuration, but it may be more than
> > > > > > > what we need.
> > > > > > >
> > > > > > > We should probably try define which pieces make sense to
> > > > > > > be
> > > > > > > solved
> > > > > > together
> > > > > > >
> > > > > > > and which pieces would be best suited to be broken out.
> > > > > > >
> > > > > > > I think the network connectivity, routing and firewalling
> > > > > > > should
> > > > > probably
> > > > > >
> > > > > > >
> > > > > > > all stay together since the majority of the tools we
> > > > > > > would
> > > > potentially
> > > > >
> > > > > >
> > > > > > use
> > > > > > >
> > > > > > > would handle all of that together in a single
> > > > > > > implementation.
> > > > > > >
> > > > > > > The password server and userdata seems like a good option
> > > > > > > for
> > > > > > > being
> > > > > > broken
> > > > > > >
> > > > > > > out and handled independently (and probably rewritten
> > > > > > > completely
> > > > since
> > > > >
> > > > > >
> > > > > > they
> > > > > > >
> > > > > > > currently have some issues).
> > > > > > >
> > > > > > > Load balancing is another that could warrant splitting
> > > > > > > out, but
> > > > > > > that depends on what direction we go and how we would be
> > > > > > > managing
> > > it.
> > > >
> > > > DHCP
> > > > >
> > > > > >
> > > > > > and
> > > > > > >
> > > > > > > DNS are others which could go either way.
> > > > > > >
> > > > > > > If we do split out services, I think we should
> > > > > > > consolidate as
> > > > > > > much as
> > > > > we
> > > > > >
> > > > > > >
> > > > > > > can into each service we break out. Ideally a network
> > > > > > > packet
> > > > > > > would
> > > > > never
> > > > > >
> > > > > > >
> > > > > > > hit more than one, maybe two, services. I don't think we
> > > > > > > should
> > > > > > > be splitting services 'just because', I think we need a
> > > > > > > valid
> > > > > > > case for splitting any service out because it adds
> > > > > > > complexity.
> > > > > > > Our project is already complex enough, we need to avoid
> > > > > > > adding
> > > > > > > complexity unless it
> > > > is
> > > > >
> > > > > >
> > > > > > >
> > > > > > > really needed.
> > > > > > >
> > > > > > > Some more of my thoughts on this anyway...
> > > > > > >
> > > > > > > *Will STEVENS*
> > > > > > > Lead Developer
> > > > > > >
> > > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > > > cloudops.com
> > > > > > > *|* tw @CloudOps_
> > > > > > >
> > > > > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sweller@e
> > > > > > > na.com>
> > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > I do agree with you that this probably isn't the right
> > > > > > > > place the
> > > > > > password
> > > > > > >
> > > > > > > >
> > > > > > > > service and user data.
> > > > > > > >
> > > > > > > >
> > > > > > > > Having said that, after taking a cursory look at the
> > > > > > > > dev docs,
> > > > > > > > it
> > > > > > doesn't
> > > > > > >
> > > > > > > >
> > > > > > > > seem that difficult to add new daemons:
> > > > https://opensnaproute.github.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > io/docs/developer.html#creating-new-component
> > > > > > > >
> > > > > > > > <https://opensnaproute.github.io/docs/developer.html#
> > > > > > > > creating-new-component>
> > > > > > > >
> > > > > > > >
> > > > > > > > They've definitely build it with a microservices
> > > > > > > > architecture in
> > > > mind,
> > > > >
> > > > > >
> > > > > > so
> > > > > > >
> > > > > > > >
> > > > > > > > each individual feature is abstracted into it's own
> > > > > > > > small daemon
> > > > > > process.
> > > > > > >
> > > > > > > >
> > > > > > > > We could just create a daemon for the password server
> > > > > > > > and the
> > > > userdata
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > components if we really had to.
> > > > > > > >
> > > > > > > >
> > > > > > > > - Si
> > > > > > > >
> > > > > > > >
> > > > > > > > ________________________________
> > > > > > > > From: williamstevens@gmail.com <williamstevens@gmail.co
> > > > > > > > m> on
> > > > > > > > behalf
> > > > > of
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Will Stevens <ws...@cloudops.com>
> > > > > > > > Sent: Thursday, September 15, 2016 9:17 AM
> > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > >
> > > > > > > > A big part of why I know about it is because it is
> > > > > > > > written in Go.
> > > > :P
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Yes, it is definitely interesting for the routing and
> > > > > > > > traffic
> > > > handling
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > aspects of the VR. We will likely have to rethink some
> > > > > > > > of the
> > > > pieces
> > > > >
> > > > > a
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > little bit like the password server and userdata if we
> > > > > > > > are to
> > > > > > > > adopt
> > > > a
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > different VR approach. This is where I think some of
> > > > > > > > JohnB and
> > > > > > Chiradeep's
> > > > > > >
> > > > > > > >
> > > > > > > > ideas make sense. In many ways, it does not make sense
> > > > > > > > for the
> > > > device
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > handling routing and network traffic to also be
> > > > > > > > responsible for
> > > > > > passwords
> > > > > > >
> > > > > > > >
> > > > > > > > and userdata.
> > > > > > > >
> > > > > > > > *Will STEVENS*
> > > > > > > > Lead Developer
> > > > > > > >
> > > > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > > > > cloudops.com
> > > > > > > > *|* tw @CloudOps_
> > > > > > > >
> > > > > > > > On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sweller@
> > > > > > > > ena.com>
> > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > I hadn't heard of Flexswitch until you mentioned it.
> > > > > > > > > It looks
> > > > pretty
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > cool!
> > > > > > > > >
> > > > > > > > > It even supports ONIE install.
> > > > > > > > >
> > > > > > > > > To be honest, the ipsec feature could be added, or we
> > > > > > > > > could
> > > > offload
> > > > >
> > > > > >
> > > > > > it to
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > separate vm if we needed to. The fact it is so
> > > > > > > > > feature rich
> > > > > > > > > from a
> > > > > > > > routing
> > > > > > > > >
> > > > > > > > > perspective (and all API driven) is really nice.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Based on the roadmap, it looks like they plan to also
> > > > > > > > > support
> > > > > > > > capabilities
> > > > > > > > >
> > > > > > > > > such as BGP-MPLS based L3VPN, EVPN, VPLS in the
> > > > > > > > > future. This
> > > > > > > > > will
> > > > be
> > > > >
> > > > > >
> > > > > > huge
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > for our carrier community that rely on these
> > > > > > > > > technologies to do
> > > > > > private
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > gateway and inter-VPC interconnections today. We
> > > > > > > > > handle this
> > > > > > > > > stuff
> > > > > on
> > > > > >
> > > > > > our
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > ASRs right now with a vlan interconnect into the VR.
> > > > > > > > > Being able
> > > > > > > > > to
> > > > > do
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > MPLS
> > > > > > > > >
> > > > > > > > > all the way to the VR would be awesome.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > It also seems to be written in GO (a language here at
> > > > > > > > > ENA we
> > > > > > > > > know
> > > > > very
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > well).
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > - Si
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > ________________________________
> > > > > > > > > From: Will Stevens <wi...@gmail.com>
> > > > > > > > > Sent: Thursday, September 15, 2016 7:06 AM
> > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > Subject: RE: [DISCUSS] Replacing the VR
> > > > > > > > >
> > > > > > > > > Ya. I don't think it covers our whole use case, but
> > > > > > > > > what it
> > > > > > > > > does
> > > > > > cover is
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > all api driven...
> > > > > > > > >
> > > > > > > > > On Sep 15, 2016 1:48 AM, "Marty Godsey" <marty@gonsou
> > > > > > > > > rce.com>
> > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Though I don’t see VPN in Snaproute.. Makes sense
> > > > > > > > > > since it was
> > > > not
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > intended to do IPSec.
> > > > > > > > > >
> > > > > > > > > > It seems as though VyOS is starting to look like
> > > > > > > > > > the best
> > > > option.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > > Marty Godsey
> > > > > > > > > > nSource Solutions
> > > > > > > > > >
> > > > > > > > > > -----Original Message-----
> > > > > > > > > > From: williamstevens@gmail.com
> > > > > > > > > > [mailto:williamstevens@gmail.com
> > > > ]
> > > > >
> > > > > On
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Behalf Of Will Stevens
> > > > > > > > > > Sent: Wednesday, September 14, 2016 11:06 PM
> > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > > > >
> > > > > > > > > > Or we could go completely crazy and go with
> > > > > > > > > > something like
> > > > > > FlexSwitch
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > from
> > > > > > > > > >
> > > > > > > > > > SnapRoute
> > > > > > > > > > - http://www.snaproute.com/
> > > > > > > > > > - https://opensnaproute.github.io/docs/apis.html
> > > > > > > > > >
> > > > > > > > > > *Will STEVENS*
> > > > > > > > > > Lead Developer
> > > > > > > > > >
> > > > > > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > > > > > > > cloudops.com
> > > > > *|*
> > > > > >
> > > > > > tw
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > @CloudOps_
> > > > > > > > > >
> > > > > > > > > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> > > > > > wstevens@cloudops.com>
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I tend to agree with Syed and Marty. I am not
> > > > > > > > > > > sure what
> > > > > problems
> > > > > >
> > > > > > are
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > solved by splitting up the function of the VR
> > > > > > > > > > > into a bunch of
> > > > > > > > separate
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > services. As Syed points out, the complexity
> > > > > > > > > > > added is
> > > > > > non-trivial.
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > We now have to manage all the intercontainer
> > > > > > > > > > > networking as
> > > > well
> > > > >
> > > > > as
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > the
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > orchestrated ACS networking.
> > > > > > > > > > >
> > > > > > > > > > > VyOS is interesting to me because it covers the
> > > > > > > > > > > majority of
> > > > our
> > > > >
> > > > > >
> > > > > > use
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > case with a single unified control plane. It
> > > > > > > > > > > also has good
> > > > > > support
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > for extending features we care about, like IPv6,
> > > > > > > > > > > VXLAN, VRRP,
> > > > > > > > > > > transactions, etc...
> > > > > > > > > > >
> > > > > > > > > > > *Will STEVENS*
> > > > > > > > > > > Lead Developer
> > > > > > > > > > >
> > > > > > > > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > > > > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > cloudops.com
> > > > >
> > > > > >
> > > > > > *|*
> > > > > > >
> > > > > > > >
> > > > > > > > tw
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > @CloudOps_
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> > > > > sahmed@cloudops.com>
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Agree with Marty, adding Docker containers to
> > > > > > > > > > > > the picture
> > > > > > although
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > can make the VR more flexible but the added
> > > > > > > > > > > > complexity is
> > > > just
> > > > >
> > > > > >
> > > > > > not
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > worth it. Not to mention we would need to take
> > > > > > > > > > > > care of
> > > > > networking
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > each container manually and given that our
> > > > > > > > > > > > iptable rules are
> > > > > very
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > unstable at the moment I don't see a big value
> > > > > > > > > > > > add.
> > > > > > > > > > > >
> > > > > > > > > > > > Vyos looks like a better solution to me. I know
> > > > > > > > > > > > that it does
> > > > > not
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > provide an api but it does fit the bill quite
> > > > > > > > > > > > well
> > > > otherwise. I
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > specially like the fact that it has a
> > > > > > > > > > > > transaction based
> > > > > > > > > > > > model
> > > > > and
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > you
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > can rollback changes if something goes wrong.
> > > > > > > > > > > > On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> > > > > > marty@gonsource.com>
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Licensing aside, I think splitting the
> > > > > > > > > > > > > various functions
> > > > into
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > containers is not a good route either. This
> > > > > > > > > > > > > will force
> > > > users
> > > > >
> > > > > to
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > have to maintain
> > > > > > > > > > > > and
> > > > > > > > > > > > >
> > > > > > > > > > > > > use containers and adds complexity to the
> > > > > > > > > > > > > networking
> > > > aspects
> > > > >
> > > > > of
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > ACS.
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Complexity decreases stability. Now I
> > > > > > > > > > > > > understand the
> > > > argument
> > > > >
> > > > > >
> > > > > > that
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > a monolithic approach also brings its own set
> > > > > > > > > > > > > of issues but
> > > > > it
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > also
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > simplifies it.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > Marty Godsey
> > > > > > > > > > > > > nSource Solutions
> > > > > > > > > > > > >
> > > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > > From: Chiradeep Vittal [mailto:chiradeepv@gma
> > > > > > > > > > > > > il.com]
> > > > > > > > > > > > > Sent: Wednesday, September 14, 2016 5:37 PM
> > > > > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > > > > > > >
> > > > > > > > > > > > > I rather doubt that the Cloudrouter will fit
> > > > > > > > > > > > > the needs
> > > > > > > > > > > > > of
> > > > the
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > CloudStack project
> > > > > > > > > > > > > - it is AGPL licensed. Many enterprises will
> > > > > > > > > > > > > not
> > > > > > > > > > > > > touch
> > > > > > anything
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > that
> > > > > > > > > > > > has
> > > > > > > > > > > > >
> > > > > > > > > > > > > AGPL
> > > > > > > > > > > > > - the github repo shows rather infrequent
> > > > > > > > > > > > > updates.
> > > > > > > > > > > > > Quite
> > > > > > likely
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > they aren't considering the use cases of the
> > > > > > > > > > > > > CloudStack
> > > > > > community
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > I'd back John B's comments on disaggregating
> > > > > > > > > > > > > the VR.
> > > > > > > > > > > > > Split
> > > > it
> > > > >
> > > > > >
> > > > > > into
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > many docker containers
> > > > > > > > > > > > > - password server
> > > > > > > > > > > > > - userdata server
> > > > > > > > > > > > > - DHCP / DNS
> > > > > > > > > > > > > - s2s VPN
> > > > > > > > > > > > > - RA VPN
> > > > > > > > > > > > > - intra-VPC routing and ACL
> > > > > > > > > > > > > - Port forwarding + NAT
> > > > > > > > > > > > > - FW
> > > > > > > > > > > > > - LB (public)
> > > > > > > > > > > > > - LB (internal),
> > > > > > > > > > > > > - secondary storage
> > > > > > > > > > > > > - agent
> > > > > > > > > > > > > Glue them together with docker compose files
> > > > > > > > > > > > > (one per
> > > > > > > > > > > > > use
> > > > > > case -
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > basic zone, isolated, VPC, SSVM, etc).
> > > > > > > > > > > > >
> > > > > > > > > > > > > The VR image then becomes a JeOS + docker.
> > > > > > > > > > > > > You can
> > > > > > > > > > > > > test
> > > > each
> > > > >
> > > > > of
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > the
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > components independently and fixing one bug
> > > > > > > > > > > > > in the
> > > > > > > > > > > > > field
> > > > (say
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > DHCP)
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > is hitless to the other components. You don't
> > > > > > > > > > > > > need to
> > > > > > > > > > > > > build per-hypervisor VRs. You could even run
> > > > > > > > > > > > > on
> > > baremetal.
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Along the way you need to figure out how to
> > > > > > > > > > > > > - make the traffic traverse the containers
> > > > > > > > > > > > > that are
> > > > > > > > > > > > > needed
> > > > > to
> > > > > >
> > > > > > be
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > traversed (in most cases just 1)
> > > > > > > > > > > > > - bootstrap the router (how does it find its
> > > > > > > > > > > > > compose
> > > file?
> > > >
> > > > >
> > > > > >
> > > > > > where
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > is the
> > > > > > > > > > > > > registry?)
> > > > > > > > > > > > > - rethink the command and control of the VR
> > > > > > > > > > > > > functions. SSH
> > > > > > works,
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > but something more declarative, idempotent
> > > > > > > > > > > > > should be
> > > > > explored.
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > As you do this, it becomes clearer which of
> > > > > > > > > > > > > the
> > > > > > > > > > > > > functions
> > > > can
> > > > >
> > > > > >
> > > > > > be
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > substituted by for example CloudRouter.
> > > > > > > > > > > > > Command and
> > > > > > > > > > > > > Control
> > > > > of
> > > > > >
> > > > > > the
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > docker
> > > > > > > > > > > > >
> > > > > > > > > > > > > containers can be moved out to another
> > > > > > > > > > > > > container. Etc.
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, Sep 14, 2016 at 12:59 AM, Marty
> > > > > > > > > > > > > Godsey
> > > > > > > > > > > > > <ma...@gonsource.com>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This one does look nice. My biggest concern
> > > > > > > > > > > > > > is the
> > > > > > > > > > > > > > lack
> > > > of
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > VXLANs. It seems that any of the ones we
> > > > > > > > > > > > > > mentioned
> > > > > > > > > > > > > > do not
> > > > > > have
> > > > > > >
> > > > > > > >
> > > > > > > > an
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > API so we may be stuck at the SSH method.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > > Marty Godsey
> > > > > > > > > > > > > > nSource Solutions
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > > > From: Abhinandan Prateek
> > > > > > > > > > > > > > [mailto:abhinandan.prateek@shapeblue.com]
> > > > > > > > > > > > > > Sent: Wednesday, September 14, 2016 2:26 AM
> > > > > > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Cloudrouter looks promising. These have
> > > > > > > > > > > > > > potential to
> > > > > > > > > > > > > > save
> > > > > > future
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > engineering effort for example on ipv6
> > > > > > > > > > > > > > routing, OSPF
> > > etc.
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > And the best part is they come with test
> > > > > > > > > > > > > > automation
> > > > > > framework.
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> > > > > > > > > > > > > > <ja...@accelerite.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Instead of replacing the VR in first
> > > > > > > > > > > > > > > place we
> > > > > > > > > > > > > > > should add VyOS/cloudrouter
> > > > > > > > > > > > > > as provider. Once it is stable, network
> > > > > > > > > > > > > > offerings
> > > > > > > > > > > > > > (on
> > > > > > upgrade)
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > can be updated to use it and we can drop
> > > > > > > > > > > > > > the VR if
> > > > > > > > > > > > > > we
> > > > want
> > > > >
> > > > > at
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > that release
> > > > > > > > > > > > > onwards.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > VR is stabilized over a period of time
> > > > > > > > > > > > > > > and some of
> > > > > > > > > > > > > > > them
> > > > > are
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > running
> > > > > > > > > > > > > > without issues. When we replicate the ACS
> > > > > > > > > > > > > > VR
> > > > > > > > > > > > > > features in
> > > > > new
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > solution it takes some to find the missing
> > > > > > > > > > > > > > pieces
> > > > > > > > > > > > > > (hidden
> > > > > > bugs).
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > Jayapal
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Sep 13, 2016, at 2:52 PM, Nux! <
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > nux@li.nux.ro> wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I like the idea.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Cloudrouter looks really promising, I'm
> > > > > > > > > > > > > > > > not too
> > > > > > > > > > > > > > > > keen
> > > > on
> > > > >
> > > > > >
> > > > > > VyOS
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > (it
> > > > > > > > > > > > > > doesn't have a proper http api etc).
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > Sent from the Delta quadrant using Borg
> > > > > > > > > > > > > > > > technology!
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Nux!
> > > > > > > > > > > > > > > > www.nux.ro
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > abhinandan.prateek@shapeblue.com
> > > > > > > > > > > > > > www.shapeblue.com<http://www.shapeblue.com>
> > > > > > > > > > > > > > ;
> > > > > > > > > > > > > > 53 Chandos Place, Covent Garden,
> > > > > > > > > > > > > > London WC2N 4HSUK
> > > > > > @shapeblue
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > ----- Original Message -----
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > From: "Will Stevens" <williamstevens@
> > > > > > > > > > > > > > > > > gmail.com>
> > > > > > > > > > > > > > > > > To: dev@cloudstack.apache.org
> > > > > > > > > > > > > > > > > Sent: Monday, 12 September, 2016
> > > > > > > > > > > > > > > > > 21:20:11
> > > > > > > > > > > > > > > > > Subject: [DISCUSS] Replacing the VR
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > *Disclaimer:* This is a thought
> > > > > > > > > > > > > > > > > experiment and
> > > > > > > > > > > > > > > > > should
> > > > > be
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > treated as
> > > > > > > > > > > > > > such.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Please weigh in with the good and bad
> > > > > > > > > > > > > > > > > of this
> > > idea...
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > A couple of us have been discussing
> > > > > > > > > > > > > > > > > the idea of
> > > > > > potentially
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > replacing the ACS VR with the VyOS
> > > > > > > > > > > > > > > > > [1] (Open
> > > > > > > > > > > > > > > > > Source
> > > > > > Vyatta
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > VM).
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > There may be a license issue because
> > > > > > > > > > > > > > > > > I think it
> > > > > > > > > > > > > > > > > is
> > > > > > licensed
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > under GPL, but for the sake of
> > > > > > > > > > > > > > > > > discussion, let's
> > > > assume
> > > > >
> > > > > >
> > > > > > we
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > can overcome any
> > > > > > > > > > > > > > license issues.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I have spent some time recently with
> > > > > > > > > > > > > > > > > the VyOS
> > > > > > > > > > > > > > > > > and I
> > > > > have
> > > > > >
> > > > > > to
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > admit, I was pretty impressed. It is
> > > > > > > > > > > > > > > > > simple and
> > > > > > intuitive
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > and it gives you a lot more options
> > > > > > > > > > > > > > > > > for auditing
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > configuration etc...
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Items of potential interest:
> > > > > > > > > > > > > > > > > - Clean up our current VR script
> > > > > > > > > > > > > > > > > spaghetti to a
> > > > simpler
> > > > >
> > > > > >
> > > > > > more
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > auditable configuration workflow.
> > > > > > > > > > > > > > > > > - Gives a cleaner path for IPv6
> > > > > > > > > > > > > > > > > support.
> > > > > > > > > > > > > > > > > - Handles VPN configuration via the
> > > > > > > > > > > > > > > > > same
> > > > configuration
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > interface.
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > - Support for OSPF & BGP.
> > > > > > > > > > > > > > > > > - VPN support through OpenVPN &
> > > > > > > > > > > > > > > > > StrongSwan.
> > > > > > > > > > > > > > > > > - Easily supports HA (redundant
> > > > > > > > > > > > > > > > > routers) through
> > > > VRRP.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > - VXLAN support.
> > > > > > > > > > > > > > > > > - Transaction based changes to the VR
> > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > rollback
> > > > on
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > error.
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Items that could be difficult to
> > > > > > > > > > > > > > > > > solve:
> > > > > > > > > > > > > > > > > - Userdata password reset workflow
> > > > > > > > > > > > > > > > > and
> > > > implementation.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > - Upgrade process.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > The VyOS is not the only option if we
> > > > > > > > > > > > > > > > > were to
> > > > consider
> > > > >
> > > > > >
> > > > > > this
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > approach.
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Another option, which I don't know as
> > > > > > > > > > > > > > > > > well,
> > > > > > > > > > > > > > > > > would be CloudRouter (AGPL
> > > > > > > > > > > > > > > > > license) [2] which is purely API
> > > > > > > > > > > > > > > > > driven.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Anyway, would love to hear your
> > > > > > > > > > > > > > > > > thoughts...
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Will
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > [1] https://vyos.io/ [2]
> > > > > > > > > > > > > > > > > https://cloudrouter.org/
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > DISCLAIMER
> > > > > > > > > > > > > > > ==========
> > > > > > > > > > > > > > > This e-mail may contain privileged and
> > > > > > > > > > > > > > > confidential
> > > > > > information
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > which is
> > > > > > > > > > > > > > the property of Accelerite, a Persistent
> > > > > > > > > > > > > > Systems
> > > > business.
> > > > >
> > > > > >
> > > > > > It is
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > intended only for the use of the individual
> > > > > > > > > > > > > > or
> > > > > > > > > > > > > > entity to
> > > > > > which
> > > > > > >
> > > > > > > >
> > > > > > > > it
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > is addressed. If you are not the intended
> > > > > > > > > > > > > > recipient,
> > > > > > > > > > > > > > you
> > > > > are
> > > > > >
> > > > > > not
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > authorized to read, retain, copy, print,
> > > > > > > > > > > > > > distribute
> > > > > > > > > > > > > > or
> > > > use
> > > > >
> > > > > >
> > > > > > this
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > message. If you have received this
> > > > > > > > > > > > > > communication in
> > > > error,
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > please
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > notify the sender and delete all copies of
> > > > > > > > > > > > > > this
> > > message.
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Accelerite, a Persistent Systems business
> > > > > > > > > > > > > > does not
> > > > > > > > > > > > > > accept
> > > > > any
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > liability for virus
> > > > > > > > > > > > > infected mails.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > >
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
On this note I also mentioned pfsense earlier.
www.pfsense.org
Regards,
Marty Godsey
-----Original Message-----
From: ilya [mailto:ilya.mailing.lists@gmail.com]
Sent: Sunday, September 18, 2016 1:09 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
Our options become much better if we consider BSD based routers.
Would that be on the table?
https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributions
On 9/16/16 12:04 PM, Will Stevens wrote:
> Ya, your points are all valid Simon. The lack of standard libraries
> to handle a lot of the details is a problem. I don't think it is an
> unsolvable problem, but if we spend the time to do that, will we have
> something that will work for us for the next 5 years? This may be the
> shortest path to getting us where we need to be for the time being.
>
> What is the best case scenario for the VR going forward which will
> last us the next 5 years? Maybe we just clean up what we have to do a
> major restructuring of the pieces and how they are implemented. We
> need to keep in mind how maintainable this implementation is because
> that is going to be key going forward IMO.
>
>
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Fri, Sep 16, 2016 at 2:29 PM, Simon Weller <sw...@ena.com> wrote:
>
>> I think our other option is to take a real look at what it would take
>> to fix the VR. In my opinion, a lot of the problems are related to
>> the monolithic python code base and the fact nothing is actually separated.
>>
>> Secondly, the python scripts (and bash scripts) don't use any
>> established libraries to complete tasks and instead shell out and run
>> commands that are both hard to track and hard to parse on return.
>>
>>
>> If we daemonized this, used a real api for Agent to VR communication,
>> used common already existing libraries for the system service and
>> network interactions and spent a bit of time separating out code into
>> distinct modules, everything would behave a lot better.
>>
>>
>> The pain and suffering is due to years and years of patches and
>> constant shelling out to complete tasks in my opinion. If we spend
>> time to rethink how we interact with the VR in general and we
>> abstract the systems and networking stuff and use well known and
>> stable libraries to do the work, the VR would be much easier to maintain.
>>
>>
>> - Si
>>
>>
>>
>>
>> ________________________________
>> From: Marty Godsey <ma...@gonsource.com>
>> Sent: Friday, September 16, 2016 12:24 PM
>> To: dev@cloudstack.apache.org
>> Subject: RE: [DISCUSS] Replacing the VR
>>
>> So based upon this discussion would it be prudent to wait on VyOS
>> 2.0? The current VR is giving us issues but would the time invested
>> in another "solution" be wasted especially if by the time another
>> option is chose, then coded, then tested, then implemented and right
>> as that time happened to be when VyOS 2.0 is released. Of course you
>> said they are just in the scoping range so this could still be a year or more out.
>>
>> Thoughts?
>>
>> Regards,
>> Marty Godsey
>> nSource Solutions
>>
>> -----Original Message-----
>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>> Behalf Of Will Stevens
>> Sent: Friday, September 16, 2016 10:31 AM
>> To: dev@cloudstack.apache.org
>> Cc: daniil@baturin.org
>> Subject: Re: [DISCUSS] Replacing the VR
>>
>> I just had a quick chat with a couple of the guys over on the VyOS chat.
>> I have CC'ed one of them in case we have more licensing questions.
>>
>> So here is the status with the license "the code inherited from
>> Vyatta and our modifications from it is GPLv2 (strict, not v2+). The
>> config reading library is GPLv2 too, so anything that links to is is GPLv2.
>> Some auxiliary components we made after the fork are more permissive,
>> LGPLv2+ or MIT."
>>
>> They are currently in the process of scoping a redesign (VyOS 2.0),
>> "we are planning a clean rewrite that will solve issues of the
>> current config system".
>> This will include the ability to configure via the API.
>>
>> If we have more questions for VyOS, they are very friendly and
>> responsive, so we should be able to get answers.
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>> @CloudOps_
>>
>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>>
>>> I agree with Will Ilya. There are so many problems with the VR right now.
>>> Most of the outages we've had recently have somehow involved the VR.
>>> We set custom iptables rules on the VR which can and have easily
>>> gone
>> wrong.
>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>> VVRP with redundant router still needs work, and not to mention the
>>> problems we will have when we introduce IPv6 into the whole picture.
>>>
>>> I think the spirit of the discussion is to rely on a 3rd party to do
>>> the networking for us (eg VyOS) and have us handle just the
>>> orchestration. All the problems that I've described have already
>>> been solved in VyOS. We also get the advantage of a potential wider
>>> community to fix and maintain the VR and given our current
>>> development velocity, it think it totally makes sense to look for a 3rd party option.
>>>
>>> -Syed
>>>
>>>
>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens
>>> <ws...@cloudops.com>
>>> wrote:
>>>
>>>> The VR has been biting us far too often recently, which is why we
>>>> have started looking into alternative implementations.
>>>>
>>>> One of the things that is nice about potentially using the VyOS is
>>>> that
>>> it
>>>> is based on Debian, so we should be able to run the other services
>>>> that
>>> we
>>>> currently have like the password server and userdata on the VyOS.
>>>> This means we would not have to change our architecture initially
>>>> and could focus on only replacing the networking paths.
>>>>
>>>> *Will STEVENS*
>>>> Lead Developer
>>>>
>>>> *CloudOps* *| *Cloud Solutions Experts
>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>> tw @CloudOps_
>>>>
>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>
>>>>> The more this is discussed the more I think we should stick with
>>>>> our
>>> VR.
>>>>>
>>>>> All these other options either seem unfinished or with
>>>>> incompatible license.
>>>>>
>>>>> VyOS looks the most promising so far, it's a serious, mature project.
>>>>> Adopting it though means we'll have to microservice our way out of
>>>>> it
>>>> with
>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>>> those
>>>>> too. Imho this adds complexity we should void.
>>>>>
>>>>> --
>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>
>>>>> Nux!
>>>>> www.nux.ro
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>> To: dev@cloudstack.apache.org
>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>
>>>>>> Ya, we would need to add a daemon for VPN as well. Load
>>>>>> balancing is another aspect which we will need to consider if we
>> went this route.
>>>>>> Something like https://traefik.io/ could potentially be a good
>>>>>> fit
>>> due
>>>>> to
>>>>>> its API driven configuration, but it may be more than what we need.
>>>>>>
>>>>>> We should probably try define which pieces make sense to be
>>>>>> solved
>>>>> together
>>>>>> and which pieces would be best suited to be broken out.
>>>>>>
>>>>>> I think the network connectivity, routing and firewalling should
>>>> probably
>>>>>> all stay together since the majority of the tools we would
>>> potentially
>>>>> use
>>>>>> would handle all of that together in a single implementation.
>>>>>>
>>>>>> The password server and userdata seems like a good option for
>>>>>> being
>>>>> broken
>>>>>> out and handled independently (and probably rewritten completely
>>> since
>>>>> they
>>>>>> currently have some issues).
>>>>>>
>>>>>> Load balancing is another that could warrant splitting out, but
>>>>>> that depends on what direction we go and how we would be managing
>> it.
>>> DHCP
>>>>> and
>>>>>> DNS are others which could go either way.
>>>>>>
>>>>>> If we do split out services, I think we should consolidate as
>>>>>> much as
>>>> we
>>>>>> can into each service we break out. Ideally a network packet
>>>>>> would
>>>> never
>>>>>> hit more than one, maybe two, services. I don't think we should
>>>>>> be splitting services 'just because', I think we need a valid
>>>>>> case for splitting any service out because it adds complexity.
>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>> complexity unless it
>>> is
>>>>>> really needed.
>>>>>>
>>>>>> Some more of my thoughts on this anyway...
>>>>>>
>>>>>> *Will STEVENS*
>>>>>> Lead Developer
>>>>>>
>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>> *|* tw @CloudOps_
>>>>>>
>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>>>> wrote:
>>>>>>
>>>>>>> I do agree with you that this probably isn't the right place the
>>>>> password
>>>>>>> service and user data.
>>>>>>>
>>>>>>>
>>>>>>> Having said that, after taking a cursory look at the dev docs,
>>>>>>> it
>>>>> doesn't
>>>>>>> seem that difficult to add new daemons:
>>> https://opensnaproute.github.
>>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>
>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>> creating-new-component>
>>>>>>>
>>>>>>>
>>>>>>> They've definitely build it with a microservices architecture in
>>> mind,
>>>>> so
>>>>>>> each individual feature is abstracted into it's own small daemon
>>>>> process.
>>>>>>> We could just create a daemon for the password server and the
>>> userdata
>>>>>>> components if we really had to.
>>>>>>>
>>>>>>>
>>>>>>> - Si
>>>>>>>
>>>>>>>
>>>>>>> ________________________________
>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
>>>>>>> behalf
>>>> of
>>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>> To: dev@cloudstack.apache.org
>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>
>>>>>>> A big part of why I know about it is because it is written in Go.
>>> :P
>>>>>>>
>>>>>>> Yes, it is definitely interesting for the routing and traffic
>>> handling
>>>>>>> aspects of the VR. We will likely have to rethink some of the
>>> pieces
>>>> a
>>>>>>> little bit like the password server and userdata if we are to
>>>>>>> adopt
>>> a
>>>>>>> different VR approach. This is where I think some of JohnB and
>>>>> Chiradeep's
>>>>>>> ideas make sense. In many ways, it does not make sense for the
>>> device
>>>>>>> handling routing and network traffic to also be responsible for
>>>>> passwords
>>>>>>> and userdata.
>>>>>>>
>>>>>>> *Will STEVENS*
>>>>>>> Lead Developer
>>>>>>>
>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>> *|* tw @CloudOps_
>>>>>>>
>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>>>> wrote:
>>>>>>>
>>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
>>> pretty
>>>>>>> cool!
>>>>>>>> It even supports ONIE install.
>>>>>>>>
>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>> offload
>>>>> it to
>>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>> from a
>>>>>>> routing
>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>
>>>>>>>>
>>>>>>>> Based on the roadmap, it looks like they plan to also support
>>>>>>> capabilities
>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>> will
>>> be
>>>>> huge
>>>>>>>> for our carrier community that rely on these technologies to do
>>>>> private
>>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>> stuff
>>>> on
>>>>> our
>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being able
>>>>>>>> to
>>>> do
>>>>>>> MPLS
>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>
>>>>>>>>
>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>> know
>>>> very
>>>>>>>> well).
>>>>>>>>
>>>>>>>>
>>>>>>>> - Si
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ________________________________
>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>
>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>> does
>>>>> cover is
>>>>>>>> all api driven...
>>>>>>>>
>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>>>> wrote:
>>>>>>>>
>>>>>>>>> Though I don’t see VPN in Snaproute.. Makes sense since it was
>>> not
>>>>>>>>> intended to do IPSec.
>>>>>>>>>
>>>>>>>>> It seems as though VyOS is starting to look like the best
>>> option.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Marty Godsey
>>>>>>>>> nSource Solutions
>>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>> [mailto:williamstevens@gmail.com
>>> ]
>>>> On
>>>>>>>>> Behalf Of Will Stevens
>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>
>>>>>>>>> Or we could go completely crazy and go with something like
>>>>> FlexSwitch
>>>>>>>> from
>>>>>>>>> SnapRoute
>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>> *|*
>>>>> tw
>>>>>>>>> @CloudOps_
>>>>>>>>>
>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>> wstevens@cloudops.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
>>>> problems
>>>>> are
>>>>>>>>>> solved by splitting up the function of the VR into a bunch of
>>>>>>> separate
>>>>>>>>>> services. As Syed points out, the complexity added is
>>>>> non-trivial.
>>>>>>>>>> We now have to manage all the intercontainer networking as
>>> well
>>>> as
>>>>>>> the
>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>
>>>>>>>>>> VyOS is interesting to me because it covers the majority of
>>> our
>>>>> use
>>>>>>>>>> case with a single unified control plane. It also has good
>>>>> support
>>>>>>>>>> for extending features we care about, like IPv6, VXLAN, VRRP,
>>>>>>>>>> transactions, etc...
>>>>>>>>>>
>>>>>>>>>> *Will STEVENS*
>>>>>>>>>> Lead Developer
>>>>>>>>>>
>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>> cloudops.com
>>>>> *|*
>>>>>>> tw
>>>>>>>>>> @CloudOps_
>>>>>>>>>>
>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>> sahmed@cloudops.com>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Agree with Marty, adding Docker containers to the picture
>>>>> although
>>>>>>>>>>> can make the VR more flexible but the added complexity is
>>> just
>>>>> not
>>>>>>>>>>> worth it. Not to mention we would need to take care of
>>>> networking
>>>>>>>>>>> each container manually and given that our iptable rules are
>>>> very
>>>>>>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>
>>>>>>>>>>> Vyos looks like a better solution to me. I know that it does
>>>> not
>>>>>>>>>>> provide an api but it does fit the bill quite well
>>> otherwise. I
>>>>>>>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>> model
>>>> and
>>>>>>> you
>>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>> marty@gonsource.com>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Licensing aside, I think splitting the various functions
>>> into
>>>>>>>>>>>> containers is not a good route either. This will force
>>> users
>>>> to
>>>>>>>>>>>> have to maintain
>>>>>>>>>>> and
>>>>>>>>>>>> use containers and adds complexity to the networking
>>> aspects
>>>> of
>>>>>>> ACS.
>>>>>>>>>>>> Complexity decreases stability. Now I understand the
>>> argument
>>>>> that
>>>>>>>>>>>> a monolithic approach also brings its own set of issues but
>>>> it
>>>>>>> also
>>>>>>>>>>>> simplifies it.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>
>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
>>>>>>>>>>>> of
>>> the
>>>>>>>>>>>> CloudStack project
>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
>>>>>>>>>>>> touch
>>>>> anything
>>>>>>>>>>>> that
>>>>>>>>>>> has
>>>>>>>>>>>> AGPL
>>>>>>>>>>>> - the github repo shows rather infrequent updates.
>>>>>>>>>>>> Quite
>>>>> likely
>>>>>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>> CloudStack
>>>>> community
>>>>>>>>>>>>
>>>>>>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>> Split
>>> it
>>>>> into
>>>>>>>>>>>> many docker containers
>>>>>>>>>>>> - password server
>>>>>>>>>>>> - userdata server
>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>> - FW
>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>> - agent
>>>>>>>>>>>> Glue them together with docker compose files (one per
>>>>>>>>>>>> use
>>>>> case -
>>>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>
>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
>>>>>>>>>>>> test
>>> each
>>>> of
>>>>>>> the
>>>>>>>>>>>> components independently and fixing one bug in the
>>>>>>>>>>>> field
>>> (say
>>>>>>> DHCP)
>>>>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>> build per-hypervisor VRs. You could even run on
>> baremetal.
>>>>>>>>>>>>
>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>> - make the traffic traverse the containers that are
>>>>>>>>>>>> needed
>>>> to
>>>>> be
>>>>>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>> - bootstrap the router (how does it find its compose
>> file?
>>>>> where
>>>>>>>>>>>> is the
>>>>>>>>>>>> registry?)
>>>>>>>>>>>> - rethink the command and control of the VR
>>>>>>>>>>>> functions. SSH
>>>>> works,
>>>>>>>>>>>> but something more declarative, idempotent should be
>>>> explored.
>>>>>>>>>>>>
>>>>>>>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>> functions
>>> can
>>>>> be
>>>>>>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>> Control
>>>> of
>>>>> the
>>>>>>>>>>> docker
>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> This one does look nice. My biggest concern is the
>>>>>>>>>>>>> lack
>>> of
>>>>>>>>>>>>> VXLANs. It seems that any of the ones we mentioned
>>>>>>>>>>>>> do not
>>>>> have
>>>>>>> an
>>>>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>> save
>>>>> future
>>>>>>>>>>>>> engineering effort for example on ipv6 routing, OSPF
>> etc.
>>>>>>>>>>>>> And the best part is they come with test automation
>>>>> framework.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Instead of replacing the VR in first place we
>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>> as provider. Once it is stable, network offerings
>>>>>>>>>>>>> (on
>>>>> upgrade)
>>>>>>>>>>>>> can be updated to use it and we can drop the VR if
>>>>>>>>>>>>> we
>>> want
>>>> at
>>>>>>>>>>>>> that release
>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>> them
>>>> are
>>>>>>>>>>>>>> running
>>>>>>>>>>>>> without issues. When we replicate the ACS VR
>>>>>>>>>>>>> features in
>>>> new
>>>>>>>>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>> (hidden
>>>>> bugs).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
>>>>>>>>>>>>>>> keen
>>> on
>>>>> VyOS
>>>>>>>>>>>>>>> (it
>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>>>> @shapeblue
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
>>>>>>>>>>>>>>>> should
>>>> be
>>>>>>>>>>>>>>>> treated as
>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>> Please weigh in with the good and bad of this
>> idea...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
>>>>> potentially
>>>>>>>>>>>>>>>> replacing the ACS VR with the VyOS [1] (Open
>>>>>>>>>>>>>>>> Source
>>>>> Vyatta
>>>>>>>> VM).
>>>>>>>>>>>>>>>> There may be a license issue because I think it
>>>>>>>>>>>>>>>> is
>>>>> licensed
>>>>>>>>>>>>>>>> under GPL, but for the sake of discussion, let's
>>> assume
>>>>> we
>>>>>>>>>>>>>>>> can overcome any
>>>>>>>>>>>>> license issues.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have spent some time recently with the VyOS
>>>>>>>>>>>>>>>> and I
>>>> have
>>>>> to
>>>>>>>>>>>>>>>> admit, I was pretty impressed. It is simple and
>>>>> intuitive
>>>>>>>>>>>>>>>> and it gives you a lot more options for auditing
>>>>>>>>>>>>>>>> the
>>>>>>>>> configuration etc...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
>>> simpler
>>>>> more
>>>>>>>>>>>>>>>> auditable configuration workflow.
>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
>>>>>>>>>>>>>>>> - Handles VPN configuration via the same
>>> configuration
>>>>>>>>> interface.
>>>>>>>>>>>>>>>> - Support for OSPF & BGP.
>>>>>>>>>>>>>>>> - VPN support through OpenVPN & StrongSwan.
>>>>>>>>>>>>>>>> - Easily supports HA (redundant routers) through
>>> VRRP.
>>>>>>>>>>>>>>>> - VXLAN support.
>>>>>>>>>>>>>>>> - Transaction based changes to the VR with
>>>>>>>>>>>>>>>> rollback
>>> on
>>>>>>> error.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Items that could be difficult to solve:
>>>>>>>>>>>>>>>> - Userdata password reset workflow and
>>> implementation.
>>>>>>>>>>>>>>>> - Upgrade process.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The VyOS is not the only option if we were to
>>> consider
>>>>> this
>>>>>>>>>>> approach.
>>>>>>>>>>>>>>>> Another option, which I don't know as well,
>>>>>>>>>>>>>>>> would be CloudRouter (AGPL
>>>>>>>>>>>>>>>> license) [2] which is purely API driven.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Anyway, would love to hear your thoughts...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1] https://vyos.io/ [2]
>>>>>>>>>>>>>>>> https://cloudrouter.org/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> DISCLAIMER
>>>>>>>>>>>>>> ==========
>>>>>>>>>>>>>> This e-mail may contain privileged and confidential
>>>>> information
>>>>>>>>>>>>>> which is
>>>>>>>>>>>>> the property of Accelerite, a Persistent Systems
>>> business.
>>>>> It is
>>>>>>>>>>>>> intended only for the use of the individual or
>>>>>>>>>>>>> entity to
>>>>> which
>>>>>>> it
>>>>>>>>>>>>> is addressed. If you are not the intended recipient,
>>>>>>>>>>>>> you
>>>> are
>>>>> not
>>>>>>>>>>>>> authorized to read, retain, copy, print, distribute
>>>>>>>>>>>>> or
>>> use
>>>>> this
>>>>>>>>>>>>> message. If you have received this communication in
>>> error,
>>>>>>> please
>>>>>>>>>>>>> notify the sender and delete all copies of this
>> message.
>>>>>>>>>>>>> Accelerite, a Persistent Systems business does not
>>>>>>>>>>>>> accept
>>>> any
>>>>>>>>>>>>> liability for virus
>>>>>>>>>>>> infected mails.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: [DISCUSS] Replacing the VR
Posted by ilya <il...@gmail.com>.
Our options become much better if we consider BSD based routers.
Would that be on the table?
https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributions
On 9/16/16 12:04 PM, Will Stevens wrote:
> Ya, your points are all valid Simon. The lack of standard libraries to
> handle a lot of the details is a problem. I don't think it is an
> unsolvable problem, but if we spend the time to do that, will we have
> something that will work for us for the next 5 years? This may be the
> shortest path to getting us where we need to be for the time being.
>
> What is the best case scenario for the VR going forward which will last us
> the next 5 years? Maybe we just clean up what we have to do a major
> restructuring of the pieces and how they are implemented. We need to keep
> in mind how maintainable this implementation is because that is going to be
> key going forward IMO.
>
>
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Fri, Sep 16, 2016 at 2:29 PM, Simon Weller <sw...@ena.com> wrote:
>
>> I think our other option is to take a real look at what it would take to
>> fix the VR. In my opinion, a lot of the problems are related to the
>> monolithic python code base and the fact nothing is actually separated.
>>
>> Secondly, the python scripts (and bash scripts) don't use any established
>> libraries to complete tasks and instead shell out and run commands that are
>> both hard to track and hard to parse on return.
>>
>>
>> If we daemonized this, used a real api for Agent to VR communication, used
>> common already existing libraries for the system service and network
>> interactions and spent a bit of time separating out code into distinct
>> modules, everything would behave a lot better.
>>
>>
>> The pain and suffering is due to years and years of patches and constant
>> shelling out to complete tasks in my opinion. If we spend time to rethink
>> how we interact with the VR in general and we abstract the systems and
>> networking stuff and use well known and stable libraries to do the work,
>> the VR would be much easier to maintain.
>>
>>
>> - Si
>>
>>
>>
>>
>> ________________________________
>> From: Marty Godsey <ma...@gonsource.com>
>> Sent: Friday, September 16, 2016 12:24 PM
>> To: dev@cloudstack.apache.org
>> Subject: RE: [DISCUSS] Replacing the VR
>>
>> So based upon this discussion would it be prudent to wait on VyOS 2.0? The
>> current VR is giving us issues but would the time invested in another
>> "solution" be wasted especially if by the time another option is chose,
>> then coded, then tested, then implemented and right as that time happened
>> to be when VyOS 2.0 is released. Of course you said they are just in the
>> scoping range so this could still be a year or more out.
>>
>> Thoughts?
>>
>> Regards,
>> Marty Godsey
>> nSource Solutions
>>
>> -----Original Message-----
>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>> Behalf Of Will Stevens
>> Sent: Friday, September 16, 2016 10:31 AM
>> To: dev@cloudstack.apache.org
>> Cc: daniil@baturin.org
>> Subject: Re: [DISCUSS] Replacing the VR
>>
>> I just had a quick chat with a couple of the guys over on the VyOS chat.
>> I have CC'ed one of them in case we have more licensing questions.
>>
>> So here is the status with the license "the code inherited from Vyatta and
>> our modifications from it is GPLv2 (strict, not v2+). The config reading
>> library is GPLv2 too, so anything that links to is is GPLv2.
>> Some auxiliary components we made after the fork are more permissive,
>> LGPLv2+ or MIT."
>>
>> They are currently in the process of scoping a redesign (VyOS 2.0), "we
>> are planning a clean rewrite that will solve issues of the current config
>> system".
>> This will include the ability to configure via the API.
>>
>> If we have more questions for VyOS, they are very friendly and responsive,
>> so we should be able to get answers.
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>> @CloudOps_
>>
>> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>>
>>> I agree with Will Ilya. There are so many problems with the VR right now.
>>> Most of the outages we've had recently have somehow involved the VR.
>>> We set custom iptables rules on the VR which can and have easily gone
>> wrong.
>>> Openswan is broken, Strongswan replacement still needs to be tested.
>>> VVRP with redundant router still needs work, and not to mention the
>>> problems we will have when we introduce IPv6 into the whole picture.
>>>
>>> I think the spirit of the discussion is to rely on a 3rd party to do
>>> the networking for us (eg VyOS) and have us handle just the
>>> orchestration. All the problems that I've described have already been
>>> solved in VyOS. We also get the advantage of a potential wider
>>> community to fix and maintain the VR and given our current development
>>> velocity, it think it totally makes sense to look for a 3rd party option.
>>>
>>> -Syed
>>>
>>>
>>> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
>>> wrote:
>>>
>>>> The VR has been biting us far too often recently, which is why we
>>>> have started looking into alternative implementations.
>>>>
>>>> One of the things that is nice about potentially using the VyOS is
>>>> that
>>> it
>>>> is based on Debian, so we should be able to run the other services
>>>> that
>>> we
>>>> currently have like the password server and userdata on the VyOS.
>>>> This means we would not have to change our architecture initially
>>>> and could focus on only replacing the networking paths.
>>>>
>>>> *Will STEVENS*
>>>> Lead Developer
>>>>
>>>> *CloudOps* *| *Cloud Solutions Experts
>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>>>> tw @CloudOps_
>>>>
>>>> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>>>>
>>>>> The more this is discussed the more I think we should stick with
>>>>> our
>>> VR.
>>>>>
>>>>> All these other options either seem unfinished or with
>>>>> incompatible license.
>>>>>
>>>>> VyOS looks the most promising so far, it's a serious, mature project.
>>>>> Adopting it though means we'll have to microservice our way out of
>>>>> it
>>>> with
>>>>> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
>>> those
>>>>> too. Imho this adds complexity we should void.
>>>>>
>>>>> --
>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>
>>>>> Nux!
>>>>> www.nux.ro
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Will Stevens" <ws...@cloudops.com>
>>>>>> To: dev@cloudstack.apache.org
>>>>>> Sent: Thursday, 15 September, 2016 17:21:28
>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>
>>>>>> Ya, we would need to add a daemon for VPN as well. Load
>>>>>> balancing is another aspect which we will need to consider if we
>> went this route.
>>>>>> Something like https://traefik.io/ could potentially be a good
>>>>>> fit
>>> due
>>>>> to
>>>>>> its API driven configuration, but it may be more than what we need.
>>>>>>
>>>>>> We should probably try define which pieces make sense to be
>>>>>> solved
>>>>> together
>>>>>> and which pieces would be best suited to be broken out.
>>>>>>
>>>>>> I think the network connectivity, routing and firewalling should
>>>> probably
>>>>>> all stay together since the majority of the tools we would
>>> potentially
>>>>> use
>>>>>> would handle all of that together in a single implementation.
>>>>>>
>>>>>> The password server and userdata seems like a good option for
>>>>>> being
>>>>> broken
>>>>>> out and handled independently (and probably rewritten completely
>>> since
>>>>> they
>>>>>> currently have some issues).
>>>>>>
>>>>>> Load balancing is another that could warrant splitting out, but
>>>>>> that depends on what direction we go and how we would be managing
>> it.
>>> DHCP
>>>>> and
>>>>>> DNS are others which could go either way.
>>>>>>
>>>>>> If we do split out services, I think we should consolidate as
>>>>>> much as
>>>> we
>>>>>> can into each service we break out. Ideally a network packet
>>>>>> would
>>>> never
>>>>>> hit more than one, maybe two, services. I don't think we should
>>>>>> be splitting services 'just because', I think we need a valid
>>>>>> case for splitting any service out because it adds complexity.
>>>>>> Our project is already complex enough, we need to avoid adding
>>>>>> complexity unless it
>>> is
>>>>>> really needed.
>>>>>>
>>>>>> Some more of my thoughts on this anyway...
>>>>>>
>>>>>> *Will STEVENS*
>>>>>> Lead Developer
>>>>>>
>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>> *|* tw @CloudOps_
>>>>>>
>>>>>> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
>>>> wrote:
>>>>>>
>>>>>>> I do agree with you that this probably isn't the right place
>>>>>>> the
>>>>> password
>>>>>>> service and user data.
>>>>>>>
>>>>>>>
>>>>>>> Having said that, after taking a cursory look at the dev docs,
>>>>>>> it
>>>>> doesn't
>>>>>>> seem that difficult to add new daemons:
>>> https://opensnaproute.github.
>>>>>>> io/docs/developer.html#creating-new-component
>>>>>>>
>>>>>>> <https://opensnaproute.github.io/docs/developer.html#
>>>>>>> creating-new-component>
>>>>>>>
>>>>>>>
>>>>>>> They've definitely build it with a microservices architecture
>>>>>>> in
>>> mind,
>>>>> so
>>>>>>> each individual feature is abstracted into it's own small
>>>>>>> daemon
>>>>> process.
>>>>>>> We could just create a daemon for the password server and the
>>> userdata
>>>>>>> components if we really had to.
>>>>>>>
>>>>>>>
>>>>>>> - Si
>>>>>>>
>>>>>>>
>>>>>>> ________________________________
>>>>>>> From: williamstevens@gmail.com <wi...@gmail.com> on
>>>>>>> behalf
>>>> of
>>>>>>> Will Stevens <ws...@cloudops.com>
>>>>>>> Sent: Thursday, September 15, 2016 9:17 AM
>>>>>>> To: dev@cloudstack.apache.org
>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>
>>>>>>> A big part of why I know about it is because it is written in Go.
>>> :P
>>>>>>>
>>>>>>> Yes, it is definitely interesting for the routing and traffic
>>> handling
>>>>>>> aspects of the VR. We will likely have to rethink some of the
>>> pieces
>>>> a
>>>>>>> little bit like the password server and userdata if we are to
>>>>>>> adopt
>>> a
>>>>>>> different VR approach. This is where I think some of JohnB and
>>>>> Chiradeep's
>>>>>>> ideas make sense. In many ways, it does not make sense for the
>>> device
>>>>>>> handling routing and network traffic to also be responsible for
>>>>> passwords
>>>>>>> and userdata.
>>>>>>>
>>>>>>> *Will STEVENS*
>>>>>>> Lead Developer
>>>>>>>
>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
>>>>>>> *|* tw @CloudOps_
>>>>>>>
>>>>>>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
>>>> wrote:
>>>>>>>
>>>>>>>> I hadn't heard of Flexswitch until you mentioned it. It looks
>>> pretty
>>>>>>> cool!
>>>>>>>> It even supports ONIE install.
>>>>>>>>
>>>>>>>> To be honest, the ipsec feature could be added, or we could
>>> offload
>>>>> it to
>>>>>>>> separate vm if we needed to. The fact it is so feature rich
>>>>>>>> from a
>>>>>>> routing
>>>>>>>> perspective (and all API driven) is really nice.
>>>>>>>>
>>>>>>>>
>>>>>>>> Based on the roadmap, it looks like they plan to also support
>>>>>>> capabilities
>>>>>>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
>>>>>>>> will
>>> be
>>>>> huge
>>>>>>>> for our carrier community that rely on these technologies to
>>>>>>>> do
>>>>> private
>>>>>>>> gateway and inter-VPC interconnections today. We handle this
>>>>>>>> stuff
>>>> on
>>>>> our
>>>>>>>> ASRs right now with a vlan interconnect into the VR. Being
>>>>>>>> able to
>>>> do
>>>>>>> MPLS
>>>>>>>> all the way to the VR would be awesome.
>>>>>>>>
>>>>>>>>
>>>>>>>> It also seems to be written in GO (a language here at ENA we
>>>>>>>> know
>>>> very
>>>>>>>> well).
>>>>>>>>
>>>>>>>>
>>>>>>>> - Si
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ________________________________
>>>>>>>> From: Will Stevens <wi...@gmail.com>
>>>>>>>> Sent: Thursday, September 15, 2016 7:06 AM
>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>> Subject: RE: [DISCUSS] Replacing the VR
>>>>>>>>
>>>>>>>> Ya. I don't think it covers our whole use case, but what it
>>>>>>>> does
>>>>> cover is
>>>>>>>> all api driven...
>>>>>>>>
>>>>>>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
>>>> wrote:
>>>>>>>>
>>>>>>>>> Though I don\u2019t see VPN in Snaproute.. Makes sense since it
>>>>>>>>> was
>>> not
>>>>>>>>> intended to do IPSec.
>>>>>>>>>
>>>>>>>>> It seems as though VyOS is starting to look like the best
>>> option.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Marty Godsey
>>>>>>>>> nSource Solutions
>>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: williamstevens@gmail.com
>>>>>>>>> [mailto:williamstevens@gmail.com
>>> ]
>>>> On
>>>>>>>>> Behalf Of Will Stevens
>>>>>>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>
>>>>>>>>> Or we could go completely crazy and go with something like
>>>>> FlexSwitch
>>>>>>>> from
>>>>>>>>> SnapRoute
>>>>>>>>> - http://www.snaproute.com/
>>>>>>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>>>>>>
>>>>>>>>> *Will STEVENS*
>>>>>>>>> Lead Developer
>>>>>>>>>
>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>>>>>>>> cloudops.com
>>>> *|*
>>>>> tw
>>>>>>>>> @CloudOps_
>>>>>>>>>
>>>>>>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
>>>>> wstevens@cloudops.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I tend to agree with Syed and Marty. I am not sure what
>>>> problems
>>>>> are
>>>>>>>>>> solved by splitting up the function of the VR into a
>>>>>>>>>> bunch of
>>>>>>> separate
>>>>>>>>>> services. As Syed points out, the complexity added is
>>>>> non-trivial.
>>>>>>>>>> We now have to manage all the intercontainer networking
>>>>>>>>>> as
>>> well
>>>> as
>>>>>>> the
>>>>>>>>>> orchestrated ACS networking.
>>>>>>>>>>
>>>>>>>>>> VyOS is interesting to me because it covers the majority
>>>>>>>>>> of
>>> our
>>>>> use
>>>>>>>>>> case with a single unified control plane. It also has
>>>>>>>>>> good
>>>>> support
>>>>>>>>>> for extending features we care about, like IPv6, VXLAN,
>>>>>>>>>> VRRP, transactions, etc...
>>>>>>>>>>
>>>>>>>>>> *Will STEVENS*
>>>>>>>>>> Lead Developer
>>>>>>>>>>
>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
>>> cloudops.com
>>>>> *|*
>>>>>>> tw
>>>>>>>>>> @CloudOps_
>>>>>>>>>>
>>>>>>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
>>>> sahmed@cloudops.com>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Agree with Marty, adding Docker containers to the
>>>>>>>>>>> picture
>>>>> although
>>>>>>>>>>> can make the VR more flexible but the added complexity
>>>>>>>>>>> is
>>> just
>>>>> not
>>>>>>>>>>> worth it. Not to mention we would need to take care of
>>>> networking
>>>>>>>>>>> each container manually and given that our iptable rules
>>>>>>>>>>> are
>>>> very
>>>>>>>>>>> unstable at the moment I don't see a big value add.
>>>>>>>>>>>
>>>>>>>>>>> Vyos looks like a better solution to me. I know that it
>>>>>>>>>>> does
>>>> not
>>>>>>>>>>> provide an api but it does fit the bill quite well
>>> otherwise. I
>>>>>>>>>>> specially like the fact that it has a transaction based
>>>>>>>>>>> model
>>>> and
>>>>>>> you
>>>>>>>>>>> can rollback changes if something goes wrong.
>>>>>>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
>>>>> marty@gonsource.com>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Licensing aside, I think splitting the various
>>>>>>>>>>>> functions
>>> into
>>>>>>>>>>>> containers is not a good route either. This will force
>>> users
>>>> to
>>>>>>>>>>>> have to maintain
>>>>>>>>>>> and
>>>>>>>>>>>> use containers and adds complexity to the networking
>>> aspects
>>>> of
>>>>>>> ACS.
>>>>>>>>>>>> Complexity decreases stability. Now I understand the
>>> argument
>>>>> that
>>>>>>>>>>>> a monolithic approach also brings its own set of
>>>>>>>>>>>> issues but
>>>> it
>>>>>>> also
>>>>>>>>>>>> simplifies it.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>
>>>>>>>>>>>> I rather doubt that the Cloudrouter will fit the needs
>>>>>>>>>>>> of
>>> the
>>>>>>>>>>>> CloudStack project
>>>>>>>>>>>> - it is AGPL licensed. Many enterprises will not
>>>>>>>>>>>> touch
>>>>> anything
>>>>>>>>>>>> that
>>>>>>>>>>> has
>>>>>>>>>>>> AGPL
>>>>>>>>>>>> - the github repo shows rather infrequent updates.
>>>>>>>>>>>> Quite
>>>>> likely
>>>>>>>>>>>> they aren't considering the use cases of the
>>>>>>>>>>>> CloudStack
>>>>> community
>>>>>>>>>>>>
>>>>>>>>>>>> I'd back John B's comments on disaggregating the VR.
>>>>>>>>>>>> Split
>>> it
>>>>> into
>>>>>>>>>>>> many docker containers
>>>>>>>>>>>> - password server
>>>>>>>>>>>> - userdata server
>>>>>>>>>>>> - DHCP / DNS
>>>>>>>>>>>> - s2s VPN
>>>>>>>>>>>> - RA VPN
>>>>>>>>>>>> - intra-VPC routing and ACL
>>>>>>>>>>>> - Port forwarding + NAT
>>>>>>>>>>>> - FW
>>>>>>>>>>>> - LB (public)
>>>>>>>>>>>> - LB (internal),
>>>>>>>>>>>> - secondary storage
>>>>>>>>>>>> - agent
>>>>>>>>>>>> Glue them together with docker compose files (one per
>>>>>>>>>>>> use
>>>>> case -
>>>>>>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>>>>>>
>>>>>>>>>>>> The VR image then becomes a JeOS + docker. You can
>>>>>>>>>>>> test
>>> each
>>>> of
>>>>>>> the
>>>>>>>>>>>> components independently and fixing one bug in the
>>>>>>>>>>>> field
>>> (say
>>>>>>> DHCP)
>>>>>>>>>>>> is hitless to the other components. You don't need to
>>>>>>>>>>>> build per-hypervisor VRs. You could even run on
>> baremetal.
>>>>>>>>>>>>
>>>>>>>>>>>> Along the way you need to figure out how to
>>>>>>>>>>>> - make the traffic traverse the containers that are
>>>>>>>>>>>> needed
>>>> to
>>>>> be
>>>>>>>>>>>> traversed (in most cases just 1)
>>>>>>>>>>>> - bootstrap the router (how does it find its compose
>> file?
>>>>> where
>>>>>>>>>>>> is the
>>>>>>>>>>>> registry?)
>>>>>>>>>>>> - rethink the command and control of the VR
>>>>>>>>>>>> functions. SSH
>>>>> works,
>>>>>>>>>>>> but something more declarative, idempotent should be
>>>> explored.
>>>>>>>>>>>>
>>>>>>>>>>>> As you do this, it becomes clearer which of the
>>>>>>>>>>>> functions
>>> can
>>>>> be
>>>>>>>>>>>> substituted by for example CloudRouter. Command and
>>>>>>>>>>>> Control
>>>> of
>>>>> the
>>>>>>>>>>> docker
>>>>>>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>>>>>>> <ma...@gonsource.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> This one does look nice. My biggest concern is the
>>>>>>>>>>>>> lack
>>> of
>>>>>>>>>>>>> VXLANs. It seems that any of the ones we mentioned
>>>>>>>>>>>>> do not
>>>>> have
>>>>>>> an
>>>>>>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Marty Godsey
>>>>>>>>>>>>> nSource Solutions
>>>>>>>>>>>>>
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: Abhinandan Prateek
>>>>>>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cloudrouter looks promising. These have potential to
>>>>>>>>>>>>> save
>>>>> future
>>>>>>>>>>>>> engineering effort for example on ipv6 routing, OSPF
>> etc.
>>>>>>>>>>>>> And the best part is they come with test automation
>>>>> framework.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>>>>>>> <ja...@accelerite.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Instead of replacing the VR in first place we
>>>>>>>>>>>>>> should add VyOS/cloudrouter
>>>>>>>>>>>>> as provider. Once it is stable, network offerings
>>>>>>>>>>>>> (on
>>>>> upgrade)
>>>>>>>>>>>>> can be updated to use it and we can drop the VR if
>>>>>>>>>>>>> we
>>> want
>>>> at
>>>>>>>>>>>>> that release
>>>>>>>>>>>> onwards.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> VR is stabilized over a period of time and some of
>>>>>>>>>>>>>> them
>>>> are
>>>>>>>>>>>>>> running
>>>>>>>>>>>>> without issues. When we replicate the ACS VR
>>>>>>>>>>>>> features in
>>>> new
>>>>>>>>>>>>> solution it takes some to find the missing pieces
>>>>>>>>>>>>> (hidden
>>>>> bugs).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Jayapal
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I like the idea.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cloudrouter looks really promising, I'm not too
>>>>>>>>>>>>>>> keen
>>> on
>>>>> VyOS
>>>>>>>>>>>>>>> (it
>>>>>>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Nux!
>>>>>>>>>>>>>>> www.nux.ro
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>>>> @shapeblue
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *Disclaimer:* This is a thought experiment and
>>>>>>>>>>>>>>>> should
>>>> be
>>>>>>>>>>>>>>>> treated as
>>>>>>>>>>>>> such.
>>>>>>>>>>>>>>>> Please weigh in with the good and bad of this
>> idea...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> A couple of us have been discussing the idea of
>>>>> potentially
>>>>>>>>>>>>>>>> replacing the ACS VR with the VyOS [1] (Open
>>>>>>>>>>>>>>>> Source
>>>>> Vyatta
>>>>>>>> VM).
>>>>>>>>>>>>>>>> There may be a license issue because I think it
>>>>>>>>>>>>>>>> is
>>>>> licensed
>>>>>>>>>>>>>>>> under GPL, but for the sake of discussion, let's
>>> assume
>>>>> we
>>>>>>>>>>>>>>>> can overcome any
>>>>>>>>>>>>> license issues.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have spent some time recently with the VyOS
>>>>>>>>>>>>>>>> and I
>>>> have
>>>>> to
>>>>>>>>>>>>>>>> admit, I was pretty impressed. It is simple and
>>>>> intuitive
>>>>>>>>>>>>>>>> and it gives you a lot more options for auditing
>>>>>>>>>>>>>>>> the
>>>>>>>>> configuration etc...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>>>>>>> - Clean up our current VR script spaghetti to a
>>> simpler
>>>>> more
>>>>>>>>>>>>>>>> auditable configuration workflow.
>>>>>>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
>>>>>>>>>>>>>>>> - Handles VPN configuration via the same
>>> configuration
>>>>>>>>> interface.
>>>>>>>>>>>>>>>> - Support for OSPF & BGP.
>>>>>>>>>>>>>>>> - VPN support through OpenVPN & StrongSwan.
>>>>>>>>>>>>>>>> - Easily supports HA (redundant routers) through
>>> VRRP.
>>>>>>>>>>>>>>>> - VXLAN support.
>>>>>>>>>>>>>>>> - Transaction based changes to the VR with
>>>>>>>>>>>>>>>> rollback
>>> on
>>>>>>> error.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Items that could be difficult to solve:
>>>>>>>>>>>>>>>> - Userdata password reset workflow and
>>> implementation.
>>>>>>>>>>>>>>>> - Upgrade process.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The VyOS is not the only option if we were to
>>> consider
>>>>> this
>>>>>>>>>>> approach.
>>>>>>>>>>>>>>>> Another option, which I don't know as well,
>>>>>>>>>>>>>>>> would be CloudRouter (AGPL
>>>>>>>>>>>>>>>> license) [2] which is purely API driven.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Anyway, would love to hear your thoughts...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1] https://vyos.io/ [2]
>>>>>>>>>>>>>>>> https://cloudrouter.org/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> DISCLAIMER
>>>>>>>>>>>>>> ==========
>>>>>>>>>>>>>> This e-mail may contain privileged and confidential
>>>>> information
>>>>>>>>>>>>>> which is
>>>>>>>>>>>>> the property of Accelerite, a Persistent Systems
>>> business.
>>>>> It is
>>>>>>>>>>>>> intended only for the use of the individual or
>>>>>>>>>>>>> entity to
>>>>> which
>>>>>>> it
>>>>>>>>>>>>> is addressed. If you are not the intended recipient,
>>>>>>>>>>>>> you
>>>> are
>>>>> not
>>>>>>>>>>>>> authorized to read, retain, copy, print, distribute
>>>>>>>>>>>>> or
>>> use
>>>>> this
>>>>>>>>>>>>> message. If you have received this communication in
>>> error,
>>>>>>> please
>>>>>>>>>>>>> notify the sender and delete all copies of this
>> message.
>>>>>>>>>>>>> Accelerite, a Persistent Systems business does not
>>>>>>>>>>>>> accept
>>>> any
>>>>>>>>>>>>> liability for virus
>>>>>>>>>>>> infected mails.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
Ya, your points are all valid Simon. The lack of standard libraries to
handle a lot of the details is a problem. I don't think it is an
unsolvable problem, but if we spend the time to do that, will we have
something that will work for us for the next 5 years? This may be the
shortest path to getting us where we need to be for the time being.
What is the best case scenario for the VR going forward which will last us
the next 5 years? Maybe we just clean up what we have to do a major
restructuring of the pieces and how they are implemented. We need to keep
in mind how maintainable this implementation is because that is going to be
key going forward IMO.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Fri, Sep 16, 2016 at 2:29 PM, Simon Weller <sw...@ena.com> wrote:
> I think our other option is to take a real look at what it would take to
> fix the VR. In my opinion, a lot of the problems are related to the
> monolithic python code base and the fact nothing is actually separated.
>
> Secondly, the python scripts (and bash scripts) don't use any established
> libraries to complete tasks and instead shell out and run commands that are
> both hard to track and hard to parse on return.
>
>
> If we daemonized this, used a real api for Agent to VR communication, used
> common already existing libraries for the system service and network
> interactions and spent a bit of time separating out code into distinct
> modules, everything would behave a lot better.
>
>
> The pain and suffering is due to years and years of patches and constant
> shelling out to complete tasks in my opinion. If we spend time to rethink
> how we interact with the VR in general and we abstract the systems and
> networking stuff and use well known and stable libraries to do the work,
> the VR would be much easier to maintain.
>
>
> - Si
>
>
>
>
> ________________________________
> From: Marty Godsey <ma...@gonsource.com>
> Sent: Friday, September 16, 2016 12:24 PM
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] Replacing the VR
>
> So based upon this discussion would it be prudent to wait on VyOS 2.0? The
> current VR is giving us issues but would the time invested in another
> "solution" be wasted especially if by the time another option is chose,
> then coded, then tested, then implemented and right as that time happened
> to be when VyOS 2.0 is released. Of course you said they are just in the
> scoping range so this could still be a year or more out.
>
> Thoughts?
>
> Regards,
> Marty Godsey
> nSource Solutions
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: Friday, September 16, 2016 10:31 AM
> To: dev@cloudstack.apache.org
> Cc: daniil@baturin.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> I just had a quick chat with a couple of the guys over on the VyOS chat.
> I have CC'ed one of them in case we have more licensing questions.
>
> So here is the status with the license "the code inherited from Vyatta and
> our modifications from it is GPLv2 (strict, not v2+). The config reading
> library is GPLv2 too, so anything that links to is is GPLv2.
> Some auxiliary components we made after the fork are more permissive,
> LGPLv2+ or MIT."
>
> They are currently in the process of scoping a redesign (VyOS 2.0), "we
> are planning a clean rewrite that will solve issues of the current config
> system".
> This will include the ability to configure via the API.
>
> If we have more questions for VyOS, they are very friendly and responsive,
> so we should be able to get answers.
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>
> > I agree with Will Ilya. There are so many problems with the VR right now.
> > Most of the outages we've had recently have somehow involved the VR.
> > We set custom iptables rules on the VR which can and have easily gone
> wrong.
> > Openswan is broken, Strongswan replacement still needs to be tested.
> > VVRP with redundant router still needs work, and not to mention the
> > problems we will have when we introduce IPv6 into the whole picture.
> >
> > I think the spirit of the discussion is to rely on a 3rd party to do
> > the networking for us (eg VyOS) and have us handle just the
> > orchestration. All the problems that I've described have already been
> > solved in VyOS. We also get the advantage of a potential wider
> > community to fix and maintain the VR and given our current development
> > velocity, it think it totally makes sense to look for a 3rd party option.
> >
> > -Syed
> >
> >
> > On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
> > wrote:
> >
> > > The VR has been biting us far too often recently, which is why we
> > > have started looking into alternative implementations.
> > >
> > > One of the things that is nice about potentially using the VyOS is
> > > that
> > it
> > > is based on Debian, so we should be able to run the other services
> > > that
> > we
> > > currently have like the password server and userdata on the VyOS.
> > > This means we would not have to change our architecture initially
> > > and could focus on only replacing the networking paths.
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > > tw @CloudOps_
> > >
> > > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> > >
> > > > The more this is discussed the more I think we should stick with
> > > > our
> > VR.
> > > >
> > > > All these other options either seem unfinished or with
> > > > incompatible license.
> > > >
> > > > VyOS looks the most promising so far, it's a serious, mature project.
> > > > Adopting it though means we'll have to microservice our way out of
> > > > it
> > > with
> > > > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
> > those
> > > > too. Imho this adds complexity we should void.
> > > >
> > > > --
> > > > Sent from the Delta quadrant using Borg technology!
> > > >
> > > > Nux!
> > > > www.nux.ro
> > > >
> > > > ----- Original Message -----
> > > > > From: "Will Stevens" <ws...@cloudops.com>
> > > > > To: dev@cloudstack.apache.org
> > > > > Sent: Thursday, 15 September, 2016 17:21:28
> > > > > Subject: Re: [DISCUSS] Replacing the VR
> > > >
> > > > > Ya, we would need to add a daemon for VPN as well. Load
> > > > > balancing is another aspect which we will need to consider if we
> went this route.
> > > > > Something like https://traefik.io/ could potentially be a good
> > > > > fit
> > due
> > > > to
> > > > > its API driven configuration, but it may be more than what we need.
> > > > >
> > > > > We should probably try define which pieces make sense to be
> > > > > solved
> > > > together
> > > > > and which pieces would be best suited to be broken out.
> > > > >
> > > > > I think the network connectivity, routing and firewalling should
> > > probably
> > > > > all stay together since the majority of the tools we would
> > potentially
> > > > use
> > > > > would handle all of that together in a single implementation.
> > > > >
> > > > > The password server and userdata seems like a good option for
> > > > > being
> > > > broken
> > > > > out and handled independently (and probably rewritten completely
> > since
> > > > they
> > > > > currently have some issues).
> > > > >
> > > > > Load balancing is another that could warrant splitting out, but
> > > > > that depends on what direction we go and how we would be managing
> it.
> > DHCP
> > > > and
> > > > > DNS are others which could go either way.
> > > > >
> > > > > If we do split out services, I think we should consolidate as
> > > > > much as
> > > we
> > > > > can into each service we break out. Ideally a network packet
> > > > > would
> > > never
> > > > > hit more than one, maybe two, services. I don't think we should
> > > > > be splitting services 'just because', I think we need a valid
> > > > > case for splitting any service out because it adds complexity.
> > > > > Our project is already complex enough, we need to avoid adding
> > > > > complexity unless it
> > is
> > > > > really needed.
> > > > >
> > > > > Some more of my thoughts on this anyway...
> > > > >
> > > > > *Will STEVENS*
> > > > > Lead Developer
> > > > >
> > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> > > > > *|* tw @CloudOps_
> > > > >
> > > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
> > > wrote:
> > > > >
> > > > >> I do agree with you that this probably isn't the right place
> > > > >> the
> > > > password
> > > > >> service and user data.
> > > > >>
> > > > >>
> > > > >> Having said that, after taking a cursory look at the dev docs,
> > > > >> it
> > > > doesn't
> > > > >> seem that difficult to add new daemons:
> > https://opensnaproute.github.
> > > > >> io/docs/developer.html#creating-new-component
> > > > >>
> > > > >> <https://opensnaproute.github.io/docs/developer.html#
> > > > >> creating-new-component>
> > > > >>
> > > > >>
> > > > >> They've definitely build it with a microservices architecture
> > > > >> in
> > mind,
> > > > so
> > > > >> each individual feature is abstracted into it's own small
> > > > >> daemon
> > > > process.
> > > > >> We could just create a daemon for the password server and the
> > userdata
> > > > >> components if we really had to.
> > > > >>
> > > > >>
> > > > >> - Si
> > > > >>
> > > > >>
> > > > >> ________________________________
> > > > >> From: williamstevens@gmail.com <wi...@gmail.com> on
> > > > >> behalf
> > > of
> > > > >> Will Stevens <ws...@cloudops.com>
> > > > >> Sent: Thursday, September 15, 2016 9:17 AM
> > > > >> To: dev@cloudstack.apache.org
> > > > >> Subject: Re: [DISCUSS] Replacing the VR
> > > > >>
> > > > >> A big part of why I know about it is because it is written in Go.
> > :P
> > > > >>
> > > > >> Yes, it is definitely interesting for the routing and traffic
> > handling
> > > > >> aspects of the VR. We will likely have to rethink some of the
> > pieces
> > > a
> > > > >> little bit like the password server and userdata if we are to
> > > > >> adopt
> > a
> > > > >> different VR approach. This is where I think some of JohnB and
> > > > Chiradeep's
> > > > >> ideas make sense. In many ways, it does not make sense for the
> > device
> > > > >> handling routing and network traffic to also be responsible for
> > > > passwords
> > > > >> and userdata.
> > > > >>
> > > > >> *Will STEVENS*
> > > > >> Lead Developer
> > > > >>
> > > > >> *CloudOps* *| *Cloud Solutions Experts
> > > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> > > > >> *|* tw @CloudOps_
> > > > >>
> > > > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
> > > wrote:
> > > > >>
> > > > >> > I hadn't heard of Flexswitch until you mentioned it. It looks
> > pretty
> > > > >> cool!
> > > > >> > It even supports ONIE install.
> > > > >> >
> > > > >> > To be honest, the ipsec feature could be added, or we could
> > offload
> > > > it to
> > > > >> > separate vm if we needed to. The fact it is so feature rich
> > > > >> > from a
> > > > >> routing
> > > > >> > perspective (and all API driven) is really nice.
> > > > >> >
> > > > >> >
> > > > >> > Based on the roadmap, it looks like they plan to also support
> > > > >> capabilities
> > > > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
> > > > >> > will
> > be
> > > > huge
> > > > >> > for our carrier community that rely on these technologies to
> > > > >> > do
> > > > private
> > > > >> > gateway and inter-VPC interconnections today. We handle this
> > > > >> > stuff
> > > on
> > > > our
> > > > >> > ASRs right now with a vlan interconnect into the VR. Being
> > > > >> > able to
> > > do
> > > > >> MPLS
> > > > >> > all the way to the VR would be awesome.
> > > > >> >
> > > > >> >
> > > > >> > It also seems to be written in GO (a language here at ENA we
> > > > >> > know
> > > very
> > > > >> > well).
> > > > >> >
> > > > >> >
> > > > >> > - Si
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > ________________________________
> > > > >> > From: Will Stevens <wi...@gmail.com>
> > > > >> > Sent: Thursday, September 15, 2016 7:06 AM
> > > > >> > To: dev@cloudstack.apache.org
> > > > >> > Subject: RE: [DISCUSS] Replacing the VR
> > > > >> >
> > > > >> > Ya. I don't think it covers our whole use case, but what it
> > > > >> > does
> > > > cover is
> > > > >> > all api driven...
> > > > >> >
> > > > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
> > > wrote:
> > > > >> >
> > > > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it
> > > > >> > > was
> > not
> > > > >> > > intended to do IPSec.
> > > > >> > >
> > > > >> > > It seems as though VyOS is starting to look like the best
> > option.
> > > > >> > >
> > > > >> > > Regards,
> > > > >> > > Marty Godsey
> > > > >> > > nSource Solutions
> > > > >> > >
> > > > >> > > -----Original Message-----
> > > > >> > > From: williamstevens@gmail.com
> > > > >> > > [mailto:williamstevens@gmail.com
> > ]
> > > On
> > > > >> > > Behalf Of Will Stevens
> > > > >> > > Sent: Wednesday, September 14, 2016 11:06 PM
> > > > >> > > To: dev@cloudstack.apache.org
> > > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > > > >> > >
> > > > >> > > Or we could go completely crazy and go with something like
> > > > FlexSwitch
> > > > >> > from
> > > > >> > > SnapRoute
> > > > >> > > - http://www.snaproute.com/
> > > > >> > > - https://opensnaproute.github.io/docs/apis.html
> > > > >> > >
> > > > >> > > *Will STEVENS*
> > > > >> > > Lead Developer
> > > > >> > >
> > > > >> > > *CloudOps* *| *Cloud Solutions Experts
> > > > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > > >> > > cloudops.com
> > > *|*
> > > > tw
> > > > >> > > @CloudOps_
> > > > >> > >
> > > > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> > > > wstevens@cloudops.com>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > I tend to agree with Syed and Marty. I am not sure what
> > > problems
> > > > are
> > > > >> > > > solved by splitting up the function of the VR into a
> > > > >> > > > bunch of
> > > > >> separate
> > > > >> > > > services. As Syed points out, the complexity added is
> > > > non-trivial.
> > > > >> > > > We now have to manage all the intercontainer networking
> > > > >> > > > as
> > well
> > > as
> > > > >> the
> > > > >> > > > orchestrated ACS networking.
> > > > >> > > >
> > > > >> > > > VyOS is interesting to me because it covers the majority
> > > > >> > > > of
> > our
> > > > use
> > > > >> > > > case with a single unified control plane. It also has
> > > > >> > > > good
> > > > support
> > > > >> > > > for extending features we care about, like IPv6, VXLAN,
> > > > >> > > > VRRP, transactions, etc...
> > > > >> > > >
> > > > >> > > > *Will STEVENS*
> > > > >> > > > Lead Developer
> > > > >> > > >
> > > > >> > > > *CloudOps* *| *Cloud Solutions Experts
> > > > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > cloudops.com
> > > > *|*
> > > > >> tw
> > > > >> > > > @CloudOps_
> > > > >> > > >
> > > > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> > > sahmed@cloudops.com>
> > > > >> > wrote:
> > > > >> > > >
> > > > >> > > >> Agree with Marty, adding Docker containers to the
> > > > >> > > >> picture
> > > > although
> > > > >> > > >> can make the VR more flexible but the added complexity
> > > > >> > > >> is
> > just
> > > > not
> > > > >> > > >> worth it. Not to mention we would need to take care of
> > > networking
> > > > >> > > >> each container manually and given that our iptable rules
> > > > >> > > >> are
> > > very
> > > > >> > > >> unstable at the moment I don't see a big value add.
> > > > >> > > >>
> > > > >> > > >> Vyos looks like a better solution to me. I know that it
> > > > >> > > >> does
> > > not
> > > > >> > > >> provide an api but it does fit the bill quite well
> > otherwise. I
> > > > >> > > >> specially like the fact that it has a transaction based
> > > > >> > > >> model
> > > and
> > > > >> you
> > > > >> > > >> can rollback changes if something goes wrong.
> > > > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> > > > marty@gonsource.com>
> > > > >> > > wrote:
> > > > >> > > >>
> > > > >> > > >> > Licensing aside, I think splitting the various
> > > > >> > > >> > functions
> > into
> > > > >> > > >> > containers is not a good route either. This will force
> > users
> > > to
> > > > >> > > >> > have to maintain
> > > > >> > > >> and
> > > > >> > > >> > use containers and adds complexity to the networking
> > aspects
> > > of
> > > > >> ACS.
> > > > >> > > >> > Complexity decreases stability. Now I understand the
> > argument
> > > > that
> > > > >> > > >> > a monolithic approach also brings its own set of
> > > > >> > > >> > issues but
> > > it
> > > > >> also
> > > > >> > > >> > simplifies it.
> > > > >> > > >> >
> > > > >> > > >> > Regards,
> > > > >> > > >> > Marty Godsey
> > > > >> > > >> > nSource Solutions
> > > > >> > > >> >
> > > > >> > > >> > -----Original Message-----
> > > > >> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> > > > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
> > > > >> > > >> > To: dev@cloudstack.apache.org
> > > > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR
> > > > >> > > >> >
> > > > >> > > >> > I rather doubt that the Cloudrouter will fit the needs
> > > > >> > > >> > of
> > the
> > > > >> > > >> > CloudStack project
> > > > >> > > >> > - it is AGPL licensed. Many enterprises will not
> > > > >> > > >> > touch
> > > > anything
> > > > >> > > >> > that
> > > > >> > > >> has
> > > > >> > > >> > AGPL
> > > > >> > > >> > - the github repo shows rather infrequent updates.
> > > > >> > > >> > Quite
> > > > likely
> > > > >> > > >> > they aren't considering the use cases of the
> > > > >> > > >> > CloudStack
> > > > community
> > > > >> > > >> >
> > > > >> > > >> > I'd back John B's comments on disaggregating the VR.
> > > > >> > > >> > Split
> > it
> > > > into
> > > > >> > > >> > many docker containers
> > > > >> > > >> > - password server
> > > > >> > > >> > - userdata server
> > > > >> > > >> > - DHCP / DNS
> > > > >> > > >> > - s2s VPN
> > > > >> > > >> > - RA VPN
> > > > >> > > >> > - intra-VPC routing and ACL
> > > > >> > > >> > - Port forwarding + NAT
> > > > >> > > >> > - FW
> > > > >> > > >> > - LB (public)
> > > > >> > > >> > - LB (internal),
> > > > >> > > >> > - secondary storage
> > > > >> > > >> > - agent
> > > > >> > > >> > Glue them together with docker compose files (one per
> > > > >> > > >> > use
> > > > case -
> > > > >> > > >> > basic zone, isolated, VPC, SSVM, etc).
> > > > >> > > >> >
> > > > >> > > >> > The VR image then becomes a JeOS + docker. You can
> > > > >> > > >> > test
> > each
> > > of
> > > > >> the
> > > > >> > > >> > components independently and fixing one bug in the
> > > > >> > > >> > field
> > (say
> > > > >> DHCP)
> > > > >> > > >> > is hitless to the other components. You don't need to
> > > > >> > > >> > build per-hypervisor VRs. You could even run on
> baremetal.
> > > > >> > > >> >
> > > > >> > > >> > Along the way you need to figure out how to
> > > > >> > > >> > - make the traffic traverse the containers that are
> > > > >> > > >> > needed
> > > to
> > > > be
> > > > >> > > >> > traversed (in most cases just 1)
> > > > >> > > >> > - bootstrap the router (how does it find its compose
> file?
> > > > where
> > > > >> > > >> > is the
> > > > >> > > >> > registry?)
> > > > >> > > >> > - rethink the command and control of the VR
> > > > >> > > >> > functions. SSH
> > > > works,
> > > > >> > > >> > but something more declarative, idempotent should be
> > > explored.
> > > > >> > > >> >
> > > > >> > > >> > As you do this, it becomes clearer which of the
> > > > >> > > >> > functions
> > can
> > > > be
> > > > >> > > >> > substituted by for example CloudRouter. Command and
> > > > >> > > >> > Control
> > > of
> > > > the
> > > > >> > > >> docker
> > > > >> > > >> > containers can be moved out to another container. Etc.
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> > > > >> > > >> > <ma...@gonsource.com>
> > > > >> > > >> > wrote:
> > > > >> > > >> >
> > > > >> > > >> > > This one does look nice. My biggest concern is the
> > > > >> > > >> > > lack
> > of
> > > > >> > > >> > > VXLANs. It seems that any of the ones we mentioned
> > > > >> > > >> > > do not
> > > > have
> > > > >> an
> > > > >> > > >> > > API so we may be stuck at the SSH method.
> > > > >> > > >> > >
> > > > >> > > >> > > Regards,
> > > > >> > > >> > > Marty Godsey
> > > > >> > > >> > > nSource Solutions
> > > > >> > > >> > >
> > > > >> > > >> > > -----Original Message-----
> > > > >> > > >> > > From: Abhinandan Prateek
> > > > >> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
> > > > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> > > > >> > > >> > > To: dev@cloudstack.apache.org
> > > > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > > > >> > > >> > >
> > > > >> > > >> > > Cloudrouter looks promising. These have potential to
> > > > >> > > >> > > save
> > > > future
> > > > >> > > >> > > engineering effort for example on ipv6 routing, OSPF
> etc.
> > > > >> > > >> > > And the best part is they come with test automation
> > > > framework.
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> > > > >> > > >> > > <ja...@accelerite.com>
> > > > >> > > >> > > wrote:
> > > > >> > > >> > >
> > > > >> > > >> > > >Hi,
> > > > >> > > >> > > >
> > > > >> > > >> > > >Instead of replacing the VR in first place we
> > > > >> > > >> > > >should add VyOS/cloudrouter
> > > > >> > > >> > > as provider. Once it is stable, network offerings
> > > > >> > > >> > > (on
> > > > upgrade)
> > > > >> > > >> > > can be updated to use it and we can drop the VR if
> > > > >> > > >> > > we
> > want
> > > at
> > > > >> > > >> > > that release
> > > > >> > > >> > onwards.
> > > > >> > > >> > > >
> > > > >> > > >> > > >VR is stabilized over a period of time and some of
> > > > >> > > >> > > >them
> > > are
> > > > >> > > >> > > >running
> > > > >> > > >> > > without issues. When we replicate the ACS VR
> > > > >> > > >> > > features in
> > > new
> > > > >> > > >> > > solution it takes some to find the missing pieces
> > > > >> > > >> > > (hidden
> > > > bugs).
> > > > >> > > >> > > >
> > > > >> > > >> > > >Thanks,
> > > > >> > > >> > > >Jayapal
> > > > >> > > >> > > >
> > > > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> > > > >> > > >> > > >
> > > > >> > > >> > > >> nux@li.nux.ro> wrote:
> > > > >> > > >> > > >>
> > > > >> > > >> > > >> Hi,
> > > > >> > > >> > > >>
> > > > >> > > >> > > >> I like the idea.
> > > > >> > > >> > > >>
> > > > >> > > >> > > >> Cloudrouter looks really promising, I'm not too
> > > > >> > > >> > > >> keen
> > on
> > > > VyOS
> > > > >> > > >> > > >> (it
> > > > >> > > >> > > doesn't have a proper http api etc).
> > > > >> > > >> > > >>
> > > > >> > > >> > > >> --
> > > > >> > > >> > > >> Sent from the Delta quadrant using Borg technology!
> > > > >> > > >> > > >>
> > > > >> > > >> > > >> Nux!
> > > > >> > > >> > > >> www.nux.ro
> > > > >> > > >> > > >>
> > > > >> > > >> > > >>
> > > > >> > > >> > > abhinandan.prateek@shapeblue.com
> > > > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
> > > > >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> > > > @shapeblue
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > >
> > > > >> > > >> > > ----- Original Message -----
> > > > >> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> > > > >> > > >> > > >>> To: dev@cloudstack.apache.org
> > > > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> > > > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
> > > > >> > > >> > > >>
> > > > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and
> > > > >> > > >> > > >>> should
> > > be
> > > > >> > > >> > > >>> treated as
> > > > >> > > >> > > such.
> > > > >> > > >> > > >>> Please weigh in with the good and bad of this
> idea...
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> A couple of us have been discussing the idea of
> > > > potentially
> > > > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open
> > > > >> > > >> > > >>> Source
> > > > Vyatta
> > > > >> > VM).
> > > > >> > > >> > > >>> There may be a license issue because I think it
> > > > >> > > >> > > >>> is
> > > > licensed
> > > > >> > > >> > > >>> under GPL, but for the sake of discussion, let's
> > assume
> > > > we
> > > > >> > > >> > > >>> can overcome any
> > > > >> > > >> > > license issues.
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> I have spent some time recently with the VyOS
> > > > >> > > >> > > >>> and I
> > > have
> > > > to
> > > > >> > > >> > > >>> admit, I was pretty impressed. It is simple and
> > > > intuitive
> > > > >> > > >> > > >>> and it gives you a lot more options for auditing
> > > > >> > > >> > > >>> the
> > > > >> > > configuration etc...
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> Items of potential interest:
> > > > >> > > >> > > >>> - Clean up our current VR script spaghetti to a
> > simpler
> > > > more
> > > > >> > > >> > > >>> auditable configuration workflow.
> > > > >> > > >> > > >>> - Gives a cleaner path for IPv6 support.
> > > > >> > > >> > > >>> - Handles VPN configuration via the same
> > configuration
> > > > >> > > interface.
> > > > >> > > >> > > >>> - Support for OSPF & BGP.
> > > > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
> > > > >> > > >> > > >>> - Easily supports HA (redundant routers) through
> > VRRP.
> > > > >> > > >> > > >>> - VXLAN support.
> > > > >> > > >> > > >>> - Transaction based changes to the VR with
> > > > >> > > >> > > >>> rollback
> > on
> > > > >> error.
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> Items that could be difficult to solve:
> > > > >> > > >> > > >>> - Userdata password reset workflow and
> > implementation.
> > > > >> > > >> > > >>> - Upgrade process.
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> The VyOS is not the only option if we were to
> > consider
> > > > this
> > > > >> > > >> approach.
> > > > >> > > >> > > >>> Another option, which I don't know as well,
> > > > >> > > >> > > >>> would be CloudRouter (AGPL
> > > > >> > > >> > > >>> license) [2] which is purely API driven.
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> Anyway, would love to hear your thoughts...
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> Will
> > > > >> > > >> > > >>>
> > > > >> > > >> > > >>> [1] https://vyos.io/ [2]
> > > > >> > > >> > > >>> https://cloudrouter.org/
> > > > >> > > >> > > >
> > > > >> > > >> > > >
> > > > >> > > >> > > >
> > > > >> > > >> > > >
> > > > >> > > >> > > >DISCLAIMER
> > > > >> > > >> > > >==========
> > > > >> > > >> > > >This e-mail may contain privileged and confidential
> > > > information
> > > > >> > > >> > > >which is
> > > > >> > > >> > > the property of Accelerite, a Persistent Systems
> > business.
> > > > It is
> > > > >> > > >> > > intended only for the use of the individual or
> > > > >> > > >> > > entity to
> > > > which
> > > > >> it
> > > > >> > > >> > > is addressed. If you are not the intended recipient,
> > > > >> > > >> > > you
> > > are
> > > > not
> > > > >> > > >> > > authorized to read, retain, copy, print, distribute
> > > > >> > > >> > > or
> > use
> > > > this
> > > > >> > > >> > > message. If you have received this communication in
> > error,
> > > > >> please
> > > > >> > > >> > > notify the sender and delete all copies of this
> message.
> > > > >> > > >> > > Accelerite, a Persistent Systems business does not
> > > > >> > > >> > > accept
> > > any
> > > > >> > > >> > > liability for virus
> > > > >> > > >> > infected mails.
> > > > >> > > >> > >
> > > > >> > > >> >
> > > > >> > > >>
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > >
> > >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Simon Weller <sw...@ena.com>.
I think our other option is to take a real look at what it would take to fix the VR. In my opinion, a lot of the problems are related to the monolithic python code base and the fact nothing is actually separated.
Secondly, the python scripts (and bash scripts) don't use any established libraries to complete tasks and instead shell out and run commands that are both hard to track and hard to parse on return.
If we daemonized this, used a real api for Agent to VR communication, used common already existing libraries for the system service and network interactions and spent a bit of time separating out code into distinct modules, everything would behave a lot better.
The pain and suffering is due to years and years of patches and constant shelling out to complete tasks in my opinion. If we spend time to rethink how we interact with the VR in general and we abstract the systems and networking stuff and use well known and stable libraries to do the work, the VR would be much easier to maintain.
- Si
________________________________
From: Marty Godsey <ma...@gonsource.com>
Sent: Friday, September 16, 2016 12:24 PM
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] Replacing the VR
So based upon this discussion would it be prudent to wait on VyOS 2.0? The current VR is giving us issues but would the time invested in another "solution" be wasted especially if by the time another option is chose, then coded, then tested, then implemented and right as that time happened to be when VyOS 2.0 is released. Of course you said they are just in the scoping range so this could still be a year or more out.
Thoughts?
Regards,
Marty Godsey
nSource Solutions
-----Original Message-----
From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
Sent: Friday, September 16, 2016 10:31 AM
To: dev@cloudstack.apache.org
Cc: daniil@baturin.org
Subject: Re: [DISCUSS] Replacing the VR
I just had a quick chat with a couple of the guys over on the VyOS chat.
I have CC'ed one of them in case we have more licensing questions.
So here is the status with the license "the code inherited from Vyatta and our modifications from it is GPLv2 (strict, not v2+). The config reading library is GPLv2 too, so anything that links to is is GPLv2.
Some auxiliary components we made after the fork are more permissive,
LGPLv2+ or MIT."
They are currently in the process of scoping a redesign (VyOS 2.0), "we are planning a clean rewrite that will solve issues of the current config system".
This will include the ability to configure via the API.
If we have more questions for VyOS, they are very friendly and responsive, so we should be able to get answers.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_
On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
> I agree with Will Ilya. There are so many problems with the VR right now.
> Most of the outages we've had recently have somehow involved the VR.
> We set custom iptables rules on the VR which can and have easily gone wrong.
> Openswan is broken, Strongswan replacement still needs to be tested.
> VVRP with redundant router still needs work, and not to mention the
> problems we will have when we introduce IPv6 into the whole picture.
>
> I think the spirit of the discussion is to rely on a 3rd party to do
> the networking for us (eg VyOS) and have us handle just the
> orchestration. All the problems that I've described have already been
> solved in VyOS. We also get the advantage of a potential wider
> community to fix and maintain the VR and given our current development
> velocity, it think it totally makes sense to look for a 3rd party option.
>
> -Syed
>
>
> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
> wrote:
>
> > The VR has been biting us far too often recently, which is why we
> > have started looking into alternative implementations.
> >
> > One of the things that is nice about potentially using the VyOS is
> > that
> it
> > is based on Debian, so we should be able to run the other services
> > that
> we
> > currently have like the password server and userdata on the VyOS.
> > This means we would not have to change our architecture initially
> > and could focus on only replacing the networking paths.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > tw @CloudOps_
> >
> > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> >
> > > The more this is discussed the more I think we should stick with
> > > our
> VR.
> > >
> > > All these other options either seem unfinished or with
> > > incompatible license.
> > >
> > > VyOS looks the most promising so far, it's a serious, mature project.
> > > Adopting it though means we'll have to microservice our way out of
> > > it
> > with
> > > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
> those
> > > too. Imho this adds complexity we should void.
> > >
> > > --
> > > Sent from the Delta quadrant using Borg technology!
> > >
> > > Nux!
> > > www.nux.ro
> > >
> > > ----- Original Message -----
> > > > From: "Will Stevens" <ws...@cloudops.com>
> > > > To: dev@cloudstack.apache.org
> > > > Sent: Thursday, 15 September, 2016 17:21:28
> > > > Subject: Re: [DISCUSS] Replacing the VR
> > >
> > > > Ya, we would need to add a daemon for VPN as well. Load
> > > > balancing is another aspect which we will need to consider if we went this route.
> > > > Something like https://traefik.io/ could potentially be a good
> > > > fit
> due
> > > to
> > > > its API driven configuration, but it may be more than what we need.
> > > >
> > > > We should probably try define which pieces make sense to be
> > > > solved
> > > together
> > > > and which pieces would be best suited to be broken out.
> > > >
> > > > I think the network connectivity, routing and firewalling should
> > probably
> > > > all stay together since the majority of the tools we would
> potentially
> > > use
> > > > would handle all of that together in a single implementation.
> > > >
> > > > The password server and userdata seems like a good option for
> > > > being
> > > broken
> > > > out and handled independently (and probably rewritten completely
> since
> > > they
> > > > currently have some issues).
> > > >
> > > > Load balancing is another that could warrant splitting out, but
> > > > that depends on what direction we go and how we would be managing it.
> DHCP
> > > and
> > > > DNS are others which could go either way.
> > > >
> > > > If we do split out services, I think we should consolidate as
> > > > much as
> > we
> > > > can into each service we break out. Ideally a network packet
> > > > would
> > never
> > > > hit more than one, maybe two, services. I don't think we should
> > > > be splitting services 'just because', I think we need a valid
> > > > case for splitting any service out because it adds complexity.
> > > > Our project is already complex enough, we need to avoid adding
> > > > complexity unless it
> is
> > > > really needed.
> > > >
> > > > Some more of my thoughts on this anyway...
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > *CloudOps* *| *Cloud Solutions Experts
> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> > > > *|* tw @CloudOps_
> > > >
> > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
> > wrote:
> > > >
> > > >> I do agree with you that this probably isn't the right place
> > > >> the
> > > password
> > > >> service and user data.
> > > >>
> > > >>
> > > >> Having said that, after taking a cursory look at the dev docs,
> > > >> it
> > > doesn't
> > > >> seem that difficult to add new daemons:
> https://opensnaproute.github.
> > > >> io/docs/developer.html#creating-new-component
> > > >>
> > > >> <https://opensnaproute.github.io/docs/developer.html#
> > > >> creating-new-component>
> > > >>
> > > >>
> > > >> They've definitely build it with a microservices architecture
> > > >> in
> mind,
> > > so
> > > >> each individual feature is abstracted into it's own small
> > > >> daemon
> > > process.
> > > >> We could just create a daemon for the password server and the
> userdata
> > > >> components if we really had to.
> > > >>
> > > >>
> > > >> - Si
> > > >>
> > > >>
> > > >> ________________________________
> > > >> From: williamstevens@gmail.com <wi...@gmail.com> on
> > > >> behalf
> > of
> > > >> Will Stevens <ws...@cloudops.com>
> > > >> Sent: Thursday, September 15, 2016 9:17 AM
> > > >> To: dev@cloudstack.apache.org
> > > >> Subject: Re: [DISCUSS] Replacing the VR
> > > >>
> > > >> A big part of why I know about it is because it is written in Go.
> :P
> > > >>
> > > >> Yes, it is definitely interesting for the routing and traffic
> handling
> > > >> aspects of the VR. We will likely have to rethink some of the
> pieces
> > a
> > > >> little bit like the password server and userdata if we are to
> > > >> adopt
> a
> > > >> different VR approach. This is where I think some of JohnB and
> > > Chiradeep's
> > > >> ideas make sense. In many ways, it does not make sense for the
> device
> > > >> handling routing and network traffic to also be responsible for
> > > passwords
> > > >> and userdata.
> > > >>
> > > >> *Will STEVENS*
> > > >> Lead Developer
> > > >>
> > > >> *CloudOps* *| *Cloud Solutions Experts
> > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> > > >> *|* tw @CloudOps_
> > > >>
> > > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
> > wrote:
> > > >>
> > > >> > I hadn't heard of Flexswitch until you mentioned it. It looks
> pretty
> > > >> cool!
> > > >> > It even supports ONIE install.
> > > >> >
> > > >> > To be honest, the ipsec feature could be added, or we could
> offload
> > > it to
> > > >> > separate vm if we needed to. The fact it is so feature rich
> > > >> > from a
> > > >> routing
> > > >> > perspective (and all API driven) is really nice.
> > > >> >
> > > >> >
> > > >> > Based on the roadmap, it looks like they plan to also support
> > > >> capabilities
> > > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
> > > >> > will
> be
> > > huge
> > > >> > for our carrier community that rely on these technologies to
> > > >> > do
> > > private
> > > >> > gateway and inter-VPC interconnections today. We handle this
> > > >> > stuff
> > on
> > > our
> > > >> > ASRs right now with a vlan interconnect into the VR. Being
> > > >> > able to
> > do
> > > >> MPLS
> > > >> > all the way to the VR would be awesome.
> > > >> >
> > > >> >
> > > >> > It also seems to be written in GO (a language here at ENA we
> > > >> > know
> > very
> > > >> > well).
> > > >> >
> > > >> >
> > > >> > - Si
> > > >> >
> > > >> >
> > > >> >
> > > >> > ________________________________
> > > >> > From: Will Stevens <wi...@gmail.com>
> > > >> > Sent: Thursday, September 15, 2016 7:06 AM
> > > >> > To: dev@cloudstack.apache.org
> > > >> > Subject: RE: [DISCUSS] Replacing the VR
> > > >> >
> > > >> > Ya. I don't think it covers our whole use case, but what it
> > > >> > does
> > > cover is
> > > >> > all api driven...
> > > >> >
> > > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
> > wrote:
> > > >> >
> > > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it
> > > >> > > was
> not
> > > >> > > intended to do IPSec.
> > > >> > >
> > > >> > > It seems as though VyOS is starting to look like the best
> option.
> > > >> > >
> > > >> > > Regards,
> > > >> > > Marty Godsey
> > > >> > > nSource Solutions
> > > >> > >
> > > >> > > -----Original Message-----
> > > >> > > From: williamstevens@gmail.com
> > > >> > > [mailto:williamstevens@gmail.com
> ]
> > On
> > > >> > > Behalf Of Will Stevens
> > > >> > > Sent: Wednesday, September 14, 2016 11:06 PM
> > > >> > > To: dev@cloudstack.apache.org
> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > > >> > >
> > > >> > > Or we could go completely crazy and go with something like
> > > FlexSwitch
> > > >> > from
> > > >> > > SnapRoute
> > > >> > > - http://www.snaproute.com/
> > > >> > > - https://opensnaproute.github.io/docs/apis.html
> > > >> > >
> > > >> > > *Will STEVENS*
> > > >> > > Lead Developer
> > > >> > >
> > > >> > > *CloudOps* *| *Cloud Solutions Experts
> > > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > >> > > cloudops.com
> > *|*
> > > tw
> > > >> > > @CloudOps_
> > > >> > >
> > > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> > > wstevens@cloudops.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > I tend to agree with Syed and Marty. I am not sure what
> > problems
> > > are
> > > >> > > > solved by splitting up the function of the VR into a
> > > >> > > > bunch of
> > > >> separate
> > > >> > > > services. As Syed points out, the complexity added is
> > > non-trivial.
> > > >> > > > We now have to manage all the intercontainer networking
> > > >> > > > as
> well
> > as
> > > >> the
> > > >> > > > orchestrated ACS networking.
> > > >> > > >
> > > >> > > > VyOS is interesting to me because it covers the majority
> > > >> > > > of
> our
> > > use
> > > >> > > > case with a single unified control plane. It also has
> > > >> > > > good
> > > support
> > > >> > > > for extending features we care about, like IPv6, VXLAN,
> > > >> > > > VRRP, transactions, etc...
> > > >> > > >
> > > >> > > > *Will STEVENS*
> > > >> > > > Lead Developer
> > > >> > > >
> > > >> > > > *CloudOps* *| *Cloud Solutions Experts
> > > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> cloudops.com
> > > *|*
> > > >> tw
> > > >> > > > @CloudOps_
> > > >> > > >
> > > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> > sahmed@cloudops.com>
> > > >> > wrote:
> > > >> > > >
> > > >> > > >> Agree with Marty, adding Docker containers to the
> > > >> > > >> picture
> > > although
> > > >> > > >> can make the VR more flexible but the added complexity
> > > >> > > >> is
> just
> > > not
> > > >> > > >> worth it. Not to mention we would need to take care of
> > networking
> > > >> > > >> each container manually and given that our iptable rules
> > > >> > > >> are
> > very
> > > >> > > >> unstable at the moment I don't see a big value add.
> > > >> > > >>
> > > >> > > >> Vyos looks like a better solution to me. I know that it
> > > >> > > >> does
> > not
> > > >> > > >> provide an api but it does fit the bill quite well
> otherwise. I
> > > >> > > >> specially like the fact that it has a transaction based
> > > >> > > >> model
> > and
> > > >> you
> > > >> > > >> can rollback changes if something goes wrong.
> > > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> > > marty@gonsource.com>
> > > >> > > wrote:
> > > >> > > >>
> > > >> > > >> > Licensing aside, I think splitting the various
> > > >> > > >> > functions
> into
> > > >> > > >> > containers is not a good route either. This will force
> users
> > to
> > > >> > > >> > have to maintain
> > > >> > > >> and
> > > >> > > >> > use containers and adds complexity to the networking
> aspects
> > of
> > > >> ACS.
> > > >> > > >> > Complexity decreases stability. Now I understand the
> argument
> > > that
> > > >> > > >> > a monolithic approach also brings its own set of
> > > >> > > >> > issues but
> > it
> > > >> also
> > > >> > > >> > simplifies it.
> > > >> > > >> >
> > > >> > > >> > Regards,
> > > >> > > >> > Marty Godsey
> > > >> > > >> > nSource Solutions
> > > >> > > >> >
> > > >> > > >> > -----Original Message-----
> > > >> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> > > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
> > > >> > > >> > To: dev@cloudstack.apache.org
> > > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR
> > > >> > > >> >
> > > >> > > >> > I rather doubt that the Cloudrouter will fit the needs
> > > >> > > >> > of
> the
> > > >> > > >> > CloudStack project
> > > >> > > >> > - it is AGPL licensed. Many enterprises will not
> > > >> > > >> > touch
> > > anything
> > > >> > > >> > that
> > > >> > > >> has
> > > >> > > >> > AGPL
> > > >> > > >> > - the github repo shows rather infrequent updates.
> > > >> > > >> > Quite
> > > likely
> > > >> > > >> > they aren't considering the use cases of the
> > > >> > > >> > CloudStack
> > > community
> > > >> > > >> >
> > > >> > > >> > I'd back John B's comments on disaggregating the VR.
> > > >> > > >> > Split
> it
> > > into
> > > >> > > >> > many docker containers
> > > >> > > >> > - password server
> > > >> > > >> > - userdata server
> > > >> > > >> > - DHCP / DNS
> > > >> > > >> > - s2s VPN
> > > >> > > >> > - RA VPN
> > > >> > > >> > - intra-VPC routing and ACL
> > > >> > > >> > - Port forwarding + NAT
> > > >> > > >> > - FW
> > > >> > > >> > - LB (public)
> > > >> > > >> > - LB (internal),
> > > >> > > >> > - secondary storage
> > > >> > > >> > - agent
> > > >> > > >> > Glue them together with docker compose files (one per
> > > >> > > >> > use
> > > case -
> > > >> > > >> > basic zone, isolated, VPC, SSVM, etc).
> > > >> > > >> >
> > > >> > > >> > The VR image then becomes a JeOS + docker. You can
> > > >> > > >> > test
> each
> > of
> > > >> the
> > > >> > > >> > components independently and fixing one bug in the
> > > >> > > >> > field
> (say
> > > >> DHCP)
> > > >> > > >> > is hitless to the other components. You don't need to
> > > >> > > >> > build per-hypervisor VRs. You could even run on baremetal.
> > > >> > > >> >
> > > >> > > >> > Along the way you need to figure out how to
> > > >> > > >> > - make the traffic traverse the containers that are
> > > >> > > >> > needed
> > to
> > > be
> > > >> > > >> > traversed (in most cases just 1)
> > > >> > > >> > - bootstrap the router (how does it find its compose file?
> > > where
> > > >> > > >> > is the
> > > >> > > >> > registry?)
> > > >> > > >> > - rethink the command and control of the VR
> > > >> > > >> > functions. SSH
> > > works,
> > > >> > > >> > but something more declarative, idempotent should be
> > explored.
> > > >> > > >> >
> > > >> > > >> > As you do this, it becomes clearer which of the
> > > >> > > >> > functions
> can
> > > be
> > > >> > > >> > substituted by for example CloudRouter. Command and
> > > >> > > >> > Control
> > of
> > > the
> > > >> > > >> docker
> > > >> > > >> > containers can be moved out to another container. Etc.
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> > > >> > > >> > <ma...@gonsource.com>
> > > >> > > >> > wrote:
> > > >> > > >> >
> > > >> > > >> > > This one does look nice. My biggest concern is the
> > > >> > > >> > > lack
> of
> > > >> > > >> > > VXLANs. It seems that any of the ones we mentioned
> > > >> > > >> > > do not
> > > have
> > > >> an
> > > >> > > >> > > API so we may be stuck at the SSH method.
> > > >> > > >> > >
> > > >> > > >> > > Regards,
> > > >> > > >> > > Marty Godsey
> > > >> > > >> > > nSource Solutions
> > > >> > > >> > >
> > > >> > > >> > > -----Original Message-----
> > > >> > > >> > > From: Abhinandan Prateek
> > > >> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
> > > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> > > >> > > >> > > To: dev@cloudstack.apache.org
> > > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > > >> > > >> > >
> > > >> > > >> > > Cloudrouter looks promising. These have potential to
> > > >> > > >> > > save
> > > future
> > > >> > > >> > > engineering effort for example on ipv6 routing, OSPF etc.
> > > >> > > >> > > And the best part is they come with test automation
> > > framework.
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> > > >> > > >> > > <ja...@accelerite.com>
> > > >> > > >> > > wrote:
> > > >> > > >> > >
> > > >> > > >> > > >Hi,
> > > >> > > >> > > >
> > > >> > > >> > > >Instead of replacing the VR in first place we
> > > >> > > >> > > >should add VyOS/cloudrouter
> > > >> > > >> > > as provider. Once it is stable, network offerings
> > > >> > > >> > > (on
> > > upgrade)
> > > >> > > >> > > can be updated to use it and we can drop the VR if
> > > >> > > >> > > we
> want
> > at
> > > >> > > >> > > that release
> > > >> > > >> > onwards.
> > > >> > > >> > > >
> > > >> > > >> > > >VR is stabilized over a period of time and some of
> > > >> > > >> > > >them
> > are
> > > >> > > >> > > >running
> > > >> > > >> > > without issues. When we replicate the ACS VR
> > > >> > > >> > > features in
> > new
> > > >> > > >> > > solution it takes some to find the missing pieces
> > > >> > > >> > > (hidden
> > > bugs).
> > > >> > > >> > > >
> > > >> > > >> > > >Thanks,
> > > >> > > >> > > >Jayapal
> > > >> > > >> > > >
> > > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> > > >> > > >> > > >
> > > >> > > >> > > >> nux@li.nux.ro> wrote:
> > > >> > > >> > > >>
> > > >> > > >> > > >> Hi,
> > > >> > > >> > > >>
> > > >> > > >> > > >> I like the idea.
> > > >> > > >> > > >>
> > > >> > > >> > > >> Cloudrouter looks really promising, I'm not too
> > > >> > > >> > > >> keen
> on
> > > VyOS
> > > >> > > >> > > >> (it
> > > >> > > >> > > doesn't have a proper http api etc).
> > > >> > > >> > > >>
> > > >> > > >> > > >> --
> > > >> > > >> > > >> Sent from the Delta quadrant using Borg technology!
> > > >> > > >> > > >>
> > > >> > > >> > > >> Nux!
> > > >> > > >> > > >> www.nux.ro
> > > >> > > >> > > >>
> > > >> > > >> > > >>
> > > >> > > >> > > abhinandan.prateek@shapeblue.com
> > > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
> > > >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> > > @shapeblue
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > > ----- Original Message -----
> > > >> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> > > >> > > >> > > >>> To: dev@cloudstack.apache.org
> > > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> > > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
> > > >> > > >> > > >>
> > > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and
> > > >> > > >> > > >>> should
> > be
> > > >> > > >> > > >>> treated as
> > > >> > > >> > > such.
> > > >> > > >> > > >>> Please weigh in with the good and bad of this idea...
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> A couple of us have been discussing the idea of
> > > potentially
> > > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open
> > > >> > > >> > > >>> Source
> > > Vyatta
> > > >> > VM).
> > > >> > > >> > > >>> There may be a license issue because I think it
> > > >> > > >> > > >>> is
> > > licensed
> > > >> > > >> > > >>> under GPL, but for the sake of discussion, let's
> assume
> > > we
> > > >> > > >> > > >>> can overcome any
> > > >> > > >> > > license issues.
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> I have spent some time recently with the VyOS
> > > >> > > >> > > >>> and I
> > have
> > > to
> > > >> > > >> > > >>> admit, I was pretty impressed. It is simple and
> > > intuitive
> > > >> > > >> > > >>> and it gives you a lot more options for auditing
> > > >> > > >> > > >>> the
> > > >> > > configuration etc...
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> Items of potential interest:
> > > >> > > >> > > >>> - Clean up our current VR script spaghetti to a
> simpler
> > > more
> > > >> > > >> > > >>> auditable configuration workflow.
> > > >> > > >> > > >>> - Gives a cleaner path for IPv6 support.
> > > >> > > >> > > >>> - Handles VPN configuration via the same
> configuration
> > > >> > > interface.
> > > >> > > >> > > >>> - Support for OSPF & BGP.
> > > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
> > > >> > > >> > > >>> - Easily supports HA (redundant routers) through
> VRRP.
> > > >> > > >> > > >>> - VXLAN support.
> > > >> > > >> > > >>> - Transaction based changes to the VR with
> > > >> > > >> > > >>> rollback
> on
> > > >> error.
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> Items that could be difficult to solve:
> > > >> > > >> > > >>> - Userdata password reset workflow and
> implementation.
> > > >> > > >> > > >>> - Upgrade process.
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> The VyOS is not the only option if we were to
> consider
> > > this
> > > >> > > >> approach.
> > > >> > > >> > > >>> Another option, which I don't know as well,
> > > >> > > >> > > >>> would be CloudRouter (AGPL
> > > >> > > >> > > >>> license) [2] which is purely API driven.
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> Anyway, would love to hear your thoughts...
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> Will
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> [1] https://vyos.io/ [2]
> > > >> > > >> > > >>> https://cloudrouter.org/
> > > >> > > >> > > >
> > > >> > > >> > > >
> > > >> > > >> > > >
> > > >> > > >> > > >
> > > >> > > >> > > >DISCLAIMER
> > > >> > > >> > > >==========
> > > >> > > >> > > >This e-mail may contain privileged and confidential
> > > information
> > > >> > > >> > > >which is
> > > >> > > >> > > the property of Accelerite, a Persistent Systems
> business.
> > > It is
> > > >> > > >> > > intended only for the use of the individual or
> > > >> > > >> > > entity to
> > > which
> > > >> it
> > > >> > > >> > > is addressed. If you are not the intended recipient,
> > > >> > > >> > > you
> > are
> > > not
> > > >> > > >> > > authorized to read, retain, copy, print, distribute
> > > >> > > >> > > or
> use
> > > this
> > > >> > > >> > > message. If you have received this communication in
> error,
> > > >> please
> > > >> > > >> > > notify the sender and delete all copies of this message.
> > > >> > > >> > > Accelerite, a Persistent Systems business does not
> > > >> > > >> > > accept
> > any
> > > >> > > >> > > liability for virus
> > > >> > > >> > infected mails.
> > > >> > > >> > >
> > > >> > > >> >
> > > >> > > >>
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > >
> >
>
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
So based upon this discussion would it be prudent to wait on VyOS 2.0? The current VR is giving us issues but would the time invested in another "solution" be wasted especially if by the time another option is chose, then coded, then tested, then implemented and right as that time happened to be when VyOS 2.0 is released. Of course you said they are just in the scoping range so this could still be a year or more out.
Thoughts?
Regards,
Marty Godsey
nSource Solutions
-----Original Message-----
From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
Sent: Friday, September 16, 2016 10:31 AM
To: dev@cloudstack.apache.org
Cc: daniil@baturin.org
Subject: Re: [DISCUSS] Replacing the VR
I just had a quick chat with a couple of the guys over on the VyOS chat.
I have CC'ed one of them in case we have more licensing questions.
So here is the status with the license "the code inherited from Vyatta and our modifications from it is GPLv2 (strict, not v2+). The config reading library is GPLv2 too, so anything that links to is is GPLv2.
Some auxiliary components we made after the fork are more permissive,
LGPLv2+ or MIT."
They are currently in the process of scoping a redesign (VyOS 2.0), "we are planning a clean rewrite that will solve issues of the current config system".
This will include the ability to configure via the API.
If we have more questions for VyOS, they are very friendly and responsive, so we should be able to get answers.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_
On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
> I agree with Will Ilya. There are so many problems with the VR right now.
> Most of the outages we've had recently have somehow involved the VR.
> We set custom iptables rules on the VR which can and have easily gone wrong.
> Openswan is broken, Strongswan replacement still needs to be tested.
> VVRP with redundant router still needs work, and not to mention the
> problems we will have when we introduce IPv6 into the whole picture.
>
> I think the spirit of the discussion is to rely on a 3rd party to do
> the networking for us (eg VyOS) and have us handle just the
> orchestration. All the problems that I've described have already been
> solved in VyOS. We also get the advantage of a potential wider
> community to fix and maintain the VR and given our current development
> velocity, it think it totally makes sense to look for a 3rd party option.
>
> -Syed
>
>
> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
> wrote:
>
> > The VR has been biting us far too often recently, which is why we
> > have started looking into alternative implementations.
> >
> > One of the things that is nice about potentially using the VyOS is
> > that
> it
> > is based on Debian, so we should be able to run the other services
> > that
> we
> > currently have like the password server and userdata on the VyOS.
> > This means we would not have to change our architecture initially
> > and could focus on only replacing the networking paths.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > tw @CloudOps_
> >
> > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> >
> > > The more this is discussed the more I think we should stick with
> > > our
> VR.
> > >
> > > All these other options either seem unfinished or with
> > > incompatible license.
> > >
> > > VyOS looks the most promising so far, it's a serious, mature project.
> > > Adopting it though means we'll have to microservice our way out of
> > > it
> > with
> > > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
> those
> > > too. Imho this adds complexity we should void.
> > >
> > > --
> > > Sent from the Delta quadrant using Borg technology!
> > >
> > > Nux!
> > > www.nux.ro
> > >
> > > ----- Original Message -----
> > > > From: "Will Stevens" <ws...@cloudops.com>
> > > > To: dev@cloudstack.apache.org
> > > > Sent: Thursday, 15 September, 2016 17:21:28
> > > > Subject: Re: [DISCUSS] Replacing the VR
> > >
> > > > Ya, we would need to add a daemon for VPN as well. Load
> > > > balancing is another aspect which we will need to consider if we went this route.
> > > > Something like https://traefik.io/ could potentially be a good
> > > > fit
> due
> > > to
> > > > its API driven configuration, but it may be more than what we need.
> > > >
> > > > We should probably try define which pieces make sense to be
> > > > solved
> > > together
> > > > and which pieces would be best suited to be broken out.
> > > >
> > > > I think the network connectivity, routing and firewalling should
> > probably
> > > > all stay together since the majority of the tools we would
> potentially
> > > use
> > > > would handle all of that together in a single implementation.
> > > >
> > > > The password server and userdata seems like a good option for
> > > > being
> > > broken
> > > > out and handled independently (and probably rewritten completely
> since
> > > they
> > > > currently have some issues).
> > > >
> > > > Load balancing is another that could warrant splitting out, but
> > > > that depends on what direction we go and how we would be managing it.
> DHCP
> > > and
> > > > DNS are others which could go either way.
> > > >
> > > > If we do split out services, I think we should consolidate as
> > > > much as
> > we
> > > > can into each service we break out. Ideally a network packet
> > > > would
> > never
> > > > hit more than one, maybe two, services. I don't think we should
> > > > be splitting services 'just because', I think we need a valid
> > > > case for splitting any service out because it adds complexity.
> > > > Our project is already complex enough, we need to avoid adding
> > > > complexity unless it
> is
> > > > really needed.
> > > >
> > > > Some more of my thoughts on this anyway...
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > *CloudOps* *| *Cloud Solutions Experts
> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> > > > *|* tw @CloudOps_
> > > >
> > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
> > wrote:
> > > >
> > > >> I do agree with you that this probably isn't the right place
> > > >> the
> > > password
> > > >> service and user data.
> > > >>
> > > >>
> > > >> Having said that, after taking a cursory look at the dev docs,
> > > >> it
> > > doesn't
> > > >> seem that difficult to add new daemons:
> https://opensnaproute.github.
> > > >> io/docs/developer.html#creating-new-component
> > > >>
> > > >> <https://opensnaproute.github.io/docs/developer.html#
> > > >> creating-new-component>
> > > >>
> > > >>
> > > >> They've definitely build it with a microservices architecture
> > > >> in
> mind,
> > > so
> > > >> each individual feature is abstracted into it's own small
> > > >> daemon
> > > process.
> > > >> We could just create a daemon for the password server and the
> userdata
> > > >> components if we really had to.
> > > >>
> > > >>
> > > >> - Si
> > > >>
> > > >>
> > > >> ________________________________
> > > >> From: williamstevens@gmail.com <wi...@gmail.com> on
> > > >> behalf
> > of
> > > >> Will Stevens <ws...@cloudops.com>
> > > >> Sent: Thursday, September 15, 2016 9:17 AM
> > > >> To: dev@cloudstack.apache.org
> > > >> Subject: Re: [DISCUSS] Replacing the VR
> > > >>
> > > >> A big part of why I know about it is because it is written in Go.
> :P
> > > >>
> > > >> Yes, it is definitely interesting for the routing and traffic
> handling
> > > >> aspects of the VR. We will likely have to rethink some of the
> pieces
> > a
> > > >> little bit like the password server and userdata if we are to
> > > >> adopt
> a
> > > >> different VR approach. This is where I think some of JohnB and
> > > Chiradeep's
> > > >> ideas make sense. In many ways, it does not make sense for the
> device
> > > >> handling routing and network traffic to also be responsible for
> > > passwords
> > > >> and userdata.
> > > >>
> > > >> *Will STEVENS*
> > > >> Lead Developer
> > > >>
> > > >> *CloudOps* *| *Cloud Solutions Experts
> > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> > > >> *|* tw @CloudOps_
> > > >>
> > > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
> > wrote:
> > > >>
> > > >> > I hadn't heard of Flexswitch until you mentioned it. It looks
> pretty
> > > >> cool!
> > > >> > It even supports ONIE install.
> > > >> >
> > > >> > To be honest, the ipsec feature could be added, or we could
> offload
> > > it to
> > > >> > separate vm if we needed to. The fact it is so feature rich
> > > >> > from a
> > > >> routing
> > > >> > perspective (and all API driven) is really nice.
> > > >> >
> > > >> >
> > > >> > Based on the roadmap, it looks like they plan to also support
> > > >> capabilities
> > > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This
> > > >> > will
> be
> > > huge
> > > >> > for our carrier community that rely on these technologies to
> > > >> > do
> > > private
> > > >> > gateway and inter-VPC interconnections today. We handle this
> > > >> > stuff
> > on
> > > our
> > > >> > ASRs right now with a vlan interconnect into the VR. Being
> > > >> > able to
> > do
> > > >> MPLS
> > > >> > all the way to the VR would be awesome.
> > > >> >
> > > >> >
> > > >> > It also seems to be written in GO (a language here at ENA we
> > > >> > know
> > very
> > > >> > well).
> > > >> >
> > > >> >
> > > >> > - Si
> > > >> >
> > > >> >
> > > >> >
> > > >> > ________________________________
> > > >> > From: Will Stevens <wi...@gmail.com>
> > > >> > Sent: Thursday, September 15, 2016 7:06 AM
> > > >> > To: dev@cloudstack.apache.org
> > > >> > Subject: RE: [DISCUSS] Replacing the VR
> > > >> >
> > > >> > Ya. I don't think it covers our whole use case, but what it
> > > >> > does
> > > cover is
> > > >> > all api driven...
> > > >> >
> > > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
> > wrote:
> > > >> >
> > > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it
> > > >> > > was
> not
> > > >> > > intended to do IPSec.
> > > >> > >
> > > >> > > It seems as though VyOS is starting to look like the best
> option.
> > > >> > >
> > > >> > > Regards,
> > > >> > > Marty Godsey
> > > >> > > nSource Solutions
> > > >> > >
> > > >> > > -----Original Message-----
> > > >> > > From: williamstevens@gmail.com
> > > >> > > [mailto:williamstevens@gmail.com
> ]
> > On
> > > >> > > Behalf Of Will Stevens
> > > >> > > Sent: Wednesday, September 14, 2016 11:06 PM
> > > >> > > To: dev@cloudstack.apache.org
> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > > >> > >
> > > >> > > Or we could go completely crazy and go with something like
> > > FlexSwitch
> > > >> > from
> > > >> > > SnapRoute
> > > >> > > - http://www.snaproute.com/
> > > >> > > - https://opensnaproute.github.io/docs/apis.html
> > > >> > >
> > > >> > > *Will STEVENS*
> > > >> > > Lead Developer
> > > >> > >
> > > >> > > *CloudOps* *| *Cloud Solutions Experts
> > > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> > > >> > > cloudops.com
> > *|*
> > > tw
> > > >> > > @CloudOps_
> > > >> > >
> > > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> > > wstevens@cloudops.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > I tend to agree with Syed and Marty. I am not sure what
> > problems
> > > are
> > > >> > > > solved by splitting up the function of the VR into a
> > > >> > > > bunch of
> > > >> separate
> > > >> > > > services. As Syed points out, the complexity added is
> > > non-trivial.
> > > >> > > > We now have to manage all the intercontainer networking
> > > >> > > > as
> well
> > as
> > > >> the
> > > >> > > > orchestrated ACS networking.
> > > >> > > >
> > > >> > > > VyOS is interesting to me because it covers the majority
> > > >> > > > of
> our
> > > use
> > > >> > > > case with a single unified control plane. It also has
> > > >> > > > good
> > > support
> > > >> > > > for extending features we care about, like IPv6, VXLAN,
> > > >> > > > VRRP, transactions, etc...
> > > >> > > >
> > > >> > > > *Will STEVENS*
> > > >> > > > Lead Developer
> > > >> > > >
> > > >> > > > *CloudOps* *| *Cloud Solutions Experts
> > > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> cloudops.com
> > > *|*
> > > >> tw
> > > >> > > > @CloudOps_
> > > >> > > >
> > > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> > sahmed@cloudops.com>
> > > >> > wrote:
> > > >> > > >
> > > >> > > >> Agree with Marty, adding Docker containers to the
> > > >> > > >> picture
> > > although
> > > >> > > >> can make the VR more flexible but the added complexity
> > > >> > > >> is
> just
> > > not
> > > >> > > >> worth it. Not to mention we would need to take care of
> > networking
> > > >> > > >> each container manually and given that our iptable rules
> > > >> > > >> are
> > very
> > > >> > > >> unstable at the moment I don't see a big value add.
> > > >> > > >>
> > > >> > > >> Vyos looks like a better solution to me. I know that it
> > > >> > > >> does
> > not
> > > >> > > >> provide an api but it does fit the bill quite well
> otherwise. I
> > > >> > > >> specially like the fact that it has a transaction based
> > > >> > > >> model
> > and
> > > >> you
> > > >> > > >> can rollback changes if something goes wrong.
> > > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> > > marty@gonsource.com>
> > > >> > > wrote:
> > > >> > > >>
> > > >> > > >> > Licensing aside, I think splitting the various
> > > >> > > >> > functions
> into
> > > >> > > >> > containers is not a good route either. This will force
> users
> > to
> > > >> > > >> > have to maintain
> > > >> > > >> and
> > > >> > > >> > use containers and adds complexity to the networking
> aspects
> > of
> > > >> ACS.
> > > >> > > >> > Complexity decreases stability. Now I understand the
> argument
> > > that
> > > >> > > >> > a monolithic approach also brings its own set of
> > > >> > > >> > issues but
> > it
> > > >> also
> > > >> > > >> > simplifies it.
> > > >> > > >> >
> > > >> > > >> > Regards,
> > > >> > > >> > Marty Godsey
> > > >> > > >> > nSource Solutions
> > > >> > > >> >
> > > >> > > >> > -----Original Message-----
> > > >> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> > > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
> > > >> > > >> > To: dev@cloudstack.apache.org
> > > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR
> > > >> > > >> >
> > > >> > > >> > I rather doubt that the Cloudrouter will fit the needs
> > > >> > > >> > of
> the
> > > >> > > >> > CloudStack project
> > > >> > > >> > - it is AGPL licensed. Many enterprises will not
> > > >> > > >> > touch
> > > anything
> > > >> > > >> > that
> > > >> > > >> has
> > > >> > > >> > AGPL
> > > >> > > >> > - the github repo shows rather infrequent updates.
> > > >> > > >> > Quite
> > > likely
> > > >> > > >> > they aren't considering the use cases of the
> > > >> > > >> > CloudStack
> > > community
> > > >> > > >> >
> > > >> > > >> > I'd back John B's comments on disaggregating the VR.
> > > >> > > >> > Split
> it
> > > into
> > > >> > > >> > many docker containers
> > > >> > > >> > - password server
> > > >> > > >> > - userdata server
> > > >> > > >> > - DHCP / DNS
> > > >> > > >> > - s2s VPN
> > > >> > > >> > - RA VPN
> > > >> > > >> > - intra-VPC routing and ACL
> > > >> > > >> > - Port forwarding + NAT
> > > >> > > >> > - FW
> > > >> > > >> > - LB (public)
> > > >> > > >> > - LB (internal),
> > > >> > > >> > - secondary storage
> > > >> > > >> > - agent
> > > >> > > >> > Glue them together with docker compose files (one per
> > > >> > > >> > use
> > > case -
> > > >> > > >> > basic zone, isolated, VPC, SSVM, etc).
> > > >> > > >> >
> > > >> > > >> > The VR image then becomes a JeOS + docker. You can
> > > >> > > >> > test
> each
> > of
> > > >> the
> > > >> > > >> > components independently and fixing one bug in the
> > > >> > > >> > field
> (say
> > > >> DHCP)
> > > >> > > >> > is hitless to the other components. You don't need to
> > > >> > > >> > build per-hypervisor VRs. You could even run on baremetal.
> > > >> > > >> >
> > > >> > > >> > Along the way you need to figure out how to
> > > >> > > >> > - make the traffic traverse the containers that are
> > > >> > > >> > needed
> > to
> > > be
> > > >> > > >> > traversed (in most cases just 1)
> > > >> > > >> > - bootstrap the router (how does it find its compose file?
> > > where
> > > >> > > >> > is the
> > > >> > > >> > registry?)
> > > >> > > >> > - rethink the command and control of the VR
> > > >> > > >> > functions. SSH
> > > works,
> > > >> > > >> > but something more declarative, idempotent should be
> > explored.
> > > >> > > >> >
> > > >> > > >> > As you do this, it becomes clearer which of the
> > > >> > > >> > functions
> can
> > > be
> > > >> > > >> > substituted by for example CloudRouter. Command and
> > > >> > > >> > Control
> > of
> > > the
> > > >> > > >> docker
> > > >> > > >> > containers can be moved out to another container. Etc.
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> > > >> > > >> > <ma...@gonsource.com>
> > > >> > > >> > wrote:
> > > >> > > >> >
> > > >> > > >> > > This one does look nice. My biggest concern is the
> > > >> > > >> > > lack
> of
> > > >> > > >> > > VXLANs. It seems that any of the ones we mentioned
> > > >> > > >> > > do not
> > > have
> > > >> an
> > > >> > > >> > > API so we may be stuck at the SSH method.
> > > >> > > >> > >
> > > >> > > >> > > Regards,
> > > >> > > >> > > Marty Godsey
> > > >> > > >> > > nSource Solutions
> > > >> > > >> > >
> > > >> > > >> > > -----Original Message-----
> > > >> > > >> > > From: Abhinandan Prateek
> > > >> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
> > > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> > > >> > > >> > > To: dev@cloudstack.apache.org
> > > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > > >> > > >> > >
> > > >> > > >> > > Cloudrouter looks promising. These have potential to
> > > >> > > >> > > save
> > > future
> > > >> > > >> > > engineering effort for example on ipv6 routing, OSPF etc.
> > > >> > > >> > > And the best part is they come with test automation
> > > framework.
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> > > >> > > >> > > <ja...@accelerite.com>
> > > >> > > >> > > wrote:
> > > >> > > >> > >
> > > >> > > >> > > >Hi,
> > > >> > > >> > > >
> > > >> > > >> > > >Instead of replacing the VR in first place we
> > > >> > > >> > > >should add VyOS/cloudrouter
> > > >> > > >> > > as provider. Once it is stable, network offerings
> > > >> > > >> > > (on
> > > upgrade)
> > > >> > > >> > > can be updated to use it and we can drop the VR if
> > > >> > > >> > > we
> want
> > at
> > > >> > > >> > > that release
> > > >> > > >> > onwards.
> > > >> > > >> > > >
> > > >> > > >> > > >VR is stabilized over a period of time and some of
> > > >> > > >> > > >them
> > are
> > > >> > > >> > > >running
> > > >> > > >> > > without issues. When we replicate the ACS VR
> > > >> > > >> > > features in
> > new
> > > >> > > >> > > solution it takes some to find the missing pieces
> > > >> > > >> > > (hidden
> > > bugs).
> > > >> > > >> > > >
> > > >> > > >> > > >Thanks,
> > > >> > > >> > > >Jayapal
> > > >> > > >> > > >
> > > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> > > >> > > >> > > >
> > > >> > > >> > > >> nux@li.nux.ro> wrote:
> > > >> > > >> > > >>
> > > >> > > >> > > >> Hi,
> > > >> > > >> > > >>
> > > >> > > >> > > >> I like the idea.
> > > >> > > >> > > >>
> > > >> > > >> > > >> Cloudrouter looks really promising, I'm not too
> > > >> > > >> > > >> keen
> on
> > > VyOS
> > > >> > > >> > > >> (it
> > > >> > > >> > > doesn't have a proper http api etc).
> > > >> > > >> > > >>
> > > >> > > >> > > >> --
> > > >> > > >> > > >> Sent from the Delta quadrant using Borg technology!
> > > >> > > >> > > >>
> > > >> > > >> > > >> Nux!
> > > >> > > >> > > >> www.nux.ro
> > > >> > > >> > > >>
> > > >> > > >> > > >>
> > > >> > > >> > > abhinandan.prateek@shapeblue.com
> > > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
> > > >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> > > @shapeblue
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > > ----- Original Message -----
> > > >> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> > > >> > > >> > > >>> To: dev@cloudstack.apache.org
> > > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> > > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
> > > >> > > >> > > >>
> > > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and
> > > >> > > >> > > >>> should
> > be
> > > >> > > >> > > >>> treated as
> > > >> > > >> > > such.
> > > >> > > >> > > >>> Please weigh in with the good and bad of this idea...
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> A couple of us have been discussing the idea of
> > > potentially
> > > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open
> > > >> > > >> > > >>> Source
> > > Vyatta
> > > >> > VM).
> > > >> > > >> > > >>> There may be a license issue because I think it
> > > >> > > >> > > >>> is
> > > licensed
> > > >> > > >> > > >>> under GPL, but for the sake of discussion, let's
> assume
> > > we
> > > >> > > >> > > >>> can overcome any
> > > >> > > >> > > license issues.
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> I have spent some time recently with the VyOS
> > > >> > > >> > > >>> and I
> > have
> > > to
> > > >> > > >> > > >>> admit, I was pretty impressed. It is simple and
> > > intuitive
> > > >> > > >> > > >>> and it gives you a lot more options for auditing
> > > >> > > >> > > >>> the
> > > >> > > configuration etc...
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> Items of potential interest:
> > > >> > > >> > > >>> - Clean up our current VR script spaghetti to a
> simpler
> > > more
> > > >> > > >> > > >>> auditable configuration workflow.
> > > >> > > >> > > >>> - Gives a cleaner path for IPv6 support.
> > > >> > > >> > > >>> - Handles VPN configuration via the same
> configuration
> > > >> > > interface.
> > > >> > > >> > > >>> - Support for OSPF & BGP.
> > > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
> > > >> > > >> > > >>> - Easily supports HA (redundant routers) through
> VRRP.
> > > >> > > >> > > >>> - VXLAN support.
> > > >> > > >> > > >>> - Transaction based changes to the VR with
> > > >> > > >> > > >>> rollback
> on
> > > >> error.
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> Items that could be difficult to solve:
> > > >> > > >> > > >>> - Userdata password reset workflow and
> implementation.
> > > >> > > >> > > >>> - Upgrade process.
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> The VyOS is not the only option if we were to
> consider
> > > this
> > > >> > > >> approach.
> > > >> > > >> > > >>> Another option, which I don't know as well,
> > > >> > > >> > > >>> would be CloudRouter (AGPL
> > > >> > > >> > > >>> license) [2] which is purely API driven.
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> Anyway, would love to hear your thoughts...
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> Will
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> [1] https://vyos.io/ [2]
> > > >> > > >> > > >>> https://cloudrouter.org/
> > > >> > > >> > > >
> > > >> > > >> > > >
> > > >> > > >> > > >
> > > >> > > >> > > >
> > > >> > > >> > > >DISCLAIMER
> > > >> > > >> > > >==========
> > > >> > > >> > > >This e-mail may contain privileged and confidential
> > > information
> > > >> > > >> > > >which is
> > > >> > > >> > > the property of Accelerite, a Persistent Systems
> business.
> > > It is
> > > >> > > >> > > intended only for the use of the individual or
> > > >> > > >> > > entity to
> > > which
> > > >> it
> > > >> > > >> > > is addressed. If you are not the intended recipient,
> > > >> > > >> > > you
> > are
> > > not
> > > >> > > >> > > authorized to read, retain, copy, print, distribute
> > > >> > > >> > > or
> use
> > > this
> > > >> > > >> > > message. If you have received this communication in
> error,
> > > >> please
> > > >> > > >> > > notify the sender and delete all copies of this message.
> > > >> > > >> > > Accelerite, a Persistent Systems business does not
> > > >> > > >> > > accept
> > any
> > > >> > > >> > > liability for virus
> > > >> > > >> > infected mails.
> > > >> > > >> > >
> > > >> > > >> >
> > > >> > > >>
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
I just had a quick chat with a couple of the guys over on the VyOS chat.
I have CC'ed one of them in case we have more licensing questions.
So here is the status with the license "the code inherited from Vyatta and
our modifications from it is GPLv2 (strict, not v2+). The config reading
library is GPLv2 too, so anything that links to is is GPLv2.
Some auxiliary components we made after the fork are more permissive,
LGPLv2+ or MIT."
They are currently in the process of scoping a redesign (VyOS 2.0), "we are
planning a clean rewrite that will solve issues of the current config system".
This will include the ability to configure via the API.
If we have more questions for VyOS, they are very friendly and responsive,
so we should be able to get answers.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Fri, Sep 16, 2016 at 9:37 AM, Syed Ahmed <sa...@cloudops.com> wrote:
> I agree with Will Ilya. There are so many problems with the VR right now.
> Most of the outages we've had recently have somehow involved the VR. We set
> custom iptables rules on the VR which can and have easily gone wrong.
> Openswan is broken, Strongswan replacement still needs to be tested. VVRP
> with redundant router still needs work, and not to mention the problems we
> will have when we introduce IPv6 into the whole picture.
>
> I think the spirit of the discussion is to rely on a 3rd party to do the
> networking for us (eg VyOS) and have us handle just the orchestration. All
> the problems that I've described have already been solved in VyOS. We also
> get the advantage of a potential wider community to fix and maintain the VR
> and given our current development velocity, it think it totally makes sense
> to look for a 3rd party option.
>
> -Syed
>
>
> On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com>
> wrote:
>
> > The VR has been biting us far too often recently, which is why we have
> > started looking into alternative implementations.
> >
> > One of the things that is nice about potentially using the VyOS is that
> it
> > is based on Debian, so we should be able to run the other services that
> we
> > currently have like the password server and userdata on the VyOS. This
> > means we would not have to change our architecture initially and could
> > focus on only replacing the networking paths.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > w cloudops.com *|* tw @CloudOps_
> >
> > On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> >
> > > The more this is discussed the more I think we should stick with our
> VR.
> > >
> > > All these other options either seem unfinished or with incompatible
> > > license.
> > >
> > > VyOS looks the most promising so far, it's a serious, mature project.
> > > Adopting it though means we'll have to microservice our way out of it
> > with
> > > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve
> those
> > > too. Imho this adds complexity we should void.
> > >
> > > --
> > > Sent from the Delta quadrant using Borg technology!
> > >
> > > Nux!
> > > www.nux.ro
> > >
> > > ----- Original Message -----
> > > > From: "Will Stevens" <ws...@cloudops.com>
> > > > To: dev@cloudstack.apache.org
> > > > Sent: Thursday, 15 September, 2016 17:21:28
> > > > Subject: Re: [DISCUSS] Replacing the VR
> > >
> > > > Ya, we would need to add a daemon for VPN as well. Load balancing is
> > > > another aspect which we will need to consider if we went this route.
> > > > Something like https://traefik.io/ could potentially be a good fit
> due
> > > to
> > > > its API driven configuration, but it may be more than what we need.
> > > >
> > > > We should probably try define which pieces make sense to be solved
> > > together
> > > > and which pieces would be best suited to be broken out.
> > > >
> > > > I think the network connectivity, routing and firewalling should
> > probably
> > > > all stay together since the majority of the tools we would
> potentially
> > > use
> > > > would handle all of that together in a single implementation.
> > > >
> > > > The password server and userdata seems like a good option for being
> > > broken
> > > > out and handled independently (and probably rewritten completely
> since
> > > they
> > > > currently have some issues).
> > > >
> > > > Load balancing is another that could warrant splitting out, but that
> > > > depends on what direction we go and how we would be managing it.
> DHCP
> > > and
> > > > DNS are others which could go either way.
> > > >
> > > > If we do split out services, I think we should consolidate as much as
> > we
> > > > can into each service we break out. Ideally a network packet would
> > never
> > > > hit more than one, maybe two, services. I don't think we should be
> > > > splitting services 'just because', I think we need a valid case for
> > > > splitting any service out because it adds complexity. Our project is
> > > > already complex enough, we need to avoid adding complexity unless it
> is
> > > > really needed.
> > > >
> > > > Some more of my thoughts on this anyway...
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > *CloudOps* *| *Cloud Solutions Experts
> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > > > w cloudops.com *|* tw @CloudOps_
> > > >
> > > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
> > wrote:
> > > >
> > > >> I do agree with you that this probably isn't the right place the
> > > password
> > > >> service and user data.
> > > >>
> > > >>
> > > >> Having said that, after taking a cursory look at the dev docs, it
> > > doesn't
> > > >> seem that difficult to add new daemons:
> https://opensnaproute.github.
> > > >> io/docs/developer.html#creating-new-component
> > > >>
> > > >> <https://opensnaproute.github.io/docs/developer.html#
> > > >> creating-new-component>
> > > >>
> > > >>
> > > >> They've definitely build it with a microservices architecture in
> mind,
> > > so
> > > >> each individual feature is abstracted into it's own small daemon
> > > process.
> > > >> We could just create a daemon for the password server and the
> userdata
> > > >> components if we really had to.
> > > >>
> > > >>
> > > >> - Si
> > > >>
> > > >>
> > > >> ________________________________
> > > >> From: williamstevens@gmail.com <wi...@gmail.com> on behalf
> > of
> > > >> Will Stevens <ws...@cloudops.com>
> > > >> Sent: Thursday, September 15, 2016 9:17 AM
> > > >> To: dev@cloudstack.apache.org
> > > >> Subject: Re: [DISCUSS] Replacing the VR
> > > >>
> > > >> A big part of why I know about it is because it is written in Go.
> :P
> > > >>
> > > >> Yes, it is definitely interesting for the routing and traffic
> handling
> > > >> aspects of the VR. We will likely have to rethink some of the
> pieces
> > a
> > > >> little bit like the password server and userdata if we are to adopt
> a
> > > >> different VR approach. This is where I think some of JohnB and
> > > Chiradeep's
> > > >> ideas make sense. In many ways, it does not make sense for the
> device
> > > >> handling routing and network traffic to also be responsible for
> > > passwords
> > > >> and userdata.
> > > >>
> > > >> *Will STEVENS*
> > > >> Lead Developer
> > > >>
> > > >> *CloudOps* *| *Cloud Solutions Experts
> > > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > > >> w cloudops.com *|* tw @CloudOps_
> > > >>
> > > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
> > wrote:
> > > >>
> > > >> > I hadn't heard of Flexswitch until you mentioned it. It looks
> pretty
> > > >> cool!
> > > >> > It even supports ONIE install.
> > > >> >
> > > >> > To be honest, the ipsec feature could be added, or we could
> offload
> > > it to
> > > >> > separate vm if we needed to. The fact it is so feature rich from a
> > > >> routing
> > > >> > perspective (and all API driven) is really nice.
> > > >> >
> > > >> >
> > > >> > Based on the roadmap, it looks like they plan to also support
> > > >> capabilities
> > > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This will
> be
> > > huge
> > > >> > for our carrier community that rely on these technologies to do
> > > private
> > > >> > gateway and inter-VPC interconnections today. We handle this stuff
> > on
> > > our
> > > >> > ASRs right now with a vlan interconnect into the VR. Being able to
> > do
> > > >> MPLS
> > > >> > all the way to the VR would be awesome.
> > > >> >
> > > >> >
> > > >> > It also seems to be written in GO (a language here at ENA we know
> > very
> > > >> > well).
> > > >> >
> > > >> >
> > > >> > - Si
> > > >> >
> > > >> >
> > > >> >
> > > >> > ________________________________
> > > >> > From: Will Stevens <wi...@gmail.com>
> > > >> > Sent: Thursday, September 15, 2016 7:06 AM
> > > >> > To: dev@cloudstack.apache.org
> > > >> > Subject: RE: [DISCUSS] Replacing the VR
> > > >> >
> > > >> > Ya. I don't think it covers our whole use case, but what it does
> > > cover is
> > > >> > all api driven...
> > > >> >
> > > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
> > wrote:
> > > >> >
> > > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it was
> not
> > > >> > > intended to do IPSec.
> > > >> > >
> > > >> > > It seems as though VyOS is starting to look like the best
> option.
> > > >> > >
> > > >> > > Regards,
> > > >> > > Marty Godsey
> > > >> > > nSource Solutions
> > > >> > >
> > > >> > > -----Original Message-----
> > > >> > > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com
> ]
> > On
> > > >> > > Behalf Of Will Stevens
> > > >> > > Sent: Wednesday, September 14, 2016 11:06 PM
> > > >> > > To: dev@cloudstack.apache.org
> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > > >> > >
> > > >> > > Or we could go completely crazy and go with something like
> > > FlexSwitch
> > > >> > from
> > > >> > > SnapRoute
> > > >> > > - http://www.snaproute.com/
> > > >> > > - https://opensnaproute.github.io/docs/apis.html
> > > >> > >
> > > >> > > *Will STEVENS*
> > > >> > > Lead Developer
> > > >> > >
> > > >> > > *CloudOps* *| *Cloud Solutions Experts
> > > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> > *|*
> > > tw
> > > >> > > @CloudOps_
> > > >> > >
> > > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> > > wstevens@cloudops.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > I tend to agree with Syed and Marty. I am not sure what
> > problems
> > > are
> > > >> > > > solved by splitting up the function of the VR into a bunch of
> > > >> separate
> > > >> > > > services. As Syed points out, the complexity added is
> > > non-trivial.
> > > >> > > > We now have to manage all the intercontainer networking as
> well
> > as
> > > >> the
> > > >> > > > orchestrated ACS networking.
> > > >> > > >
> > > >> > > > VyOS is interesting to me because it covers the majority of
> our
> > > use
> > > >> > > > case with a single unified control plane. It also has good
> > > support
> > > >> > > > for extending features we care about, like IPv6, VXLAN, VRRP,
> > > >> > > > transactions, etc...
> > > >> > > >
> > > >> > > > *Will STEVENS*
> > > >> > > > Lead Developer
> > > >> > > >
> > > >> > > > *CloudOps* *| *Cloud Solutions Experts
> > > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w
> cloudops.com
> > > *|*
> > > >> tw
> > > >> > > > @CloudOps_
> > > >> > > >
> > > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> > sahmed@cloudops.com>
> > > >> > wrote:
> > > >> > > >
> > > >> > > >> Agree with Marty, adding Docker containers to the picture
> > > although
> > > >> > > >> can make the VR more flexible but the added complexity is
> just
> > > not
> > > >> > > >> worth it. Not to mention we would need to take care of
> > networking
> > > >> > > >> each container manually and given that our iptable rules are
> > very
> > > >> > > >> unstable at the moment I don't see a big value add.
> > > >> > > >>
> > > >> > > >> Vyos looks like a better solution to me. I know that it does
> > not
> > > >> > > >> provide an api but it does fit the bill quite well
> otherwise. I
> > > >> > > >> specially like the fact that it has a transaction based model
> > and
> > > >> you
> > > >> > > >> can rollback changes if something goes wrong.
> > > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> > > marty@gonsource.com>
> > > >> > > wrote:
> > > >> > > >>
> > > >> > > >> > Licensing aside, I think splitting the various functions
> into
> > > >> > > >> > containers is not a good route either. This will force
> users
> > to
> > > >> > > >> > have to maintain
> > > >> > > >> and
> > > >> > > >> > use containers and adds complexity to the networking
> aspects
> > of
> > > >> ACS.
> > > >> > > >> > Complexity decreases stability. Now I understand the
> argument
> > > that
> > > >> > > >> > a monolithic approach also brings its own set of issues but
> > it
> > > >> also
> > > >> > > >> > simplifies it.
> > > >> > > >> >
> > > >> > > >> > Regards,
> > > >> > > >> > Marty Godsey
> > > >> > > >> > nSource Solutions
> > > >> > > >> >
> > > >> > > >> > -----Original Message-----
> > > >> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> > > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
> > > >> > > >> > To: dev@cloudstack.apache.org
> > > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR
> > > >> > > >> >
> > > >> > > >> > I rather doubt that the Cloudrouter will fit the needs of
> the
> > > >> > > >> > CloudStack project
> > > >> > > >> > - it is AGPL licensed. Many enterprises will not touch
> > > anything
> > > >> > > >> > that
> > > >> > > >> has
> > > >> > > >> > AGPL
> > > >> > > >> > - the github repo shows rather infrequent updates. Quite
> > > likely
> > > >> > > >> > they aren't considering the use cases of the CloudStack
> > > community
> > > >> > > >> >
> > > >> > > >> > I'd back John B's comments on disaggregating the VR. Split
> it
> > > into
> > > >> > > >> > many docker containers
> > > >> > > >> > - password server
> > > >> > > >> > - userdata server
> > > >> > > >> > - DHCP / DNS
> > > >> > > >> > - s2s VPN
> > > >> > > >> > - RA VPN
> > > >> > > >> > - intra-VPC routing and ACL
> > > >> > > >> > - Port forwarding + NAT
> > > >> > > >> > - FW
> > > >> > > >> > - LB (public)
> > > >> > > >> > - LB (internal),
> > > >> > > >> > - secondary storage
> > > >> > > >> > - agent
> > > >> > > >> > Glue them together with docker compose files (one per use
> > > case -
> > > >> > > >> > basic zone, isolated, VPC, SSVM, etc).
> > > >> > > >> >
> > > >> > > >> > The VR image then becomes a JeOS + docker. You can test
> each
> > of
> > > >> the
> > > >> > > >> > components independently and fixing one bug in the field
> (say
> > > >> DHCP)
> > > >> > > >> > is hitless to the other components. You don't need to build
> > > >> > > >> > per-hypervisor VRs. You could even run on baremetal.
> > > >> > > >> >
> > > >> > > >> > Along the way you need to figure out how to
> > > >> > > >> > - make the traffic traverse the containers that are needed
> > to
> > > be
> > > >> > > >> > traversed (in most cases just 1)
> > > >> > > >> > - bootstrap the router (how does it find its compose file?
> > > where
> > > >> > > >> > is the
> > > >> > > >> > registry?)
> > > >> > > >> > - rethink the command and control of the VR functions. SSH
> > > works,
> > > >> > > >> > but something more declarative, idempotent should be
> > explored.
> > > >> > > >> >
> > > >> > > >> > As you do this, it becomes clearer which of the functions
> can
> > > be
> > > >> > > >> > substituted by for example CloudRouter. Command and Control
> > of
> > > the
> > > >> > > >> docker
> > > >> > > >> > containers can be moved out to another container. Etc.
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> > > >> > > >> > <ma...@gonsource.com>
> > > >> > > >> > wrote:
> > > >> > > >> >
> > > >> > > >> > > This one does look nice. My biggest concern is the lack
> of
> > > >> > > >> > > VXLANs. It seems that any of the ones we mentioned do not
> > > have
> > > >> an
> > > >> > > >> > > API so we may be stuck at the SSH method.
> > > >> > > >> > >
> > > >> > > >> > > Regards,
> > > >> > > >> > > Marty Godsey
> > > >> > > >> > > nSource Solutions
> > > >> > > >> > >
> > > >> > > >> > > -----Original Message-----
> > > >> > > >> > > From: Abhinandan Prateek
> > > >> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
> > > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> > > >> > > >> > > To: dev@cloudstack.apache.org
> > > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > > >> > > >> > >
> > > >> > > >> > > Cloudrouter looks promising. These have potential to save
> > > future
> > > >> > > >> > > engineering effort for example on ipv6 routing, OSPF etc.
> > > >> > > >> > > And the best part is they come with test automation
> > > framework.
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> > > >> > > >> > > <ja...@accelerite.com>
> > > >> > > >> > > wrote:
> > > >> > > >> > >
> > > >> > > >> > > >Hi,
> > > >> > > >> > > >
> > > >> > > >> > > >Instead of replacing the VR in first place we should add
> > > >> > > >> > > >VyOS/cloudrouter
> > > >> > > >> > > as provider. Once it is stable, network offerings (on
> > > upgrade)
> > > >> > > >> > > can be updated to use it and we can drop the VR if we
> want
> > at
> > > >> > > >> > > that release
> > > >> > > >> > onwards.
> > > >> > > >> > > >
> > > >> > > >> > > >VR is stabilized over a period of time and some of them
> > are
> > > >> > > >> > > >running
> > > >> > > >> > > without issues. When we replicate the ACS VR features in
> > new
> > > >> > > >> > > solution it takes some to find the missing pieces (hidden
> > > bugs).
> > > >> > > >> > > >
> > > >> > > >> > > >Thanks,
> > > >> > > >> > > >Jayapal
> > > >> > > >> > > >
> > > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> > > >> > > >> > > >
> > > >> > > >> > > >> nux@li.nux.ro> wrote:
> > > >> > > >> > > >>
> > > >> > > >> > > >> Hi,
> > > >> > > >> > > >>
> > > >> > > >> > > >> I like the idea.
> > > >> > > >> > > >>
> > > >> > > >> > > >> Cloudrouter looks really promising, I'm not too keen
> on
> > > VyOS
> > > >> > > >> > > >> (it
> > > >> > > >> > > doesn't have a proper http api etc).
> > > >> > > >> > > >>
> > > >> > > >> > > >> --
> > > >> > > >> > > >> Sent from the Delta quadrant using Borg technology!
> > > >> > > >> > > >>
> > > >> > > >> > > >> Nux!
> > > >> > > >> > > >> www.nux.ro
> > > >> > > >> > > >>
> > > >> > > >> > > >>
> > > >> > > >> > > abhinandan.prateek@shapeblue.com
> > > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
> > > >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> > > @shapeblue
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> > > ----- Original Message -----
> > > >> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> > > >> > > >> > > >>> To: dev@cloudstack.apache.org
> > > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> > > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
> > > >> > > >> > > >>
> > > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and should
> > be
> > > >> > > >> > > >>> treated as
> > > >> > > >> > > such.
> > > >> > > >> > > >>> Please weigh in with the good and bad of this idea...
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> A couple of us have been discussing the idea of
> > > potentially
> > > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open Source
> > > Vyatta
> > > >> > VM).
> > > >> > > >> > > >>> There may be a license issue because I think it is
> > > licensed
> > > >> > > >> > > >>> under GPL, but for the sake of discussion, let's
> assume
> > > we
> > > >> > > >> > > >>> can overcome any
> > > >> > > >> > > license issues.
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> I have spent some time recently with the VyOS and I
> > have
> > > to
> > > >> > > >> > > >>> admit, I was pretty impressed. It is simple and
> > > intuitive
> > > >> > > >> > > >>> and it gives you a lot more options for auditing the
> > > >> > > configuration etc...
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> Items of potential interest:
> > > >> > > >> > > >>> - Clean up our current VR script spaghetti to a
> simpler
> > > more
> > > >> > > >> > > >>> auditable configuration workflow.
> > > >> > > >> > > >>> - Gives a cleaner path for IPv6 support.
> > > >> > > >> > > >>> - Handles VPN configuration via the same
> configuration
> > > >> > > interface.
> > > >> > > >> > > >>> - Support for OSPF & BGP.
> > > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
> > > >> > > >> > > >>> - Easily supports HA (redundant routers) through
> VRRP.
> > > >> > > >> > > >>> - VXLAN support.
> > > >> > > >> > > >>> - Transaction based changes to the VR with rollback
> on
> > > >> error.
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> Items that could be difficult to solve:
> > > >> > > >> > > >>> - Userdata password reset workflow and
> implementation.
> > > >> > > >> > > >>> - Upgrade process.
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> The VyOS is not the only option if we were to
> consider
> > > this
> > > >> > > >> approach.
> > > >> > > >> > > >>> Another option, which I don't know as well, would be
> > > >> > > >> > > >>> CloudRouter (AGPL
> > > >> > > >> > > >>> license) [2] which is purely API driven.
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> Anyway, would love to hear your thoughts...
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> Will
> > > >> > > >> > > >>>
> > > >> > > >> > > >>> [1] https://vyos.io/
> > > >> > > >> > > >>> [2] https://cloudrouter.org/
> > > >> > > >> > > >
> > > >> > > >> > > >
> > > >> > > >> > > >
> > > >> > > >> > > >
> > > >> > > >> > > >DISCLAIMER
> > > >> > > >> > > >==========
> > > >> > > >> > > >This e-mail may contain privileged and confidential
> > > information
> > > >> > > >> > > >which is
> > > >> > > >> > > the property of Accelerite, a Persistent Systems
> business.
> > > It is
> > > >> > > >> > > intended only for the use of the individual or entity to
> > > which
> > > >> it
> > > >> > > >> > > is addressed. If you are not the intended recipient, you
> > are
> > > not
> > > >> > > >> > > authorized to read, retain, copy, print, distribute or
> use
> > > this
> > > >> > > >> > > message. If you have received this communication in
> error,
> > > >> please
> > > >> > > >> > > notify the sender and delete all copies of this message.
> > > >> > > >> > > Accelerite, a Persistent Systems business does not accept
> > any
> > > >> > > >> > > liability for virus
> > > >> > > >> > infected mails.
> > > >> > > >> > >
> > > >> > > >> >
> > > >> > > >>
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Syed Ahmed <sa...@cloudops.com>.
I agree with Will Ilya. There are so many problems with the VR right now.
Most of the outages we've had recently have somehow involved the VR. We set
custom iptables rules on the VR which can and have easily gone wrong.
Openswan is broken, Strongswan replacement still needs to be tested. VVRP
with redundant router still needs work, and not to mention the problems we
will have when we introduce IPv6 into the whole picture.
I think the spirit of the discussion is to rely on a 3rd party to do the
networking for us (eg VyOS) and have us handle just the orchestration. All
the problems that I've described have already been solved in VyOS. We also
get the advantage of a potential wider community to fix and maintain the VR
and given our current development velocity, it think it totally makes sense
to look for a 3rd party option.
-Syed
On Fri, Sep 16, 2016 at 9:18 AM, Will Stevens <ws...@cloudops.com> wrote:
> The VR has been biting us far too often recently, which is why we have
> started looking into alternative implementations.
>
> One of the things that is nice about potentially using the VyOS is that it
> is based on Debian, so we should be able to run the other services that we
> currently have like the password server and userdata on the VyOS. This
> means we would not have to change our architecture initially and could
> focus on only replacing the networking paths.
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
>
> > The more this is discussed the more I think we should stick with our VR.
> >
> > All these other options either seem unfinished or with incompatible
> > license.
> >
> > VyOS looks the most promising so far, it's a serious, mature project.
> > Adopting it though means we'll have to microservice our way out of it
> with
> > extra machines for DNS/USERDATA/etc, unless we can make VyOS serve those
> > too. Imho this adds complexity we should void.
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > ----- Original Message -----
> > > From: "Will Stevens" <ws...@cloudops.com>
> > > To: dev@cloudstack.apache.org
> > > Sent: Thursday, 15 September, 2016 17:21:28
> > > Subject: Re: [DISCUSS] Replacing the VR
> >
> > > Ya, we would need to add a daemon for VPN as well. Load balancing is
> > > another aspect which we will need to consider if we went this route.
> > > Something like https://traefik.io/ could potentially be a good fit due
> > to
> > > its API driven configuration, but it may be more than what we need.
> > >
> > > We should probably try define which pieces make sense to be solved
> > together
> > > and which pieces would be best suited to be broken out.
> > >
> > > I think the network connectivity, routing and firewalling should
> probably
> > > all stay together since the majority of the tools we would potentially
> > use
> > > would handle all of that together in a single implementation.
> > >
> > > The password server and userdata seems like a good option for being
> > broken
> > > out and handled independently (and probably rewritten completely since
> > they
> > > currently have some issues).
> > >
> > > Load balancing is another that could warrant splitting out, but that
> > > depends on what direction we go and how we would be managing it. DHCP
> > and
> > > DNS are others which could go either way.
> > >
> > > If we do split out services, I think we should consolidate as much as
> we
> > > can into each service we break out. Ideally a network packet would
> never
> > > hit more than one, maybe two, services. I don't think we should be
> > > splitting services 'just because', I think we need a valid case for
> > > splitting any service out because it adds complexity. Our project is
> > > already complex enough, we need to avoid adding complexity unless it is
> > > really needed.
> > >
> > > Some more of my thoughts on this anyway...
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > > w cloudops.com *|* tw @CloudOps_
> > >
> > > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com>
> wrote:
> > >
> > >> I do agree with you that this probably isn't the right place the
> > password
> > >> service and user data.
> > >>
> > >>
> > >> Having said that, after taking a cursory look at the dev docs, it
> > doesn't
> > >> seem that difficult to add new daemons: https://opensnaproute.github.
> > >> io/docs/developer.html#creating-new-component
> > >>
> > >> <https://opensnaproute.github.io/docs/developer.html#
> > >> creating-new-component>
> > >>
> > >>
> > >> They've definitely build it with a microservices architecture in mind,
> > so
> > >> each individual feature is abstracted into it's own small daemon
> > process.
> > >> We could just create a daemon for the password server and the userdata
> > >> components if we really had to.
> > >>
> > >>
> > >> - Si
> > >>
> > >>
> > >> ________________________________
> > >> From: williamstevens@gmail.com <wi...@gmail.com> on behalf
> of
> > >> Will Stevens <ws...@cloudops.com>
> > >> Sent: Thursday, September 15, 2016 9:17 AM
> > >> To: dev@cloudstack.apache.org
> > >> Subject: Re: [DISCUSS] Replacing the VR
> > >>
> > >> A big part of why I know about it is because it is written in Go. :P
> > >>
> > >> Yes, it is definitely interesting for the routing and traffic handling
> > >> aspects of the VR. We will likely have to rethink some of the pieces
> a
> > >> little bit like the password server and userdata if we are to adopt a
> > >> different VR approach. This is where I think some of JohnB and
> > Chiradeep's
> > >> ideas make sense. In many ways, it does not make sense for the device
> > >> handling routing and network traffic to also be responsible for
> > passwords
> > >> and userdata.
> > >>
> > >> *Will STEVENS*
> > >> Lead Developer
> > >>
> > >> *CloudOps* *| *Cloud Solutions Experts
> > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > >> w cloudops.com *|* tw @CloudOps_
> > >>
> > >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com>
> wrote:
> > >>
> > >> > I hadn't heard of Flexswitch until you mentioned it. It looks pretty
> > >> cool!
> > >> > It even supports ONIE install.
> > >> >
> > >> > To be honest, the ipsec feature could be added, or we could offload
> > it to
> > >> > separate vm if we needed to. The fact it is so feature rich from a
> > >> routing
> > >> > perspective (and all API driven) is really nice.
> > >> >
> > >> >
> > >> > Based on the roadmap, it looks like they plan to also support
> > >> capabilities
> > >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This will be
> > huge
> > >> > for our carrier community that rely on these technologies to do
> > private
> > >> > gateway and inter-VPC interconnections today. We handle this stuff
> on
> > our
> > >> > ASRs right now with a vlan interconnect into the VR. Being able to
> do
> > >> MPLS
> > >> > all the way to the VR would be awesome.
> > >> >
> > >> >
> > >> > It also seems to be written in GO (a language here at ENA we know
> very
> > >> > well).
> > >> >
> > >> >
> > >> > - Si
> > >> >
> > >> >
> > >> >
> > >> > ________________________________
> > >> > From: Will Stevens <wi...@gmail.com>
> > >> > Sent: Thursday, September 15, 2016 7:06 AM
> > >> > To: dev@cloudstack.apache.org
> > >> > Subject: RE: [DISCUSS] Replacing the VR
> > >> >
> > >> > Ya. I don't think it covers our whole use case, but what it does
> > cover is
> > >> > all api driven...
> > >> >
> > >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com>
> wrote:
> > >> >
> > >> > > Though I don’t see VPN in Snaproute.. Makes sense since it was not
> > >> > > intended to do IPSec.
> > >> > >
> > >> > > It seems as though VyOS is starting to look like the best option.
> > >> > >
> > >> > > Regards,
> > >> > > Marty Godsey
> > >> > > nSource Solutions
> > >> > >
> > >> > > -----Original Message-----
> > >> > > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com]
> On
> > >> > > Behalf Of Will Stevens
> > >> > > Sent: Wednesday, September 14, 2016 11:06 PM
> > >> > > To: dev@cloudstack.apache.org
> > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > >> > >
> > >> > > Or we could go completely crazy and go with something like
> > FlexSwitch
> > >> > from
> > >> > > SnapRoute
> > >> > > - http://www.snaproute.com/
> > >> > > - https://opensnaproute.github.io/docs/apis.html
> > >> > >
> > >> > > *Will STEVENS*
> > >> > > Lead Developer
> > >> > >
> > >> > > *CloudOps* *| *Cloud Solutions Experts
> > >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> *|*
> > tw
> > >> > > @CloudOps_
> > >> > >
> > >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> > wstevens@cloudops.com>
> > >> > > wrote:
> > >> > >
> > >> > > > I tend to agree with Syed and Marty. I am not sure what
> problems
> > are
> > >> > > > solved by splitting up the function of the VR into a bunch of
> > >> separate
> > >> > > > services. As Syed points out, the complexity added is
> > non-trivial.
> > >> > > > We now have to manage all the intercontainer networking as well
> as
> > >> the
> > >> > > > orchestrated ACS networking.
> > >> > > >
> > >> > > > VyOS is interesting to me because it covers the majority of our
> > use
> > >> > > > case with a single unified control plane. It also has good
> > support
> > >> > > > for extending features we care about, like IPv6, VXLAN, VRRP,
> > >> > > > transactions, etc...
> > >> > > >
> > >> > > > *Will STEVENS*
> > >> > > > Lead Developer
> > >> > > >
> > >> > > > *CloudOps* *| *Cloud Solutions Experts
> > >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> > *|*
> > >> tw
> > >> > > > @CloudOps_
> > >> > > >
> > >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <
> sahmed@cloudops.com>
> > >> > wrote:
> > >> > > >
> > >> > > >> Agree with Marty, adding Docker containers to the picture
> > although
> > >> > > >> can make the VR more flexible but the added complexity is just
> > not
> > >> > > >> worth it. Not to mention we would need to take care of
> networking
> > >> > > >> each container manually and given that our iptable rules are
> very
> > >> > > >> unstable at the moment I don't see a big value add.
> > >> > > >>
> > >> > > >> Vyos looks like a better solution to me. I know that it does
> not
> > >> > > >> provide an api but it does fit the bill quite well otherwise. I
> > >> > > >> specially like the fact that it has a transaction based model
> and
> > >> you
> > >> > > >> can rollback changes if something goes wrong.
> > >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> > marty@gonsource.com>
> > >> > > wrote:
> > >> > > >>
> > >> > > >> > Licensing aside, I think splitting the various functions into
> > >> > > >> > containers is not a good route either. This will force users
> to
> > >> > > >> > have to maintain
> > >> > > >> and
> > >> > > >> > use containers and adds complexity to the networking aspects
> of
> > >> ACS.
> > >> > > >> > Complexity decreases stability. Now I understand the argument
> > that
> > >> > > >> > a monolithic approach also brings its own set of issues but
> it
> > >> also
> > >> > > >> > simplifies it.
> > >> > > >> >
> > >> > > >> > Regards,
> > >> > > >> > Marty Godsey
> > >> > > >> > nSource Solutions
> > >> > > >> >
> > >> > > >> > -----Original Message-----
> > >> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> > >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
> > >> > > >> > To: dev@cloudstack.apache.org
> > >> > > >> > Subject: Re: [DISCUSS] Replacing the VR
> > >> > > >> >
> > >> > > >> > I rather doubt that the Cloudrouter will fit the needs of the
> > >> > > >> > CloudStack project
> > >> > > >> > - it is AGPL licensed. Many enterprises will not touch
> > anything
> > >> > > >> > that
> > >> > > >> has
> > >> > > >> > AGPL
> > >> > > >> > - the github repo shows rather infrequent updates. Quite
> > likely
> > >> > > >> > they aren't considering the use cases of the CloudStack
> > community
> > >> > > >> >
> > >> > > >> > I'd back John B's comments on disaggregating the VR. Split it
> > into
> > >> > > >> > many docker containers
> > >> > > >> > - password server
> > >> > > >> > - userdata server
> > >> > > >> > - DHCP / DNS
> > >> > > >> > - s2s VPN
> > >> > > >> > - RA VPN
> > >> > > >> > - intra-VPC routing and ACL
> > >> > > >> > - Port forwarding + NAT
> > >> > > >> > - FW
> > >> > > >> > - LB (public)
> > >> > > >> > - LB (internal),
> > >> > > >> > - secondary storage
> > >> > > >> > - agent
> > >> > > >> > Glue them together with docker compose files (one per use
> > case -
> > >> > > >> > basic zone, isolated, VPC, SSVM, etc).
> > >> > > >> >
> > >> > > >> > The VR image then becomes a JeOS + docker. You can test each
> of
> > >> the
> > >> > > >> > components independently and fixing one bug in the field (say
> > >> DHCP)
> > >> > > >> > is hitless to the other components. You don't need to build
> > >> > > >> > per-hypervisor VRs. You could even run on baremetal.
> > >> > > >> >
> > >> > > >> > Along the way you need to figure out how to
> > >> > > >> > - make the traffic traverse the containers that are needed
> to
> > be
> > >> > > >> > traversed (in most cases just 1)
> > >> > > >> > - bootstrap the router (how does it find its compose file?
> > where
> > >> > > >> > is the
> > >> > > >> > registry?)
> > >> > > >> > - rethink the command and control of the VR functions. SSH
> > works,
> > >> > > >> > but something more declarative, idempotent should be
> explored.
> > >> > > >> >
> > >> > > >> > As you do this, it becomes clearer which of the functions can
> > be
> > >> > > >> > substituted by for example CloudRouter. Command and Control
> of
> > the
> > >> > > >> docker
> > >> > > >> > containers can be moved out to another container. Etc.
> > >> > > >> >
> > >> > > >> >
> > >> > > >> >
> > >> > > >> >
> > >> > > >> >
> > >> > > >> >
> > >> > > >> >
> > >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> > >> > > >> > <ma...@gonsource.com>
> > >> > > >> > wrote:
> > >> > > >> >
> > >> > > >> > > This one does look nice. My biggest concern is the lack of
> > >> > > >> > > VXLANs. It seems that any of the ones we mentioned do not
> > have
> > >> an
> > >> > > >> > > API so we may be stuck at the SSH method.
> > >> > > >> > >
> > >> > > >> > > Regards,
> > >> > > >> > > Marty Godsey
> > >> > > >> > > nSource Solutions
> > >> > > >> > >
> > >> > > >> > > -----Original Message-----
> > >> > > >> > > From: Abhinandan Prateek
> > >> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
> > >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> > >> > > >> > > To: dev@cloudstack.apache.org
> > >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > >> > > >> > >
> > >> > > >> > > Cloudrouter looks promising. These have potential to save
> > future
> > >> > > >> > > engineering effort for example on ipv6 routing, OSPF etc.
> > >> > > >> > > And the best part is they come with test automation
> > framework.
> > >> > > >> > >
> > >> > > >> > >
> > >> > > >> > >
> > >> > > >> > >
> > >> > > >> > >
> > >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> > >> > > >> > > <ja...@accelerite.com>
> > >> > > >> > > wrote:
> > >> > > >> > >
> > >> > > >> > > >Hi,
> > >> > > >> > > >
> > >> > > >> > > >Instead of replacing the VR in first place we should add
> > >> > > >> > > >VyOS/cloudrouter
> > >> > > >> > > as provider. Once it is stable, network offerings (on
> > upgrade)
> > >> > > >> > > can be updated to use it and we can drop the VR if we want
> at
> > >> > > >> > > that release
> > >> > > >> > onwards.
> > >> > > >> > > >
> > >> > > >> > > >VR is stabilized over a period of time and some of them
> are
> > >> > > >> > > >running
> > >> > > >> > > without issues. When we replicate the ACS VR features in
> new
> > >> > > >> > > solution it takes some to find the missing pieces (hidden
> > bugs).
> > >> > > >> > > >
> > >> > > >> > > >Thanks,
> > >> > > >> > > >Jayapal
> > >> > > >> > > >
> > >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> > >> > > >> > > >
> > >> > > >> > > >> nux@li.nux.ro> wrote:
> > >> > > >> > > >>
> > >> > > >> > > >> Hi,
> > >> > > >> > > >>
> > >> > > >> > > >> I like the idea.
> > >> > > >> > > >>
> > >> > > >> > > >> Cloudrouter looks really promising, I'm not too keen on
> > VyOS
> > >> > > >> > > >> (it
> > >> > > >> > > doesn't have a proper http api etc).
> > >> > > >> > > >>
> > >> > > >> > > >> --
> > >> > > >> > > >> Sent from the Delta quadrant using Borg technology!
> > >> > > >> > > >>
> > >> > > >> > > >> Nux!
> > >> > > >> > > >> www.nux.ro
> > >> > > >> > > >>
> > >> > > >> > > >>
> > >> > > >> > > abhinandan.prateek@shapeblue.com
> > >> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
> > >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> > @shapeblue
> > >> > > >> > >
> > >> > > >> > >
> > >> > > >> > >
> > >> > > >> > > ----- Original Message -----
> > >> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> > >> > > >> > > >>> To: dev@cloudstack.apache.org
> > >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> > >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
> > >> > > >> > > >>
> > >> > > >> > > >>> *Disclaimer:* This is a thought experiment and should
> be
> > >> > > >> > > >>> treated as
> > >> > > >> > > such.
> > >> > > >> > > >>> Please weigh in with the good and bad of this idea...
> > >> > > >> > > >>>
> > >> > > >> > > >>> A couple of us have been discussing the idea of
> > potentially
> > >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open Source
> > Vyatta
> > >> > VM).
> > >> > > >> > > >>> There may be a license issue because I think it is
> > licensed
> > >> > > >> > > >>> under GPL, but for the sake of discussion, let's assume
> > we
> > >> > > >> > > >>> can overcome any
> > >> > > >> > > license issues.
> > >> > > >> > > >>>
> > >> > > >> > > >>> I have spent some time recently with the VyOS and I
> have
> > to
> > >> > > >> > > >>> admit, I was pretty impressed. It is simple and
> > intuitive
> > >> > > >> > > >>> and it gives you a lot more options for auditing the
> > >> > > configuration etc...
> > >> > > >> > > >>>
> > >> > > >> > > >>> Items of potential interest:
> > >> > > >> > > >>> - Clean up our current VR script spaghetti to a simpler
> > more
> > >> > > >> > > >>> auditable configuration workflow.
> > >> > > >> > > >>> - Gives a cleaner path for IPv6 support.
> > >> > > >> > > >>> - Handles VPN configuration via the same configuration
> > >> > > interface.
> > >> > > >> > > >>> - Support for OSPF & BGP.
> > >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
> > >> > > >> > > >>> - Easily supports HA (redundant routers) through VRRP.
> > >> > > >> > > >>> - VXLAN support.
> > >> > > >> > > >>> - Transaction based changes to the VR with rollback on
> > >> error.
> > >> > > >> > > >>>
> > >> > > >> > > >>> Items that could be difficult to solve:
> > >> > > >> > > >>> - Userdata password reset workflow and implementation.
> > >> > > >> > > >>> - Upgrade process.
> > >> > > >> > > >>>
> > >> > > >> > > >>> The VyOS is not the only option if we were to consider
> > this
> > >> > > >> approach.
> > >> > > >> > > >>> Another option, which I don't know as well, would be
> > >> > > >> > > >>> CloudRouter (AGPL
> > >> > > >> > > >>> license) [2] which is purely API driven.
> > >> > > >> > > >>>
> > >> > > >> > > >>> Anyway, would love to hear your thoughts...
> > >> > > >> > > >>>
> > >> > > >> > > >>> Will
> > >> > > >> > > >>>
> > >> > > >> > > >>> [1] https://vyos.io/
> > >> > > >> > > >>> [2] https://cloudrouter.org/
> > >> > > >> > > >
> > >> > > >> > > >
> > >> > > >> > > >
> > >> > > >> > > >
> > >> > > >> > > >DISCLAIMER
> > >> > > >> > > >==========
> > >> > > >> > > >This e-mail may contain privileged and confidential
> > information
> > >> > > >> > > >which is
> > >> > > >> > > the property of Accelerite, a Persistent Systems business.
> > It is
> > >> > > >> > > intended only for the use of the individual or entity to
> > which
> > >> it
> > >> > > >> > > is addressed. If you are not the intended recipient, you
> are
> > not
> > >> > > >> > > authorized to read, retain, copy, print, distribute or use
> > this
> > >> > > >> > > message. If you have received this communication in error,
> > >> please
> > >> > > >> > > notify the sender and delete all copies of this message.
> > >> > > >> > > Accelerite, a Persistent Systems business does not accept
> any
> > >> > > >> > > liability for virus
> > >> > > >> > infected mails.
> > >> > > >> > >
> > >> > > >> >
> > >> > > >>
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
The VR has been biting us far too often recently, which is why we have
started looking into alternative implementations.
One of the things that is nice about potentially using the VyOS is that it
is based on Debian, so we should be able to run the other services that we
currently have like the password server and userdata on the VyOS. This
means we would not have to change our architecture initially and could
focus on only replacing the networking paths.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Fri, Sep 16, 2016 at 6:20 AM, Nux! <nu...@li.nux.ro> wrote:
> The more this is discussed the more I think we should stick with our VR.
>
> All these other options either seem unfinished or with incompatible
> license.
>
> VyOS looks the most promising so far, it's a serious, mature project.
> Adopting it though means we'll have to microservice our way out of it with
> extra machines for DNS/USERDATA/etc, unless we can make VyOS serve those
> too. Imho this adds complexity we should void.
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Will Stevens" <ws...@cloudops.com>
> > To: dev@cloudstack.apache.org
> > Sent: Thursday, 15 September, 2016 17:21:28
> > Subject: Re: [DISCUSS] Replacing the VR
>
> > Ya, we would need to add a daemon for VPN as well. Load balancing is
> > another aspect which we will need to consider if we went this route.
> > Something like https://traefik.io/ could potentially be a good fit due
> to
> > its API driven configuration, but it may be more than what we need.
> >
> > We should probably try define which pieces make sense to be solved
> together
> > and which pieces would be best suited to be broken out.
> >
> > I think the network connectivity, routing and firewalling should probably
> > all stay together since the majority of the tools we would potentially
> use
> > would handle all of that together in a single implementation.
> >
> > The password server and userdata seems like a good option for being
> broken
> > out and handled independently (and probably rewritten completely since
> they
> > currently have some issues).
> >
> > Load balancing is another that could warrant splitting out, but that
> > depends on what direction we go and how we would be managing it. DHCP
> and
> > DNS are others which could go either way.
> >
> > If we do split out services, I think we should consolidate as much as we
> > can into each service we break out. Ideally a network packet would never
> > hit more than one, maybe two, services. I don't think we should be
> > splitting services 'just because', I think we need a valid case for
> > splitting any service out because it adds complexity. Our project is
> > already complex enough, we need to avoid adding complexity unless it is
> > really needed.
> >
> > Some more of my thoughts on this anyway...
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > w cloudops.com *|* tw @CloudOps_
> >
> > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com> wrote:
> >
> >> I do agree with you that this probably isn't the right place the
> password
> >> service and user data.
> >>
> >>
> >> Having said that, after taking a cursory look at the dev docs, it
> doesn't
> >> seem that difficult to add new daemons: https://opensnaproute.github.
> >> io/docs/developer.html#creating-new-component
> >>
> >> <https://opensnaproute.github.io/docs/developer.html#
> >> creating-new-component>
> >>
> >>
> >> They've definitely build it with a microservices architecture in mind,
> so
> >> each individual feature is abstracted into it's own small daemon
> process.
> >> We could just create a daemon for the password server and the userdata
> >> components if we really had to.
> >>
> >>
> >> - Si
> >>
> >>
> >> ________________________________
> >> From: williamstevens@gmail.com <wi...@gmail.com> on behalf of
> >> Will Stevens <ws...@cloudops.com>
> >> Sent: Thursday, September 15, 2016 9:17 AM
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: [DISCUSS] Replacing the VR
> >>
> >> A big part of why I know about it is because it is written in Go. :P
> >>
> >> Yes, it is definitely interesting for the routing and traffic handling
> >> aspects of the VR. We will likely have to rethink some of the pieces a
> >> little bit like the password server and userdata if we are to adopt a
> >> different VR approach. This is where I think some of JohnB and
> Chiradeep's
> >> ideas make sense. In many ways, it does not make sense for the device
> >> handling routing and network traffic to also be responsible for
> passwords
> >> and userdata.
> >>
> >> *Will STEVENS*
> >> Lead Developer
> >>
> >> *CloudOps* *| *Cloud Solutions Experts
> >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >> w cloudops.com *|* tw @CloudOps_
> >>
> >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com> wrote:
> >>
> >> > I hadn't heard of Flexswitch until you mentioned it. It looks pretty
> >> cool!
> >> > It even supports ONIE install.
> >> >
> >> > To be honest, the ipsec feature could be added, or we could offload
> it to
> >> > separate vm if we needed to. The fact it is so feature rich from a
> >> routing
> >> > perspective (and all API driven) is really nice.
> >> >
> >> >
> >> > Based on the roadmap, it looks like they plan to also support
> >> capabilities
> >> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This will be
> huge
> >> > for our carrier community that rely on these technologies to do
> private
> >> > gateway and inter-VPC interconnections today. We handle this stuff on
> our
> >> > ASRs right now with a vlan interconnect into the VR. Being able to do
> >> MPLS
> >> > all the way to the VR would be awesome.
> >> >
> >> >
> >> > It also seems to be written in GO (a language here at ENA we know very
> >> > well).
> >> >
> >> >
> >> > - Si
> >> >
> >> >
> >> >
> >> > ________________________________
> >> > From: Will Stevens <wi...@gmail.com>
> >> > Sent: Thursday, September 15, 2016 7:06 AM
> >> > To: dev@cloudstack.apache.org
> >> > Subject: RE: [DISCUSS] Replacing the VR
> >> >
> >> > Ya. I don't think it covers our whole use case, but what it does
> cover is
> >> > all api driven...
> >> >
> >> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com> wrote:
> >> >
> >> > > Though I don’t see VPN in Snaproute.. Makes sense since it was not
> >> > > intended to do IPSec.
> >> > >
> >> > > It seems as though VyOS is starting to look like the best option.
> >> > >
> >> > > Regards,
> >> > > Marty Godsey
> >> > > nSource Solutions
> >> > >
> >> > > -----Original Message-----
> >> > > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> >> > > Behalf Of Will Stevens
> >> > > Sent: Wednesday, September 14, 2016 11:06 PM
> >> > > To: dev@cloudstack.apache.org
> >> > > Subject: Re: [DISCUSS] Replacing the VR
> >> > >
> >> > > Or we could go completely crazy and go with something like
> FlexSwitch
> >> > from
> >> > > SnapRoute
> >> > > - http://www.snaproute.com/
> >> > > - https://opensnaproute.github.io/docs/apis.html
> >> > >
> >> > > *Will STEVENS*
> >> > > Lead Developer
> >> > >
> >> > > *CloudOps* *| *Cloud Solutions Experts
> >> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> tw
> >> > > @CloudOps_
> >> > >
> >> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <
> wstevens@cloudops.com>
> >> > > wrote:
> >> > >
> >> > > > I tend to agree with Syed and Marty. I am not sure what problems
> are
> >> > > > solved by splitting up the function of the VR into a bunch of
> >> separate
> >> > > > services. As Syed points out, the complexity added is
> non-trivial.
> >> > > > We now have to manage all the intercontainer networking as well as
> >> the
> >> > > > orchestrated ACS networking.
> >> > > >
> >> > > > VyOS is interesting to me because it covers the majority of our
> use
> >> > > > case with a single unified control plane. It also has good
> support
> >> > > > for extending features we care about, like IPv6, VXLAN, VRRP,
> >> > > > transactions, etc...
> >> > > >
> >> > > > *Will STEVENS*
> >> > > > Lead Developer
> >> > > >
> >> > > > *CloudOps* *| *Cloud Solutions Experts
> >> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> *|*
> >> tw
> >> > > > @CloudOps_
> >> > > >
> >> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <sa...@cloudops.com>
> >> > wrote:
> >> > > >
> >> > > >> Agree with Marty, adding Docker containers to the picture
> although
> >> > > >> can make the VR more flexible but the added complexity is just
> not
> >> > > >> worth it. Not to mention we would need to take care of networking
> >> > > >> each container manually and given that our iptable rules are very
> >> > > >> unstable at the moment I don't see a big value add.
> >> > > >>
> >> > > >> Vyos looks like a better solution to me. I know that it does not
> >> > > >> provide an api but it does fit the bill quite well otherwise. I
> >> > > >> specially like the fact that it has a transaction based model and
> >> you
> >> > > >> can rollback changes if something goes wrong.
> >> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <
> marty@gonsource.com>
> >> > > wrote:
> >> > > >>
> >> > > >> > Licensing aside, I think splitting the various functions into
> >> > > >> > containers is not a good route either. This will force users to
> >> > > >> > have to maintain
> >> > > >> and
> >> > > >> > use containers and adds complexity to the networking aspects of
> >> ACS.
> >> > > >> > Complexity decreases stability. Now I understand the argument
> that
> >> > > >> > a monolithic approach also brings its own set of issues but it
> >> also
> >> > > >> > simplifies it.
> >> > > >> >
> >> > > >> > Regards,
> >> > > >> > Marty Godsey
> >> > > >> > nSource Solutions
> >> > > >> >
> >> > > >> > -----Original Message-----
> >> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> >> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
> >> > > >> > To: dev@cloudstack.apache.org
> >> > > >> > Subject: Re: [DISCUSS] Replacing the VR
> >> > > >> >
> >> > > >> > I rather doubt that the Cloudrouter will fit the needs of the
> >> > > >> > CloudStack project
> >> > > >> > - it is AGPL licensed. Many enterprises will not touch
> anything
> >> > > >> > that
> >> > > >> has
> >> > > >> > AGPL
> >> > > >> > - the github repo shows rather infrequent updates. Quite
> likely
> >> > > >> > they aren't considering the use cases of the CloudStack
> community
> >> > > >> >
> >> > > >> > I'd back John B's comments on disaggregating the VR. Split it
> into
> >> > > >> > many docker containers
> >> > > >> > - password server
> >> > > >> > - userdata server
> >> > > >> > - DHCP / DNS
> >> > > >> > - s2s VPN
> >> > > >> > - RA VPN
> >> > > >> > - intra-VPC routing and ACL
> >> > > >> > - Port forwarding + NAT
> >> > > >> > - FW
> >> > > >> > - LB (public)
> >> > > >> > - LB (internal),
> >> > > >> > - secondary storage
> >> > > >> > - agent
> >> > > >> > Glue them together with docker compose files (one per use
> case -
> >> > > >> > basic zone, isolated, VPC, SSVM, etc).
> >> > > >> >
> >> > > >> > The VR image then becomes a JeOS + docker. You can test each of
> >> the
> >> > > >> > components independently and fixing one bug in the field (say
> >> DHCP)
> >> > > >> > is hitless to the other components. You don't need to build
> >> > > >> > per-hypervisor VRs. You could even run on baremetal.
> >> > > >> >
> >> > > >> > Along the way you need to figure out how to
> >> > > >> > - make the traffic traverse the containers that are needed to
> be
> >> > > >> > traversed (in most cases just 1)
> >> > > >> > - bootstrap the router (how does it find its compose file?
> where
> >> > > >> > is the
> >> > > >> > registry?)
> >> > > >> > - rethink the command and control of the VR functions. SSH
> works,
> >> > > >> > but something more declarative, idempotent should be explored.
> >> > > >> >
> >> > > >> > As you do this, it becomes clearer which of the functions can
> be
> >> > > >> > substituted by for example CloudRouter. Command and Control of
> the
> >> > > >> docker
> >> > > >> > containers can be moved out to another container. Etc.
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> >> > > >> > <ma...@gonsource.com>
> >> > > >> > wrote:
> >> > > >> >
> >> > > >> > > This one does look nice. My biggest concern is the lack of
> >> > > >> > > VXLANs. It seems that any of the ones we mentioned do not
> have
> >> an
> >> > > >> > > API so we may be stuck at the SSH method.
> >> > > >> > >
> >> > > >> > > Regards,
> >> > > >> > > Marty Godsey
> >> > > >> > > nSource Solutions
> >> > > >> > >
> >> > > >> > > -----Original Message-----
> >> > > >> > > From: Abhinandan Prateek
> >> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
> >> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> >> > > >> > > To: dev@cloudstack.apache.org
> >> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> >> > > >> > >
> >> > > >> > > Cloudrouter looks promising. These have potential to save
> future
> >> > > >> > > engineering effort for example on ipv6 routing, OSPF etc.
> >> > > >> > > And the best part is they come with test automation
> framework.
> >> > > >> > >
> >> > > >> > >
> >> > > >> > >
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> >> > > >> > > <ja...@accelerite.com>
> >> > > >> > > wrote:
> >> > > >> > >
> >> > > >> > > >Hi,
> >> > > >> > > >
> >> > > >> > > >Instead of replacing the VR in first place we should add
> >> > > >> > > >VyOS/cloudrouter
> >> > > >> > > as provider. Once it is stable, network offerings (on
> upgrade)
> >> > > >> > > can be updated to use it and we can drop the VR if we want at
> >> > > >> > > that release
> >> > > >> > onwards.
> >> > > >> > > >
> >> > > >> > > >VR is stabilized over a period of time and some of them are
> >> > > >> > > >running
> >> > > >> > > without issues. When we replicate the ACS VR features in new
> >> > > >> > > solution it takes some to find the missing pieces (hidden
> bugs).
> >> > > >> > > >
> >> > > >> > > >Thanks,
> >> > > >> > > >Jayapal
> >> > > >> > > >
> >> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> >> > > >> > > >
> >> > > >> > > >> nux@li.nux.ro> wrote:
> >> > > >> > > >>
> >> > > >> > > >> Hi,
> >> > > >> > > >>
> >> > > >> > > >> I like the idea.
> >> > > >> > > >>
> >> > > >> > > >> Cloudrouter looks really promising, I'm not too keen on
> VyOS
> >> > > >> > > >> (it
> >> > > >> > > doesn't have a proper http api etc).
> >> > > >> > > >>
> >> > > >> > > >> --
> >> > > >> > > >> Sent from the Delta quadrant using Borg technology!
> >> > > >> > > >>
> >> > > >> > > >> Nux!
> >> > > >> > > >> www.nux.ro
> >> > > >> > > >>
> >> > > >> > > >>
> >> > > >> > > abhinandan.prateek@shapeblue.com
> >> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
> >> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
> >> > > >> > >
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > ----- Original Message -----
> >> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> >> > > >> > > >>> To: dev@cloudstack.apache.org
> >> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> >> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
> >> > > >> > > >>
> >> > > >> > > >>> *Disclaimer:* This is a thought experiment and should be
> >> > > >> > > >>> treated as
> >> > > >> > > such.
> >> > > >> > > >>> Please weigh in with the good and bad of this idea...
> >> > > >> > > >>>
> >> > > >> > > >>> A couple of us have been discussing the idea of
> potentially
> >> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open Source
> Vyatta
> >> > VM).
> >> > > >> > > >>> There may be a license issue because I think it is
> licensed
> >> > > >> > > >>> under GPL, but for the sake of discussion, let's assume
> we
> >> > > >> > > >>> can overcome any
> >> > > >> > > license issues.
> >> > > >> > > >>>
> >> > > >> > > >>> I have spent some time recently with the VyOS and I have
> to
> >> > > >> > > >>> admit, I was pretty impressed. It is simple and
> intuitive
> >> > > >> > > >>> and it gives you a lot more options for auditing the
> >> > > configuration etc...
> >> > > >> > > >>>
> >> > > >> > > >>> Items of potential interest:
> >> > > >> > > >>> - Clean up our current VR script spaghetti to a simpler
> more
> >> > > >> > > >>> auditable configuration workflow.
> >> > > >> > > >>> - Gives a cleaner path for IPv6 support.
> >> > > >> > > >>> - Handles VPN configuration via the same configuration
> >> > > interface.
> >> > > >> > > >>> - Support for OSPF & BGP.
> >> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
> >> > > >> > > >>> - Easily supports HA (redundant routers) through VRRP.
> >> > > >> > > >>> - VXLAN support.
> >> > > >> > > >>> - Transaction based changes to the VR with rollback on
> >> error.
> >> > > >> > > >>>
> >> > > >> > > >>> Items that could be difficult to solve:
> >> > > >> > > >>> - Userdata password reset workflow and implementation.
> >> > > >> > > >>> - Upgrade process.
> >> > > >> > > >>>
> >> > > >> > > >>> The VyOS is not the only option if we were to consider
> this
> >> > > >> approach.
> >> > > >> > > >>> Another option, which I don't know as well, would be
> >> > > >> > > >>> CloudRouter (AGPL
> >> > > >> > > >>> license) [2] which is purely API driven.
> >> > > >> > > >>>
> >> > > >> > > >>> Anyway, would love to hear your thoughts...
> >> > > >> > > >>>
> >> > > >> > > >>> Will
> >> > > >> > > >>>
> >> > > >> > > >>> [1] https://vyos.io/
> >> > > >> > > >>> [2] https://cloudrouter.org/
> >> > > >> > > >
> >> > > >> > > >
> >> > > >> > > >
> >> > > >> > > >
> >> > > >> > > >DISCLAIMER
> >> > > >> > > >==========
> >> > > >> > > >This e-mail may contain privileged and confidential
> information
> >> > > >> > > >which is
> >> > > >> > > the property of Accelerite, a Persistent Systems business.
> It is
> >> > > >> > > intended only for the use of the individual or entity to
> which
> >> it
> >> > > >> > > is addressed. If you are not the intended recipient, you are
> not
> >> > > >> > > authorized to read, retain, copy, print, distribute or use
> this
> >> > > >> > > message. If you have received this communication in error,
> >> please
> >> > > >> > > notify the sender and delete all copies of this message.
> >> > > >> > > Accelerite, a Persistent Systems business does not accept any
> >> > > >> > > liability for virus
> >> > > >> > infected mails.
> >> > > >> > >
> >> > > >> >
> >> > > >>
> >> > > >
> >> > > >
> >> > >
> >> >
>
Re: [DISCUSS] Replacing the VR
Posted by Nux! <nu...@li.nux.ro>.
The more this is discussed the more I think we should stick with our VR.
All these other options either seem unfinished or with incompatible license.
VyOS looks the most promising so far, it's a serious, mature project.
Adopting it though means we'll have to microservice our way out of it with extra machines for DNS/USERDATA/etc, unless we can make VyOS serve those too. Imho this adds complexity we should void.
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Will Stevens" <ws...@cloudops.com>
> To: dev@cloudstack.apache.org
> Sent: Thursday, 15 September, 2016 17:21:28
> Subject: Re: [DISCUSS] Replacing the VR
> Ya, we would need to add a daemon for VPN as well. Load balancing is
> another aspect which we will need to consider if we went this route.
> Something like https://traefik.io/ could potentially be a good fit due to
> its API driven configuration, but it may be more than what we need.
>
> We should probably try define which pieces make sense to be solved together
> and which pieces would be best suited to be broken out.
>
> I think the network connectivity, routing and firewalling should probably
> all stay together since the majority of the tools we would potentially use
> would handle all of that together in a single implementation.
>
> The password server and userdata seems like a good option for being broken
> out and handled independently (and probably rewritten completely since they
> currently have some issues).
>
> Load balancing is another that could warrant splitting out, but that
> depends on what direction we go and how we would be managing it. DHCP and
> DNS are others which could go either way.
>
> If we do split out services, I think we should consolidate as much as we
> can into each service we break out. Ideally a network packet would never
> hit more than one, maybe two, services. I don't think we should be
> splitting services 'just because', I think we need a valid case for
> splitting any service out because it adds complexity. Our project is
> already complex enough, we need to avoid adding complexity unless it is
> really needed.
>
> Some more of my thoughts on this anyway...
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com> wrote:
>
>> I do agree with you that this probably isn't the right place the password
>> service and user data.
>>
>>
>> Having said that, after taking a cursory look at the dev docs, it doesn't
>> seem that difficult to add new daemons: https://opensnaproute.github.
>> io/docs/developer.html#creating-new-component
>>
>> <https://opensnaproute.github.io/docs/developer.html#
>> creating-new-component>
>>
>>
>> They've definitely build it with a microservices architecture in mind, so
>> each individual feature is abstracted into it's own small daemon process.
>> We could just create a daemon for the password server and the userdata
>> components if we really had to.
>>
>>
>> - Si
>>
>>
>> ________________________________
>> From: williamstevens@gmail.com <wi...@gmail.com> on behalf of
>> Will Stevens <ws...@cloudops.com>
>> Sent: Thursday, September 15, 2016 9:17 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [DISCUSS] Replacing the VR
>>
>> A big part of why I know about it is because it is written in Go. :P
>>
>> Yes, it is definitely interesting for the routing and traffic handling
>> aspects of the VR. We will likely have to rethink some of the pieces a
>> little bit like the password server and userdata if we are to adopt a
>> different VR approach. This is where I think some of JohnB and Chiradeep's
>> ideas make sense. In many ways, it does not make sense for the device
>> handling routing and network traffic to also be responsible for passwords
>> and userdata.
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> w cloudops.com *|* tw @CloudOps_
>>
>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com> wrote:
>>
>> > I hadn't heard of Flexswitch until you mentioned it. It looks pretty
>> cool!
>> > It even supports ONIE install.
>> >
>> > To be honest, the ipsec feature could be added, or we could offload it to
>> > separate vm if we needed to. The fact it is so feature rich from a
>> routing
>> > perspective (and all API driven) is really nice.
>> >
>> >
>> > Based on the roadmap, it looks like they plan to also support
>> capabilities
>> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This will be huge
>> > for our carrier community that rely on these technologies to do private
>> > gateway and inter-VPC interconnections today. We handle this stuff on our
>> > ASRs right now with a vlan interconnect into the VR. Being able to do
>> MPLS
>> > all the way to the VR would be awesome.
>> >
>> >
>> > It also seems to be written in GO (a language here at ENA we know very
>> > well).
>> >
>> >
>> > - Si
>> >
>> >
>> >
>> > ________________________________
>> > From: Will Stevens <wi...@gmail.com>
>> > Sent: Thursday, September 15, 2016 7:06 AM
>> > To: dev@cloudstack.apache.org
>> > Subject: RE: [DISCUSS] Replacing the VR
>> >
>> > Ya. I don't think it covers our whole use case, but what it does cover is
>> > all api driven...
>> >
>> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com> wrote:
>> >
>> > > Though I don’t see VPN in Snaproute.. Makes sense since it was not
>> > > intended to do IPSec.
>> > >
>> > > It seems as though VyOS is starting to look like the best option.
>> > >
>> > > Regards,
>> > > Marty Godsey
>> > > nSource Solutions
>> > >
>> > > -----Original Message-----
>> > > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>> > > Behalf Of Will Stevens
>> > > Sent: Wednesday, September 14, 2016 11:06 PM
>> > > To: dev@cloudstack.apache.org
>> > > Subject: Re: [DISCUSS] Replacing the VR
>> > >
>> > > Or we could go completely crazy and go with something like FlexSwitch
>> > from
>> > > SnapRoute
>> > > - http://www.snaproute.com/
>> > > - https://opensnaproute.github.io/docs/apis.html
>> > >
>> > > *Will STEVENS*
>> > > Lead Developer
>> > >
>> > > *CloudOps* *| *Cloud Solutions Experts
>> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>> > > @CloudOps_
>> > >
>> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <ws...@cloudops.com>
>> > > wrote:
>> > >
>> > > > I tend to agree with Syed and Marty. I am not sure what problems are
>> > > > solved by splitting up the function of the VR into a bunch of
>> separate
>> > > > services. As Syed points out, the complexity added is non-trivial.
>> > > > We now have to manage all the intercontainer networking as well as
>> the
>> > > > orchestrated ACS networking.
>> > > >
>> > > > VyOS is interesting to me because it covers the majority of our use
>> > > > case with a single unified control plane. It also has good support
>> > > > for extending features we care about, like IPv6, VXLAN, VRRP,
>> > > > transactions, etc...
>> > > >
>> > > > *Will STEVENS*
>> > > > Lead Developer
>> > > >
>> > > > *CloudOps* *| *Cloud Solutions Experts
>> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>> tw
>> > > > @CloudOps_
>> > > >
>> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <sa...@cloudops.com>
>> > wrote:
>> > > >
>> > > >> Agree with Marty, adding Docker containers to the picture although
>> > > >> can make the VR more flexible but the added complexity is just not
>> > > >> worth it. Not to mention we would need to take care of networking
>> > > >> each container manually and given that our iptable rules are very
>> > > >> unstable at the moment I don't see a big value add.
>> > > >>
>> > > >> Vyos looks like a better solution to me. I know that it does not
>> > > >> provide an api but it does fit the bill quite well otherwise. I
>> > > >> specially like the fact that it has a transaction based model and
>> you
>> > > >> can rollback changes if something goes wrong.
>> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <ma...@gonsource.com>
>> > > wrote:
>> > > >>
>> > > >> > Licensing aside, I think splitting the various functions into
>> > > >> > containers is not a good route either. This will force users to
>> > > >> > have to maintain
>> > > >> and
>> > > >> > use containers and adds complexity to the networking aspects of
>> ACS.
>> > > >> > Complexity decreases stability. Now I understand the argument that
>> > > >> > a monolithic approach also brings its own set of issues but it
>> also
>> > > >> > simplifies it.
>> > > >> >
>> > > >> > Regards,
>> > > >> > Marty Godsey
>> > > >> > nSource Solutions
>> > > >> >
>> > > >> > -----Original Message-----
>> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
>> > > >> > To: dev@cloudstack.apache.org
>> > > >> > Subject: Re: [DISCUSS] Replacing the VR
>> > > >> >
>> > > >> > I rather doubt that the Cloudrouter will fit the needs of the
>> > > >> > CloudStack project
>> > > >> > - it is AGPL licensed. Many enterprises will not touch anything
>> > > >> > that
>> > > >> has
>> > > >> > AGPL
>> > > >> > - the github repo shows rather infrequent updates. Quite likely
>> > > >> > they aren't considering the use cases of the CloudStack community
>> > > >> >
>> > > >> > I'd back John B's comments on disaggregating the VR. Split it into
>> > > >> > many docker containers
>> > > >> > - password server
>> > > >> > - userdata server
>> > > >> > - DHCP / DNS
>> > > >> > - s2s VPN
>> > > >> > - RA VPN
>> > > >> > - intra-VPC routing and ACL
>> > > >> > - Port forwarding + NAT
>> > > >> > - FW
>> > > >> > - LB (public)
>> > > >> > - LB (internal),
>> > > >> > - secondary storage
>> > > >> > - agent
>> > > >> > Glue them together with docker compose files (one per use case -
>> > > >> > basic zone, isolated, VPC, SSVM, etc).
>> > > >> >
>> > > >> > The VR image then becomes a JeOS + docker. You can test each of
>> the
>> > > >> > components independently and fixing one bug in the field (say
>> DHCP)
>> > > >> > is hitless to the other components. You don't need to build
>> > > >> > per-hypervisor VRs. You could even run on baremetal.
>> > > >> >
>> > > >> > Along the way you need to figure out how to
>> > > >> > - make the traffic traverse the containers that are needed to be
>> > > >> > traversed (in most cases just 1)
>> > > >> > - bootstrap the router (how does it find its compose file? where
>> > > >> > is the
>> > > >> > registry?)
>> > > >> > - rethink the command and control of the VR functions. SSH works,
>> > > >> > but something more declarative, idempotent should be explored.
>> > > >> >
>> > > >> > As you do this, it becomes clearer which of the functions can be
>> > > >> > substituted by for example CloudRouter. Command and Control of the
>> > > >> docker
>> > > >> > containers can be moved out to another container. Etc.
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>> > > >> > <ma...@gonsource.com>
>> > > >> > wrote:
>> > > >> >
>> > > >> > > This one does look nice. My biggest concern is the lack of
>> > > >> > > VXLANs. It seems that any of the ones we mentioned do not have
>> an
>> > > >> > > API so we may be stuck at the SSH method.
>> > > >> > >
>> > > >> > > Regards,
>> > > >> > > Marty Godsey
>> > > >> > > nSource Solutions
>> > > >> > >
>> > > >> > > -----Original Message-----
>> > > >> > > From: Abhinandan Prateek
>> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
>> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
>> > > >> > > To: dev@cloudstack.apache.org
>> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
>> > > >> > >
>> > > >> > > Cloudrouter looks promising. These have potential to save future
>> > > >> > > engineering effort for example on ipv6 routing, OSPF etc.
>> > > >> > > And the best part is they come with test automation framework.
>> > > >> > >
>> > > >> > >
>> > > >> > >
>> > > >> > >
>> > > >> > >
>> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
>> > > >> > > <ja...@accelerite.com>
>> > > >> > > wrote:
>> > > >> > >
>> > > >> > > >Hi,
>> > > >> > > >
>> > > >> > > >Instead of replacing the VR in first place we should add
>> > > >> > > >VyOS/cloudrouter
>> > > >> > > as provider. Once it is stable, network offerings (on upgrade)
>> > > >> > > can be updated to use it and we can drop the VR if we want at
>> > > >> > > that release
>> > > >> > onwards.
>> > > >> > > >
>> > > >> > > >VR is stabilized over a period of time and some of them are
>> > > >> > > >running
>> > > >> > > without issues. When we replicate the ACS VR features in new
>> > > >> > > solution it takes some to find the missing pieces (hidden bugs).
>> > > >> > > >
>> > > >> > > >Thanks,
>> > > >> > > >Jayapal
>> > > >> > > >
>> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
>> > > >> > > >
>> > > >> > > >> nux@li.nux.ro> wrote:
>> > > >> > > >>
>> > > >> > > >> Hi,
>> > > >> > > >>
>> > > >> > > >> I like the idea.
>> > > >> > > >>
>> > > >> > > >> Cloudrouter looks really promising, I'm not too keen on VyOS
>> > > >> > > >> (it
>> > > >> > > doesn't have a proper http api etc).
>> > > >> > > >>
>> > > >> > > >> --
>> > > >> > > >> Sent from the Delta quadrant using Borg technology!
>> > > >> > > >>
>> > > >> > > >> Nux!
>> > > >> > > >> www.nux.ro
>> > > >> > > >>
>> > > >> > > >>
>> > > >> > > abhinandan.prateek@shapeblue.com
>> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
>> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
>> > > >> > >
>> > > >> > >
>> > > >> > >
>> > > >> > > ----- Original Message -----
>> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
>> > > >> > > >>> To: dev@cloudstack.apache.org
>> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
>> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
>> > > >> > > >>
>> > > >> > > >>> *Disclaimer:* This is a thought experiment and should be
>> > > >> > > >>> treated as
>> > > >> > > such.
>> > > >> > > >>> Please weigh in with the good and bad of this idea...
>> > > >> > > >>>
>> > > >> > > >>> A couple of us have been discussing the idea of potentially
>> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta
>> > VM).
>> > > >> > > >>> There may be a license issue because I think it is licensed
>> > > >> > > >>> under GPL, but for the sake of discussion, let's assume we
>> > > >> > > >>> can overcome any
>> > > >> > > license issues.
>> > > >> > > >>>
>> > > >> > > >>> I have spent some time recently with the VyOS and I have to
>> > > >> > > >>> admit, I was pretty impressed. It is simple and intuitive
>> > > >> > > >>> and it gives you a lot more options for auditing the
>> > > configuration etc...
>> > > >> > > >>>
>> > > >> > > >>> Items of potential interest:
>> > > >> > > >>> - Clean up our current VR script spaghetti to a simpler more
>> > > >> > > >>> auditable configuration workflow.
>> > > >> > > >>> - Gives a cleaner path for IPv6 support.
>> > > >> > > >>> - Handles VPN configuration via the same configuration
>> > > interface.
>> > > >> > > >>> - Support for OSPF & BGP.
>> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
>> > > >> > > >>> - Easily supports HA (redundant routers) through VRRP.
>> > > >> > > >>> - VXLAN support.
>> > > >> > > >>> - Transaction based changes to the VR with rollback on
>> error.
>> > > >> > > >>>
>> > > >> > > >>> Items that could be difficult to solve:
>> > > >> > > >>> - Userdata password reset workflow and implementation.
>> > > >> > > >>> - Upgrade process.
>> > > >> > > >>>
>> > > >> > > >>> The VyOS is not the only option if we were to consider this
>> > > >> approach.
>> > > >> > > >>> Another option, which I don't know as well, would be
>> > > >> > > >>> CloudRouter (AGPL
>> > > >> > > >>> license) [2] which is purely API driven.
>> > > >> > > >>>
>> > > >> > > >>> Anyway, would love to hear your thoughts...
>> > > >> > > >>>
>> > > >> > > >>> Will
>> > > >> > > >>>
>> > > >> > > >>> [1] https://vyos.io/
>> > > >> > > >>> [2] https://cloudrouter.org/
>> > > >> > > >
>> > > >> > > >
>> > > >> > > >
>> > > >> > > >
>> > > >> > > >DISCLAIMER
>> > > >> > > >==========
>> > > >> > > >This e-mail may contain privileged and confidential information
>> > > >> > > >which is
>> > > >> > > the property of Accelerite, a Persistent Systems business. It is
>> > > >> > > intended only for the use of the individual or entity to which
>> it
>> > > >> > > is addressed. If you are not the intended recipient, you are not
>> > > >> > > authorized to read, retain, copy, print, distribute or use this
>> > > >> > > message. If you have received this communication in error,
>> please
>> > > >> > > notify the sender and delete all copies of this message.
>> > > >> > > Accelerite, a Persistent Systems business does not accept any
>> > > >> > > liability for virus
>> > > >> > infected mails.
>> > > >> > >
>> > > >> >
>> > > >>
>> > > >
>> > > >
>> > >
>> >
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
Ya, I hear ya Ilya. I have been trying not to focus on the license
initially in order to get as much input as possible.
I guess at this point we should start understanding what we need from a
license. On Wikipedia it has the license as "License Free software
licenses (mainly GPL)" for VyOS. If we are interested in VyOS as an
option, we may want to contact them and see if we can work something out
regarding license which fits what we need. Not sure if that is an option,
but worth checking into...
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Fri, Sep 16, 2016 at 1:49 AM, ilya <il...@gmail.com> wrote:
> Hi folks, please kindly keep in mind the open source licensing when
> choosing the next router. AGPL or GPL v3 are "no go" for many shops.
>
> I could not find the license for Vyatta, i do know Vyatta dev folks were
> open to working with ACS few years back, but somehow the initiative was
> dropped.
>
>
> On 9/15/16 9:21 AM, Will Stevens wrote:
> > Ya, we would need to add a daemon for VPN as well. Load balancing is
> > another aspect which we will need to consider if we went this route.
> > Something like https://traefik.io/ could potentially be a good fit due
> to
> > its API driven configuration, but it may be more than what we need.
> >
> > We should probably try define which pieces make sense to be solved
> together
> > and which pieces would be best suited to be broken out.
> >
> > I think the network connectivity, routing and firewalling should probably
> > all stay together since the majority of the tools we would potentially
> use
> > would handle all of that together in a single implementation.
> >
> > The password server and userdata seems like a good option for being
> broken
> > out and handled independently (and probably rewritten completely since
> they
> > currently have some issues).
> >
> > Load balancing is another that could warrant splitting out, but that
> > depends on what direction we go and how we would be managing it. DHCP
> and
> > DNS are others which could go either way.
> >
> > If we do split out services, I think we should consolidate as much as we
> > can into each service we break out. Ideally a network packet would never
> > hit more than one, maybe two, services. I don't think we should be
> > splitting services 'just because', I think we need a valid case for
> > splitting any service out because it adds complexity. Our project is
> > already complex enough, we need to avoid adding complexity unless it is
> > really needed.
> >
> > Some more of my thoughts on this anyway...
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > w cloudops.com *|* tw @CloudOps_
> >
> > On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com> wrote:
> >
> >> I do agree with you that this probably isn't the right place the
> password
> >> service and user data.
> >>
> >>
> >> Having said that, after taking a cursory look at the dev docs, it
> doesn't
> >> seem that difficult to add new daemons: https://opensnaproute.github.
> >> io/docs/developer.html#creating-new-component
> >>
> >> <https://opensnaproute.github.io/docs/developer.html#
> >> creating-new-component>
> >>
> >>
> >> They've definitely build it with a microservices architecture in mind,
> so
> >> each individual feature is abstracted into it's own small daemon
> process.
> >> We could just create a daemon for the password server and the userdata
> >> components if we really had to.
> >>
> >>
> >> - Si
> >>
> >>
> >> ________________________________
> >> From: williamstevens@gmail.com <wi...@gmail.com> on behalf of
> >> Will Stevens <ws...@cloudops.com>
> >> Sent: Thursday, September 15, 2016 9:17 AM
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: [DISCUSS] Replacing the VR
> >>
> >> A big part of why I know about it is because it is written in Go. :P
> >>
> >> Yes, it is definitely interesting for the routing and traffic handling
> >> aspects of the VR. We will likely have to rethink some of the pieces a
> >> little bit like the password server and userdata if we are to adopt a
> >> different VR approach. This is where I think some of JohnB and
> Chiradeep's
> >> ideas make sense. In many ways, it does not make sense for the device
> >> handling routing and network traffic to also be responsible for
> passwords
> >> and userdata.
> >>
> >> *Will STEVENS*
> >> Lead Developer
> >>
> >> *CloudOps* *| *Cloud Solutions Experts
> >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >> w cloudops.com *|* tw @CloudOps_
> >>
> >> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com> wrote:
> >>
> >>> I hadn't heard of Flexswitch until you mentioned it. It looks pretty
> >> cool!
> >>> It even supports ONIE install.
> >>>
> >>> To be honest, the ipsec feature could be added, or we could offload it
> to
> >>> separate vm if we needed to. The fact it is so feature rich from a
> >> routing
> >>> perspective (and all API driven) is really nice.
> >>>
> >>>
> >>> Based on the roadmap, it looks like they plan to also support
> >> capabilities
> >>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This will be
> huge
> >>> for our carrier community that rely on these technologies to do private
> >>> gateway and inter-VPC interconnections today. We handle this stuff on
> our
> >>> ASRs right now with a vlan interconnect into the VR. Being able to do
> >> MPLS
> >>> all the way to the VR would be awesome.
> >>>
> >>>
> >>> It also seems to be written in GO (a language here at ENA we know very
> >>> well).
> >>>
> >>>
> >>> - Si
> >>>
> >>>
> >>>
> >>> ________________________________
> >>> From: Will Stevens <wi...@gmail.com>
> >>> Sent: Thursday, September 15, 2016 7:06 AM
> >>> To: dev@cloudstack.apache.org
> >>> Subject: RE: [DISCUSS] Replacing the VR
> >>>
> >>> Ya. I don't think it covers our whole use case, but what it does cover
> is
> >>> all api driven...
> >>>
> >>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com> wrote:
> >>>
> >>>> Though I don’t see VPN in Snaproute.. Makes sense since it was not
> >>>> intended to do IPSec.
> >>>>
> >>>> It seems as though VyOS is starting to look like the best option.
> >>>>
> >>>> Regards,
> >>>> Marty Godsey
> >>>> nSource Solutions
> >>>>
> >>>> -----Original Message-----
> >>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> >>>> Behalf Of Will Stevens
> >>>> Sent: Wednesday, September 14, 2016 11:06 PM
> >>>> To: dev@cloudstack.apache.org
> >>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>
> >>>> Or we could go completely crazy and go with something like FlexSwitch
> >>> from
> >>>> SnapRoute
> >>>> - http://www.snaproute.com/
> >>>> - https://opensnaproute.github.io/docs/apis.html
> >>>>
> >>>> *Will STEVENS*
> >>>> Lead Developer
> >>>>
> >>>> *CloudOps* *| *Cloud Solutions Experts
> >>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> >>>> @CloudOps_
> >>>>
> >>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <wstevens@cloudops.com
> >
> >>>> wrote:
> >>>>
> >>>>> I tend to agree with Syed and Marty. I am not sure what problems are
> >>>>> solved by splitting up the function of the VR into a bunch of
> >> separate
> >>>>> services. As Syed points out, the complexity added is non-trivial.
> >>>>> We now have to manage all the intercontainer networking as well as
> >> the
> >>>>> orchestrated ACS networking.
> >>>>>
> >>>>> VyOS is interesting to me because it covers the majority of our use
> >>>>> case with a single unified control plane. It also has good support
> >>>>> for extending features we care about, like IPv6, VXLAN, VRRP,
> >>>>> transactions, etc...
> >>>>>
> >>>>> *Will STEVENS*
> >>>>> Lead Developer
> >>>>>
> >>>>> *CloudOps* *| *Cloud Solutions Experts
> >>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> >> tw
> >>>>> @CloudOps_
> >>>>>
> >>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <sa...@cloudops.com>
> >>> wrote:
> >>>>>
> >>>>>> Agree with Marty, adding Docker containers to the picture although
> >>>>>> can make the VR more flexible but the added complexity is just not
> >>>>>> worth it. Not to mention we would need to take care of networking
> >>>>>> each container manually and given that our iptable rules are very
> >>>>>> unstable at the moment I don't see a big value add.
> >>>>>>
> >>>>>> Vyos looks like a better solution to me. I know that it does not
> >>>>>> provide an api but it does fit the bill quite well otherwise. I
> >>>>>> specially like the fact that it has a transaction based model and
> >> you
> >>>>>> can rollback changes if something goes wrong.
> >>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <ma...@gonsource.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Licensing aside, I think splitting the various functions into
> >>>>>>> containers is not a good route either. This will force users to
> >>>>>>> have to maintain
> >>>>>> and
> >>>>>>> use containers and adds complexity to the networking aspects of
> >> ACS.
> >>>>>>> Complexity decreases stability. Now I understand the argument that
> >>>>>>> a monolithic approach also brings its own set of issues but it
> >> also
> >>>>>>> simplifies it.
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> Marty Godsey
> >>>>>>> nSource Solutions
> >>>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> >>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
> >>>>>>> To: dev@cloudstack.apache.org
> >>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>
> >>>>>>> I rather doubt that the Cloudrouter will fit the needs of the
> >>>>>>> CloudStack project
> >>>>>>> - it is AGPL licensed. Many enterprises will not touch anything
> >>>>>>> that
> >>>>>> has
> >>>>>>> AGPL
> >>>>>>> - the github repo shows rather infrequent updates. Quite likely
> >>>>>>> they aren't considering the use cases of the CloudStack community
> >>>>>>>
> >>>>>>> I'd back John B's comments on disaggregating the VR. Split it into
> >>>>>>> many docker containers
> >>>>>>> - password server
> >>>>>>> - userdata server
> >>>>>>> - DHCP / DNS
> >>>>>>> - s2s VPN
> >>>>>>> - RA VPN
> >>>>>>> - intra-VPC routing and ACL
> >>>>>>> - Port forwarding + NAT
> >>>>>>> - FW
> >>>>>>> - LB (public)
> >>>>>>> - LB (internal),
> >>>>>>> - secondary storage
> >>>>>>> - agent
> >>>>>>> Glue them together with docker compose files (one per use case -
> >>>>>>> basic zone, isolated, VPC, SSVM, etc).
> >>>>>>>
> >>>>>>> The VR image then becomes a JeOS + docker. You can test each of
> >> the
> >>>>>>> components independently and fixing one bug in the field (say
> >> DHCP)
> >>>>>>> is hitless to the other components. You don't need to build
> >>>>>>> per-hypervisor VRs. You could even run on baremetal.
> >>>>>>>
> >>>>>>> Along the way you need to figure out how to
> >>>>>>> - make the traffic traverse the containers that are needed to be
> >>>>>>> traversed (in most cases just 1)
> >>>>>>> - bootstrap the router (how does it find its compose file? where
> >>>>>>> is the
> >>>>>>> registry?)
> >>>>>>> - rethink the command and control of the VR functions. SSH works,
> >>>>>>> but something more declarative, idempotent should be explored.
> >>>>>>>
> >>>>>>> As you do this, it becomes clearer which of the functions can be
> >>>>>>> substituted by for example CloudRouter. Command and Control of the
> >>>>>> docker
> >>>>>>> containers can be moved out to another container. Etc.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> >>>>>>> <ma...@gonsource.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> This one does look nice. My biggest concern is the lack of
> >>>>>>>> VXLANs. It seems that any of the ones we mentioned do not have
> >> an
> >>>>>>>> API so we may be stuck at the SSH method.
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Marty Godsey
> >>>>>>>> nSource Solutions
> >>>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Abhinandan Prateek
> >>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
> >>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
> >>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
> >>>>>>>>
> >>>>>>>> Cloudrouter looks promising. These have potential to save future
> >>>>>>>> engineering effort for example on ipv6 routing, OSPF etc.
> >>>>>>>> And the best part is they come with test automation framework.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
> >>>>>>>> <ja...@accelerite.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> Instead of replacing the VR in first place we should add
> >>>>>>>>> VyOS/cloudrouter
> >>>>>>>> as provider. Once it is stable, network offerings (on upgrade)
> >>>>>>>> can be updated to use it and we can drop the VR if we want at
> >>>>>>>> that release
> >>>>>>> onwards.
> >>>>>>>>>
> >>>>>>>>> VR is stabilized over a period of time and some of them are
> >>>>>>>>> running
> >>>>>>>> without issues. When we replicate the ACS VR features in new
> >>>>>>>> solution it takes some to find the missing pieces (hidden bugs).
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Jayapal
> >>>>>>>>>
> >>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
> >>>>>>>>>
> >>>>>>>>>> nux@li.nux.ro> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hi,
> >>>>>>>>>>
> >>>>>>>>>> I like the idea.
> >>>>>>>>>>
> >>>>>>>>>> Cloudrouter looks really promising, I'm not too keen on VyOS
> >>>>>>>>>> (it
> >>>>>>>> doesn't have a proper http api etc).
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Sent from the Delta quadrant using Borg technology!
> >>>>>>>>>>
> >>>>>>>>>> Nux!
> >>>>>>>>>> www.nux.ro
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>> abhinandan.prateek@shapeblue.com
> >>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
> >>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> ----- Original Message -----
> >>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
> >>>>>>>>>>> To: dev@cloudstack.apache.org
> >>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
> >>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
> >>>>>>>>>>
> >>>>>>>>>>> *Disclaimer:* This is a thought experiment and should be
> >>>>>>>>>>> treated as
> >>>>>>>> such.
> >>>>>>>>>>> Please weigh in with the good and bad of this idea...
> >>>>>>>>>>>
> >>>>>>>>>>> A couple of us have been discussing the idea of potentially
> >>>>>>>>>>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta
> >>> VM).
> >>>>>>>>>>> There may be a license issue because I think it is licensed
> >>>>>>>>>>> under GPL, but for the sake of discussion, let's assume we
> >>>>>>>>>>> can overcome any
> >>>>>>>> license issues.
> >>>>>>>>>>>
> >>>>>>>>>>> I have spent some time recently with the VyOS and I have to
> >>>>>>>>>>> admit, I was pretty impressed. It is simple and intuitive
> >>>>>>>>>>> and it gives you a lot more options for auditing the
> >>>> configuration etc...
> >>>>>>>>>>>
> >>>>>>>>>>> Items of potential interest:
> >>>>>>>>>>> - Clean up our current VR script spaghetti to a simpler more
> >>>>>>>>>>> auditable configuration workflow.
> >>>>>>>>>>> - Gives a cleaner path for IPv6 support.
> >>>>>>>>>>> - Handles VPN configuration via the same configuration
> >>>> interface.
> >>>>>>>>>>> - Support for OSPF & BGP.
> >>>>>>>>>>> - VPN support through OpenVPN & StrongSwan.
> >>>>>>>>>>> - Easily supports HA (redundant routers) through VRRP.
> >>>>>>>>>>> - VXLAN support.
> >>>>>>>>>>> - Transaction based changes to the VR with rollback on
> >> error.
> >>>>>>>>>>>
> >>>>>>>>>>> Items that could be difficult to solve:
> >>>>>>>>>>> - Userdata password reset workflow and implementation.
> >>>>>>>>>>> - Upgrade process.
> >>>>>>>>>>>
> >>>>>>>>>>> The VyOS is not the only option if we were to consider this
> >>>>>> approach.
> >>>>>>>>>>> Another option, which I don't know as well, would be
> >>>>>>>>>>> CloudRouter (AGPL
> >>>>>>>>>>> license) [2] which is purely API driven.
> >>>>>>>>>>>
> >>>>>>>>>>> Anyway, would love to hear your thoughts...
> >>>>>>>>>>>
> >>>>>>>>>>> Will
> >>>>>>>>>>>
> >>>>>>>>>>> [1] https://vyos.io/
> >>>>>>>>>>> [2] https://cloudrouter.org/
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> DISCLAIMER
> >>>>>>>>> ==========
> >>>>>>>>> This e-mail may contain privileged and confidential information
> >>>>>>>>> which is
> >>>>>>>> the property of Accelerite, a Persistent Systems business. It is
> >>>>>>>> intended only for the use of the individual or entity to which
> >> it
> >>>>>>>> is addressed. If you are not the intended recipient, you are not
> >>>>>>>> authorized to read, retain, copy, print, distribute or use this
> >>>>>>>> message. If you have received this communication in error,
> >> please
> >>>>>>>> notify the sender and delete all copies of this message.
> >>>>>>>> Accelerite, a Persistent Systems business does not accept any
> >>>>>>>> liability for virus
> >>>>>>> infected mails.
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
Re: [DISCUSS] Replacing the VR
Posted by ilya <il...@gmail.com>.
Hi folks, please kindly keep in mind the open source licensing when
choosing the next router. AGPL or GPL v3 are "no go" for many shops.
I could not find the license for Vyatta, i do know Vyatta dev folks were
open to working with ACS few years back, but somehow the initiative was
dropped.
On 9/15/16 9:21 AM, Will Stevens wrote:
> Ya, we would need to add a daemon for VPN as well. Load balancing is
> another aspect which we will need to consider if we went this route.
> Something like https://traefik.io/ could potentially be a good fit due to
> its API driven configuration, but it may be more than what we need.
>
> We should probably try define which pieces make sense to be solved together
> and which pieces would be best suited to be broken out.
>
> I think the network connectivity, routing and firewalling should probably
> all stay together since the majority of the tools we would potentially use
> would handle all of that together in a single implementation.
>
> The password server and userdata seems like a good option for being broken
> out and handled independently (and probably rewritten completely since they
> currently have some issues).
>
> Load balancing is another that could warrant splitting out, but that
> depends on what direction we go and how we would be managing it. DHCP and
> DNS are others which could go either way.
>
> If we do split out services, I think we should consolidate as much as we
> can into each service we break out. Ideally a network packet would never
> hit more than one, maybe two, services. I don't think we should be
> splitting services 'just because', I think we need a valid case for
> splitting any service out because it adds complexity. Our project is
> already complex enough, we need to avoid adding complexity unless it is
> really needed.
>
> Some more of my thoughts on this anyway...
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com> wrote:
>
>> I do agree with you that this probably isn't the right place the password
>> service and user data.
>>
>>
>> Having said that, after taking a cursory look at the dev docs, it doesn't
>> seem that difficult to add new daemons: https://opensnaproute.github.
>> io/docs/developer.html#creating-new-component
>>
>> <https://opensnaproute.github.io/docs/developer.html#
>> creating-new-component>
>>
>>
>> They've definitely build it with a microservices architecture in mind, so
>> each individual feature is abstracted into it's own small daemon process.
>> We could just create a daemon for the password server and the userdata
>> components if we really had to.
>>
>>
>> - Si
>>
>>
>> ________________________________
>> From: williamstevens@gmail.com <wi...@gmail.com> on behalf of
>> Will Stevens <ws...@cloudops.com>
>> Sent: Thursday, September 15, 2016 9:17 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [DISCUSS] Replacing the VR
>>
>> A big part of why I know about it is because it is written in Go. :P
>>
>> Yes, it is definitely interesting for the routing and traffic handling
>> aspects of the VR. We will likely have to rethink some of the pieces a
>> little bit like the password server and userdata if we are to adopt a
>> different VR approach. This is where I think some of JohnB and Chiradeep's
>> ideas make sense. In many ways, it does not make sense for the device
>> handling routing and network traffic to also be responsible for passwords
>> and userdata.
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> w cloudops.com *|* tw @CloudOps_
>>
>> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com> wrote:
>>
>>> I hadn't heard of Flexswitch until you mentioned it. It looks pretty
>> cool!
>>> It even supports ONIE install.
>>>
>>> To be honest, the ipsec feature could be added, or we could offload it to
>>> separate vm if we needed to. The fact it is so feature rich from a
>> routing
>>> perspective (and all API driven) is really nice.
>>>
>>>
>>> Based on the roadmap, it looks like they plan to also support
>> capabilities
>>> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This will be huge
>>> for our carrier community that rely on these technologies to do private
>>> gateway and inter-VPC interconnections today. We handle this stuff on our
>>> ASRs right now with a vlan interconnect into the VR. Being able to do
>> MPLS
>>> all the way to the VR would be awesome.
>>>
>>>
>>> It also seems to be written in GO (a language here at ENA we know very
>>> well).
>>>
>>>
>>> - Si
>>>
>>>
>>>
>>> ________________________________
>>> From: Will Stevens <wi...@gmail.com>
>>> Sent: Thursday, September 15, 2016 7:06 AM
>>> To: dev@cloudstack.apache.org
>>> Subject: RE: [DISCUSS] Replacing the VR
>>>
>>> Ya. I don't think it covers our whole use case, but what it does cover is
>>> all api driven...
>>>
>>> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com> wrote:
>>>
>>>> Though I don\u2019t see VPN in Snaproute.. Makes sense since it was not
>>>> intended to do IPSec.
>>>>
>>>> It seems as though VyOS is starting to look like the best option.
>>>>
>>>> Regards,
>>>> Marty Godsey
>>>> nSource Solutions
>>>>
>>>> -----Original Message-----
>>>> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
>>>> Behalf Of Will Stevens
>>>> Sent: Wednesday, September 14, 2016 11:06 PM
>>>> To: dev@cloudstack.apache.org
>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>
>>>> Or we could go completely crazy and go with something like FlexSwitch
>>> from
>>>> SnapRoute
>>>> - http://www.snaproute.com/
>>>> - https://opensnaproute.github.io/docs/apis.html
>>>>
>>>> *Will STEVENS*
>>>> Lead Developer
>>>>
>>>> *CloudOps* *| *Cloud Solutions Experts
>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
>>>> @CloudOps_
>>>>
>>>> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <ws...@cloudops.com>
>>>> wrote:
>>>>
>>>>> I tend to agree with Syed and Marty. I am not sure what problems are
>>>>> solved by splitting up the function of the VR into a bunch of
>> separate
>>>>> services. As Syed points out, the complexity added is non-trivial.
>>>>> We now have to manage all the intercontainer networking as well as
>> the
>>>>> orchestrated ACS networking.
>>>>>
>>>>> VyOS is interesting to me because it covers the majority of our use
>>>>> case with a single unified control plane. It also has good support
>>>>> for extending features we care about, like IPv6, VXLAN, VRRP,
>>>>> transactions, etc...
>>>>>
>>>>> *Will STEVENS*
>>>>> Lead Developer
>>>>>
>>>>> *CloudOps* *| *Cloud Solutions Experts
>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>> tw
>>>>> @CloudOps_
>>>>>
>>>>> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <sa...@cloudops.com>
>>> wrote:
>>>>>
>>>>>> Agree with Marty, adding Docker containers to the picture although
>>>>>> can make the VR more flexible but the added complexity is just not
>>>>>> worth it. Not to mention we would need to take care of networking
>>>>>> each container manually and given that our iptable rules are very
>>>>>> unstable at the moment I don't see a big value add.
>>>>>>
>>>>>> Vyos looks like a better solution to me. I know that it does not
>>>>>> provide an api but it does fit the bill quite well otherwise. I
>>>>>> specially like the fact that it has a transaction based model and
>> you
>>>>>> can rollback changes if something goes wrong.
>>>>>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <ma...@gonsource.com>
>>>> wrote:
>>>>>>
>>>>>>> Licensing aside, I think splitting the various functions into
>>>>>>> containers is not a good route either. This will force users to
>>>>>>> have to maintain
>>>>>> and
>>>>>>> use containers and adds complexity to the networking aspects of
>> ACS.
>>>>>>> Complexity decreases stability. Now I understand the argument that
>>>>>>> a monolithic approach also brings its own set of issues but it
>> also
>>>>>>> simplifies it.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Marty Godsey
>>>>>>> nSource Solutions
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>>>>>>> Sent: Wednesday, September 14, 2016 5:37 PM
>>>>>>> To: dev@cloudstack.apache.org
>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>
>>>>>>> I rather doubt that the Cloudrouter will fit the needs of the
>>>>>>> CloudStack project
>>>>>>> - it is AGPL licensed. Many enterprises will not touch anything
>>>>>>> that
>>>>>> has
>>>>>>> AGPL
>>>>>>> - the github repo shows rather infrequent updates. Quite likely
>>>>>>> they aren't considering the use cases of the CloudStack community
>>>>>>>
>>>>>>> I'd back John B's comments on disaggregating the VR. Split it into
>>>>>>> many docker containers
>>>>>>> - password server
>>>>>>> - userdata server
>>>>>>> - DHCP / DNS
>>>>>>> - s2s VPN
>>>>>>> - RA VPN
>>>>>>> - intra-VPC routing and ACL
>>>>>>> - Port forwarding + NAT
>>>>>>> - FW
>>>>>>> - LB (public)
>>>>>>> - LB (internal),
>>>>>>> - secondary storage
>>>>>>> - agent
>>>>>>> Glue them together with docker compose files (one per use case -
>>>>>>> basic zone, isolated, VPC, SSVM, etc).
>>>>>>>
>>>>>>> The VR image then becomes a JeOS + docker. You can test each of
>> the
>>>>>>> components independently and fixing one bug in the field (say
>> DHCP)
>>>>>>> is hitless to the other components. You don't need to build
>>>>>>> per-hypervisor VRs. You could even run on baremetal.
>>>>>>>
>>>>>>> Along the way you need to figure out how to
>>>>>>> - make the traffic traverse the containers that are needed to be
>>>>>>> traversed (in most cases just 1)
>>>>>>> - bootstrap the router (how does it find its compose file? where
>>>>>>> is the
>>>>>>> registry?)
>>>>>>> - rethink the command and control of the VR functions. SSH works,
>>>>>>> but something more declarative, idempotent should be explored.
>>>>>>>
>>>>>>> As you do this, it becomes clearer which of the functions can be
>>>>>>> substituted by for example CloudRouter. Command and Control of the
>>>>>> docker
>>>>>>> containers can be moved out to another container. Etc.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>>>>>>> <ma...@gonsource.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> This one does look nice. My biggest concern is the lack of
>>>>>>>> VXLANs. It seems that any of the ones we mentioned do not have
>> an
>>>>>>>> API so we may be stuck at the SSH method.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Marty Godsey
>>>>>>>> nSource Solutions
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Abhinandan Prateek
>>>>>>>> [mailto:abhinandan.prateek@shapeblue.com]
>>>>>>>> Sent: Wednesday, September 14, 2016 2:26 AM
>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>> Subject: Re: [DISCUSS] Replacing the VR
>>>>>>>>
>>>>>>>> Cloudrouter looks promising. These have potential to save future
>>>>>>>> engineering effort for example on ipv6 routing, OSPF etc.
>>>>>>>> And the best part is they come with test automation framework.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 13/09/16, 4:22 PM, "Jayapal Uradi"
>>>>>>>> <ja...@accelerite.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Instead of replacing the VR in first place we should add
>>>>>>>>> VyOS/cloudrouter
>>>>>>>> as provider. Once it is stable, network offerings (on upgrade)
>>>>>>>> can be updated to use it and we can drop the VR if we want at
>>>>>>>> that release
>>>>>>> onwards.
>>>>>>>>>
>>>>>>>>> VR is stabilized over a period of time and some of them are
>>>>>>>>> running
>>>>>>>> without issues. When we replicate the ACS VR features in new
>>>>>>>> solution it takes some to find the missing pieces (hidden bugs).
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Jayapal
>>>>>>>>>
>>>>>>>>>> On Sep 13, 2016, at 2:52 PM, Nux! <
>>>>>>>>>
>>>>>>>>>> nux@li.nux.ro> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I like the idea.
>>>>>>>>>>
>>>>>>>>>> Cloudrouter looks really promising, I'm not too keen on VyOS
>>>>>>>>>> (it
>>>>>>>> doesn't have a proper http api etc).
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Sent from the Delta quadrant using Borg technology!
>>>>>>>>>>
>>>>>>>>>> Nux!
>>>>>>>>>> www.nux.ro
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> abhinandan.prateek@shapeblue.com
>>>>>>>> www.shapeblue.com<http://www.shapeblue.com>
>>>>>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>>>>> From: "Will Stevens" <wi...@gmail.com>
>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>> Sent: Monday, 12 September, 2016 21:20:11
>>>>>>>>>>> Subject: [DISCUSS] Replacing the VR
>>>>>>>>>>
>>>>>>>>>>> *Disclaimer:* This is a thought experiment and should be
>>>>>>>>>>> treated as
>>>>>>>> such.
>>>>>>>>>>> Please weigh in with the good and bad of this idea...
>>>>>>>>>>>
>>>>>>>>>>> A couple of us have been discussing the idea of potentially
>>>>>>>>>>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta
>>> VM).
>>>>>>>>>>> There may be a license issue because I think it is licensed
>>>>>>>>>>> under GPL, but for the sake of discussion, let's assume we
>>>>>>>>>>> can overcome any
>>>>>>>> license issues.
>>>>>>>>>>>
>>>>>>>>>>> I have spent some time recently with the VyOS and I have to
>>>>>>>>>>> admit, I was pretty impressed. It is simple and intuitive
>>>>>>>>>>> and it gives you a lot more options for auditing the
>>>> configuration etc...
>>>>>>>>>>>
>>>>>>>>>>> Items of potential interest:
>>>>>>>>>>> - Clean up our current VR script spaghetti to a simpler more
>>>>>>>>>>> auditable configuration workflow.
>>>>>>>>>>> - Gives a cleaner path for IPv6 support.
>>>>>>>>>>> - Handles VPN configuration via the same configuration
>>>> interface.
>>>>>>>>>>> - Support for OSPF & BGP.
>>>>>>>>>>> - VPN support through OpenVPN & StrongSwan.
>>>>>>>>>>> - Easily supports HA (redundant routers) through VRRP.
>>>>>>>>>>> - VXLAN support.
>>>>>>>>>>> - Transaction based changes to the VR with rollback on
>> error.
>>>>>>>>>>>
>>>>>>>>>>> Items that could be difficult to solve:
>>>>>>>>>>> - Userdata password reset workflow and implementation.
>>>>>>>>>>> - Upgrade process.
>>>>>>>>>>>
>>>>>>>>>>> The VyOS is not the only option if we were to consider this
>>>>>> approach.
>>>>>>>>>>> Another option, which I don't know as well, would be
>>>>>>>>>>> CloudRouter (AGPL
>>>>>>>>>>> license) [2] which is purely API driven.
>>>>>>>>>>>
>>>>>>>>>>> Anyway, would love to hear your thoughts...
>>>>>>>>>>>
>>>>>>>>>>> Will
>>>>>>>>>>>
>>>>>>>>>>> [1] https://vyos.io/
>>>>>>>>>>> [2] https://cloudrouter.org/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> DISCLAIMER
>>>>>>>>> ==========
>>>>>>>>> This e-mail may contain privileged and confidential information
>>>>>>>>> which is
>>>>>>>> the property of Accelerite, a Persistent Systems business. It is
>>>>>>>> intended only for the use of the individual or entity to which
>> it
>>>>>>>> is addressed. If you are not the intended recipient, you are not
>>>>>>>> authorized to read, retain, copy, print, distribute or use this
>>>>>>>> message. If you have received this communication in error,
>> please
>>>>>>>> notify the sender and delete all copies of this message.
>>>>>>>> Accelerite, a Persistent Systems business does not accept any
>>>>>>>> liability for virus
>>>>>>> infected mails.
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
Ya, we would need to add a daemon for VPN as well. Load balancing is
another aspect which we will need to consider if we went this route.
Something like https://traefik.io/ could potentially be a good fit due to
its API driven configuration, but it may be more than what we need.
We should probably try define which pieces make sense to be solved together
and which pieces would be best suited to be broken out.
I think the network connectivity, routing and firewalling should probably
all stay together since the majority of the tools we would potentially use
would handle all of that together in a single implementation.
The password server and userdata seems like a good option for being broken
out and handled independently (and probably rewritten completely since they
currently have some issues).
Load balancing is another that could warrant splitting out, but that
depends on what direction we go and how we would be managing it. DHCP and
DNS are others which could go either way.
If we do split out services, I think we should consolidate as much as we
can into each service we break out. Ideally a network packet would never
hit more than one, maybe two, services. I don't think we should be
splitting services 'just because', I think we need a valid case for
splitting any service out because it adds complexity. Our project is
already complex enough, we need to avoid adding complexity unless it is
really needed.
Some more of my thoughts on this anyway...
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Thu, Sep 15, 2016 at 10:28 AM, Simon Weller <sw...@ena.com> wrote:
> I do agree with you that this probably isn't the right place the password
> service and user data.
>
>
> Having said that, after taking a cursory look at the dev docs, it doesn't
> seem that difficult to add new daemons: https://opensnaproute.github.
> io/docs/developer.html#creating-new-component
>
> <https://opensnaproute.github.io/docs/developer.html#
> creating-new-component>
>
>
> They've definitely build it with a microservices architecture in mind, so
> each individual feature is abstracted into it's own small daemon process.
> We could just create a daemon for the password server and the userdata
> components if we really had to.
>
>
> - Si
>
>
> ________________________________
> From: williamstevens@gmail.com <wi...@gmail.com> on behalf of
> Will Stevens <ws...@cloudops.com>
> Sent: Thursday, September 15, 2016 9:17 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> A big part of why I know about it is because it is written in Go. :P
>
> Yes, it is definitely interesting for the routing and traffic handling
> aspects of the VR. We will likely have to rethink some of the pieces a
> little bit like the password server and userdata if we are to adopt a
> different VR approach. This is where I think some of JohnB and Chiradeep's
> ideas make sense. In many ways, it does not make sense for the device
> handling routing and network traffic to also be responsible for passwords
> and userdata.
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com> wrote:
>
> > I hadn't heard of Flexswitch until you mentioned it. It looks pretty
> cool!
> > It even supports ONIE install.
> >
> > To be honest, the ipsec feature could be added, or we could offload it to
> > separate vm if we needed to. The fact it is so feature rich from a
> routing
> > perspective (and all API driven) is really nice.
> >
> >
> > Based on the roadmap, it looks like they plan to also support
> capabilities
> > such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This will be huge
> > for our carrier community that rely on these technologies to do private
> > gateway and inter-VPC interconnections today. We handle this stuff on our
> > ASRs right now with a vlan interconnect into the VR. Being able to do
> MPLS
> > all the way to the VR would be awesome.
> >
> >
> > It also seems to be written in GO (a language here at ENA we know very
> > well).
> >
> >
> > - Si
> >
> >
> >
> > ________________________________
> > From: Will Stevens <wi...@gmail.com>
> > Sent: Thursday, September 15, 2016 7:06 AM
> > To: dev@cloudstack.apache.org
> > Subject: RE: [DISCUSS] Replacing the VR
> >
> > Ya. I don't think it covers our whole use case, but what it does cover is
> > all api driven...
> >
> > On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com> wrote:
> >
> > > Though I don’t see VPN in Snaproute.. Makes sense since it was not
> > > intended to do IPSec.
> > >
> > > It seems as though VyOS is starting to look like the best option.
> > >
> > > Regards,
> > > Marty Godsey
> > > nSource Solutions
> > >
> > > -----Original Message-----
> > > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> > > Behalf Of Will Stevens
> > > Sent: Wednesday, September 14, 2016 11:06 PM
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: [DISCUSS] Replacing the VR
> > >
> > > Or we could go completely crazy and go with something like FlexSwitch
> > from
> > > SnapRoute
> > > - http://www.snaproute.com/
> > > - https://opensnaproute.github.io/docs/apis.html
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > > @CloudOps_
> > >
> > > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <ws...@cloudops.com>
> > > wrote:
> > >
> > > > I tend to agree with Syed and Marty. I am not sure what problems are
> > > > solved by splitting up the function of the VR into a bunch of
> separate
> > > > services. As Syed points out, the complexity added is non-trivial.
> > > > We now have to manage all the intercontainer networking as well as
> the
> > > > orchestrated ACS networking.
> > > >
> > > > VyOS is interesting to me because it covers the majority of our use
> > > > case with a single unified control plane. It also has good support
> > > > for extending features we care about, like IPv6, VXLAN, VRRP,
> > > > transactions, etc...
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > *CloudOps* *| *Cloud Solutions Experts
> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> tw
> > > > @CloudOps_
> > > >
> > > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <sa...@cloudops.com>
> > wrote:
> > > >
> > > >> Agree with Marty, adding Docker containers to the picture although
> > > >> can make the VR more flexible but the added complexity is just not
> > > >> worth it. Not to mention we would need to take care of networking
> > > >> each container manually and given that our iptable rules are very
> > > >> unstable at the moment I don't see a big value add.
> > > >>
> > > >> Vyos looks like a better solution to me. I know that it does not
> > > >> provide an api but it does fit the bill quite well otherwise. I
> > > >> specially like the fact that it has a transaction based model and
> you
> > > >> can rollback changes if something goes wrong.
> > > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <ma...@gonsource.com>
> > > wrote:
> > > >>
> > > >> > Licensing aside, I think splitting the various functions into
> > > >> > containers is not a good route either. This will force users to
> > > >> > have to maintain
> > > >> and
> > > >> > use containers and adds complexity to the networking aspects of
> ACS.
> > > >> > Complexity decreases stability. Now I understand the argument that
> > > >> > a monolithic approach also brings its own set of issues but it
> also
> > > >> > simplifies it.
> > > >> >
> > > >> > Regards,
> > > >> > Marty Godsey
> > > >> > nSource Solutions
> > > >> >
> > > >> > -----Original Message-----
> > > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> > > >> > Sent: Wednesday, September 14, 2016 5:37 PM
> > > >> > To: dev@cloudstack.apache.org
> > > >> > Subject: Re: [DISCUSS] Replacing the VR
> > > >> >
> > > >> > I rather doubt that the Cloudrouter will fit the needs of the
> > > >> > CloudStack project
> > > >> > - it is AGPL licensed. Many enterprises will not touch anything
> > > >> > that
> > > >> has
> > > >> > AGPL
> > > >> > - the github repo shows rather infrequent updates. Quite likely
> > > >> > they aren't considering the use cases of the CloudStack community
> > > >> >
> > > >> > I'd back John B's comments on disaggregating the VR. Split it into
> > > >> > many docker containers
> > > >> > - password server
> > > >> > - userdata server
> > > >> > - DHCP / DNS
> > > >> > - s2s VPN
> > > >> > - RA VPN
> > > >> > - intra-VPC routing and ACL
> > > >> > - Port forwarding + NAT
> > > >> > - FW
> > > >> > - LB (public)
> > > >> > - LB (internal),
> > > >> > - secondary storage
> > > >> > - agent
> > > >> > Glue them together with docker compose files (one per use case -
> > > >> > basic zone, isolated, VPC, SSVM, etc).
> > > >> >
> > > >> > The VR image then becomes a JeOS + docker. You can test each of
> the
> > > >> > components independently and fixing one bug in the field (say
> DHCP)
> > > >> > is hitless to the other components. You don't need to build
> > > >> > per-hypervisor VRs. You could even run on baremetal.
> > > >> >
> > > >> > Along the way you need to figure out how to
> > > >> > - make the traffic traverse the containers that are needed to be
> > > >> > traversed (in most cases just 1)
> > > >> > - bootstrap the router (how does it find its compose file? where
> > > >> > is the
> > > >> > registry?)
> > > >> > - rethink the command and control of the VR functions. SSH works,
> > > >> > but something more declarative, idempotent should be explored.
> > > >> >
> > > >> > As you do this, it becomes clearer which of the functions can be
> > > >> > substituted by for example CloudRouter. Command and Control of the
> > > >> docker
> > > >> > containers can be moved out to another container. Etc.
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> > > >> > <ma...@gonsource.com>
> > > >> > wrote:
> > > >> >
> > > >> > > This one does look nice. My biggest concern is the lack of
> > > >> > > VXLANs. It seems that any of the ones we mentioned do not have
> an
> > > >> > > API so we may be stuck at the SSH method.
> > > >> > >
> > > >> > > Regards,
> > > >> > > Marty Godsey
> > > >> > > nSource Solutions
> > > >> > >
> > > >> > > -----Original Message-----
> > > >> > > From: Abhinandan Prateek
> > > >> > > [mailto:abhinandan.prateek@shapeblue.com]
> > > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> > > >> > > To: dev@cloudstack.apache.org
> > > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > > >> > >
> > > >> > > Cloudrouter looks promising. These have potential to save future
> > > >> > > engineering effort for example on ipv6 routing, OSPF etc.
> > > >> > > And the best part is they come with test automation framework.
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> > > >> > > <ja...@accelerite.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > >Hi,
> > > >> > > >
> > > >> > > >Instead of replacing the VR in first place we should add
> > > >> > > >VyOS/cloudrouter
> > > >> > > as provider. Once it is stable, network offerings (on upgrade)
> > > >> > > can be updated to use it and we can drop the VR if we want at
> > > >> > > that release
> > > >> > onwards.
> > > >> > > >
> > > >> > > >VR is stabilized over a period of time and some of them are
> > > >> > > >running
> > > >> > > without issues. When we replicate the ACS VR features in new
> > > >> > > solution it takes some to find the missing pieces (hidden bugs).
> > > >> > > >
> > > >> > > >Thanks,
> > > >> > > >Jayapal
> > > >> > > >
> > > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> > > >> > > >
> > > >> > > >> nux@li.nux.ro> wrote:
> > > >> > > >>
> > > >> > > >> Hi,
> > > >> > > >>
> > > >> > > >> I like the idea.
> > > >> > > >>
> > > >> > > >> Cloudrouter looks really promising, I'm not too keen on VyOS
> > > >> > > >> (it
> > > >> > > doesn't have a proper http api etc).
> > > >> > > >>
> > > >> > > >> --
> > > >> > > >> Sent from the Delta quadrant using Borg technology!
> > > >> > > >>
> > > >> > > >> Nux!
> > > >> > > >> www.nux.ro
> > > >> > > >>
> > > >> > > >>
> > > >> > > abhinandan.prateek@shapeblue.com
> > > >> > > www.shapeblue.com<http://www.shapeblue.com>
> > > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > ----- Original Message -----
> > > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> > > >> > > >>> To: dev@cloudstack.apache.org
> > > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> > > >> > > >>> Subject: [DISCUSS] Replacing the VR
> > > >> > > >>
> > > >> > > >>> *Disclaimer:* This is a thought experiment and should be
> > > >> > > >>> treated as
> > > >> > > such.
> > > >> > > >>> Please weigh in with the good and bad of this idea...
> > > >> > > >>>
> > > >> > > >>> A couple of us have been discussing the idea of potentially
> > > >> > > >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta
> > VM).
> > > >> > > >>> There may be a license issue because I think it is licensed
> > > >> > > >>> under GPL, but for the sake of discussion, let's assume we
> > > >> > > >>> can overcome any
> > > >> > > license issues.
> > > >> > > >>>
> > > >> > > >>> I have spent some time recently with the VyOS and I have to
> > > >> > > >>> admit, I was pretty impressed. It is simple and intuitive
> > > >> > > >>> and it gives you a lot more options for auditing the
> > > configuration etc...
> > > >> > > >>>
> > > >> > > >>> Items of potential interest:
> > > >> > > >>> - Clean up our current VR script spaghetti to a simpler more
> > > >> > > >>> auditable configuration workflow.
> > > >> > > >>> - Gives a cleaner path for IPv6 support.
> > > >> > > >>> - Handles VPN configuration via the same configuration
> > > interface.
> > > >> > > >>> - Support for OSPF & BGP.
> > > >> > > >>> - VPN support through OpenVPN & StrongSwan.
> > > >> > > >>> - Easily supports HA (redundant routers) through VRRP.
> > > >> > > >>> - VXLAN support.
> > > >> > > >>> - Transaction based changes to the VR with rollback on
> error.
> > > >> > > >>>
> > > >> > > >>> Items that could be difficult to solve:
> > > >> > > >>> - Userdata password reset workflow and implementation.
> > > >> > > >>> - Upgrade process.
> > > >> > > >>>
> > > >> > > >>> The VyOS is not the only option if we were to consider this
> > > >> approach.
> > > >> > > >>> Another option, which I don't know as well, would be
> > > >> > > >>> CloudRouter (AGPL
> > > >> > > >>> license) [2] which is purely API driven.
> > > >> > > >>>
> > > >> > > >>> Anyway, would love to hear your thoughts...
> > > >> > > >>>
> > > >> > > >>> Will
> > > >> > > >>>
> > > >> > > >>> [1] https://vyos.io/
> > > >> > > >>> [2] https://cloudrouter.org/
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >DISCLAIMER
> > > >> > > >==========
> > > >> > > >This e-mail may contain privileged and confidential information
> > > >> > > >which is
> > > >> > > the property of Accelerite, a Persistent Systems business. It is
> > > >> > > intended only for the use of the individual or entity to which
> it
> > > >> > > is addressed. If you are not the intended recipient, you are not
> > > >> > > authorized to read, retain, copy, print, distribute or use this
> > > >> > > message. If you have received this communication in error,
> please
> > > >> > > notify the sender and delete all copies of this message.
> > > >> > > Accelerite, a Persistent Systems business does not accept any
> > > >> > > liability for virus
> > > >> > infected mails.
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Simon Weller <sw...@ena.com>.
I do agree with you that this probably isn't the right place the password service and user data.
Having said that, after taking a cursory look at the dev docs, it doesn't seem that difficult to add new daemons: https://opensnaproute.github.io/docs/developer.html#creating-new-component
<https://opensnaproute.github.io/docs/developer.html#creating-new-component>
They've definitely build it with a microservices architecture in mind, so each individual feature is abstracted into it's own small daemon process. We could just create a daemon for the password server and the userdata components if we really had to.
- Si
________________________________
From: williamstevens@gmail.com <wi...@gmail.com> on behalf of Will Stevens <ws...@cloudops.com>
Sent: Thursday, September 15, 2016 9:17 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
A big part of why I know about it is because it is written in Go. :P
Yes, it is definitely interesting for the routing and traffic handling
aspects of the VR. We will likely have to rethink some of the pieces a
little bit like the password server and userdata if we are to adopt a
different VR approach. This is where I think some of JohnB and Chiradeep's
ideas make sense. In many ways, it does not make sense for the device
handling routing and network traffic to also be responsible for passwords
and userdata.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com> wrote:
> I hadn't heard of Flexswitch until you mentioned it. It looks pretty cool!
> It even supports ONIE install.
>
> To be honest, the ipsec feature could be added, or we could offload it to
> separate vm if we needed to. The fact it is so feature rich from a routing
> perspective (and all API driven) is really nice.
>
>
> Based on the roadmap, it looks like they plan to also support capabilities
> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This will be huge
> for our carrier community that rely on these technologies to do private
> gateway and inter-VPC interconnections today. We handle this stuff on our
> ASRs right now with a vlan interconnect into the VR. Being able to do MPLS
> all the way to the VR would be awesome.
>
>
> It also seems to be written in GO (a language here at ENA we know very
> well).
>
>
> - Si
>
>
>
> ________________________________
> From: Will Stevens <wi...@gmail.com>
> Sent: Thursday, September 15, 2016 7:06 AM
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] Replacing the VR
>
> Ya. I don't think it covers our whole use case, but what it does cover is
> all api driven...
>
> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com> wrote:
>
> > Though I don’t see VPN in Snaproute.. Makes sense since it was not
> > intended to do IPSec.
> >
> > It seems as though VyOS is starting to look like the best option.
> >
> > Regards,
> > Marty Godsey
> > nSource Solutions
> >
> > -----Original Message-----
> > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> > Behalf Of Will Stevens
> > Sent: Wednesday, September 14, 2016 11:06 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] Replacing the VR
> >
> > Or we could go completely crazy and go with something like FlexSwitch
> from
> > SnapRoute
> > - http://www.snaproute.com/
> > - https://opensnaproute.github.io/docs/apis.html
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > @CloudOps_
> >
> > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <ws...@cloudops.com>
> > wrote:
> >
> > > I tend to agree with Syed and Marty. I am not sure what problems are
> > > solved by splitting up the function of the VR into a bunch of separate
> > > services. As Syed points out, the complexity added is non-trivial.
> > > We now have to manage all the intercontainer networking as well as the
> > > orchestrated ACS networking.
> > >
> > > VyOS is interesting to me because it covers the majority of our use
> > > case with a single unified control plane. It also has good support
> > > for extending features we care about, like IPv6, VXLAN, VRRP,
> > > transactions, etc...
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > > @CloudOps_
> > >
> > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <sa...@cloudops.com>
> wrote:
> > >
> > >> Agree with Marty, adding Docker containers to the picture although
> > >> can make the VR more flexible but the added complexity is just not
> > >> worth it. Not to mention we would need to take care of networking
> > >> each container manually and given that our iptable rules are very
> > >> unstable at the moment I don't see a big value add.
> > >>
> > >> Vyos looks like a better solution to me. I know that it does not
> > >> provide an api but it does fit the bill quite well otherwise. I
> > >> specially like the fact that it has a transaction based model and you
> > >> can rollback changes if something goes wrong.
> > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <ma...@gonsource.com>
> > wrote:
> > >>
> > >> > Licensing aside, I think splitting the various functions into
> > >> > containers is not a good route either. This will force users to
> > >> > have to maintain
> > >> and
> > >> > use containers and adds complexity to the networking aspects of ACS.
> > >> > Complexity decreases stability. Now I understand the argument that
> > >> > a monolithic approach also brings its own set of issues but it also
> > >> > simplifies it.
> > >> >
> > >> > Regards,
> > >> > Marty Godsey
> > >> > nSource Solutions
> > >> >
> > >> > -----Original Message-----
> > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> > >> > Sent: Wednesday, September 14, 2016 5:37 PM
> > >> > To: dev@cloudstack.apache.org
> > >> > Subject: Re: [DISCUSS] Replacing the VR
> > >> >
> > >> > I rather doubt that the Cloudrouter will fit the needs of the
> > >> > CloudStack project
> > >> > - it is AGPL licensed. Many enterprises will not touch anything
> > >> > that
> > >> has
> > >> > AGPL
> > >> > - the github repo shows rather infrequent updates. Quite likely
> > >> > they aren't considering the use cases of the CloudStack community
> > >> >
> > >> > I'd back John B's comments on disaggregating the VR. Split it into
> > >> > many docker containers
> > >> > - password server
> > >> > - userdata server
> > >> > - DHCP / DNS
> > >> > - s2s VPN
> > >> > - RA VPN
> > >> > - intra-VPC routing and ACL
> > >> > - Port forwarding + NAT
> > >> > - FW
> > >> > - LB (public)
> > >> > - LB (internal),
> > >> > - secondary storage
> > >> > - agent
> > >> > Glue them together with docker compose files (one per use case -
> > >> > basic zone, isolated, VPC, SSVM, etc).
> > >> >
> > >> > The VR image then becomes a JeOS + docker. You can test each of the
> > >> > components independently and fixing one bug in the field (say DHCP)
> > >> > is hitless to the other components. You don't need to build
> > >> > per-hypervisor VRs. You could even run on baremetal.
> > >> >
> > >> > Along the way you need to figure out how to
> > >> > - make the traffic traverse the containers that are needed to be
> > >> > traversed (in most cases just 1)
> > >> > - bootstrap the router (how does it find its compose file? where
> > >> > is the
> > >> > registry?)
> > >> > - rethink the command and control of the VR functions. SSH works,
> > >> > but something more declarative, idempotent should be explored.
> > >> >
> > >> > As you do this, it becomes clearer which of the functions can be
> > >> > substituted by for example CloudRouter. Command and Control of the
> > >> docker
> > >> > containers can be moved out to another container. Etc.
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> > >> > <ma...@gonsource.com>
> > >> > wrote:
> > >> >
> > >> > > This one does look nice. My biggest concern is the lack of
> > >> > > VXLANs. It seems that any of the ones we mentioned do not have an
> > >> > > API so we may be stuck at the SSH method.
> > >> > >
> > >> > > Regards,
> > >> > > Marty Godsey
> > >> > > nSource Solutions
> > >> > >
> > >> > > -----Original Message-----
> > >> > > From: Abhinandan Prateek
> > >> > > [mailto:abhinandan.prateek@shapeblue.com]
> > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> > >> > > To: dev@cloudstack.apache.org
> > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > >> > >
> > >> > > Cloudrouter looks promising. These have potential to save future
> > >> > > engineering effort for example on ipv6 routing, OSPF etc.
> > >> > > And the best part is they come with test automation framework.
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> > >> > > <ja...@accelerite.com>
> > >> > > wrote:
> > >> > >
> > >> > > >Hi,
> > >> > > >
> > >> > > >Instead of replacing the VR in first place we should add
> > >> > > >VyOS/cloudrouter
> > >> > > as provider. Once it is stable, network offerings (on upgrade)
> > >> > > can be updated to use it and we can drop the VR if we want at
> > >> > > that release
> > >> > onwards.
> > >> > > >
> > >> > > >VR is stabilized over a period of time and some of them are
> > >> > > >running
> > >> > > without issues. When we replicate the ACS VR features in new
> > >> > > solution it takes some to find the missing pieces (hidden bugs).
> > >> > > >
> > >> > > >Thanks,
> > >> > > >Jayapal
> > >> > > >
> > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> > >> > > >
> > >> > > >> nux@li.nux.ro> wrote:
> > >> > > >>
> > >> > > >> Hi,
> > >> > > >>
> > >> > > >> I like the idea.
> > >> > > >>
> > >> > > >> Cloudrouter looks really promising, I'm not too keen on VyOS
> > >> > > >> (it
> > >> > > doesn't have a proper http api etc).
> > >> > > >>
> > >> > > >> --
> > >> > > >> Sent from the Delta quadrant using Borg technology!
> > >> > > >>
> > >> > > >> Nux!
> > >> > > >> www.nux.ro
> > >> > > >>
> > >> > > >>
> > >> > > abhinandan.prateek@shapeblue.com
> > >> > > www.shapeblue.com<http://www.shapeblue.com>
> > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
> > >> > >
> > >> > >
> > >> > >
> > >> > > ----- Original Message -----
> > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> > >> > > >>> To: dev@cloudstack.apache.org
> > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> > >> > > >>> Subject: [DISCUSS] Replacing the VR
> > >> > > >>
> > >> > > >>> *Disclaimer:* This is a thought experiment and should be
> > >> > > >>> treated as
> > >> > > such.
> > >> > > >>> Please weigh in with the good and bad of this idea...
> > >> > > >>>
> > >> > > >>> A couple of us have been discussing the idea of potentially
> > >> > > >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta
> VM).
> > >> > > >>> There may be a license issue because I think it is licensed
> > >> > > >>> under GPL, but for the sake of discussion, let's assume we
> > >> > > >>> can overcome any
> > >> > > license issues.
> > >> > > >>>
> > >> > > >>> I have spent some time recently with the VyOS and I have to
> > >> > > >>> admit, I was pretty impressed. It is simple and intuitive
> > >> > > >>> and it gives you a lot more options for auditing the
> > configuration etc...
> > >> > > >>>
> > >> > > >>> Items of potential interest:
> > >> > > >>> - Clean up our current VR script spaghetti to a simpler more
> > >> > > >>> auditable configuration workflow.
> > >> > > >>> - Gives a cleaner path for IPv6 support.
> > >> > > >>> - Handles VPN configuration via the same configuration
> > interface.
> > >> > > >>> - Support for OSPF & BGP.
> > >> > > >>> - VPN support through OpenVPN & StrongSwan.
> > >> > > >>> - Easily supports HA (redundant routers) through VRRP.
> > >> > > >>> - VXLAN support.
> > >> > > >>> - Transaction based changes to the VR with rollback on error.
> > >> > > >>>
> > >> > > >>> Items that could be difficult to solve:
> > >> > > >>> - Userdata password reset workflow and implementation.
> > >> > > >>> - Upgrade process.
> > >> > > >>>
> > >> > > >>> The VyOS is not the only option if we were to consider this
> > >> approach.
> > >> > > >>> Another option, which I don't know as well, would be
> > >> > > >>> CloudRouter (AGPL
> > >> > > >>> license) [2] which is purely API driven.
> > >> > > >>>
> > >> > > >>> Anyway, would love to hear your thoughts...
> > >> > > >>>
> > >> > > >>> Will
> > >> > > >>>
> > >> > > >>> [1] https://vyos.io/
> > >> > > >>> [2] https://cloudrouter.org/
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >DISCLAIMER
> > >> > > >==========
> > >> > > >This e-mail may contain privileged and confidential information
> > >> > > >which is
> > >> > > the property of Accelerite, a Persistent Systems business. It is
> > >> > > intended only for the use of the individual or entity to which it
> > >> > > is addressed. If you are not the intended recipient, you are not
> > >> > > authorized to read, retain, copy, print, distribute or use this
> > >> > > message. If you have received this communication in error, please
> > >> > > notify the sender and delete all copies of this message.
> > >> > > Accelerite, a Persistent Systems business does not accept any
> > >> > > liability for virus
> > >> > infected mails.
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
A big part of why I know about it is because it is written in Go. :P
Yes, it is definitely interesting for the routing and traffic handling
aspects of the VR. We will likely have to rethink some of the pieces a
little bit like the password server and userdata if we are to adopt a
different VR approach. This is where I think some of JohnB and Chiradeep's
ideas make sense. In many ways, it does not make sense for the device
handling routing and network traffic to also be responsible for passwords
and userdata.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Thu, Sep 15, 2016 at 9:10 AM, Simon Weller <sw...@ena.com> wrote:
> I hadn't heard of Flexswitch until you mentioned it. It looks pretty cool!
> It even supports ONIE install.
>
> To be honest, the ipsec feature could be added, or we could offload it to
> separate vm if we needed to. The fact it is so feature rich from a routing
> perspective (and all API driven) is really nice.
>
>
> Based on the roadmap, it looks like they plan to also support capabilities
> such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This will be huge
> for our carrier community that rely on these technologies to do private
> gateway and inter-VPC interconnections today. We handle this stuff on our
> ASRs right now with a vlan interconnect into the VR. Being able to do MPLS
> all the way to the VR would be awesome.
>
>
> It also seems to be written in GO (a language here at ENA we know very
> well).
>
>
> - Si
>
>
>
> ________________________________
> From: Will Stevens <wi...@gmail.com>
> Sent: Thursday, September 15, 2016 7:06 AM
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] Replacing the VR
>
> Ya. I don't think it covers our whole use case, but what it does cover is
> all api driven...
>
> On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com> wrote:
>
> > Though I don’t see VPN in Snaproute.. Makes sense since it was not
> > intended to do IPSec.
> >
> > It seems as though VyOS is starting to look like the best option.
> >
> > Regards,
> > Marty Godsey
> > nSource Solutions
> >
> > -----Original Message-----
> > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> > Behalf Of Will Stevens
> > Sent: Wednesday, September 14, 2016 11:06 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] Replacing the VR
> >
> > Or we could go completely crazy and go with something like FlexSwitch
> from
> > SnapRoute
> > - http://www.snaproute.com/
> > - https://opensnaproute.github.io/docs/apis.html
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > @CloudOps_
> >
> > On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <ws...@cloudops.com>
> > wrote:
> >
> > > I tend to agree with Syed and Marty. I am not sure what problems are
> > > solved by splitting up the function of the VR into a bunch of separate
> > > services. As Syed points out, the complexity added is non-trivial.
> > > We now have to manage all the intercontainer networking as well as the
> > > orchestrated ACS networking.
> > >
> > > VyOS is interesting to me because it covers the majority of our use
> > > case with a single unified control plane. It also has good support
> > > for extending features we care about, like IPv6, VXLAN, VRRP,
> > > transactions, etc...
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > *CloudOps* *| *Cloud Solutions Experts
> > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > > @CloudOps_
> > >
> > > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <sa...@cloudops.com>
> wrote:
> > >
> > >> Agree with Marty, adding Docker containers to the picture although
> > >> can make the VR more flexible but the added complexity is just not
> > >> worth it. Not to mention we would need to take care of networking
> > >> each container manually and given that our iptable rules are very
> > >> unstable at the moment I don't see a big value add.
> > >>
> > >> Vyos looks like a better solution to me. I know that it does not
> > >> provide an api but it does fit the bill quite well otherwise. I
> > >> specially like the fact that it has a transaction based model and you
> > >> can rollback changes if something goes wrong.
> > >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <ma...@gonsource.com>
> > wrote:
> > >>
> > >> > Licensing aside, I think splitting the various functions into
> > >> > containers is not a good route either. This will force users to
> > >> > have to maintain
> > >> and
> > >> > use containers and adds complexity to the networking aspects of ACS.
> > >> > Complexity decreases stability. Now I understand the argument that
> > >> > a monolithic approach also brings its own set of issues but it also
> > >> > simplifies it.
> > >> >
> > >> > Regards,
> > >> > Marty Godsey
> > >> > nSource Solutions
> > >> >
> > >> > -----Original Message-----
> > >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> > >> > Sent: Wednesday, September 14, 2016 5:37 PM
> > >> > To: dev@cloudstack.apache.org
> > >> > Subject: Re: [DISCUSS] Replacing the VR
> > >> >
> > >> > I rather doubt that the Cloudrouter will fit the needs of the
> > >> > CloudStack project
> > >> > - it is AGPL licensed. Many enterprises will not touch anything
> > >> > that
> > >> has
> > >> > AGPL
> > >> > - the github repo shows rather infrequent updates. Quite likely
> > >> > they aren't considering the use cases of the CloudStack community
> > >> >
> > >> > I'd back John B's comments on disaggregating the VR. Split it into
> > >> > many docker containers
> > >> > - password server
> > >> > - userdata server
> > >> > - DHCP / DNS
> > >> > - s2s VPN
> > >> > - RA VPN
> > >> > - intra-VPC routing and ACL
> > >> > - Port forwarding + NAT
> > >> > - FW
> > >> > - LB (public)
> > >> > - LB (internal),
> > >> > - secondary storage
> > >> > - agent
> > >> > Glue them together with docker compose files (one per use case -
> > >> > basic zone, isolated, VPC, SSVM, etc).
> > >> >
> > >> > The VR image then becomes a JeOS + docker. You can test each of the
> > >> > components independently and fixing one bug in the field (say DHCP)
> > >> > is hitless to the other components. You don't need to build
> > >> > per-hypervisor VRs. You could even run on baremetal.
> > >> >
> > >> > Along the way you need to figure out how to
> > >> > - make the traffic traverse the containers that are needed to be
> > >> > traversed (in most cases just 1)
> > >> > - bootstrap the router (how does it find its compose file? where
> > >> > is the
> > >> > registry?)
> > >> > - rethink the command and control of the VR functions. SSH works,
> > >> > but something more declarative, idempotent should be explored.
> > >> >
> > >> > As you do this, it becomes clearer which of the functions can be
> > >> > substituted by for example CloudRouter. Command and Control of the
> > >> docker
> > >> > containers can be moved out to another container. Etc.
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> > >> > <ma...@gonsource.com>
> > >> > wrote:
> > >> >
> > >> > > This one does look nice. My biggest concern is the lack of
> > >> > > VXLANs. It seems that any of the ones we mentioned do not have an
> > >> > > API so we may be stuck at the SSH method.
> > >> > >
> > >> > > Regards,
> > >> > > Marty Godsey
> > >> > > nSource Solutions
> > >> > >
> > >> > > -----Original Message-----
> > >> > > From: Abhinandan Prateek
> > >> > > [mailto:abhinandan.prateek@shapeblue.com]
> > >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> > >> > > To: dev@cloudstack.apache.org
> > >> > > Subject: Re: [DISCUSS] Replacing the VR
> > >> > >
> > >> > > Cloudrouter looks promising. These have potential to save future
> > >> > > engineering effort for example on ipv6 routing, OSPF etc.
> > >> > > And the best part is they come with test automation framework.
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> > >> > > <ja...@accelerite.com>
> > >> > > wrote:
> > >> > >
> > >> > > >Hi,
> > >> > > >
> > >> > > >Instead of replacing the VR in first place we should add
> > >> > > >VyOS/cloudrouter
> > >> > > as provider. Once it is stable, network offerings (on upgrade)
> > >> > > can be updated to use it and we can drop the VR if we want at
> > >> > > that release
> > >> > onwards.
> > >> > > >
> > >> > > >VR is stabilized over a period of time and some of them are
> > >> > > >running
> > >> > > without issues. When we replicate the ACS VR features in new
> > >> > > solution it takes some to find the missing pieces (hidden bugs).
> > >> > > >
> > >> > > >Thanks,
> > >> > > >Jayapal
> > >> > > >
> > >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> > >> > > >
> > >> > > >> nux@li.nux.ro> wrote:
> > >> > > >>
> > >> > > >> Hi,
> > >> > > >>
> > >> > > >> I like the idea.
> > >> > > >>
> > >> > > >> Cloudrouter looks really promising, I'm not too keen on VyOS
> > >> > > >> (it
> > >> > > doesn't have a proper http api etc).
> > >> > > >>
> > >> > > >> --
> > >> > > >> Sent from the Delta quadrant using Borg technology!
> > >> > > >>
> > >> > > >> Nux!
> > >> > > >> www.nux.ro
> > >> > > >>
> > >> > > >>
> > >> > > abhinandan.prateek@shapeblue.com
> > >> > > www.shapeblue.com<http://www.shapeblue.com>
> > >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
> > >> > >
> > >> > >
> > >> > >
> > >> > > ----- Original Message -----
> > >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> > >> > > >>> To: dev@cloudstack.apache.org
> > >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> > >> > > >>> Subject: [DISCUSS] Replacing the VR
> > >> > > >>
> > >> > > >>> *Disclaimer:* This is a thought experiment and should be
> > >> > > >>> treated as
> > >> > > such.
> > >> > > >>> Please weigh in with the good and bad of this idea...
> > >> > > >>>
> > >> > > >>> A couple of us have been discussing the idea of potentially
> > >> > > >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta
> VM).
> > >> > > >>> There may be a license issue because I think it is licensed
> > >> > > >>> under GPL, but for the sake of discussion, let's assume we
> > >> > > >>> can overcome any
> > >> > > license issues.
> > >> > > >>>
> > >> > > >>> I have spent some time recently with the VyOS and I have to
> > >> > > >>> admit, I was pretty impressed. It is simple and intuitive
> > >> > > >>> and it gives you a lot more options for auditing the
> > configuration etc...
> > >> > > >>>
> > >> > > >>> Items of potential interest:
> > >> > > >>> - Clean up our current VR script spaghetti to a simpler more
> > >> > > >>> auditable configuration workflow.
> > >> > > >>> - Gives a cleaner path for IPv6 support.
> > >> > > >>> - Handles VPN configuration via the same configuration
> > interface.
> > >> > > >>> - Support for OSPF & BGP.
> > >> > > >>> - VPN support through OpenVPN & StrongSwan.
> > >> > > >>> - Easily supports HA (redundant routers) through VRRP.
> > >> > > >>> - VXLAN support.
> > >> > > >>> - Transaction based changes to the VR with rollback on error.
> > >> > > >>>
> > >> > > >>> Items that could be difficult to solve:
> > >> > > >>> - Userdata password reset workflow and implementation.
> > >> > > >>> - Upgrade process.
> > >> > > >>>
> > >> > > >>> The VyOS is not the only option if we were to consider this
> > >> approach.
> > >> > > >>> Another option, which I don't know as well, would be
> > >> > > >>> CloudRouter (AGPL
> > >> > > >>> license) [2] which is purely API driven.
> > >> > > >>>
> > >> > > >>> Anyway, would love to hear your thoughts...
> > >> > > >>>
> > >> > > >>> Will
> > >> > > >>>
> > >> > > >>> [1] https://vyos.io/
> > >> > > >>> [2] https://cloudrouter.org/
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >DISCLAIMER
> > >> > > >==========
> > >> > > >This e-mail may contain privileged and confidential information
> > >> > > >which is
> > >> > > the property of Accelerite, a Persistent Systems business. It is
> > >> > > intended only for the use of the individual or entity to which it
> > >> > > is addressed. If you are not the intended recipient, you are not
> > >> > > authorized to read, retain, copy, print, distribute or use this
> > >> > > message. If you have received this communication in error, please
> > >> > > notify the sender and delete all copies of this message.
> > >> > > Accelerite, a Persistent Systems business does not accept any
> > >> > > liability for virus
> > >> > infected mails.
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Simon Weller <sw...@ena.com>.
I hadn't heard of Flexswitch until you mentioned it. It looks pretty cool! It even supports ONIE install.
To be honest, the ipsec feature could be added, or we could offload it to separate vm if we needed to. The fact it is so feature rich from a routing perspective (and all API driven) is really nice.
Based on the roadmap, it looks like they plan to also support capabilities such as BGP-MPLS based L3VPN, EVPN, VPLS in the future. This will be huge for our carrier community that rely on these technologies to do private gateway and inter-VPC interconnections today. We handle this stuff on our ASRs right now with a vlan interconnect into the VR. Being able to do MPLS all the way to the VR would be awesome.
It also seems to be written in GO (a language here at ENA we know very well).
- Si
________________________________
From: Will Stevens <wi...@gmail.com>
Sent: Thursday, September 15, 2016 7:06 AM
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] Replacing the VR
Ya. I don't think it covers our whole use case, but what it does cover is
all api driven...
On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com> wrote:
> Though I don’t see VPN in Snaproute.. Makes sense since it was not
> intended to do IPSec.
>
> It seems as though VyOS is starting to look like the best option.
>
> Regards,
> Marty Godsey
> nSource Solutions
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: Wednesday, September 14, 2016 11:06 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Or we could go completely crazy and go with something like FlexSwitch from
> SnapRoute
> - http://www.snaproute.com/
> - https://opensnaproute.github.io/docs/apis.html
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <ws...@cloudops.com>
> wrote:
>
> > I tend to agree with Syed and Marty. I am not sure what problems are
> > solved by splitting up the function of the VR into a bunch of separate
> > services. As Syed points out, the complexity added is non-trivial.
> > We now have to manage all the intercontainer networking as well as the
> > orchestrated ACS networking.
> >
> > VyOS is interesting to me because it covers the majority of our use
> > case with a single unified control plane. It also has good support
> > for extending features we care about, like IPv6, VXLAN, VRRP,
> > transactions, etc...
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > @CloudOps_
> >
> > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <sa...@cloudops.com> wrote:
> >
> >> Agree with Marty, adding Docker containers to the picture although
> >> can make the VR more flexible but the added complexity is just not
> >> worth it. Not to mention we would need to take care of networking
> >> each container manually and given that our iptable rules are very
> >> unstable at the moment I don't see a big value add.
> >>
> >> Vyos looks like a better solution to me. I know that it does not
> >> provide an api but it does fit the bill quite well otherwise. I
> >> specially like the fact that it has a transaction based model and you
> >> can rollback changes if something goes wrong.
> >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <ma...@gonsource.com>
> wrote:
> >>
> >> > Licensing aside, I think splitting the various functions into
> >> > containers is not a good route either. This will force users to
> >> > have to maintain
> >> and
> >> > use containers and adds complexity to the networking aspects of ACS.
> >> > Complexity decreases stability. Now I understand the argument that
> >> > a monolithic approach also brings its own set of issues but it also
> >> > simplifies it.
> >> >
> >> > Regards,
> >> > Marty Godsey
> >> > nSource Solutions
> >> >
> >> > -----Original Message-----
> >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> >> > Sent: Wednesday, September 14, 2016 5:37 PM
> >> > To: dev@cloudstack.apache.org
> >> > Subject: Re: [DISCUSS] Replacing the VR
> >> >
> >> > I rather doubt that the Cloudrouter will fit the needs of the
> >> > CloudStack project
> >> > - it is AGPL licensed. Many enterprises will not touch anything
> >> > that
> >> has
> >> > AGPL
> >> > - the github repo shows rather infrequent updates. Quite likely
> >> > they aren't considering the use cases of the CloudStack community
> >> >
> >> > I'd back John B's comments on disaggregating the VR. Split it into
> >> > many docker containers
> >> > - password server
> >> > - userdata server
> >> > - DHCP / DNS
> >> > - s2s VPN
> >> > - RA VPN
> >> > - intra-VPC routing and ACL
> >> > - Port forwarding + NAT
> >> > - FW
> >> > - LB (public)
> >> > - LB (internal),
> >> > - secondary storage
> >> > - agent
> >> > Glue them together with docker compose files (one per use case -
> >> > basic zone, isolated, VPC, SSVM, etc).
> >> >
> >> > The VR image then becomes a JeOS + docker. You can test each of the
> >> > components independently and fixing one bug in the field (say DHCP)
> >> > is hitless to the other components. You don't need to build
> >> > per-hypervisor VRs. You could even run on baremetal.
> >> >
> >> > Along the way you need to figure out how to
> >> > - make the traffic traverse the containers that are needed to be
> >> > traversed (in most cases just 1)
> >> > - bootstrap the router (how does it find its compose file? where
> >> > is the
> >> > registry?)
> >> > - rethink the command and control of the VR functions. SSH works,
> >> > but something more declarative, idempotent should be explored.
> >> >
> >> > As you do this, it becomes clearer which of the functions can be
> >> > substituted by for example CloudRouter. Command and Control of the
> >> docker
> >> > containers can be moved out to another container. Etc.
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> >> > <ma...@gonsource.com>
> >> > wrote:
> >> >
> >> > > This one does look nice. My biggest concern is the lack of
> >> > > VXLANs. It seems that any of the ones we mentioned do not have an
> >> > > API so we may be stuck at the SSH method.
> >> > >
> >> > > Regards,
> >> > > Marty Godsey
> >> > > nSource Solutions
> >> > >
> >> > > -----Original Message-----
> >> > > From: Abhinandan Prateek
> >> > > [mailto:abhinandan.prateek@shapeblue.com]
> >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> >> > > To: dev@cloudstack.apache.org
> >> > > Subject: Re: [DISCUSS] Replacing the VR
> >> > >
> >> > > Cloudrouter looks promising. These have potential to save future
> >> > > engineering effort for example on ipv6 routing, OSPF etc.
> >> > > And the best part is they come with test automation framework.
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> >> > > <ja...@accelerite.com>
> >> > > wrote:
> >> > >
> >> > > >Hi,
> >> > > >
> >> > > >Instead of replacing the VR in first place we should add
> >> > > >VyOS/cloudrouter
> >> > > as provider. Once it is stable, network offerings (on upgrade)
> >> > > can be updated to use it and we can drop the VR if we want at
> >> > > that release
> >> > onwards.
> >> > > >
> >> > > >VR is stabilized over a period of time and some of them are
> >> > > >running
> >> > > without issues. When we replicate the ACS VR features in new
> >> > > solution it takes some to find the missing pieces (hidden bugs).
> >> > > >
> >> > > >Thanks,
> >> > > >Jayapal
> >> > > >
> >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> >> > > >
> >> > > >> nux@li.nux.ro> wrote:
> >> > > >>
> >> > > >> Hi,
> >> > > >>
> >> > > >> I like the idea.
> >> > > >>
> >> > > >> Cloudrouter looks really promising, I'm not too keen on VyOS
> >> > > >> (it
> >> > > doesn't have a proper http api etc).
> >> > > >>
> >> > > >> --
> >> > > >> Sent from the Delta quadrant using Borg technology!
> >> > > >>
> >> > > >> Nux!
> >> > > >> www.nux.ro
> >> > > >>
> >> > > >>
> >> > > abhinandan.prateek@shapeblue.com
> >> > > www.shapeblue.com<http://www.shapeblue.com>
> >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
> >> > >
> >> > >
> >> > >
> >> > > ----- Original Message -----
> >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> >> > > >>> To: dev@cloudstack.apache.org
> >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> >> > > >>> Subject: [DISCUSS] Replacing the VR
> >> > > >>
> >> > > >>> *Disclaimer:* This is a thought experiment and should be
> >> > > >>> treated as
> >> > > such.
> >> > > >>> Please weigh in with the good and bad of this idea...
> >> > > >>>
> >> > > >>> A couple of us have been discussing the idea of potentially
> >> > > >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
> >> > > >>> There may be a license issue because I think it is licensed
> >> > > >>> under GPL, but for the sake of discussion, let's assume we
> >> > > >>> can overcome any
> >> > > license issues.
> >> > > >>>
> >> > > >>> I have spent some time recently with the VyOS and I have to
> >> > > >>> admit, I was pretty impressed. It is simple and intuitive
> >> > > >>> and it gives you a lot more options for auditing the
> configuration etc...
> >> > > >>>
> >> > > >>> Items of potential interest:
> >> > > >>> - Clean up our current VR script spaghetti to a simpler more
> >> > > >>> auditable configuration workflow.
> >> > > >>> - Gives a cleaner path for IPv6 support.
> >> > > >>> - Handles VPN configuration via the same configuration
> interface.
> >> > > >>> - Support for OSPF & BGP.
> >> > > >>> - VPN support through OpenVPN & StrongSwan.
> >> > > >>> - Easily supports HA (redundant routers) through VRRP.
> >> > > >>> - VXLAN support.
> >> > > >>> - Transaction based changes to the VR with rollback on error.
> >> > > >>>
> >> > > >>> Items that could be difficult to solve:
> >> > > >>> - Userdata password reset workflow and implementation.
> >> > > >>> - Upgrade process.
> >> > > >>>
> >> > > >>> The VyOS is not the only option if we were to consider this
> >> approach.
> >> > > >>> Another option, which I don't know as well, would be
> >> > > >>> CloudRouter (AGPL
> >> > > >>> license) [2] which is purely API driven.
> >> > > >>>
> >> > > >>> Anyway, would love to hear your thoughts...
> >> > > >>>
> >> > > >>> Will
> >> > > >>>
> >> > > >>> [1] https://vyos.io/
> >> > > >>> [2] https://cloudrouter.org/
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >DISCLAIMER
> >> > > >==========
> >> > > >This e-mail may contain privileged and confidential information
> >> > > >which is
> >> > > the property of Accelerite, a Persistent Systems business. It is
> >> > > intended only for the use of the individual or entity to which it
> >> > > is addressed. If you are not the intended recipient, you are not
> >> > > authorized to read, retain, copy, print, distribute or use this
> >> > > message. If you have received this communication in error, please
> >> > > notify the sender and delete all copies of this message.
> >> > > Accelerite, a Persistent Systems business does not accept any
> >> > > liability for virus
> >> > infected mails.
> >> > >
> >> >
> >>
> >
> >
>
RE: [DISCUSS] Replacing the VR
Posted by Will Stevens <wi...@gmail.com>.
Ya. I don't think it covers our whole use case, but what it does cover is
all api driven...
On Sep 15, 2016 1:48 AM, "Marty Godsey" <ma...@gonsource.com> wrote:
> Though I don’t see VPN in Snaproute.. Makes sense since it was not
> intended to do IPSec.
>
> It seems as though VyOS is starting to look like the best option.
>
> Regards,
> Marty Godsey
> nSource Solutions
>
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: Wednesday, September 14, 2016 11:06 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Or we could go completely crazy and go with something like FlexSwitch from
> SnapRoute
> - http://www.snaproute.com/
> - https://opensnaproute.github.io/docs/apis.html
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <ws...@cloudops.com>
> wrote:
>
> > I tend to agree with Syed and Marty. I am not sure what problems are
> > solved by splitting up the function of the VR into a bunch of separate
> > services. As Syed points out, the complexity added is non-trivial.
> > We now have to manage all the intercontainer networking as well as the
> > orchestrated ACS networking.
> >
> > VyOS is interesting to me because it covers the majority of our use
> > case with a single unified control plane. It also has good support
> > for extending features we care about, like IPv6, VXLAN, VRRP,
> > transactions, etc...
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > @CloudOps_
> >
> > On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <sa...@cloudops.com> wrote:
> >
> >> Agree with Marty, adding Docker containers to the picture although
> >> can make the VR more flexible but the added complexity is just not
> >> worth it. Not to mention we would need to take care of networking
> >> each container manually and given that our iptable rules are very
> >> unstable at the moment I don't see a big value add.
> >>
> >> Vyos looks like a better solution to me. I know that it does not
> >> provide an api but it does fit the bill quite well otherwise. I
> >> specially like the fact that it has a transaction based model and you
> >> can rollback changes if something goes wrong.
> >> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <ma...@gonsource.com>
> wrote:
> >>
> >> > Licensing aside, I think splitting the various functions into
> >> > containers is not a good route either. This will force users to
> >> > have to maintain
> >> and
> >> > use containers and adds complexity to the networking aspects of ACS.
> >> > Complexity decreases stability. Now I understand the argument that
> >> > a monolithic approach also brings its own set of issues but it also
> >> > simplifies it.
> >> >
> >> > Regards,
> >> > Marty Godsey
> >> > nSource Solutions
> >> >
> >> > -----Original Message-----
> >> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> >> > Sent: Wednesday, September 14, 2016 5:37 PM
> >> > To: dev@cloudstack.apache.org
> >> > Subject: Re: [DISCUSS] Replacing the VR
> >> >
> >> > I rather doubt that the Cloudrouter will fit the needs of the
> >> > CloudStack project
> >> > - it is AGPL licensed. Many enterprises will not touch anything
> >> > that
> >> has
> >> > AGPL
> >> > - the github repo shows rather infrequent updates. Quite likely
> >> > they aren't considering the use cases of the CloudStack community
> >> >
> >> > I'd back John B's comments on disaggregating the VR. Split it into
> >> > many docker containers
> >> > - password server
> >> > - userdata server
> >> > - DHCP / DNS
> >> > - s2s VPN
> >> > - RA VPN
> >> > - intra-VPC routing and ACL
> >> > - Port forwarding + NAT
> >> > - FW
> >> > - LB (public)
> >> > - LB (internal),
> >> > - secondary storage
> >> > - agent
> >> > Glue them together with docker compose files (one per use case -
> >> > basic zone, isolated, VPC, SSVM, etc).
> >> >
> >> > The VR image then becomes a JeOS + docker. You can test each of the
> >> > components independently and fixing one bug in the field (say DHCP)
> >> > is hitless to the other components. You don't need to build
> >> > per-hypervisor VRs. You could even run on baremetal.
> >> >
> >> > Along the way you need to figure out how to
> >> > - make the traffic traverse the containers that are needed to be
> >> > traversed (in most cases just 1)
> >> > - bootstrap the router (how does it find its compose file? where
> >> > is the
> >> > registry?)
> >> > - rethink the command and control of the VR functions. SSH works,
> >> > but something more declarative, idempotent should be explored.
> >> >
> >> > As you do this, it becomes clearer which of the functions can be
> >> > substituted by for example CloudRouter. Command and Control of the
> >> docker
> >> > containers can be moved out to another container. Etc.
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
> >> > <ma...@gonsource.com>
> >> > wrote:
> >> >
> >> > > This one does look nice. My biggest concern is the lack of
> >> > > VXLANs. It seems that any of the ones we mentioned do not have an
> >> > > API so we may be stuck at the SSH method.
> >> > >
> >> > > Regards,
> >> > > Marty Godsey
> >> > > nSource Solutions
> >> > >
> >> > > -----Original Message-----
> >> > > From: Abhinandan Prateek
> >> > > [mailto:abhinandan.prateek@shapeblue.com]
> >> > > Sent: Wednesday, September 14, 2016 2:26 AM
> >> > > To: dev@cloudstack.apache.org
> >> > > Subject: Re: [DISCUSS] Replacing the VR
> >> > >
> >> > > Cloudrouter looks promising. These have potential to save future
> >> > > engineering effort for example on ipv6 routing, OSPF etc.
> >> > > And the best part is they come with test automation framework.
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
> >> > > <ja...@accelerite.com>
> >> > > wrote:
> >> > >
> >> > > >Hi,
> >> > > >
> >> > > >Instead of replacing the VR in first place we should add
> >> > > >VyOS/cloudrouter
> >> > > as provider. Once it is stable, network offerings (on upgrade)
> >> > > can be updated to use it and we can drop the VR if we want at
> >> > > that release
> >> > onwards.
> >> > > >
> >> > > >VR is stabilized over a period of time and some of them are
> >> > > >running
> >> > > without issues. When we replicate the ACS VR features in new
> >> > > solution it takes some to find the missing pieces (hidden bugs).
> >> > > >
> >> > > >Thanks,
> >> > > >Jayapal
> >> > > >
> >> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> >> > > >
> >> > > >> nux@li.nux.ro> wrote:
> >> > > >>
> >> > > >> Hi,
> >> > > >>
> >> > > >> I like the idea.
> >> > > >>
> >> > > >> Cloudrouter looks really promising, I'm not too keen on VyOS
> >> > > >> (it
> >> > > doesn't have a proper http api etc).
> >> > > >>
> >> > > >> --
> >> > > >> Sent from the Delta quadrant using Borg technology!
> >> > > >>
> >> > > >> Nux!
> >> > > >> www.nux.ro
> >> > > >>
> >> > > >>
> >> > > abhinandan.prateek@shapeblue.com
> >> > > www.shapeblue.com
> >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
> >> > >
> >> > >
> >> > >
> >> > > ----- Original Message -----
> >> > > >>> From: "Will Stevens" <wi...@gmail.com>
> >> > > >>> To: dev@cloudstack.apache.org
> >> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> >> > > >>> Subject: [DISCUSS] Replacing the VR
> >> > > >>
> >> > > >>> *Disclaimer:* This is a thought experiment and should be
> >> > > >>> treated as
> >> > > such.
> >> > > >>> Please weigh in with the good and bad of this idea...
> >> > > >>>
> >> > > >>> A couple of us have been discussing the idea of potentially
> >> > > >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
> >> > > >>> There may be a license issue because I think it is licensed
> >> > > >>> under GPL, but for the sake of discussion, let's assume we
> >> > > >>> can overcome any
> >> > > license issues.
> >> > > >>>
> >> > > >>> I have spent some time recently with the VyOS and I have to
> >> > > >>> admit, I was pretty impressed. It is simple and intuitive
> >> > > >>> and it gives you a lot more options for auditing the
> configuration etc...
> >> > > >>>
> >> > > >>> Items of potential interest:
> >> > > >>> - Clean up our current VR script spaghetti to a simpler more
> >> > > >>> auditable configuration workflow.
> >> > > >>> - Gives a cleaner path for IPv6 support.
> >> > > >>> - Handles VPN configuration via the same configuration
> interface.
> >> > > >>> - Support for OSPF & BGP.
> >> > > >>> - VPN support through OpenVPN & StrongSwan.
> >> > > >>> - Easily supports HA (redundant routers) through VRRP.
> >> > > >>> - VXLAN support.
> >> > > >>> - Transaction based changes to the VR with rollback on error.
> >> > > >>>
> >> > > >>> Items that could be difficult to solve:
> >> > > >>> - Userdata password reset workflow and implementation.
> >> > > >>> - Upgrade process.
> >> > > >>>
> >> > > >>> The VyOS is not the only option if we were to consider this
> >> approach.
> >> > > >>> Another option, which I don't know as well, would be
> >> > > >>> CloudRouter (AGPL
> >> > > >>> license) [2] which is purely API driven.
> >> > > >>>
> >> > > >>> Anyway, would love to hear your thoughts...
> >> > > >>>
> >> > > >>> Will
> >> > > >>>
> >> > > >>> [1] https://vyos.io/
> >> > > >>> [2] https://cloudrouter.org/
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >DISCLAIMER
> >> > > >==========
> >> > > >This e-mail may contain privileged and confidential information
> >> > > >which is
> >> > > the property of Accelerite, a Persistent Systems business. It is
> >> > > intended only for the use of the individual or entity to which it
> >> > > is addressed. If you are not the intended recipient, you are not
> >> > > authorized to read, retain, copy, print, distribute or use this
> >> > > message. If you have received this communication in error, please
> >> > > notify the sender and delete all copies of this message.
> >> > > Accelerite, a Persistent Systems business does not accept any
> >> > > liability for virus
> >> > infected mails.
> >> > >
> >> >
> >>
> >
> >
>
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
Though I don’t see VPN in Snaproute.. Makes sense since it was not intended to do IPSec.
It seems as though VyOS is starting to look like the best option.
Regards,
Marty Godsey
nSource Solutions
-----Original Message-----
From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
Sent: Wednesday, September 14, 2016 11:06 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
Or we could go completely crazy and go with something like FlexSwitch from SnapRoute
- http://www.snaproute.com/
- https://opensnaproute.github.io/docs/apis.html
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_
On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <ws...@cloudops.com>
wrote:
> I tend to agree with Syed and Marty. I am not sure what problems are
> solved by splitting up the function of the VR into a bunch of separate
> services. As Syed points out, the complexity added is non-trivial.
> We now have to manage all the intercontainer networking as well as the
> orchestrated ACS networking.
>
> VyOS is interesting to me because it covers the majority of our use
> case with a single unified control plane. It also has good support
> for extending features we care about, like IPv6, VXLAN, VRRP,
> transactions, etc...
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <sa...@cloudops.com> wrote:
>
>> Agree with Marty, adding Docker containers to the picture although
>> can make the VR more flexible but the added complexity is just not
>> worth it. Not to mention we would need to take care of networking
>> each container manually and given that our iptable rules are very
>> unstable at the moment I don't see a big value add.
>>
>> Vyos looks like a better solution to me. I know that it does not
>> provide an api but it does fit the bill quite well otherwise. I
>> specially like the fact that it has a transaction based model and you
>> can rollback changes if something goes wrong.
>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <ma...@gonsource.com> wrote:
>>
>> > Licensing aside, I think splitting the various functions into
>> > containers is not a good route either. This will force users to
>> > have to maintain
>> and
>> > use containers and adds complexity to the networking aspects of ACS.
>> > Complexity decreases stability. Now I understand the argument that
>> > a monolithic approach also brings its own set of issues but it also
>> > simplifies it.
>> >
>> > Regards,
>> > Marty Godsey
>> > nSource Solutions
>> >
>> > -----Original Message-----
>> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>> > Sent: Wednesday, September 14, 2016 5:37 PM
>> > To: dev@cloudstack.apache.org
>> > Subject: Re: [DISCUSS] Replacing the VR
>> >
>> > I rather doubt that the Cloudrouter will fit the needs of the
>> > CloudStack project
>> > - it is AGPL licensed. Many enterprises will not touch anything
>> > that
>> has
>> > AGPL
>> > - the github repo shows rather infrequent updates. Quite likely
>> > they aren't considering the use cases of the CloudStack community
>> >
>> > I'd back John B's comments on disaggregating the VR. Split it into
>> > many docker containers
>> > - password server
>> > - userdata server
>> > - DHCP / DNS
>> > - s2s VPN
>> > - RA VPN
>> > - intra-VPC routing and ACL
>> > - Port forwarding + NAT
>> > - FW
>> > - LB (public)
>> > - LB (internal),
>> > - secondary storage
>> > - agent
>> > Glue them together with docker compose files (one per use case -
>> > basic zone, isolated, VPC, SSVM, etc).
>> >
>> > The VR image then becomes a JeOS + docker. You can test each of the
>> > components independently and fixing one bug in the field (say DHCP)
>> > is hitless to the other components. You don't need to build
>> > per-hypervisor VRs. You could even run on baremetal.
>> >
>> > Along the way you need to figure out how to
>> > - make the traffic traverse the containers that are needed to be
>> > traversed (in most cases just 1)
>> > - bootstrap the router (how does it find its compose file? where
>> > is the
>> > registry?)
>> > - rethink the command and control of the VR functions. SSH works,
>> > but something more declarative, idempotent should be explored.
>> >
>> > As you do this, it becomes clearer which of the functions can be
>> > substituted by for example CloudRouter. Command and Control of the
>> docker
>> > containers can be moved out to another container. Etc.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>> > <ma...@gonsource.com>
>> > wrote:
>> >
>> > > This one does look nice. My biggest concern is the lack of
>> > > VXLANs. It seems that any of the ones we mentioned do not have an
>> > > API so we may be stuck at the SSH method.
>> > >
>> > > Regards,
>> > > Marty Godsey
>> > > nSource Solutions
>> > >
>> > > -----Original Message-----
>> > > From: Abhinandan Prateek
>> > > [mailto:abhinandan.prateek@shapeblue.com]
>> > > Sent: Wednesday, September 14, 2016 2:26 AM
>> > > To: dev@cloudstack.apache.org
>> > > Subject: Re: [DISCUSS] Replacing the VR
>> > >
>> > > Cloudrouter looks promising. These have potential to save future
>> > > engineering effort for example on ipv6 routing, OSPF etc.
>> > > And the best part is they come with test automation framework.
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
>> > > <ja...@accelerite.com>
>> > > wrote:
>> > >
>> > > >Hi,
>> > > >
>> > > >Instead of replacing the VR in first place we should add
>> > > >VyOS/cloudrouter
>> > > as provider. Once it is stable, network offerings (on upgrade)
>> > > can be updated to use it and we can drop the VR if we want at
>> > > that release
>> > onwards.
>> > > >
>> > > >VR is stabilized over a period of time and some of them are
>> > > >running
>> > > without issues. When we replicate the ACS VR features in new
>> > > solution it takes some to find the missing pieces (hidden bugs).
>> > > >
>> > > >Thanks,
>> > > >Jayapal
>> > > >
>> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
>> > > >
>> > > >> nux@li.nux.ro> wrote:
>> > > >>
>> > > >> Hi,
>> > > >>
>> > > >> I like the idea.
>> > > >>
>> > > >> Cloudrouter looks really promising, I'm not too keen on VyOS
>> > > >> (it
>> > > doesn't have a proper http api etc).
>> > > >>
>> > > >> --
>> > > >> Sent from the Delta quadrant using Borg technology!
>> > > >>
>> > > >> Nux!
>> > > >> www.nux.ro
>> > > >>
>> > > >>
>> > > abhinandan.prateek@shapeblue.com
>> > > www.shapeblue.com
>> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
>> > >
>> > >
>> > >
>> > > ----- Original Message -----
>> > > >>> From: "Will Stevens" <wi...@gmail.com>
>> > > >>> To: dev@cloudstack.apache.org
>> > > >>> Sent: Monday, 12 September, 2016 21:20:11
>> > > >>> Subject: [DISCUSS] Replacing the VR
>> > > >>
>> > > >>> *Disclaimer:* This is a thought experiment and should be
>> > > >>> treated as
>> > > such.
>> > > >>> Please weigh in with the good and bad of this idea...
>> > > >>>
>> > > >>> A couple of us have been discussing the idea of potentially
>> > > >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
>> > > >>> There may be a license issue because I think it is licensed
>> > > >>> under GPL, but for the sake of discussion, let's assume we
>> > > >>> can overcome any
>> > > license issues.
>> > > >>>
>> > > >>> I have spent some time recently with the VyOS and I have to
>> > > >>> admit, I was pretty impressed. It is simple and intuitive
>> > > >>> and it gives you a lot more options for auditing the configuration etc...
>> > > >>>
>> > > >>> Items of potential interest:
>> > > >>> - Clean up our current VR script spaghetti to a simpler more
>> > > >>> auditable configuration workflow.
>> > > >>> - Gives a cleaner path for IPv6 support.
>> > > >>> - Handles VPN configuration via the same configuration interface.
>> > > >>> - Support for OSPF & BGP.
>> > > >>> - VPN support through OpenVPN & StrongSwan.
>> > > >>> - Easily supports HA (redundant routers) through VRRP.
>> > > >>> - VXLAN support.
>> > > >>> - Transaction based changes to the VR with rollback on error.
>> > > >>>
>> > > >>> Items that could be difficult to solve:
>> > > >>> - Userdata password reset workflow and implementation.
>> > > >>> - Upgrade process.
>> > > >>>
>> > > >>> The VyOS is not the only option if we were to consider this
>> approach.
>> > > >>> Another option, which I don't know as well, would be
>> > > >>> CloudRouter (AGPL
>> > > >>> license) [2] which is purely API driven.
>> > > >>>
>> > > >>> Anyway, would love to hear your thoughts...
>> > > >>>
>> > > >>> Will
>> > > >>>
>> > > >>> [1] https://vyos.io/
>> > > >>> [2] https://cloudrouter.org/
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >DISCLAIMER
>> > > >==========
>> > > >This e-mail may contain privileged and confidential information
>> > > >which is
>> > > the property of Accelerite, a Persistent Systems business. It is
>> > > intended only for the use of the individual or entity to which it
>> > > is addressed. If you are not the intended recipient, you are not
>> > > authorized to read, retain, copy, print, distribute or use this
>> > > message. If you have received this communication in error, please
>> > > notify the sender and delete all copies of this message.
>> > > Accelerite, a Persistent Systems business does not accept any
>> > > liability for virus
>> > infected mails.
>> > >
>> >
>>
>
>
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
Honestly, this looks interesting. I have never used it myself but it "reads" very much like what we are wanting to do.
Regards,
Marty Godsey
nSource Solutions
-----Original Message-----
From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
Sent: Wednesday, September 14, 2016 11:06 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
Or we could go completely crazy and go with something like FlexSwitch from SnapRoute
- http://www.snaproute.com/
- https://opensnaproute.github.io/docs/apis.html
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_
On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <ws...@cloudops.com>
wrote:
> I tend to agree with Syed and Marty. I am not sure what problems are
> solved by splitting up the function of the VR into a bunch of separate
> services. As Syed points out, the complexity added is non-trivial.
> We now have to manage all the intercontainer networking as well as the
> orchestrated ACS networking.
>
> VyOS is interesting to me because it covers the majority of our use
> case with a single unified control plane. It also has good support
> for extending features we care about, like IPv6, VXLAN, VRRP,
> transactions, etc...
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <sa...@cloudops.com> wrote:
>
>> Agree with Marty, adding Docker containers to the picture although
>> can make the VR more flexible but the added complexity is just not
>> worth it. Not to mention we would need to take care of networking
>> each container manually and given that our iptable rules are very
>> unstable at the moment I don't see a big value add.
>>
>> Vyos looks like a better solution to me. I know that it does not
>> provide an api but it does fit the bill quite well otherwise. I
>> specially like the fact that it has a transaction based model and you
>> can rollback changes if something goes wrong.
>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <ma...@gonsource.com> wrote:
>>
>> > Licensing aside, I think splitting the various functions into
>> > containers is not a good route either. This will force users to
>> > have to maintain
>> and
>> > use containers and adds complexity to the networking aspects of ACS.
>> > Complexity decreases stability. Now I understand the argument that
>> > a monolithic approach also brings its own set of issues but it also
>> > simplifies it.
>> >
>> > Regards,
>> > Marty Godsey
>> > nSource Solutions
>> >
>> > -----Original Message-----
>> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>> > Sent: Wednesday, September 14, 2016 5:37 PM
>> > To: dev@cloudstack.apache.org
>> > Subject: Re: [DISCUSS] Replacing the VR
>> >
>> > I rather doubt that the Cloudrouter will fit the needs of the
>> > CloudStack project
>> > - it is AGPL licensed. Many enterprises will not touch anything
>> > that
>> has
>> > AGPL
>> > - the github repo shows rather infrequent updates. Quite likely
>> > they aren't considering the use cases of the CloudStack community
>> >
>> > I'd back John B's comments on disaggregating the VR. Split it into
>> > many docker containers
>> > - password server
>> > - userdata server
>> > - DHCP / DNS
>> > - s2s VPN
>> > - RA VPN
>> > - intra-VPC routing and ACL
>> > - Port forwarding + NAT
>> > - FW
>> > - LB (public)
>> > - LB (internal),
>> > - secondary storage
>> > - agent
>> > Glue them together with docker compose files (one per use case -
>> > basic zone, isolated, VPC, SSVM, etc).
>> >
>> > The VR image then becomes a JeOS + docker. You can test each of the
>> > components independently and fixing one bug in the field (say DHCP)
>> > is hitless to the other components. You don't need to build
>> > per-hypervisor VRs. You could even run on baremetal.
>> >
>> > Along the way you need to figure out how to
>> > - make the traffic traverse the containers that are needed to be
>> > traversed (in most cases just 1)
>> > - bootstrap the router (how does it find its compose file? where
>> > is the
>> > registry?)
>> > - rethink the command and control of the VR functions. SSH works,
>> > but something more declarative, idempotent should be explored.
>> >
>> > As you do this, it becomes clearer which of the functions can be
>> > substituted by for example CloudRouter. Command and Control of the
>> docker
>> > containers can be moved out to another container. Etc.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey
>> > <ma...@gonsource.com>
>> > wrote:
>> >
>> > > This one does look nice. My biggest concern is the lack of
>> > > VXLANs. It seems that any of the ones we mentioned do not have an
>> > > API so we may be stuck at the SSH method.
>> > >
>> > > Regards,
>> > > Marty Godsey
>> > > nSource Solutions
>> > >
>> > > -----Original Message-----
>> > > From: Abhinandan Prateek
>> > > [mailto:abhinandan.prateek@shapeblue.com]
>> > > Sent: Wednesday, September 14, 2016 2:26 AM
>> > > To: dev@cloudstack.apache.org
>> > > Subject: Re: [DISCUSS] Replacing the VR
>> > >
>> > > Cloudrouter looks promising. These have potential to save future
>> > > engineering effort for example on ipv6 routing, OSPF etc.
>> > > And the best part is they come with test automation framework.
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On 13/09/16, 4:22 PM, "Jayapal Uradi"
>> > > <ja...@accelerite.com>
>> > > wrote:
>> > >
>> > > >Hi,
>> > > >
>> > > >Instead of replacing the VR in first place we should add
>> > > >VyOS/cloudrouter
>> > > as provider. Once it is stable, network offerings (on upgrade)
>> > > can be updated to use it and we can drop the VR if we want at
>> > > that release
>> > onwards.
>> > > >
>> > > >VR is stabilized over a period of time and some of them are
>> > > >running
>> > > without issues. When we replicate the ACS VR features in new
>> > > solution it takes some to find the missing pieces (hidden bugs).
>> > > >
>> > > >Thanks,
>> > > >Jayapal
>> > > >
>> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
>> > > >
>> > > >> nux@li.nux.ro> wrote:
>> > > >>
>> > > >> Hi,
>> > > >>
>> > > >> I like the idea.
>> > > >>
>> > > >> Cloudrouter looks really promising, I'm not too keen on VyOS
>> > > >> (it
>> > > doesn't have a proper http api etc).
>> > > >>
>> > > >> --
>> > > >> Sent from the Delta quadrant using Borg technology!
>> > > >>
>> > > >> Nux!
>> > > >> www.nux.ro
>> > > >>
>> > > >>
>> > > abhinandan.prateek@shapeblue.com
>> > > www.shapeblue.com
>> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
>> > >
>> > >
>> > >
>> > > ----- Original Message -----
>> > > >>> From: "Will Stevens" <wi...@gmail.com>
>> > > >>> To: dev@cloudstack.apache.org
>> > > >>> Sent: Monday, 12 September, 2016 21:20:11
>> > > >>> Subject: [DISCUSS] Replacing the VR
>> > > >>
>> > > >>> *Disclaimer:* This is a thought experiment and should be
>> > > >>> treated as
>> > > such.
>> > > >>> Please weigh in with the good and bad of this idea...
>> > > >>>
>> > > >>> A couple of us have been discussing the idea of potentially
>> > > >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
>> > > >>> There may be a license issue because I think it is licensed
>> > > >>> under GPL, but for the sake of discussion, let's assume we
>> > > >>> can overcome any
>> > > license issues.
>> > > >>>
>> > > >>> I have spent some time recently with the VyOS and I have to
>> > > >>> admit, I was pretty impressed. It is simple and intuitive
>> > > >>> and it gives you a lot more options for auditing the configuration etc...
>> > > >>>
>> > > >>> Items of potential interest:
>> > > >>> - Clean up our current VR script spaghetti to a simpler more
>> > > >>> auditable configuration workflow.
>> > > >>> - Gives a cleaner path for IPv6 support.
>> > > >>> - Handles VPN configuration via the same configuration interface.
>> > > >>> - Support for OSPF & BGP.
>> > > >>> - VPN support through OpenVPN & StrongSwan.
>> > > >>> - Easily supports HA (redundant routers) through VRRP.
>> > > >>> - VXLAN support.
>> > > >>> - Transaction based changes to the VR with rollback on error.
>> > > >>>
>> > > >>> Items that could be difficult to solve:
>> > > >>> - Userdata password reset workflow and implementation.
>> > > >>> - Upgrade process.
>> > > >>>
>> > > >>> The VyOS is not the only option if we were to consider this
>> approach.
>> > > >>> Another option, which I don't know as well, would be
>> > > >>> CloudRouter (AGPL
>> > > >>> license) [2] which is purely API driven.
>> > > >>>
>> > > >>> Anyway, would love to hear your thoughts...
>> > > >>>
>> > > >>> Will
>> > > >>>
>> > > >>> [1] https://vyos.io/
>> > > >>> [2] https://cloudrouter.org/
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >DISCLAIMER
>> > > >==========
>> > > >This e-mail may contain privileged and confidential information
>> > > >which is
>> > > the property of Accelerite, a Persistent Systems business. It is
>> > > intended only for the use of the individual or entity to which it
>> > > is addressed. If you are not the intended recipient, you are not
>> > > authorized to read, retain, copy, print, distribute or use this
>> > > message. If you have received this communication in error, please
>> > > notify the sender and delete all copies of this message.
>> > > Accelerite, a Persistent Systems business does not accept any
>> > > liability for virus
>> > infected mails.
>> > >
>> >
>>
>
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
Or we could go completely crazy and go with something like FlexSwitch from
SnapRoute
- http://www.snaproute.com/
- https://opensnaproute.github.io/docs/apis.html
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Wed, Sep 14, 2016 at 10:55 PM, Will Stevens <ws...@cloudops.com>
wrote:
> I tend to agree with Syed and Marty. I am not sure what problems are
> solved by splitting up the function of the VR into a bunch of separate
> services. As Syed points out, the complexity added is non-trivial. We now
> have to manage all the intercontainer networking as well as the
> orchestrated ACS networking.
>
> VyOS is interesting to me because it covers the majority of our use case
> with a single unified control plane. It also has good support for
> extending features we care about, like IPv6, VXLAN, VRRP, transactions,
> etc...
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <sa...@cloudops.com> wrote:
>
>> Agree with Marty, adding Docker containers to the picture although can
>> make
>> the VR more flexible but the added complexity is just not worth it. Not to
>> mention we would need to take care of networking each container manually
>> and given that our iptable rules are very unstable at the moment I don't
>> see a big value add.
>>
>> Vyos looks like a better solution to me. I know that it does not provide
>> an
>> api but it does fit the bill quite well otherwise. I specially like the
>> fact that it has a transaction based model and you can rollback changes if
>> something goes wrong.
>> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <ma...@gonsource.com> wrote:
>>
>> > Licensing aside, I think splitting the various functions into containers
>> > is not a good route either. This will force users to have to maintain
>> and
>> > use containers and adds complexity to the networking aspects of ACS.
>> > Complexity decreases stability. Now I understand the argument that a
>> > monolithic approach also brings its own set of issues but it also
>> > simplifies it.
>> >
>> > Regards,
>> > Marty Godsey
>> > nSource Solutions
>> >
>> > -----Original Message-----
>> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
>> > Sent: Wednesday, September 14, 2016 5:37 PM
>> > To: dev@cloudstack.apache.org
>> > Subject: Re: [DISCUSS] Replacing the VR
>> >
>> > I rather doubt that the Cloudrouter will fit the needs of the CloudStack
>> > project
>> > - it is AGPL licensed. Many enterprises will not touch anything that
>> has
>> > AGPL
>> > - the github repo shows rather infrequent updates. Quite likely they
>> > aren't considering the use cases of the CloudStack community
>> >
>> > I'd back John B's comments on disaggregating the VR. Split it into many
>> > docker containers
>> > - password server
>> > - userdata server
>> > - DHCP / DNS
>> > - s2s VPN
>> > - RA VPN
>> > - intra-VPC routing and ACL
>> > - Port forwarding + NAT
>> > - FW
>> > - LB (public)
>> > - LB (internal),
>> > - secondary storage
>> > - agent
>> > Glue them together with docker compose files (one per use case - basic
>> > zone, isolated, VPC, SSVM, etc).
>> >
>> > The VR image then becomes a JeOS + docker. You can test each of the
>> > components independently and fixing one bug in the field (say DHCP) is
>> > hitless to the other components. You don't need to build per-hypervisor
>> > VRs. You could even run on baremetal.
>> >
>> > Along the way you need to figure out how to
>> > - make the traffic traverse the containers that are needed to be
>> > traversed (in most cases just 1)
>> > - bootstrap the router (how does it find its compose file? where is the
>> > registry?)
>> > - rethink the command and control of the VR functions. SSH works, but
>> > something more declarative, idempotent should be explored.
>> >
>> > As you do this, it becomes clearer which of the functions can be
>> > substituted by for example CloudRouter. Command and Control of the
>> docker
>> > containers can be moved out to another container. Etc.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey <ma...@gonsource.com>
>> > wrote:
>> >
>> > > This one does look nice. My biggest concern is the lack of VXLANs. It
>> > > seems that any of the ones we mentioned do not have an API so we may
>> > > be stuck at the SSH method.
>> > >
>> > > Regards,
>> > > Marty Godsey
>> > > nSource Solutions
>> > >
>> > > -----Original Message-----
>> > > From: Abhinandan Prateek [mailto:abhinandan.prateek@shapeblue.com]
>> > > Sent: Wednesday, September 14, 2016 2:26 AM
>> > > To: dev@cloudstack.apache.org
>> > > Subject: Re: [DISCUSS] Replacing the VR
>> > >
>> > > Cloudrouter looks promising. These have potential to save future
>> > > engineering effort for example on ipv6 routing, OSPF etc.
>> > > And the best part is they come with test automation framework.
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On 13/09/16, 4:22 PM, "Jayapal Uradi" <ja...@accelerite.com>
>> > > wrote:
>> > >
>> > > >Hi,
>> > > >
>> > > >Instead of replacing the VR in first place we should add
>> > > >VyOS/cloudrouter
>> > > as provider. Once it is stable, network offerings (on upgrade) can be
>> > > updated to use it and we can drop the VR if we want at that release
>> > onwards.
>> > > >
>> > > >VR is stabilized over a period of time and some of them are running
>> > > without issues. When we replicate the ACS VR features in new solution
>> > > it takes some to find the missing pieces (hidden bugs).
>> > > >
>> > > >Thanks,
>> > > >Jayapal
>> > > >
>> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
>> > > >
>> > > >> nux@li.nux.ro> wrote:
>> > > >>
>> > > >> Hi,
>> > > >>
>> > > >> I like the idea.
>> > > >>
>> > > >> Cloudrouter looks really promising, I'm not too keen on VyOS (it
>> > > doesn't have a proper http api etc).
>> > > >>
>> > > >> --
>> > > >> Sent from the Delta quadrant using Borg technology!
>> > > >>
>> > > >> Nux!
>> > > >> www.nux.ro
>> > > >>
>> > > >>
>> > > abhinandan.prateek@shapeblue.com
>> > > www.shapeblue.com
>> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
>> > >
>> > >
>> > >
>> > > ----- Original Message -----
>> > > >>> From: "Will Stevens" <wi...@gmail.com>
>> > > >>> To: dev@cloudstack.apache.org
>> > > >>> Sent: Monday, 12 September, 2016 21:20:11
>> > > >>> Subject: [DISCUSS] Replacing the VR
>> > > >>
>> > > >>> *Disclaimer:* This is a thought experiment and should be treated
>> > > >>> as
>> > > such.
>> > > >>> Please weigh in with the good and bad of this idea...
>> > > >>>
>> > > >>> A couple of us have been discussing the idea of potentially
>> > > >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
>> > > >>> There may be a license issue because I think it is licensed under
>> > > >>> GPL, but for the sake of discussion, let's assume we can overcome
>> > > >>> any
>> > > license issues.
>> > > >>>
>> > > >>> I have spent some time recently with the VyOS and I have to admit,
>> > > >>> I was pretty impressed. It is simple and intuitive and it gives
>> > > >>> you a lot more options for auditing the configuration etc...
>> > > >>>
>> > > >>> Items of potential interest:
>> > > >>> - Clean up our current VR script spaghetti to a simpler more
>> > > >>> auditable configuration workflow.
>> > > >>> - Gives a cleaner path for IPv6 support.
>> > > >>> - Handles VPN configuration via the same configuration interface.
>> > > >>> - Support for OSPF & BGP.
>> > > >>> - VPN support through OpenVPN & StrongSwan.
>> > > >>> - Easily supports HA (redundant routers) through VRRP.
>> > > >>> - VXLAN support.
>> > > >>> - Transaction based changes to the VR with rollback on error.
>> > > >>>
>> > > >>> Items that could be difficult to solve:
>> > > >>> - Userdata password reset workflow and implementation.
>> > > >>> - Upgrade process.
>> > > >>>
>> > > >>> The VyOS is not the only option if we were to consider this
>> approach.
>> > > >>> Another option, which I don't know as well, would be CloudRouter
>> > > >>> (AGPL
>> > > >>> license) [2] which is purely API driven.
>> > > >>>
>> > > >>> Anyway, would love to hear your thoughts...
>> > > >>>
>> > > >>> Will
>> > > >>>
>> > > >>> [1] https://vyos.io/
>> > > >>> [2] https://cloudrouter.org/
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >DISCLAIMER
>> > > >==========
>> > > >This e-mail may contain privileged and confidential information which
>> > > >is
>> > > the property of Accelerite, a Persistent Systems business. It is
>> > > intended only for the use of the individual or entity to which it is
>> > > addressed. If you are not the intended recipient, you are not
>> > > authorized to read, retain, copy, print, distribute or use this
>> > > message. If you have received this communication in error, please
>> > > notify the sender and delete all copies of this message. Accelerite, a
>> > > Persistent Systems business does not accept any liability for virus
>> > infected mails.
>> > >
>> >
>>
>
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
I tend to agree with Syed and Marty. I am not sure what problems are
solved by splitting up the function of the VR into a bunch of separate
services. As Syed points out, the complexity added is non-trivial. We now
have to manage all the intercontainer networking as well as the
orchestrated ACS networking.
VyOS is interesting to me because it covers the majority of our use case
with a single unified control plane. It also has good support for
extending features we care about, like IPv6, VXLAN, VRRP, transactions,
etc...
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Wed, Sep 14, 2016 at 9:49 PM, Syed Ahmed <sa...@cloudops.com> wrote:
> Agree with Marty, adding Docker containers to the picture although can make
> the VR more flexible but the added complexity is just not worth it. Not to
> mention we would need to take care of networking each container manually
> and given that our iptable rules are very unstable at the moment I don't
> see a big value add.
>
> Vyos looks like a better solution to me. I know that it does not provide an
> api but it does fit the bill quite well otherwise. I specially like the
> fact that it has a transaction based model and you can rollback changes if
> something goes wrong.
> On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <ma...@gonsource.com> wrote:
>
> > Licensing aside, I think splitting the various functions into containers
> > is not a good route either. This will force users to have to maintain and
> > use containers and adds complexity to the networking aspects of ACS.
> > Complexity decreases stability. Now I understand the argument that a
> > monolithic approach also brings its own set of issues but it also
> > simplifies it.
> >
> > Regards,
> > Marty Godsey
> > nSource Solutions
> >
> > -----Original Message-----
> > From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> > Sent: Wednesday, September 14, 2016 5:37 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] Replacing the VR
> >
> > I rather doubt that the Cloudrouter will fit the needs of the CloudStack
> > project
> > - it is AGPL licensed. Many enterprises will not touch anything that has
> > AGPL
> > - the github repo shows rather infrequent updates. Quite likely they
> > aren't considering the use cases of the CloudStack community
> >
> > I'd back John B's comments on disaggregating the VR. Split it into many
> > docker containers
> > - password server
> > - userdata server
> > - DHCP / DNS
> > - s2s VPN
> > - RA VPN
> > - intra-VPC routing and ACL
> > - Port forwarding + NAT
> > - FW
> > - LB (public)
> > - LB (internal),
> > - secondary storage
> > - agent
> > Glue them together with docker compose files (one per use case - basic
> > zone, isolated, VPC, SSVM, etc).
> >
> > The VR image then becomes a JeOS + docker. You can test each of the
> > components independently and fixing one bug in the field (say DHCP) is
> > hitless to the other components. You don't need to build per-hypervisor
> > VRs. You could even run on baremetal.
> >
> > Along the way you need to figure out how to
> > - make the traffic traverse the containers that are needed to be
> > traversed (in most cases just 1)
> > - bootstrap the router (how does it find its compose file? where is the
> > registry?)
> > - rethink the command and control of the VR functions. SSH works, but
> > something more declarative, idempotent should be explored.
> >
> > As you do this, it becomes clearer which of the functions can be
> > substituted by for example CloudRouter. Command and Control of the docker
> > containers can be moved out to another container. Etc.
> >
> >
> >
> >
> >
> >
> >
> > On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey <ma...@gonsource.com>
> > wrote:
> >
> > > This one does look nice. My biggest concern is the lack of VXLANs. It
> > > seems that any of the ones we mentioned do not have an API so we may
> > > be stuck at the SSH method.
> > >
> > > Regards,
> > > Marty Godsey
> > > nSource Solutions
> > >
> > > -----Original Message-----
> > > From: Abhinandan Prateek [mailto:abhinandan.prateek@shapeblue.com]
> > > Sent: Wednesday, September 14, 2016 2:26 AM
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: [DISCUSS] Replacing the VR
> > >
> > > Cloudrouter looks promising. These have potential to save future
> > > engineering effort for example on ipv6 routing, OSPF etc.
> > > And the best part is they come with test automation framework.
> > >
> > >
> > >
> > >
> > >
> > > On 13/09/16, 4:22 PM, "Jayapal Uradi" <ja...@accelerite.com>
> > > wrote:
> > >
> > > >Hi,
> > > >
> > > >Instead of replacing the VR in first place we should add
> > > >VyOS/cloudrouter
> > > as provider. Once it is stable, network offerings (on upgrade) can be
> > > updated to use it and we can drop the VR if we want at that release
> > onwards.
> > > >
> > > >VR is stabilized over a period of time and some of them are running
> > > without issues. When we replicate the ACS VR features in new solution
> > > it takes some to find the missing pieces (hidden bugs).
> > > >
> > > >Thanks,
> > > >Jayapal
> > > >
> > > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> > > >
> > > >> nux@li.nux.ro> wrote:
> > > >>
> > > >> Hi,
> > > >>
> > > >> I like the idea.
> > > >>
> > > >> Cloudrouter looks really promising, I'm not too keen on VyOS (it
> > > doesn't have a proper http api etc).
> > > >>
> > > >> --
> > > >> Sent from the Delta quadrant using Borg technology!
> > > >>
> > > >> Nux!
> > > >> www.nux.ro
> > > >>
> > > >>
> > > abhinandan.prateek@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
> > >
> > >
> > >
> > > ----- Original Message -----
> > > >>> From: "Will Stevens" <wi...@gmail.com>
> > > >>> To: dev@cloudstack.apache.org
> > > >>> Sent: Monday, 12 September, 2016 21:20:11
> > > >>> Subject: [DISCUSS] Replacing the VR
> > > >>
> > > >>> *Disclaimer:* This is a thought experiment and should be treated
> > > >>> as
> > > such.
> > > >>> Please weigh in with the good and bad of this idea...
> > > >>>
> > > >>> A couple of us have been discussing the idea of potentially
> > > >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
> > > >>> There may be a license issue because I think it is licensed under
> > > >>> GPL, but for the sake of discussion, let's assume we can overcome
> > > >>> any
> > > license issues.
> > > >>>
> > > >>> I have spent some time recently with the VyOS and I have to admit,
> > > >>> I was pretty impressed. It is simple and intuitive and it gives
> > > >>> you a lot more options for auditing the configuration etc...
> > > >>>
> > > >>> Items of potential interest:
> > > >>> - Clean up our current VR script spaghetti to a simpler more
> > > >>> auditable configuration workflow.
> > > >>> - Gives a cleaner path for IPv6 support.
> > > >>> - Handles VPN configuration via the same configuration interface.
> > > >>> - Support for OSPF & BGP.
> > > >>> - VPN support through OpenVPN & StrongSwan.
> > > >>> - Easily supports HA (redundant routers) through VRRP.
> > > >>> - VXLAN support.
> > > >>> - Transaction based changes to the VR with rollback on error.
> > > >>>
> > > >>> Items that could be difficult to solve:
> > > >>> - Userdata password reset workflow and implementation.
> > > >>> - Upgrade process.
> > > >>>
> > > >>> The VyOS is not the only option if we were to consider this
> approach.
> > > >>> Another option, which I don't know as well, would be CloudRouter
> > > >>> (AGPL
> > > >>> license) [2] which is purely API driven.
> > > >>>
> > > >>> Anyway, would love to hear your thoughts...
> > > >>>
> > > >>> Will
> > > >>>
> > > >>> [1] https://vyos.io/
> > > >>> [2] https://cloudrouter.org/
> > > >
> > > >
> > > >
> > > >
> > > >DISCLAIMER
> > > >==========
> > > >This e-mail may contain privileged and confidential information which
> > > >is
> > > the property of Accelerite, a Persistent Systems business. It is
> > > intended only for the use of the individual or entity to which it is
> > > addressed. If you are not the intended recipient, you are not
> > > authorized to read, retain, copy, print, distribute or use this
> > > message. If you have received this communication in error, please
> > > notify the sender and delete all copies of this message. Accelerite, a
> > > Persistent Systems business does not accept any liability for virus
> > infected mails.
> > >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Syed Ahmed <sa...@cloudops.com>.
Agree with Marty, adding Docker containers to the picture although can make
the VR more flexible but the added complexity is just not worth it. Not to
mention we would need to take care of networking each container manually
and given that our iptable rules are very unstable at the moment I don't
see a big value add.
Vyos looks like a better solution to me. I know that it does not provide an
api but it does fit the bill quite well otherwise. I specially like the
fact that it has a transaction based model and you can rollback changes if
something goes wrong.
On Wed, Sep 14, 2016 at 9:06 PM Marty Godsey <ma...@gonsource.com> wrote:
> Licensing aside, I think splitting the various functions into containers
> is not a good route either. This will force users to have to maintain and
> use containers and adds complexity to the networking aspects of ACS.
> Complexity decreases stability. Now I understand the argument that a
> monolithic approach also brings its own set of issues but it also
> simplifies it.
>
> Regards,
> Marty Godsey
> nSource Solutions
>
> -----Original Message-----
> From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
> Sent: Wednesday, September 14, 2016 5:37 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> I rather doubt that the Cloudrouter will fit the needs of the CloudStack
> project
> - it is AGPL licensed. Many enterprises will not touch anything that has
> AGPL
> - the github repo shows rather infrequent updates. Quite likely they
> aren't considering the use cases of the CloudStack community
>
> I'd back John B's comments on disaggregating the VR. Split it into many
> docker containers
> - password server
> - userdata server
> - DHCP / DNS
> - s2s VPN
> - RA VPN
> - intra-VPC routing and ACL
> - Port forwarding + NAT
> - FW
> - LB (public)
> - LB (internal),
> - secondary storage
> - agent
> Glue them together with docker compose files (one per use case - basic
> zone, isolated, VPC, SSVM, etc).
>
> The VR image then becomes a JeOS + docker. You can test each of the
> components independently and fixing one bug in the field (say DHCP) is
> hitless to the other components. You don't need to build per-hypervisor
> VRs. You could even run on baremetal.
>
> Along the way you need to figure out how to
> - make the traffic traverse the containers that are needed to be
> traversed (in most cases just 1)
> - bootstrap the router (how does it find its compose file? where is the
> registry?)
> - rethink the command and control of the VR functions. SSH works, but
> something more declarative, idempotent should be explored.
>
> As you do this, it becomes clearer which of the functions can be
> substituted by for example CloudRouter. Command and Control of the docker
> containers can be moved out to another container. Etc.
>
>
>
>
>
>
>
> On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey <ma...@gonsource.com>
> wrote:
>
> > This one does look nice. My biggest concern is the lack of VXLANs. It
> > seems that any of the ones we mentioned do not have an API so we may
> > be stuck at the SSH method.
> >
> > Regards,
> > Marty Godsey
> > nSource Solutions
> >
> > -----Original Message-----
> > From: Abhinandan Prateek [mailto:abhinandan.prateek@shapeblue.com]
> > Sent: Wednesday, September 14, 2016 2:26 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] Replacing the VR
> >
> > Cloudrouter looks promising. These have potential to save future
> > engineering effort for example on ipv6 routing, OSPF etc.
> > And the best part is they come with test automation framework.
> >
> >
> >
> >
> >
> > On 13/09/16, 4:22 PM, "Jayapal Uradi" <ja...@accelerite.com>
> > wrote:
> >
> > >Hi,
> > >
> > >Instead of replacing the VR in first place we should add
> > >VyOS/cloudrouter
> > as provider. Once it is stable, network offerings (on upgrade) can be
> > updated to use it and we can drop the VR if we want at that release
> onwards.
> > >
> > >VR is stabilized over a period of time and some of them are running
> > without issues. When we replicate the ACS VR features in new solution
> > it takes some to find the missing pieces (hidden bugs).
> > >
> > >Thanks,
> > >Jayapal
> > >
> > >> On Sep 13, 2016, at 2:52 PM, Nux! <
> > >
> > >> nux@li.nux.ro> wrote:
> > >>
> > >> Hi,
> > >>
> > >> I like the idea.
> > >>
> > >> Cloudrouter looks really promising, I'm not too keen on VyOS (it
> > doesn't have a proper http api etc).
> > >>
> > >> --
> > >> Sent from the Delta quadrant using Borg technology!
> > >>
> > >> Nux!
> > >> www.nux.ro
> > >>
> > >>
> > abhinandan.prateek@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
> >
> >
> >
> > ----- Original Message -----
> > >>> From: "Will Stevens" <wi...@gmail.com>
> > >>> To: dev@cloudstack.apache.org
> > >>> Sent: Monday, 12 September, 2016 21:20:11
> > >>> Subject: [DISCUSS] Replacing the VR
> > >>
> > >>> *Disclaimer:* This is a thought experiment and should be treated
> > >>> as
> > such.
> > >>> Please weigh in with the good and bad of this idea...
> > >>>
> > >>> A couple of us have been discussing the idea of potentially
> > >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
> > >>> There may be a license issue because I think it is licensed under
> > >>> GPL, but for the sake of discussion, let's assume we can overcome
> > >>> any
> > license issues.
> > >>>
> > >>> I have spent some time recently with the VyOS and I have to admit,
> > >>> I was pretty impressed. It is simple and intuitive and it gives
> > >>> you a lot more options for auditing the configuration etc...
> > >>>
> > >>> Items of potential interest:
> > >>> - Clean up our current VR script spaghetti to a simpler more
> > >>> auditable configuration workflow.
> > >>> - Gives a cleaner path for IPv6 support.
> > >>> - Handles VPN configuration via the same configuration interface.
> > >>> - Support for OSPF & BGP.
> > >>> - VPN support through OpenVPN & StrongSwan.
> > >>> - Easily supports HA (redundant routers) through VRRP.
> > >>> - VXLAN support.
> > >>> - Transaction based changes to the VR with rollback on error.
> > >>>
> > >>> Items that could be difficult to solve:
> > >>> - Userdata password reset workflow and implementation.
> > >>> - Upgrade process.
> > >>>
> > >>> The VyOS is not the only option if we were to consider this approach.
> > >>> Another option, which I don't know as well, would be CloudRouter
> > >>> (AGPL
> > >>> license) [2] which is purely API driven.
> > >>>
> > >>> Anyway, would love to hear your thoughts...
> > >>>
> > >>> Will
> > >>>
> > >>> [1] https://vyos.io/
> > >>> [2] https://cloudrouter.org/
> > >
> > >
> > >
> > >
> > >DISCLAIMER
> > >==========
> > >This e-mail may contain privileged and confidential information which
> > >is
> > the property of Accelerite, a Persistent Systems business. It is
> > intended only for the use of the individual or entity to which it is
> > addressed. If you are not the intended recipient, you are not
> > authorized to read, retain, copy, print, distribute or use this
> > message. If you have received this communication in error, please
> > notify the sender and delete all copies of this message. Accelerite, a
> > Persistent Systems business does not accept any liability for virus
> infected mails.
> >
>
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
Licensing aside, I think splitting the various functions into containers is not a good route either. This will force users to have to maintain and use containers and adds complexity to the networking aspects of ACS. Complexity decreases stability. Now I understand the argument that a monolithic approach also brings its own set of issues but it also simplifies it.
Regards,
Marty Godsey
nSource Solutions
-----Original Message-----
From: Chiradeep Vittal [mailto:chiradeepv@gmail.com]
Sent: Wednesday, September 14, 2016 5:37 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
I rather doubt that the Cloudrouter will fit the needs of the CloudStack project
- it is AGPL licensed. Many enterprises will not touch anything that has AGPL
- the github repo shows rather infrequent updates. Quite likely they aren't considering the use cases of the CloudStack community
I'd back John B's comments on disaggregating the VR. Split it into many docker containers
- password server
- userdata server
- DHCP / DNS
- s2s VPN
- RA VPN
- intra-VPC routing and ACL
- Port forwarding + NAT
- FW
- LB (public)
- LB (internal),
- secondary storage
- agent
Glue them together with docker compose files (one per use case - basic zone, isolated, VPC, SSVM, etc).
The VR image then becomes a JeOS + docker. You can test each of the components independently and fixing one bug in the field (say DHCP) is hitless to the other components. You don't need to build per-hypervisor VRs. You could even run on baremetal.
Along the way you need to figure out how to
- make the traffic traverse the containers that are needed to be traversed (in most cases just 1)
- bootstrap the router (how does it find its compose file? where is the
registry?)
- rethink the command and control of the VR functions. SSH works, but something more declarative, idempotent should be explored.
As you do this, it becomes clearer which of the functions can be substituted by for example CloudRouter. Command and Control of the docker containers can be moved out to another container. Etc.
On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey <ma...@gonsource.com> wrote:
> This one does look nice. My biggest concern is the lack of VXLANs. It
> seems that any of the ones we mentioned do not have an API so we may
> be stuck at the SSH method.
>
> Regards,
> Marty Godsey
> nSource Solutions
>
> -----Original Message-----
> From: Abhinandan Prateek [mailto:abhinandan.prateek@shapeblue.com]
> Sent: Wednesday, September 14, 2016 2:26 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Cloudrouter looks promising. These have potential to save future
> engineering effort for example on ipv6 routing, OSPF etc.
> And the best part is they come with test automation framework.
>
>
>
>
>
> On 13/09/16, 4:22 PM, "Jayapal Uradi" <ja...@accelerite.com>
> wrote:
>
> >Hi,
> >
> >Instead of replacing the VR in first place we should add
> >VyOS/cloudrouter
> as provider. Once it is stable, network offerings (on upgrade) can be
> updated to use it and we can drop the VR if we want at that release onwards.
> >
> >VR is stabilized over a period of time and some of them are running
> without issues. When we replicate the ACS VR features in new solution
> it takes some to find the missing pieces (hidden bugs).
> >
> >Thanks,
> >Jayapal
> >
> >> On Sep 13, 2016, at 2:52 PM, Nux! <
> >
> >> nux@li.nux.ro> wrote:
> >>
> >> Hi,
> >>
> >> I like the idea.
> >>
> >> Cloudrouter looks really promising, I'm not too keen on VyOS (it
> doesn't have a proper http api etc).
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
> >>
> >>
> abhinandan.prateek@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
>
>
>
> ----- Original Message -----
> >>> From: "Will Stevens" <wi...@gmail.com>
> >>> To: dev@cloudstack.apache.org
> >>> Sent: Monday, 12 September, 2016 21:20:11
> >>> Subject: [DISCUSS] Replacing the VR
> >>
> >>> *Disclaimer:* This is a thought experiment and should be treated
> >>> as
> such.
> >>> Please weigh in with the good and bad of this idea...
> >>>
> >>> A couple of us have been discussing the idea of potentially
> >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
> >>> There may be a license issue because I think it is licensed under
> >>> GPL, but for the sake of discussion, let's assume we can overcome
> >>> any
> license issues.
> >>>
> >>> I have spent some time recently with the VyOS and I have to admit,
> >>> I was pretty impressed. It is simple and intuitive and it gives
> >>> you a lot more options for auditing the configuration etc...
> >>>
> >>> Items of potential interest:
> >>> - Clean up our current VR script spaghetti to a simpler more
> >>> auditable configuration workflow.
> >>> - Gives a cleaner path for IPv6 support.
> >>> - Handles VPN configuration via the same configuration interface.
> >>> - Support for OSPF & BGP.
> >>> - VPN support through OpenVPN & StrongSwan.
> >>> - Easily supports HA (redundant routers) through VRRP.
> >>> - VXLAN support.
> >>> - Transaction based changes to the VR with rollback on error.
> >>>
> >>> Items that could be difficult to solve:
> >>> - Userdata password reset workflow and implementation.
> >>> - Upgrade process.
> >>>
> >>> The VyOS is not the only option if we were to consider this approach.
> >>> Another option, which I don't know as well, would be CloudRouter
> >>> (AGPL
> >>> license) [2] which is purely API driven.
> >>>
> >>> Anyway, would love to hear your thoughts...
> >>>
> >>> Will
> >>>
> >>> [1] https://vyos.io/
> >>> [2] https://cloudrouter.org/
> >
> >
> >
> >
> >DISCLAIMER
> >==========
> >This e-mail may contain privileged and confidential information which
> >is
> the property of Accelerite, a Persistent Systems business. It is
> intended only for the use of the individual or entity to which it is
> addressed. If you are not the intended recipient, you are not
> authorized to read, retain, copy, print, distribute or use this
> message. If you have received this communication in error, please
> notify the sender and delete all copies of this message. Accelerite, a
> Persistent Systems business does not accept any liability for virus infected mails.
>
Re: [DISCUSS] Replacing the VR
Posted by Chiradeep Vittal <ch...@gmail.com>.
I rather doubt that the Cloudrouter will fit the needs of the CloudStack
project
- it is AGPL licensed. Many enterprises will not touch anything that has
AGPL
- the github repo shows rather infrequent updates. Quite likely they
aren't considering the use cases of the CloudStack community
I'd back John B's comments on disaggregating the VR. Split it into many
docker containers
- password server
- userdata server
- DHCP / DNS
- s2s VPN
- RA VPN
- intra-VPC routing and ACL
- Port forwarding + NAT
- FW
- LB (public)
- LB (internal),
- secondary storage
- agent
Glue them together with docker compose files (one per use case - basic
zone, isolated, VPC, SSVM, etc).
The VR image then becomes a JeOS + docker. You can test each of the
components independently and fixing one bug in the field (say DHCP) is
hitless to the other components. You don't need to build per-hypervisor
VRs. You could even run on baremetal.
Along the way you need to figure out how to
- make the traffic traverse the containers that are needed to be traversed
(in most cases just 1)
- bootstrap the router (how does it find its compose file? where is the
registry?)
- rethink the command and control of the VR functions. SSH works, but
something more declarative, idempotent should be explored.
As you do this, it becomes clearer which of the functions can be
substituted by for example CloudRouter. Command and Control of the docker
containers can be moved out to another container. Etc.
On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey <ma...@gonsource.com> wrote:
> This one does look nice. My biggest concern is the lack of VXLANs. It
> seems that any of the ones we mentioned do not have an API so we may be
> stuck at the SSH method.
>
> Regards,
> Marty Godsey
> nSource Solutions
>
> -----Original Message-----
> From: Abhinandan Prateek [mailto:abhinandan.prateek@shapeblue.com]
> Sent: Wednesday, September 14, 2016 2:26 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Cloudrouter looks promising. These have potential to save future
> engineering effort for example on ipv6 routing, OSPF etc.
> And the best part is they come with test automation framework.
>
>
>
>
>
> On 13/09/16, 4:22 PM, "Jayapal Uradi" <ja...@accelerite.com>
> wrote:
>
> >Hi,
> >
> >Instead of replacing the VR in first place we should add VyOS/cloudrouter
> as provider. Once it is stable, network offerings (on upgrade) can be
> updated to use it and we can drop the VR if we want at that release onwards.
> >
> >VR is stabilized over a period of time and some of them are running
> without issues. When we replicate the ACS VR features in new solution it
> takes some to find the missing pieces (hidden bugs).
> >
> >Thanks,
> >Jayapal
> >
> >> On Sep 13, 2016, at 2:52 PM, Nux! <
> >
> >> nux@li.nux.ro> wrote:
> >>
> >> Hi,
> >>
> >> I like the idea.
> >>
> >> Cloudrouter looks really promising, I'm not too keen on VyOS (it
> doesn't have a proper http api etc).
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
> >>
> >>
> abhinandan.prateek@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
>
>
>
> ----- Original Message -----
> >>> From: "Will Stevens" <wi...@gmail.com>
> >>> To: dev@cloudstack.apache.org
> >>> Sent: Monday, 12 September, 2016 21:20:11
> >>> Subject: [DISCUSS] Replacing the VR
> >>
> >>> *Disclaimer:* This is a thought experiment and should be treated as
> such.
> >>> Please weigh in with the good and bad of this idea...
> >>>
> >>> A couple of us have been discussing the idea of potentially
> >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
> >>> There may be a license issue because I think it is licensed under
> >>> GPL, but for the sake of discussion, let's assume we can overcome any
> license issues.
> >>>
> >>> I have spent some time recently with the VyOS and I have to admit, I
> >>> was pretty impressed. It is simple and intuitive and it gives you a
> >>> lot more options for auditing the configuration etc...
> >>>
> >>> Items of potential interest:
> >>> - Clean up our current VR script spaghetti to a simpler more
> >>> auditable configuration workflow.
> >>> - Gives a cleaner path for IPv6 support.
> >>> - Handles VPN configuration via the same configuration interface.
> >>> - Support for OSPF & BGP.
> >>> - VPN support through OpenVPN & StrongSwan.
> >>> - Easily supports HA (redundant routers) through VRRP.
> >>> - VXLAN support.
> >>> - Transaction based changes to the VR with rollback on error.
> >>>
> >>> Items that could be difficult to solve:
> >>> - Userdata password reset workflow and implementation.
> >>> - Upgrade process.
> >>>
> >>> The VyOS is not the only option if we were to consider this approach.
> >>> Another option, which I don't know as well, would be CloudRouter
> >>> (AGPL
> >>> license) [2] which is purely API driven.
> >>>
> >>> Anyway, would love to hear your thoughts...
> >>>
> >>> Will
> >>>
> >>> [1] https://vyos.io/
> >>> [2] https://cloudrouter.org/
> >
> >
> >
> >
> >DISCLAIMER
> >==========
> >This e-mail may contain privileged and confidential information which is
> the property of Accelerite, a Persistent Systems business. It is intended
> only for the use of the individual or entity to which it is addressed. If
> you are not the intended recipient, you are not authorized to read, retain,
> copy, print, distribute or use this message. If you have received this
> communication in error, please notify the sender and delete all copies of
> this message. Accelerite, a Persistent Systems business does not accept any
> liability for virus infected mails.
>
RE: [DISCUSS] Replacing the VR
Posted by Marty Godsey <ma...@gonsource.com>.
This one does look nice. My biggest concern is the lack of VXLANs. It seems that any of the ones we mentioned do not have an API so we may be stuck at the SSH method.
Regards,
Marty Godsey
nSource Solutions
-----Original Message-----
From: Abhinandan Prateek [mailto:abhinandan.prateek@shapeblue.com]
Sent: Wednesday, September 14, 2016 2:26 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Replacing the VR
Cloudrouter looks promising. These have potential to save future engineering effort for example on ipv6 routing, OSPF etc.
And the best part is they come with test automation framework.
On 13/09/16, 4:22 PM, "Jayapal Uradi" <ja...@accelerite.com> wrote:
>Hi,
>
>Instead of replacing the VR in first place we should add VyOS/cloudrouter as provider. Once it is stable, network offerings (on upgrade) can be updated to use it and we can drop the VR if we want at that release onwards.
>
>VR is stabilized over a period of time and some of them are running without issues. When we replicate the ACS VR features in new solution it takes some to find the missing pieces (hidden bugs).
>
>Thanks,
>Jayapal
>
>> On Sep 13, 2016, at 2:52 PM, Nux! <
>
>> nux@li.nux.ro> wrote:
>>
>> Hi,
>>
>> I like the idea.
>>
>> Cloudrouter looks really promising, I'm not too keen on VyOS (it doesn't have a proper http api etc).
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>>
abhinandan.prateek@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
----- Original Message -----
>>> From: "Will Stevens" <wi...@gmail.com>
>>> To: dev@cloudstack.apache.org
>>> Sent: Monday, 12 September, 2016 21:20:11
>>> Subject: [DISCUSS] Replacing the VR
>>
>>> *Disclaimer:* This is a thought experiment and should be treated as such.
>>> Please weigh in with the good and bad of this idea...
>>>
>>> A couple of us have been discussing the idea of potentially
>>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
>>> There may be a license issue because I think it is licensed under
>>> GPL, but for the sake of discussion, let's assume we can overcome any license issues.
>>>
>>> I have spent some time recently with the VyOS and I have to admit, I
>>> was pretty impressed. It is simple and intuitive and it gives you a
>>> lot more options for auditing the configuration etc...
>>>
>>> Items of potential interest:
>>> - Clean up our current VR script spaghetti to a simpler more
>>> auditable configuration workflow.
>>> - Gives a cleaner path for IPv6 support.
>>> - Handles VPN configuration via the same configuration interface.
>>> - Support for OSPF & BGP.
>>> - VPN support through OpenVPN & StrongSwan.
>>> - Easily supports HA (redundant routers) through VRRP.
>>> - VXLAN support.
>>> - Transaction based changes to the VR with rollback on error.
>>>
>>> Items that could be difficult to solve:
>>> - Userdata password reset workflow and implementation.
>>> - Upgrade process.
>>>
>>> The VyOS is not the only option if we were to consider this approach.
>>> Another option, which I don't know as well, would be CloudRouter
>>> (AGPL
>>> license) [2] which is purely API driven.
>>>
>>> Anyway, would love to hear your thoughts...
>>>
>>> Will
>>>
>>> [1] https://vyos.io/
>>> [2] https://cloudrouter.org/
>
>
>
>
>DISCLAIMER
>==========
>This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
Re: [DISCUSS] Replacing the VR
Posted by Abhinandan Prateek <ab...@shapeblue.com>.
Cloudrouter looks promising. These have potential to save future engineering effort for example on ipv6 routing, OSPF etc.
And the best part is they come with test automation framework.
On 13/09/16, 4:22 PM, "Jayapal Uradi" <ja...@accelerite.com> wrote:
>Hi,
>
>Instead of replacing the VR in first place we should add VyOS/cloudrouter as provider. Once it is stable, network offerings (on upgrade) can be updated to use it and we can drop the VR if we want at that release onwards.
>
>VR is stabilized over a period of time and some of them are running without issues. When we replicate the ACS VR features in new solution it takes some to find the missing pieces (hidden bugs).
>
>Thanks,
>Jayapal
>
>> On Sep 13, 2016, at 2:52 PM, Nux! <
>
>> nux@li.nux.ro> wrote:
>>
>> Hi,
>>
>> I like the idea.
>>
>> Cloudrouter looks really promising, I'm not too keen on VyOS (it doesn't have a proper http api etc).
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>>
abhinandan.prateek@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
----- Original Message -----
>>> From: "Will Stevens" <wi...@gmail.com>
>>> To: dev@cloudstack.apache.org
>>> Sent: Monday, 12 September, 2016 21:20:11
>>> Subject: [DISCUSS] Replacing the VR
>>
>>> *Disclaimer:* This is a thought experiment and should be treated as such.
>>> Please weigh in with the good and bad of this idea...
>>>
>>> A couple of us have been discussing the idea of potentially replacing the
>>> ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a license
>>> issue because I think it is licensed under GPL, but for the sake of
>>> discussion, let's assume we can overcome any license issues.
>>>
>>> I have spent some time recently with the VyOS and I have to admit, I was
>>> pretty impressed. It is simple and intuitive and it gives you a lot more
>>> options for auditing the configuration etc...
>>>
>>> Items of potential interest:
>>> - Clean up our current VR script spaghetti to a simpler more auditable
>>> configuration workflow.
>>> - Gives a cleaner path for IPv6 support.
>>> - Handles VPN configuration via the same configuration interface.
>>> - Support for OSPF & BGP.
>>> - VPN support through OpenVPN & StrongSwan.
>>> - Easily supports HA (redundant routers) through VRRP.
>>> - VXLAN support.
>>> - Transaction based changes to the VR with rollback on error.
>>>
>>> Items that could be difficult to solve:
>>> - Userdata password reset workflow and implementation.
>>> - Upgrade process.
>>>
>>> The VyOS is not the only option if we were to consider this approach.
>>> Another option, which I don't know as well, would be CloudRouter (AGPL
>>> license) [2] which is purely API driven.
>>>
>>> Anyway, would love to hear your thoughts...
>>>
>>> Will
>>>
>>> [1] https://vyos.io/
>>> [2] https://cloudrouter.org/
>
>
>
>
>DISCLAIMER
>==========
>This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
Re: [DISCUSS] Replacing the VR
Posted by Jayapal Uradi <ja...@accelerite.com>.
Hi,
Instead of replacing the VR in first place we should add VyOS/cloudrouter as provider. Once it is stable, network offerings (on upgrade) can be updated to use it and we can drop the VR if we want at that release onwards.
VR is stabilized over a period of time and some of them are running without issues. When we replicate the ACS VR features in new solution it takes some to find the missing pieces (hidden bugs).
Thanks,
Jayapal
> On Sep 13, 2016, at 2:52 PM, Nux! <
> nux@li.nux.ro> wrote:
>
> Hi,
>
> I like the idea.
>
> Cloudrouter looks really promising, I'm not too keen on VyOS (it doesn't have a proper http api etc).
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
>> From: "Will Stevens" <wi...@gmail.com>
>> To: dev@cloudstack.apache.org
>> Sent: Monday, 12 September, 2016 21:20:11
>> Subject: [DISCUSS] Replacing the VR
>
>> *Disclaimer:* This is a thought experiment and should be treated as such.
>> Please weigh in with the good and bad of this idea...
>>
>> A couple of us have been discussing the idea of potentially replacing the
>> ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a license
>> issue because I think it is licensed under GPL, but for the sake of
>> discussion, let's assume we can overcome any license issues.
>>
>> I have spent some time recently with the VyOS and I have to admit, I was
>> pretty impressed. It is simple and intuitive and it gives you a lot more
>> options for auditing the configuration etc...
>>
>> Items of potential interest:
>> - Clean up our current VR script spaghetti to a simpler more auditable
>> configuration workflow.
>> - Gives a cleaner path for IPv6 support.
>> - Handles VPN configuration via the same configuration interface.
>> - Support for OSPF & BGP.
>> - VPN support through OpenVPN & StrongSwan.
>> - Easily supports HA (redundant routers) through VRRP.
>> - VXLAN support.
>> - Transaction based changes to the VR with rollback on error.
>>
>> Items that could be difficult to solve:
>> - Userdata password reset workflow and implementation.
>> - Upgrade process.
>>
>> The VyOS is not the only option if we were to consider this approach.
>> Another option, which I don't know as well, would be CloudRouter (AGPL
>> license) [2] which is purely API driven.
>>
>> Anyway, would love to hear your thoughts...
>>
>> Will
>>
>> [1] https://vyos.io/
>> [2] https://cloudrouter.org/
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
Re: [DISCUSS] Replacing the VR
Posted by Nux! <nu...@li.nux.ro>.
Hi,
I like the idea.
Cloudrouter looks really promising, I'm not too keen on VyOS (it doesn't have a proper http api etc).
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Will Stevens" <wi...@gmail.com>
> To: dev@cloudstack.apache.org
> Sent: Monday, 12 September, 2016 21:20:11
> Subject: [DISCUSS] Replacing the VR
> *Disclaimer:* This is a thought experiment and should be treated as such.
> Please weigh in with the good and bad of this idea...
>
> A couple of us have been discussing the idea of potentially replacing the
> ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a license
> issue because I think it is licensed under GPL, but for the sake of
> discussion, let's assume we can overcome any license issues.
>
> I have spent some time recently with the VyOS and I have to admit, I was
> pretty impressed. It is simple and intuitive and it gives you a lot more
> options for auditing the configuration etc...
>
> Items of potential interest:
> - Clean up our current VR script spaghetti to a simpler more auditable
> configuration workflow.
> - Gives a cleaner path for IPv6 support.
> - Handles VPN configuration via the same configuration interface.
> - Support for OSPF & BGP.
> - VPN support through OpenVPN & StrongSwan.
> - Easily supports HA (redundant routers) through VRRP.
> - VXLAN support.
> - Transaction based changes to the VR with rollback on error.
>
> Items that could be difficult to solve:
> - Userdata password reset workflow and implementation.
> - Upgrade process.
>
> The VyOS is not the only option if we were to consider this approach.
> Another option, which I don't know as well, would be CloudRouter (AGPL
> license) [2] which is purely API driven.
>
> Anyway, would love to hear your thoughts...
>
> Will
>
> [1] https://vyos.io/
> [2] https://cloudrouter.org/
Re: [DISCUSS] Replacing the VR
Posted by John Burwell <jo...@shapeblue.com>.
Will,
Typo. “application model” was meant to be “appliance model”.
Thanks,
-John
>
john.burwell@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
@shapeblue
On Sep 12, 2016, at 4:35 PM, John Burwell <jo...@shapeblue.com> wrote:
>
> Will,
>
> I agree that we need to replace the VR, but I am not convinced that continuing with the notion of a monolithic application model is a best direction. The problem with the current model is that it lacks flexibility. Some users only need to deploy DHCP and DNS across a zone where others need a much wider range of services at the pod or cluster level. With the monolithic appliance, we are forced to build to the lowest common denominator.
>
> I would like to see the VR’s functions disambiguated likely into containers (Zones/LXC-style rather than requiring the Docker/rkt runtime). With this subdivision, we could then adopt the service chain model and allow users to compose networks services to better fit their use cases.
>
> My thinking is that if we are going to through the (continuing) pain of another VR replacement, we should take the opportunity to re-evaluate the entire model.
>
> Thanks,
> -John
>
>>
> john.burwell@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> @shapeblue
>
>
>
> On Sep 12, 2016, at 4:20 PM, Will Stevens <wi...@gmail.com> wrote:
>>
>> *Disclaimer:* This is a thought experiment and should be treated as such.
>> Please weigh in with the good and bad of this idea...
>>
>> A couple of us have been discussing the idea of potentially replacing the
>> ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a license
>> issue because I think it is licensed under GPL, but for the sake of
>> discussion, let's assume we can overcome any license issues.
>>
>> I have spent some time recently with the VyOS and I have to admit, I was
>> pretty impressed. It is simple and intuitive and it gives you a lot more
>> options for auditing the configuration etc...
>>
>> Items of potential interest:
>> - Clean up our current VR script spaghetti to a simpler more auditable
>> configuration workflow.
>> - Gives a cleaner path for IPv6 support.
>> - Handles VPN configuration via the same configuration interface.
>> - Support for OSPF & BGP.
>> - VPN support through OpenVPN & StrongSwan.
>> - Easily supports HA (redundant routers) through VRRP.
>> - VXLAN support.
>> - Transaction based changes to the VR with rollback on error.
>>
>> Items that could be difficult to solve:
>> - Userdata password reset workflow and implementation.
>> - Upgrade process.
>>
>> The VyOS is not the only option if we were to consider this approach.
>> Another option, which I don't know as well, would be CloudRouter (AGPL
>> license) [2] which is purely API driven.
>>
>> Anyway, would love to hear your thoughts...
>>
>> Will
>>
>> [1] https://vyos.io/
>> [2] https://cloudrouter.org/
>
Re: [DISCUSS] Replacing the VR
Posted by Syed Ahmed <sa...@cloudops.com>.
John,
When you say to decompose the services to multiple containers? Where do you
envision the containers be running? Surely, they must be running in some VM
running on top of the hypervisor otherwise you would not be able to support
all hypervisors. Now the question is, does each individual service require
its own VM? Could you elaborate this further?
Thanks,
-Syed
On Mon, Sep 12, 2016 at 4:52 PM, Will Stevens <ws...@cloudops.com> wrote:
> Those are fair points John. I was going down the thought process of "if we
> have a VR, let's use an existing proven technology and not build our own".
>
> I think ACS needs an easy-to-use, out-of-the box default which anyone can
> use without having to think too much about it. It would be great if it was
> also implemented using a composition of distinct parts, but my initial goal
> was to solve for a drop in replacement.
>
> Let's broaden the discussion to include the potential of introducing
> composable pieces. If we are rewriting all of this, it would make sense to
> consider that at the same time. The one thing that I think could be
> challenging if we do that is the backwards compatibility of the plugin
> implementations for the current hardware appliances because the plugin
> hooks may have to change.
>
> Let's keep the conversation going... :P
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Mon, Sep 12, 2016 at 4:35 PM, John Burwell <jo...@shapeblue.com>
> wrote:
>
> > Will,
> >
> > I agree that we need to replace the VR, but I am not convinced that
> > continuing with the notion of a monolithic application model is a best
> > direction. The problem with the current model is that it lacks
> > flexibility. Some users only need to deploy DHCP and DNS across a zone
> > where others need a much wider range of services at the pod or cluster
> > level. With the monolithic appliance, we are forced to build to the
> lowest
> > common denominator.
> >
> > I would like to see the VR’s functions disambiguated likely into
> > containers (Zones/LXC-style rather than requiring the Docker/rkt
> runtime).
> > With this subdivision, we could then adopt the service chain model and
> > allow users to compose networks services to better fit their use cases.
> >
> > My thinking is that if we are going to through the (continuing) pain of
> > another VR replacement, we should take the opportunity to re-evaluate the
> > entire model.
> >
> > Thanks,
> > -John
> >
> > >
> > john.burwell@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> > On Sep 12, 2016, at 4:20 PM, Will Stevens <wi...@gmail.com>
> > wrote:
> > >
> > > *Disclaimer:* This is a thought experiment and should be treated as
> such.
> > > Please weigh in with the good and bad of this idea...
> > >
> > > A couple of us have been discussing the idea of potentially replacing
> the
> > > ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a
> license
> > > issue because I think it is licensed under GPL, but for the sake of
> > > discussion, let's assume we can overcome any license issues.
> > >
> > > I have spent some time recently with the VyOS and I have to admit, I
> was
> > > pretty impressed. It is simple and intuitive and it gives you a lot
> more
> > > options for auditing the configuration etc...
> > >
> > > Items of potential interest:
> > > - Clean up our current VR script spaghetti to a simpler more auditable
> > > configuration workflow.
> > > - Gives a cleaner path for IPv6 support.
> > > - Handles VPN configuration via the same configuration interface.
> > > - Support for OSPF & BGP.
> > > - VPN support through OpenVPN & StrongSwan.
> > > - Easily supports HA (redundant routers) through VRRP.
> > > - VXLAN support.
> > > - Transaction based changes to the VR with rollback on error.
> > >
> > > Items that could be difficult to solve:
> > > - Userdata password reset workflow and implementation.
> > > - Upgrade process.
> > >
> > > The VyOS is not the only option if we were to consider this approach.
> > > Another option, which I don't know as well, would be CloudRouter (AGPL
> > > license) [2] which is purely API driven.
> > >
> > > Anyway, would love to hear your thoughts...
> > >
> > > Will
> > >
> > > [1] https://vyos.io/
> > > [2] https://cloudrouter.org/
> >
> >
>
Re: [DISCUSS] Replacing the VR
Posted by Will Stevens <ws...@cloudops.com>.
Those are fair points John. I was going down the thought process of "if we
have a VR, let's use an existing proven technology and not build our own".
I think ACS needs an easy-to-use, out-of-the box default which anyone can
use without having to think too much about it. It would be great if it was
also implemented using a composition of distinct parts, but my initial goal
was to solve for a drop in replacement.
Let's broaden the discussion to include the potential of introducing
composable pieces. If we are rewriting all of this, it would make sense to
consider that at the same time. The one thing that I think could be
challenging if we do that is the backwards compatibility of the plugin
implementations for the current hardware appliances because the plugin
hooks may have to change.
Let's keep the conversation going... :P
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Mon, Sep 12, 2016 at 4:35 PM, John Burwell <jo...@shapeblue.com>
wrote:
> Will,
>
> I agree that we need to replace the VR, but I am not convinced that
> continuing with the notion of a monolithic application model is a best
> direction. The problem with the current model is that it lacks
> flexibility. Some users only need to deploy DHCP and DNS across a zone
> where others need a much wider range of services at the pod or cluster
> level. With the monolithic appliance, we are forced to build to the lowest
> common denominator.
>
> I would like to see the VR’s functions disambiguated likely into
> containers (Zones/LXC-style rather than requiring the Docker/rkt runtime).
> With this subdivision, we could then adopt the service chain model and
> allow users to compose networks services to better fit their use cases.
>
> My thinking is that if we are going to through the (continuing) pain of
> another VR replacement, we should take the opportunity to re-evaluate the
> entire model.
>
> Thanks,
> -John
>
> >
> john.burwell@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> @shapeblue
>
>
>
> On Sep 12, 2016, at 4:20 PM, Will Stevens <wi...@gmail.com>
> wrote:
> >
> > *Disclaimer:* This is a thought experiment and should be treated as such.
> > Please weigh in with the good and bad of this idea...
> >
> > A couple of us have been discussing the idea of potentially replacing the
> > ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a license
> > issue because I think it is licensed under GPL, but for the sake of
> > discussion, let's assume we can overcome any license issues.
> >
> > I have spent some time recently with the VyOS and I have to admit, I was
> > pretty impressed. It is simple and intuitive and it gives you a lot more
> > options for auditing the configuration etc...
> >
> > Items of potential interest:
> > - Clean up our current VR script spaghetti to a simpler more auditable
> > configuration workflow.
> > - Gives a cleaner path for IPv6 support.
> > - Handles VPN configuration via the same configuration interface.
> > - Support for OSPF & BGP.
> > - VPN support through OpenVPN & StrongSwan.
> > - Easily supports HA (redundant routers) through VRRP.
> > - VXLAN support.
> > - Transaction based changes to the VR with rollback on error.
> >
> > Items that could be difficult to solve:
> > - Userdata password reset workflow and implementation.
> > - Upgrade process.
> >
> > The VyOS is not the only option if we were to consider this approach.
> > Another option, which I don't know as well, would be CloudRouter (AGPL
> > license) [2] which is purely API driven.
> >
> > Anyway, would love to hear your thoughts...
> >
> > Will
> >
> > [1] https://vyos.io/
> > [2] https://cloudrouter.org/
>
>
Re: [DISCUSS] Replacing the VR
Posted by John Burwell <jo...@shapeblue.com>.
Will,
I agree that we need to replace the VR, but I am not convinced that continuing with the notion of a monolithic application model is a best direction. The problem with the current model is that it lacks flexibility. Some users only need to deploy DHCP and DNS across a zone where others need a much wider range of services at the pod or cluster level. With the monolithic appliance, we are forced to build to the lowest common denominator.
I would like to see the VR’s functions disambiguated likely into containers (Zones/LXC-style rather than requiring the Docker/rkt runtime). With this subdivision, we could then adopt the service chain model and allow users to compose networks services to better fit their use cases.
My thinking is that if we are going to through the (continuing) pain of another VR replacement, we should take the opportunity to re-evaluate the entire model.
Thanks,
-John
>
john.burwell@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
@shapeblue
On Sep 12, 2016, at 4:20 PM, Will Stevens <wi...@gmail.com> wrote:
>
> *Disclaimer:* This is a thought experiment and should be treated as such.
> Please weigh in with the good and bad of this idea...
>
> A couple of us have been discussing the idea of potentially replacing the
> ACS VR with the VyOS [1] (Open Source Vyatta VM). There may be a license
> issue because I think it is licensed under GPL, but for the sake of
> discussion, let's assume we can overcome any license issues.
>
> I have spent some time recently with the VyOS and I have to admit, I was
> pretty impressed. It is simple and intuitive and it gives you a lot more
> options for auditing the configuration etc...
>
> Items of potential interest:
> - Clean up our current VR script spaghetti to a simpler more auditable
> configuration workflow.
> - Gives a cleaner path for IPv6 support.
> - Handles VPN configuration via the same configuration interface.
> - Support for OSPF & BGP.
> - VPN support through OpenVPN & StrongSwan.
> - Easily supports HA (redundant routers) through VRRP.
> - VXLAN support.
> - Transaction based changes to the VR with rollback on error.
>
> Items that could be difficult to solve:
> - Userdata password reset workflow and implementation.
> - Upgrade process.
>
> The VyOS is not the only option if we were to consider this approach.
> Another option, which I don't know as well, would be CloudRouter (AGPL
> license) [2] which is purely API driven.
>
> Anyway, would love to hear your thoughts...
>
> Will
>
> [1] https://vyos.io/
> [2] https://cloudrouter.org/