You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Dennis Meyer <sn...@gmail.com> on 2017/08/14 09:46:51 UTC
Error on Terraform with creating many networks
Hi,
i try to create a vpc and some networks with terraform.
Thats my main.tf:
# creating the vpc
resource "cloudstack_vpc" "myvpc" {
name = "myvpc"
cidr = "10.230.0.0/16"
vpc_offering = "Default VPC offering"
zone = "${var.zone}"
project = "${var.project}"
}
# create our network inside the vpc
resource "cloudstack_network" "public" {
name = "public"
cidr = "10.230.200.0/24"
network_offering = "Default VPC Network with LB"
zone = "${var.zone}"
vpc_id = "${cloudstack_vpc.myvpc.id}"
project = "${var.project}"
acl_id = "ebefcc96-75f5-11e7-adb3-e2bd27d4977e" # default
static id of cloudstack
depends_on = ["cloudstack_vpc.myvpc"]
}
# create our network inside the vpc
resource "cloudstack_network" "network1" {
name = "network1"
cidr = "10.230.1.0/24"
gateway = "10.230.1.1"
network_offering = "${var.network_offering_internalwithrouter}"
zone = "${var.zone}"
vpc_id = "${cloudstack_vpc.myvpc.id}"
project = "${var.project}"
acl_id = "ebefcc96-75f5-11e7-adb3-e2bd27d4977e" # default
static id of cloudstack
depends_on = ["cloudstack_network.public"]
}
# create our network inside the vpc
resource "cloudstack_network" "network2" {
name = "network2"
cidr = "10.230.2.0/24"
gateway = "10.230.2.1"
network_offering = "${var.network_offering_internalwithrouter}"
zone = "${var.zone}"
vpc_id = "${cloudstack_vpc.myvpc.id}"
project = "${var.project}"
acl_id = "ebefcc96-75f5-11e7-adb3-e2bd27d4977e" # default
static id of cloudstack
depends_on = ["cloudstack_network.network1"]
}
# create our network inside the vpc
resource "cloudstack_network" "network3" {
name = "network3"
cidr = "10.230.3.0/24"
gateway = "10.230.3.1"
network_offering = "${var.network_offering_internalwithrouter}"
zone = "${var.zone}"
vpc_id = "${cloudstack_vpc.myvpc.id}"
project = "${var.project}"
acl_id = "ebefcc96-75f5-11e7-adb3-e2bd27d4977e" # default
static id of cloudstack
depends_on = ["cloudstack_network.network2"]
}
# create our network inside the vpc
resource "cloudstack_network" "network4" {
name = "network4"
cidr = "10.230.4.0/24"
gateway = "10.230.4.1"
network_offering = "${var.network_offering_internalwithrouter}"
zone = "${var.zone}"
vpc_id = "${cloudstack_vpc.myvpc.id}"
project = "${var.project}"
acl_id = "ebefcc96-75f5-11e7-adb3-e2bd27d4977e" # default
static id of cloudstack
depends_on = ["cloudstack_network.network3"]
}
# create our network inside the vpc
resource "cloudstack_network" "network5" {
name = "network5"
cidr = "10.230.5.0/24"
gateway = "10.230.5.1"
network_offering = "${var.network_offering_internalwithrouter}"
zone = "${var.zone}"
vpc_id = "${cloudstack_vpc.myvpc.id}"
project = "${var.project}"
acl_id = "ebefcc96-75f5-11e7-adb3-e2bd27d4977e" # default
static id of cloudstack
depends_on = ["cloudstack_network.network4"]
}
# create our network inside the vpc
resource "cloudstack_network" "network6" {
name = "network6"
cidr = "10.230.6.0/24"
gateway = "10.230.6.1"
network_offering = "${var.network_offering_internalwithrouter}"
zone = "${var.zone}"
vpc_id = "${cloudstack_vpc.myvpc.id}"
project = "${var.project}"
acl_id = "ebefcc96-75f5-11e7-adb3-e2bd27d4977e" # default
static id of cloudstack
depends_on = ["cloudstack_network.network5"]
}
And thats the terraform error i get after applying this:
cloudstack_network.network6: Creating...
acl_id: "" => "ebefcc96-75f5-11e7-adb3-e2bd27d4977e"
cidr: "" => "10.230.6.0/24"
display_text: "" => "<computed>"
endip: "" => "<computed>"
gateway: "" => "10.230.6.1"
name: "" => "network6"
network_domain: "" => "<computed>"
network_offering: "" => "VPC Default Network"
project: "" => "Meyer_TF"
startip: "" => "<computed>"
tags.%: "" => "<computed>"
vpc_id: "" => "5a418049-6e9a-49ef-af98-953f53a3262d"
zone: "" => "Germany Nuernberg QSC"
Error applying plan:
1 error(s) occurred:
* cloudstack_network.network6: 1 error(s) occurred:
* cloudstack_network.network6: Error creating network network6: CloudStack
API error 530 (CSExceptionErrorCode: 4250): Internal error executing
command, please contact your system administrator
Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.
All six Networks are deployed, even network6 which throws an error.
I have looked at the cloudstack-management.log and found this, is that a
bug?:
2017-08-14 11:26:28,850 DEBUG [o.a.c.a.BaseCmd]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Ignoring paremeter displaynetwork as the caller is not authorized to pass
it in
2017-08-14 11:26:28,856 DEBUG [o.a.c.a.BaseCmd]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Ignoring paremeter displaynetwork as the caller is not authorized to pass
it in
2017-08-14 11:26:28,869 DEBUG [c.c.u.AccountManagerImpl]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Access granted to
Acct[09422506-a4a4-4a03-9fa4-62f3fcdd23a6-terraform_meyer] to Domain:1/ by
AffinityGroupAccessChecker
2017-08-14 11:26:28,884 DEBUG [c.c.u.AccountManagerImpl]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Access to [VPC [110-myvpc] granted to
Acct[09422506-a4a4-4a03-9fa4-62f3fcdd23a6-terraform_meyer] by DomainChecker
2017-08-14 11:26:28,998 DEBUG [c.c.n.v.VpcManagerImpl]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Releasing lock for [VPC [110-myvpc]
2017-08-14 11:26:29,237 DEBUG [c.c.n.g.BigSwitchBcfGuestNetworkGuru]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Refusing to design this network, the physical isolation type is not
BCF_SEGMENT
2017-08-14 11:26:29,238 DEBUG [o.a.c.n.c.m.ContrailGuru]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Refusing to design this network
2017-08-14 11:26:29,238 DEBUG [c.c.n.g.MidoNetGuestNetworkGuru]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
design called
2017-08-14 11:26:29,239 DEBUG [c.c.n.g.MidoNetGuestNetworkGuru]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Refusing to design this network, the physical isolation type is not MIDO
2017-08-14 11:26:29,240 DEBUG [c.c.n.g.NiciraNvpGuestNetworkGuru]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Refusing to design this network
2017-08-14 11:26:29,241 DEBUG [o.a.c.n.o.OpendaylightGuestNetworkGuru]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Refusing to design this network
2017-08-14 11:26:29,243 DEBUG [c.c.n.g.OvsGuestNetworkGuru]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Refusing to design this network
2017-08-14 11:26:29,417 DEBUG [o.a.c.n.g.SspGuestNetworkGuru]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
SSP not configured to be active
2017-08-14 11:26:29,418 DEBUG [c.c.n.g.BrocadeVcsGuestNetworkGuru]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Refusing to design this network
2017-08-14 11:26:29,419 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Releasing lock for
Acct[0501ceac-6472-4f7f-baa9-3609baf51304-PrjAcct-Meyer_TF-1]
2017-08-14 11:26:29,829 DEBUG [c.c.n.NetworkServiceImpl]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Implementing network Ntwk[401|Guest|46] as a part of network provision for
persistent network
2017-08-14 11:26:29,890 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Lock is acquired for network id 401 as a part of network implement
2017-08-14 11:26:29,890 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Asking ExternalGuestNetworkGuru to implement Ntwk[401|Guest|46]
2017-08-14 11:26:30,183 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Asking VpcVirtualRouter to implemenet Ntwk[401|Guest|46]
2017-08-14 11:26:30,188 INFO [c.c.n.e.VpcVirtualRouterElement]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Adding VPC routers to Guest Network: 1 to be added!
2017-08-14 11:26:30,322 WARN [o.a.c.f.j.AsyncJobExecutionContext]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Job is executed without a context, setup psudo job for the executing thread
2017-08-14 11:26:30,447 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Sync job-4694 execution on object VmWorkJobQueue.152
2017-08-14 11:26:33,895 WARN
[c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Failed to add router VM[DomainRouter|r-152-VM] to network
Ntwk[401|Guest|46] due to
java.lang.RuntimeException: Unexpected job execution result
at
com.cloud.vm.VirtualMachineManagerImpl.addVmToNetwork(VirtualMachineManagerImpl.java:3001)
at
com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.addVpcRouterToGuestNetwork(VpcVirtualNetworkApplianceManagerImpl.java:139)
at sun.reflect.GeneratedMethodAccessor364.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy223.addVpcRouterToGuestNetwork(Unknown Source)
at
com.cloud.network.element.VpcVirtualRouterElement.configureGuestNetwork(VpcVirtualRouterElement.java:227)
at
com.cloud.network.element.VpcVirtualRouterElement.implement(VpcVirtualRouterElement.java:211)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElementsAndResources(NetworkOrchestrator.java:1116)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetwork(NetworkOrchestrator.java:1020)
at
com.cloud.network.NetworkServiceImpl.createGuestNetwork(NetworkServiceImpl.java:1323)
at sun.reflect.GeneratedMethodAccessor380.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at
com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy167.createGuestNetwork(Unknown Source)
at
org.apache.cloudstack.api.command.user.network.CreateNetworkCmd.execute(CreateNetworkCmd.java:304)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:709)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:533)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:299)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:129)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:126)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:88)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1753)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1712)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
2017-08-14 11:26:33,897 DEBUG
[c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Removing the router VM[DomainRouter|r-152-VM] from network
Ntwk[401|Guest|46] as a part of cleanup
2017-08-14 11:26:33,902 DEBUG
[c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
(catalina-exec-82:ctx-7ac528a8 ctx-ea30722a ctx-c6442941) (logid:49de96f6)
Router VM[DomainRouter|r-152-VM] is not a part of the Guest network
Ntwk[401|Guest|46]
Re: Error on Terraform with creating many networks
Posted by Simon Weller <sw...@ena.com.INVALID>.
Dennis,
Cloudstack is an Orchestrator, so it works well with all of the major hypervisors (KVM, XenServer, VMWare et al). Each hypervisor has it's own limitations and all of them work slightly differently in terms of how they support various features.
- Si
________________________________
From: Dennis Meyer <sn...@gmail.com>
Sent: Monday, August 14, 2017 6:32 AM
To: users@cloudstack.apache.org
Subject: Re: Error on Terraform with creating many networks
Is this limitation only on XenServer? What Hypervisor works best with
CloudStack?
2017-08-14 13:27 GMT+02:00 Paul Angus <pa...@shapeblue.com>:
> XenServer only supports 7 vNICs on a guest which obviously limits the
> number of tiers, and although vSphere supports 10, there seems to be a code
> bug which means guests can't start if they have more than 7 NICs attached
> (although you can attach more once the instances has started).
>
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com<http://www.shapeblue.com>
[http://www.shapeblue.com/wp-content/uploads/2017/06/logo.png]<http://www.shapeblue.com/>
Shapeblue - The CloudStack Company<http://www.shapeblue.com/>
www.shapeblue.com
Rapid deployment framework for Apache CloudStack IaaS Clouds. CSForge is a framework developed by ShapeBlue to deliver the rapid deployment of a standardised ...
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
>
>
>
> -----Original Message-----
> From: Rene Moser [mailto:mail@renemoser.net]
> Sent: 14 August 2017 11:12
> To: users@cloudstack.apache.org
> Subject: Re: Error on Terraform with creating many networks
>
> Hi Dennis
>
> On 08/14/2017 11:46 AM, Dennis Meyer wrote:
> > Hi,
> > i try to create a vpc and some networks with terraform.
>
> Not sure but wasn't there a issue with more than (or equal 8) VPC networks?
>
> https://issues.apache.org/jira/browse/CLOUDSTACK-9999
>
> However, not sure which versions of cloudstack is affected.
>
> René
>
>
Re: Error on Terraform with creating many networks
Posted by Dennis Meyer <sn...@gmail.com>.
Is this limitation only on XenServer? What Hypervisor works best with
CloudStack?
2017-08-14 13:27 GMT+02:00 Paul Angus <pa...@shapeblue.com>:
> XenServer only supports 7 vNICs on a guest which obviously limits the
> number of tiers, and although vSphere supports 10, there seems to be a code
> bug which means guests can't start if they have more than 7 NICs attached
> (although you can attach more once the instances has started).
>
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
>
>
>
> -----Original Message-----
> From: Rene Moser [mailto:mail@renemoser.net]
> Sent: 14 August 2017 11:12
> To: users@cloudstack.apache.org
> Subject: Re: Error on Terraform with creating many networks
>
> Hi Dennis
>
> On 08/14/2017 11:46 AM, Dennis Meyer wrote:
> > Hi,
> > i try to create a vpc and some networks with terraform.
>
> Not sure but wasn't there a issue with more than (or equal 8) VPC networks?
>
> https://issues.apache.org/jira/browse/CLOUDSTACK-9999
>
> However, not sure which versions of cloudstack is affected.
>
> René
>
>
RE: Error on Terraform with creating many networks
Posted by Paul Angus <pa...@shapeblue.com>.
XenServer only supports 7 vNICs on a guest which obviously limits the number of tiers, and although vSphere supports 10, there seems to be a code bug which means guests can't start if they have more than 7 NICs attached (although you can attach more once the instances has started).
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: Rene Moser [mailto:mail@renemoser.net]
Sent: 14 August 2017 11:12
To: users@cloudstack.apache.org
Subject: Re: Error on Terraform with creating many networks
Hi Dennis
On 08/14/2017 11:46 AM, Dennis Meyer wrote:
> Hi,
> i try to create a vpc and some networks with terraform.
Not sure but wasn't there a issue with more than (or equal 8) VPC networks?
https://issues.apache.org/jira/browse/CLOUDSTACK-9999
However, not sure which versions of cloudstack is affected.
René
Re: Error on Terraform with creating many networks
Posted by Rene Moser <ma...@renemoser.net>.
Hi Dennis
On 08/14/2017 11:46 AM, Dennis Meyer wrote:
> Hi,
> i try to create a vpc and some networks with terraform.
Not sure but wasn't there a issue with more than (or equal 8) VPC networks?
https://issues.apache.org/jira/browse/CLOUDSTACK-9999
However, not sure which versions of cloudstack is affected.
René