You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by "Vash_X@gmx.de" <Va...@gmx.de> on 2022/01/04 01:49:54 UTC
Public IP is stucked in state "released" after usage for static nat
hi everyone,
sadly i am encountering some problems in my CS 4.16 set-up.
today i wanted to take a look into static-nat for the routers of my vpc.
Sadly i missed, that the VM i wanted to assaign to the static public ip had
was already targeted by an port forwarding.
As you can imagine i got an error that static can't be applied for this VM
/ default interface.
Thats when strange things happend.
The static NAT configuration was successfully applied.
I wanted to revert everything so i tried to delete the static NAT And i got
the following error:
2022-01-04 00:55:58,394 DEBUG [o.a.c.n.t.AdvancedNetworkTopology]
(API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74) (logid:90c06d44)
APPLYING VPC IP RULES
2022-01-04 00:55:58,399 DEBUG [o.a.c.n.t.BasicNetworkTopology]
(API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74) (logid:90c06d44)
Applying vpc ip association in network Ntwk[246|Guest|30]
2022-01-04 00:55:58,442 DEBUG [c.c.a.t.Request]
(API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74) (logid:90c06d44)
Seq 5-4862198747700030042: Sending { Cmd , MgmtId: 90520740759984, via:
5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 100001,
[{"com.cloud.agent.api.routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":"5","publicIp":"10.129.0.37","sourceNat":"true","add":"false","oneToOneNat":"true","firstIP":"true","broadcastUri":"vlan://2990","vlanGateway":"10.129.0.1","vlanNetmask":"255.255.255.128","vifMacAddress":"1e:00:59:00:00:b2","networkRate":"-1","trafficType":"Public","networkName":"br3_CS_Pub","newNic":"false","isPrivateGateway":"false"}],"accessDetails":{"
router.name":"r-62-VM","router.guest.ip":"10.129.0.42","network.public.last.ip":"false","router.ip":"169.254.1.55","zone.network.type":"Advanced"},"wait":"0","bypassHostMaintenance":"false"}}]
}
2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-3:null) (logid:) Seq 5-4862198747700030042:
Processing: { Ans: , MgmtId: 90520740759984, via: 5, Ver: v1, Flags: 0,
[{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null - success:
Creating file in VR, with ip: 169.254.1.55, file:
ip_associations.json.167dc5ba-c89a-48c8-9b99-81b375e59691","null - success:
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
Error: Nexthop has invalid gateway.
"],"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
(API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74) (logid:90c06d44)
Seq 5-4862198747700030042: Received: { Ans: , MgmtId: 90520740759984, via:
5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 0, { GroupAnswer } }
2022-01-04 00:55:59,486 ERROR [c.c.a.ApiAsyncJobDispatcher]
(API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Unexpected
exception while executing
org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd
com.cloud.exception.UnsupportedServiceException: Service UserData is not
supported in the network id=246
at
com.cloud.network.dao.NetworkServiceMapDaoImpl.getProviderForServiceInNetwork(NetworkServiceMapDaoImpl.java:126)
at jdk.internal.reflect.GeneratedMethodAccessor1754.invoke(Unknown Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy61.getProviderForServiceInNetwork(Unknown Source)
at
com.cloud.network.NetworkModelImpl.getUserDataUpdateProvider(NetworkModelImpl.java:902)
at
com.cloud.network.rules.RulesManagerImpl.applyUserData(RulesManagerImpl.java:633)
at
com.cloud.network.rules.RulesManagerImpl.revokeAllPFAndStaticNatRulesForIp(RulesManagerImpl.java:1147)
at
com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1328)
at
com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1290)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at
com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy122.disableStaticNat(Unknown Source)
at
org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd.execute(DisableStaticNatCmd.java:88)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
at
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-01-04 00:55:59,488 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Complete async
job-2135, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Service
UserData is not supported in the network id=246"}
2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Publish async
job-2135 complete on message bus
2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up jobs
related to job-2135
2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Update db
status for job-2135
2022-01-04 00:55:59,490 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up jobs
joined with job-2135 and disjoin all subjobs created from job- 2135
2022-01-04 00:55:59,494 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Done executing
org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd for job-2135
2022-01-04 00:55:59,494 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Remove
job-2135 from job monitoring
As far as i understand there is a problem with the "service user data".
That's correct as i created my own offerings for VPC and the Tier-Networks
which don't offer the service.
How ever now i have the situation that i can't dele the static nat via the
GUI and the public ip is stuck in the state "released", is not availeable
for further usage and the static NAT is still shown as configured.
Maybe someone can point me on how to release the IP properly? May i need to
work directly in the DB to clean this "mess" ?
Any suggestion is appreciated.
With regards,
Chris
Re: Public IP is stucked in state "released" after usage for static nat
Posted by "Vash_X@gmx.de" <Va...@gmx.de>.
Today, after spending some more time investigating the database for
dependencies as well as looking through the live configuration of the VPC
routers, i decided to change the values in the database directly.
Everything seems back to normal. IP is again availeable, can be aquired and
used for portforwarding and released.
Nevertheless i hope someone more qualified and skilled will find some time
to take a look into this.
Thanks for your help,
regards
Chris
Am Mi., 5. Jan. 2022 um 15:31 Uhr schrieb Vash_X@gmx.de <Va...@gmx.de>:
> @ Wei
>
> As suggested i wanted to trie to update the network / tier to an offering
> with userdata.
> Sadly i don't find a way to update the serviceoffering for an existing vpc
> in the gui. Updateing the serviceoffering for the network wasn't
> successfull as well. Got the error that the serviceprovider (the existing
> VPC) doesn't support service user data. Therefore i couldn't update the
> network as well.
> So this seems to be a dead-lock.
>
> Regards,
> Chris
>
> Am Di., 4. Jan. 2022 um 16:41 Uhr schrieb Vash_X@gmx.de <Va...@gmx.de>:
>
>> Hi Wei,
>>
>> Will have to think that through as I am not sure if I could revert back
>> an service offering without Service User data. Also do you guess It could
>> be sufficent to only upgrade the effected network (as the error is pointing
>> at the network id)?
>> Why not using userdata... Well short: not needed in the current usecase
>> of the VPC as it is quiet static in regards of deploying vm's.
>> Also I have to argue such decisions with our compliance team which
>> basically comes to: If not needed then disable it. Especially in Our
>> CUrrent usecase.
>> As the GUI as well as the docs didn't suggested the actual need for this
>> service we are not deploying it.
>>
>>
>> Wei ZHOU <us...@gmail.com> schrieb am Di., 4. Jan. 2022, 14:41:
>>
>>> Hi Chris,
>>>
>>> You can try to update the network/vpc tier to an offering with Userdata
>>> and then disable static nat.
>>> May I ask why you decided not to use Userdata ?
>>>
>>> -Wei
>>>
>>>
>>>
>>>
>>> On Tue, 4 Jan 2022 at 14:35, Vash_X@gmx.de <Va...@gmx.de> wrote:
>>>
>>>> As i will need to fix this - i guess somewhat during the reset of the
>>>> week
>>>> are there any ideas on how to "remove" this from CS? (BackUP Jobs will
>>>> be
>>>> started for DB and i wouldn't have the chance to perform a binarylog
>>>> rollback / recovery)
>>>>
>>>> The "clean" way i guess would be a DB Rollback to a point prior to
>>>> enableing static NAT.
>>>> Taking a look into the DB itself, i only found one entry in the table
>>>> "user_ip_address": Entry with rulestate / state "Releasing" for the ip
>>>> is
>>>> present. one_to_one_nat = 1 and the dnat_vmip is present.
>>>> Does anyone has some experience on cleaning this up directly in db?
>>>>
>>>> Regards,
>>>> Chris
>>>>
>>>> Am Di., 4. Jan. 2022 um 13:16 Uhr schrieb Vash_X@gmx.de <Vash_X@gmx.de
>>>> >:
>>>>
>>>> > Hi Wei,
>>>> >
>>>> > of course! :-)
>>>> >
>>>> > https://github.com/apache/cloudstack/issues/5824
>>>> >
>>>> > Regards,
>>>> > Chris
>>>> >
>>>> > Am Di., 4. Jan. 2022 um 11:32 Uhr schrieb Wei ZHOU <
>>>> ustcweizhou@gmail.com
>>>> > >:
>>>> >
>>>> >> Hi Chris,
>>>> >>
>>>> >> This seems to be a bug. Can you file an issue on
>>>> >> https://github.com/apache/cloudstack/issues ?
>>>> >>
>>>> >> -Wei
>>>> >>
>>>> >> On Tue, 4 Jan 2022 at 11:16, Vash_X@gmx.de <Va...@gmx.de> wrote:
>>>> >>
>>>> >>> Hi Dan,
>>>> >>>
>>>> >>> Sorry for the misunderstanding.
>>>> >>> The VPC was created using a "custome" service offering which is not
>>>> >>> using /
>>>> >>> offering "service user data" . The same goes for the NW-Offering
>>>> which is
>>>> >>> used for creation of the tiers. There is no service user data in
>>>> this
>>>> >>> VPC.
>>>> >>>
>>>> >>> The default / out-of-the-box VPC offering has the service "user
>>>> data"
>>>> >>> included. Same goes for the default nework offering for VPC Tiers.
>>>> So
>>>> >>> this
>>>> >>> service should be supported at all from my understanding.
>>>> >>> That's why this error seems a bit "bogus" to me. I was understanding
>>>> >>> somethin like "static nat can't be disabled due to missing service
>>>> "user
>>>> >>> data" for the network".
>>>> >>>
>>>> >>> Deleting and building the whole thing from scratch will be the last
>>>> >>> resort,
>>>> >>> yes. Even taking the risk that it won't be deletable as the
>>>> allocated
>>>> >>> ressourcces can't be released properly at the moment, e.g. using the
>>>> >>> cloudstack.api.command.user.nat.DisableStaticNatCmd.
>>>> >>> Seems like i found a configuration which doesn't work well but can
>>>> be
>>>> >>> created via the GUI. Even if i don't understand the connection
>>>> between
>>>> >>> the
>>>> >>> service user data and the service stativ nat.
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>> Daan Hoogland <da...@gmail.com> schrieb am Di., 4. Jan.
>>>> 2022,
>>>> >>> 09:21:
>>>> >>>
>>>> >>> > Chris, it looks to me like you should delete the VPC and then
>>>> check if
>>>> >>> all
>>>> >>> > IPs are available before creating a new one.
>>>> >>> >
>>>> >>> > As for the user data service, the message already says that user
>>>> data
>>>> >>> is
>>>> >>> > not supported, so why create another offering?
>>>> >>> >
>>>> >>> > On Tue, Jan 4, 2022 at 2:50 AM Vash_X@gmx.de <Va...@gmx.de>
>>>> wrote:
>>>> >>> >
>>>> >>> >> hi everyone,
>>>> >>> >>
>>>> >>> >> sadly i am encountering some problems in my CS 4.16 set-up.
>>>> >>> >>
>>>> >>> >> today i wanted to take a look into static-nat for the routers of
>>>> my
>>>> >>> vpc.
>>>> >>> >> Sadly i missed, that the VM i wanted to assaign to the static
>>>> public
>>>> >>> ip
>>>> >>> >> had
>>>> >>> >> was already targeted by an port forwarding.
>>>> >>> >> As you can imagine i got an error that static can't be applied
>>>> for
>>>> >>> this VM
>>>> >>> >> / default interface.
>>>> >>> >>
>>>> >>> >> Thats when strange things happend.
>>>> >>> >> The static NAT configuration was successfully applied.
>>>> >>> >> I wanted to revert everything so i tried to delete the static
>>>> NAT And
>>>> >>> i
>>>> >>> >> got
>>>> >>> >> the following error:
>>>> >>> >>
>>>> >>> >> 2022-01-04 00:55:58,394 DEBUG [o.a.c.n.t.AdvancedNetworkTopology]
>>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>>>> >>> (logid:90c06d44)
>>>> >>> >> APPLYING VPC IP RULES
>>>> >>> >> 2022-01-04 00:55:58,399 DEBUG [o.a.c.n.t.BasicNetworkTopology]
>>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>>>> >>> (logid:90c06d44)
>>>> >>> >> Applying vpc ip association in network Ntwk[246|Guest|30]
>>>> >>> >> 2022-01-04 00:55:58,442 DEBUG [c.c.a.t.Request]
>>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>>>> >>> (logid:90c06d44)
>>>> >>> >> Seq 5-4862198747700030042: Sending { Cmd , MgmtId:
>>>> 90520740759984,
>>>> >>> via:
>>>> >>> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 100001,
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> [{"com.cloud.agent.api.routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":"5","publicIp":"10.129.0.37","sourceNat":"true","add":"false","oneToOneNat":"true","firstIP":"true","broadcastUri":"vlan://2990","vlanGateway":"10.129.0.1","vlanNetmask":"255.255.255.128","vifMacAddress":"1e:00:59:00:00:b2","networkRate":"-1","trafficType":"Public","networkName":"br3_CS_Pub","newNic":"false","isPrivateGateway":"false"}],"accessDetails":{"
>>>> >>> >> router.name
>>>> >>> >>
>>>> >>>
>>>> ":"r-62-VM","router.guest.ip":"10.129.0.42","network.public.last.ip":"false","router.ip":"169.254.1.55","zone.network.type":"Advanced"},"wait":"0","bypassHostMaintenance":"false"}}]
>>>> >>> >> }
>>>> >>> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
>>>> >>> >> (AgentManager-Handler-3:null) (logid:) Seq 5-4862198747700030042:
>>>> >>> >> Processing: { Ans: , MgmtId: 90520740759984, via: 5, Ver: v1,
>>>> Flags:
>>>> >>> 0,
>>>> >>> >> [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null -
>>>> >>> success:
>>>> >>> >> Creating file in VR, with ip: 169.254.1.55, file:
>>>> >>> >> ip_associations.json.167dc5ba-c89a-48c8-9b99-81b375e59691","null
>>>> -
>>>> >>> >> success:
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> Error: Nexthop has invalid gateway.
>>>> >>> >> "],"result":"true","wait":"0","bypassHostMaintenance":"false"}}]
>>>> }
>>>> >>> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
>>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>>>> >>> (logid:90c06d44)
>>>> >>> >> Seq 5-4862198747700030042: Received: { Ans: , MgmtId:
>>>> 90520740759984,
>>>> >>> >> via:
>>>> >>> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 0, {
>>>> GroupAnswer } }
>>>> >>> >> 2022-01-04 00:55:59,486 ERROR [c.c.a.ApiAsyncJobDispatcher]
>>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>>>> >>> Unexpected
>>>> >>> >> exception while executing
>>>> >>> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd
>>>> >>> >> com.cloud.exception.UnsupportedServiceException: Service
>>>> UserData is
>>>> >>> not
>>>> >>> >> supported in the network id=246
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> com.cloud.network.dao.NetworkServiceMapDaoImpl.getProviderForServiceInNetwork(NetworkServiceMapDaoImpl.java:126)
>>>> >>> >> at
>>>> jdk.internal.reflect.GeneratedMethodAccessor1754.invoke(Unknown
>>>> >>> Source)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> >>> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
>>>> >>> >> at com.sun.proxy.$Proxy61.getProviderForServiceInNetwork(Unknown
>>>> >>> Source)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> com.cloud.network.NetworkModelImpl.getUserDataUpdateProvider(NetworkModelImpl.java:902)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> com.cloud.network.rules.RulesManagerImpl.applyUserData(RulesManagerImpl.java:633)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> com.cloud.network.rules.RulesManagerImpl.revokeAllPFAndStaticNatRulesForIp(RulesManagerImpl.java:1147)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1328)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1290)
>>>> >>> >> at
>>>> >>>
>>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>> >>> >> Method)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> >>> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
>>>> >>> >> at com.sun.proxy.$Proxy122.disableStaticNat(Unknown Source)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd.execute(DisableStaticNatCmd.java:88)
>>>> >>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
>>>> >>> >> at
>>>> >>> >>
>>>> >>>
>>>> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>>>> >>> >> at
>>>> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>>>> >>> >> at
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>>>> >>> >> at java.base/java.lang.Thread.run(Thread.java:829)
>>>> >>> >> 2022-01-04 00:55:59,488 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>>>> Complete
>>>> >>> >> async
>>>> >>> >> job-2135, jobStatus: FAILED, resultCode: 530, result:
>>>> >>> >>
>>>> >>> >>
>>>> >>>
>>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Service
>>>> >>> >> UserData is not supported in the network id=246"}
>>>> >>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>>>> Publish
>>>> >>> async
>>>> >>> >> job-2135 complete on message bus
>>>> >>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>>>> Wake up
>>>> >>> jobs
>>>> >>> >> related to job-2135
>>>> >>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>>>> Update db
>>>> >>> >> status for job-2135
>>>> >>> >> 2022-01-04 00:55:59,490 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>>>> Wake up
>>>> >>> jobs
>>>> >>> >> joined with job-2135 and disjoin all subjobs created from job-
>>>> 2135
>>>> >>> >> 2022-01-04 00:55:59,494 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Done
>>>> >>> >> executing
>>>> >>> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd
>>>> for
>>>> >>> >> job-2135
>>>> >>> >> 2022-01-04 00:55:59,494 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>>>> Remove
>>>> >>> >> job-2135 from job monitoring
>>>> >>> >>
>>>> >>> >> As far as i understand there is a problem with the "service user
>>>> >>> data".
>>>> >>> >> That's correct as i created my own offerings for VPC and the
>>>> >>> Tier-Networks
>>>> >>> >> which don't offer the service.
>>>> >>> >> How ever now i have the situation that i can't dele the static
>>>> nat
>>>> >>> via the
>>>> >>> >> GUI and the public ip is stuck in the state "released", is not
>>>> >>> availeable
>>>> >>> >> for further usage and the static NAT is still shown as
>>>> configured.
>>>> >>> >>
>>>> >>> >> Maybe someone can point me on how to release the IP properly?
>>>> May i
>>>> >>> need
>>>> >>> >> to
>>>> >>> >> work directly in the DB to clean this "mess" ?
>>>> >>> >> Any suggestion is appreciated.
>>>> >>> >>
>>>> >>> >> With regards,
>>>> >>> >> Chris
>>>> >>> >>
>>>> >>> >
>>>> >>> >
>>>> >>> > --
>>>> >>> > Daan
>>>> >>> >
>>>> >>>
>>>> >>
>>>>
>>>
Re: Public IP is stucked in state "released" after usage for static nat
Posted by "Vash_X@gmx.de" <Va...@gmx.de>.
@ Wei
As suggested i wanted to trie to update the network / tier to an offering
with userdata.
Sadly i don't find a way to update the serviceoffering for an existing vpc
in the gui. Updateing the serviceoffering for the network wasn't
successfull as well. Got the error that the serviceprovider (the existing
VPC) doesn't support service user data. Therefore i couldn't update the
network as well.
So this seems to be a dead-lock.
Regards,
Chris
Am Di., 4. Jan. 2022 um 16:41 Uhr schrieb Vash_X@gmx.de <Va...@gmx.de>:
> Hi Wei,
>
> Will have to think that through as I am not sure if I could revert back an
> service offering without Service User data. Also do you guess It could be
> sufficent to only upgrade the effected network (as the error is pointing at
> the network id)?
> Why not using userdata... Well short: not needed in the current usecase of
> the VPC as it is quiet static in regards of deploying vm's.
> Also I have to argue such decisions with our compliance team which
> basically comes to: If not needed then disable it. Especially in Our
> CUrrent usecase.
> As the GUI as well as the docs didn't suggested the actual need for this
> service we are not deploying it.
>
>
> Wei ZHOU <us...@gmail.com> schrieb am Di., 4. Jan. 2022, 14:41:
>
>> Hi Chris,
>>
>> You can try to update the network/vpc tier to an offering with Userdata
>> and then disable static nat.
>> May I ask why you decided not to use Userdata ?
>>
>> -Wei
>>
>>
>>
>>
>> On Tue, 4 Jan 2022 at 14:35, Vash_X@gmx.de <Va...@gmx.de> wrote:
>>
>>> As i will need to fix this - i guess somewhat during the reset of the
>>> week
>>> are there any ideas on how to "remove" this from CS? (BackUP Jobs will be
>>> started for DB and i wouldn't have the chance to perform a binarylog
>>> rollback / recovery)
>>>
>>> The "clean" way i guess would be a DB Rollback to a point prior to
>>> enableing static NAT.
>>> Taking a look into the DB itself, i only found one entry in the table
>>> "user_ip_address": Entry with rulestate / state "Releasing" for the ip is
>>> present. one_to_one_nat = 1 and the dnat_vmip is present.
>>> Does anyone has some experience on cleaning this up directly in db?
>>>
>>> Regards,
>>> Chris
>>>
>>> Am Di., 4. Jan. 2022 um 13:16 Uhr schrieb Vash_X@gmx.de <Va...@gmx.de>:
>>>
>>> > Hi Wei,
>>> >
>>> > of course! :-)
>>> >
>>> > https://github.com/apache/cloudstack/issues/5824
>>> >
>>> > Regards,
>>> > Chris
>>> >
>>> > Am Di., 4. Jan. 2022 um 11:32 Uhr schrieb Wei ZHOU <
>>> ustcweizhou@gmail.com
>>> > >:
>>> >
>>> >> Hi Chris,
>>> >>
>>> >> This seems to be a bug. Can you file an issue on
>>> >> https://github.com/apache/cloudstack/issues ?
>>> >>
>>> >> -Wei
>>> >>
>>> >> On Tue, 4 Jan 2022 at 11:16, Vash_X@gmx.de <Va...@gmx.de> wrote:
>>> >>
>>> >>> Hi Dan,
>>> >>>
>>> >>> Sorry for the misunderstanding.
>>> >>> The VPC was created using a "custome" service offering which is not
>>> >>> using /
>>> >>> offering "service user data" . The same goes for the NW-Offering
>>> which is
>>> >>> used for creation of the tiers. There is no service user data in this
>>> >>> VPC.
>>> >>>
>>> >>> The default / out-of-the-box VPC offering has the service "user data"
>>> >>> included. Same goes for the default nework offering for VPC Tiers. So
>>> >>> this
>>> >>> service should be supported at all from my understanding.
>>> >>> That's why this error seems a bit "bogus" to me. I was understanding
>>> >>> somethin like "static nat can't be disabled due to missing service
>>> "user
>>> >>> data" for the network".
>>> >>>
>>> >>> Deleting and building the whole thing from scratch will be the last
>>> >>> resort,
>>> >>> yes. Even taking the risk that it won't be deletable as the allocated
>>> >>> ressourcces can't be released properly at the moment, e.g. using the
>>> >>> cloudstack.api.command.user.nat.DisableStaticNatCmd.
>>> >>> Seems like i found a configuration which doesn't work well but can be
>>> >>> created via the GUI. Even if i don't understand the connection
>>> between
>>> >>> the
>>> >>> service user data and the service stativ nat.
>>> >>>
>>> >>>
>>> >>>
>>> >>> Daan Hoogland <da...@gmail.com> schrieb am Di., 4. Jan.
>>> 2022,
>>> >>> 09:21:
>>> >>>
>>> >>> > Chris, it looks to me like you should delete the VPC and then
>>> check if
>>> >>> all
>>> >>> > IPs are available before creating a new one.
>>> >>> >
>>> >>> > As for the user data service, the message already says that user
>>> data
>>> >>> is
>>> >>> > not supported, so why create another offering?
>>> >>> >
>>> >>> > On Tue, Jan 4, 2022 at 2:50 AM Vash_X@gmx.de <Va...@gmx.de>
>>> wrote:
>>> >>> >
>>> >>> >> hi everyone,
>>> >>> >>
>>> >>> >> sadly i am encountering some problems in my CS 4.16 set-up.
>>> >>> >>
>>> >>> >> today i wanted to take a look into static-nat for the routers of
>>> my
>>> >>> vpc.
>>> >>> >> Sadly i missed, that the VM i wanted to assaign to the static
>>> public
>>> >>> ip
>>> >>> >> had
>>> >>> >> was already targeted by an port forwarding.
>>> >>> >> As you can imagine i got an error that static can't be applied for
>>> >>> this VM
>>> >>> >> / default interface.
>>> >>> >>
>>> >>> >> Thats when strange things happend.
>>> >>> >> The static NAT configuration was successfully applied.
>>> >>> >> I wanted to revert everything so i tried to delete the static NAT
>>> And
>>> >>> i
>>> >>> >> got
>>> >>> >> the following error:
>>> >>> >>
>>> >>> >> 2022-01-04 00:55:58,394 DEBUG [o.a.c.n.t.AdvancedNetworkTopology]
>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>>> >>> (logid:90c06d44)
>>> >>> >> APPLYING VPC IP RULES
>>> >>> >> 2022-01-04 00:55:58,399 DEBUG [o.a.c.n.t.BasicNetworkTopology]
>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>>> >>> (logid:90c06d44)
>>> >>> >> Applying vpc ip association in network Ntwk[246|Guest|30]
>>> >>> >> 2022-01-04 00:55:58,442 DEBUG [c.c.a.t.Request]
>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>>> >>> (logid:90c06d44)
>>> >>> >> Seq 5-4862198747700030042: Sending { Cmd , MgmtId:
>>> 90520740759984,
>>> >>> via:
>>> >>> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 100001,
>>> >>> >>
>>> >>> >>
>>> >>>
>>> [{"com.cloud.agent.api.routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":"5","publicIp":"10.129.0.37","sourceNat":"true","add":"false","oneToOneNat":"true","firstIP":"true","broadcastUri":"vlan://2990","vlanGateway":"10.129.0.1","vlanNetmask":"255.255.255.128","vifMacAddress":"1e:00:59:00:00:b2","networkRate":"-1","trafficType":"Public","networkName":"br3_CS_Pub","newNic":"false","isPrivateGateway":"false"}],"accessDetails":{"
>>> >>> >> router.name
>>> >>> >>
>>> >>>
>>> ":"r-62-VM","router.guest.ip":"10.129.0.42","network.public.last.ip":"false","router.ip":"169.254.1.55","zone.network.type":"Advanced"},"wait":"0","bypassHostMaintenance":"false"}}]
>>> >>> >> }
>>> >>> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
>>> >>> >> (AgentManager-Handler-3:null) (logid:) Seq 5-4862198747700030042:
>>> >>> >> Processing: { Ans: , MgmtId: 90520740759984, via: 5, Ver: v1,
>>> Flags:
>>> >>> 0,
>>> >>> >> [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null -
>>> >>> success:
>>> >>> >> Creating file in VR, with ip: 169.254.1.55, file:
>>> >>> >> ip_associations.json.167dc5ba-c89a-48c8-9b99-81b375e59691","null -
>>> >>> >> success:
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> Error: Nexthop has invalid gateway.
>>> >>> >> "],"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
>>> >>> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>>> >>> (logid:90c06d44)
>>> >>> >> Seq 5-4862198747700030042: Received: { Ans: , MgmtId:
>>> 90520740759984,
>>> >>> >> via:
>>> >>> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 0, { GroupAnswer
>>> } }
>>> >>> >> 2022-01-04 00:55:59,486 ERROR [c.c.a.ApiAsyncJobDispatcher]
>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>>> >>> Unexpected
>>> >>> >> exception while executing
>>> >>> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd
>>> >>> >> com.cloud.exception.UnsupportedServiceException: Service UserData
>>> is
>>> >>> not
>>> >>> >> supported in the network id=246
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> com.cloud.network.dao.NetworkServiceMapDaoImpl.getProviderForServiceInNetwork(NetworkServiceMapDaoImpl.java:126)
>>> >>> >> at jdk.internal.reflect.GeneratedMethodAccessor1754.invoke(Unknown
>>> >>> Source)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> >>> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
>>> >>> >> at com.sun.proxy.$Proxy61.getProviderForServiceInNetwork(Unknown
>>> >>> Source)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> com.cloud.network.NetworkModelImpl.getUserDataUpdateProvider(NetworkModelImpl.java:902)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> com.cloud.network.rules.RulesManagerImpl.applyUserData(RulesManagerImpl.java:633)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> com.cloud.network.rules.RulesManagerImpl.revokeAllPFAndStaticNatRulesForIp(RulesManagerImpl.java:1147)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1328)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1290)
>>> >>> >> at
>>> >>>
>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> >>> >> Method)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> >>> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
>>> >>> >> at com.sun.proxy.$Proxy122.disableStaticNat(Unknown Source)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd.execute(DisableStaticNatCmd.java:88)
>>> >>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
>>> >>> >> at
>>> >>> >>
>>> >>>
>>> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>>> >>> >> at
>>> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>>> >>> >> at
>>> >>> >>
>>> >>> >>
>>> >>>
>>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>>> >>> >> at java.base/java.lang.Thread.run(Thread.java:829)
>>> >>> >> 2022-01-04 00:55:59,488 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>>> Complete
>>> >>> >> async
>>> >>> >> job-2135, jobStatus: FAILED, resultCode: 530, result:
>>> >>> >>
>>> >>> >>
>>> >>>
>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Service
>>> >>> >> UserData is not supported in the network id=246"}
>>> >>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>>> Publish
>>> >>> async
>>> >>> >> job-2135 complete on message bus
>>> >>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake
>>> up
>>> >>> jobs
>>> >>> >> related to job-2135
>>> >>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>>> Update db
>>> >>> >> status for job-2135
>>> >>> >> 2022-01-04 00:55:59,490 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake
>>> up
>>> >>> jobs
>>> >>> >> joined with job-2135 and disjoin all subjobs created from job-
>>> 2135
>>> >>> >> 2022-01-04 00:55:59,494 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Done
>>> >>> >> executing
>>> >>> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd for
>>> >>> >> job-2135
>>> >>> >> 2022-01-04 00:55:59,494 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>>> Remove
>>> >>> >> job-2135 from job monitoring
>>> >>> >>
>>> >>> >> As far as i understand there is a problem with the "service user
>>> >>> data".
>>> >>> >> That's correct as i created my own offerings for VPC and the
>>> >>> Tier-Networks
>>> >>> >> which don't offer the service.
>>> >>> >> How ever now i have the situation that i can't dele the static nat
>>> >>> via the
>>> >>> >> GUI and the public ip is stuck in the state "released", is not
>>> >>> availeable
>>> >>> >> for further usage and the static NAT is still shown as configured.
>>> >>> >>
>>> >>> >> Maybe someone can point me on how to release the IP properly? May
>>> i
>>> >>> need
>>> >>> >> to
>>> >>> >> work directly in the DB to clean this "mess" ?
>>> >>> >> Any suggestion is appreciated.
>>> >>> >>
>>> >>> >> With regards,
>>> >>> >> Chris
>>> >>> >>
>>> >>> >
>>> >>> >
>>> >>> > --
>>> >>> > Daan
>>> >>> >
>>> >>>
>>> >>
>>>
>>
Re: Public IP is stucked in state "released" after usage for static nat
Posted by "Vash_X@gmx.de" <Va...@gmx.de>.
Hi Wei,
Will have to think that through as I am not sure if I could revert back an
service offering without Service User data. Also do you guess It could be
sufficent to only upgrade the effected network (as the error is pointing at
the network id)?
Why not using userdata... Well short: not needed in the current usecase of
the VPC as it is quiet static in regards of deploying vm's.
Also I have to argue such decisions with our compliance team which
basically comes to: If not needed then disable it. Especially in Our
CUrrent usecase.
As the GUI as well as the docs didn't suggested the actual need for this
service we are not deploying it.
Wei ZHOU <us...@gmail.com> schrieb am Di., 4. Jan. 2022, 14:41:
> Hi Chris,
>
> You can try to update the network/vpc tier to an offering with Userdata
> and then disable static nat.
> May I ask why you decided not to use Userdata ?
>
> -Wei
>
>
>
>
> On Tue, 4 Jan 2022 at 14:35, Vash_X@gmx.de <Va...@gmx.de> wrote:
>
>> As i will need to fix this - i guess somewhat during the reset of the week
>> are there any ideas on how to "remove" this from CS? (BackUP Jobs will be
>> started for DB and i wouldn't have the chance to perform a binarylog
>> rollback / recovery)
>>
>> The "clean" way i guess would be a DB Rollback to a point prior to
>> enableing static NAT.
>> Taking a look into the DB itself, i only found one entry in the table
>> "user_ip_address": Entry with rulestate / state "Releasing" for the ip is
>> present. one_to_one_nat = 1 and the dnat_vmip is present.
>> Does anyone has some experience on cleaning this up directly in db?
>>
>> Regards,
>> Chris
>>
>> Am Di., 4. Jan. 2022 um 13:16 Uhr schrieb Vash_X@gmx.de <Va...@gmx.de>:
>>
>> > Hi Wei,
>> >
>> > of course! :-)
>> >
>> > https://github.com/apache/cloudstack/issues/5824
>> >
>> > Regards,
>> > Chris
>> >
>> > Am Di., 4. Jan. 2022 um 11:32 Uhr schrieb Wei ZHOU <
>> ustcweizhou@gmail.com
>> > >:
>> >
>> >> Hi Chris,
>> >>
>> >> This seems to be a bug. Can you file an issue on
>> >> https://github.com/apache/cloudstack/issues ?
>> >>
>> >> -Wei
>> >>
>> >> On Tue, 4 Jan 2022 at 11:16, Vash_X@gmx.de <Va...@gmx.de> wrote:
>> >>
>> >>> Hi Dan,
>> >>>
>> >>> Sorry for the misunderstanding.
>> >>> The VPC was created using a "custome" service offering which is not
>> >>> using /
>> >>> offering "service user data" . The same goes for the NW-Offering
>> which is
>> >>> used for creation of the tiers. There is no service user data in this
>> >>> VPC.
>> >>>
>> >>> The default / out-of-the-box VPC offering has the service "user data"
>> >>> included. Same goes for the default nework offering for VPC Tiers. So
>> >>> this
>> >>> service should be supported at all from my understanding.
>> >>> That's why this error seems a bit "bogus" to me. I was understanding
>> >>> somethin like "static nat can't be disabled due to missing service
>> "user
>> >>> data" for the network".
>> >>>
>> >>> Deleting and building the whole thing from scratch will be the last
>> >>> resort,
>> >>> yes. Even taking the risk that it won't be deletable as the allocated
>> >>> ressourcces can't be released properly at the moment, e.g. using the
>> >>> cloudstack.api.command.user.nat.DisableStaticNatCmd.
>> >>> Seems like i found a configuration which doesn't work well but can be
>> >>> created via the GUI. Even if i don't understand the connection between
>> >>> the
>> >>> service user data and the service stativ nat.
>> >>>
>> >>>
>> >>>
>> >>> Daan Hoogland <da...@gmail.com> schrieb am Di., 4. Jan. 2022,
>> >>> 09:21:
>> >>>
>> >>> > Chris, it looks to me like you should delete the VPC and then check
>> if
>> >>> all
>> >>> > IPs are available before creating a new one.
>> >>> >
>> >>> > As for the user data service, the message already says that user
>> data
>> >>> is
>> >>> > not supported, so why create another offering?
>> >>> >
>> >>> > On Tue, Jan 4, 2022 at 2:50 AM Vash_X@gmx.de <Va...@gmx.de> wrote:
>> >>> >
>> >>> >> hi everyone,
>> >>> >>
>> >>> >> sadly i am encountering some problems in my CS 4.16 set-up.
>> >>> >>
>> >>> >> today i wanted to take a look into static-nat for the routers of my
>> >>> vpc.
>> >>> >> Sadly i missed, that the VM i wanted to assaign to the static
>> public
>> >>> ip
>> >>> >> had
>> >>> >> was already targeted by an port forwarding.
>> >>> >> As you can imagine i got an error that static can't be applied for
>> >>> this VM
>> >>> >> / default interface.
>> >>> >>
>> >>> >> Thats when strange things happend.
>> >>> >> The static NAT configuration was successfully applied.
>> >>> >> I wanted to revert everything so i tried to delete the static NAT
>> And
>> >>> i
>> >>> >> got
>> >>> >> the following error:
>> >>> >>
>> >>> >> 2022-01-04 00:55:58,394 DEBUG [o.a.c.n.t.AdvancedNetworkTopology]
>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>> >>> (logid:90c06d44)
>> >>> >> APPLYING VPC IP RULES
>> >>> >> 2022-01-04 00:55:58,399 DEBUG [o.a.c.n.t.BasicNetworkTopology]
>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>> >>> (logid:90c06d44)
>> >>> >> Applying vpc ip association in network Ntwk[246|Guest|30]
>> >>> >> 2022-01-04 00:55:58,442 DEBUG [c.c.a.t.Request]
>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>> >>> (logid:90c06d44)
>> >>> >> Seq 5-4862198747700030042: Sending { Cmd , MgmtId: 90520740759984,
>> >>> via:
>> >>> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 100001,
>> >>> >>
>> >>> >>
>> >>>
>> [{"com.cloud.agent.api.routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":"5","publicIp":"10.129.0.37","sourceNat":"true","add":"false","oneToOneNat":"true","firstIP":"true","broadcastUri":"vlan://2990","vlanGateway":"10.129.0.1","vlanNetmask":"255.255.255.128","vifMacAddress":"1e:00:59:00:00:b2","networkRate":"-1","trafficType":"Public","networkName":"br3_CS_Pub","newNic":"false","isPrivateGateway":"false"}],"accessDetails":{"
>> >>> >> router.name
>> >>> >>
>> >>>
>> ":"r-62-VM","router.guest.ip":"10.129.0.42","network.public.last.ip":"false","router.ip":"169.254.1.55","zone.network.type":"Advanced"},"wait":"0","bypassHostMaintenance":"false"}}]
>> >>> >> }
>> >>> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
>> >>> >> (AgentManager-Handler-3:null) (logid:) Seq 5-4862198747700030042:
>> >>> >> Processing: { Ans: , MgmtId: 90520740759984, via: 5, Ver: v1,
>> Flags:
>> >>> 0,
>> >>> >> [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null -
>> >>> success:
>> >>> >> Creating file in VR, with ip: 169.254.1.55, file:
>> >>> >> ip_associations.json.167dc5ba-c89a-48c8-9b99-81b375e59691","null -
>> >>> >> success:
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> Error: Nexthop has invalid gateway.
>> >>> >> "],"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
>> >>> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>> >>> (logid:90c06d44)
>> >>> >> Seq 5-4862198747700030042: Received: { Ans: , MgmtId:
>> 90520740759984,
>> >>> >> via:
>> >>> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 0, { GroupAnswer
>> } }
>> >>> >> 2022-01-04 00:55:59,486 ERROR [c.c.a.ApiAsyncJobDispatcher]
>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>> >>> Unexpected
>> >>> >> exception while executing
>> >>> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd
>> >>> >> com.cloud.exception.UnsupportedServiceException: Service UserData
>> is
>> >>> not
>> >>> >> supported in the network id=246
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> com.cloud.network.dao.NetworkServiceMapDaoImpl.getProviderForServiceInNetwork(NetworkServiceMapDaoImpl.java:126)
>> >>> >> at jdk.internal.reflect.GeneratedMethodAccessor1754.invoke(Unknown
>> >>> Source)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >>> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
>> >>> >> at com.sun.proxy.$Proxy61.getProviderForServiceInNetwork(Unknown
>> >>> Source)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> com.cloud.network.NetworkModelImpl.getUserDataUpdateProvider(NetworkModelImpl.java:902)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> com.cloud.network.rules.RulesManagerImpl.applyUserData(RulesManagerImpl.java:633)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> com.cloud.network.rules.RulesManagerImpl.revokeAllPFAndStaticNatRulesForIp(RulesManagerImpl.java:1147)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1328)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1290)
>> >>> >> at
>> >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
>> >>> >> Method)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >>> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
>> >>> >> at com.sun.proxy.$Proxy122.disableStaticNat(Unknown Source)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd.execute(DisableStaticNatCmd.java:88)
>> >>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
>> >>> >> at
>> >>> >>
>> >>>
>> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>> >>> >> at
>> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>> >>> >> at
>> >>> >>
>> >>> >>
>> >>>
>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>> >>> >> at java.base/java.lang.Thread.run(Thread.java:829)
>> >>> >> 2022-01-04 00:55:59,488 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>> Complete
>> >>> >> async
>> >>> >> job-2135, jobStatus: FAILED, resultCode: 530, result:
>> >>> >>
>> >>> >>
>> >>>
>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Service
>> >>> >> UserData is not supported in the network id=246"}
>> >>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>> Publish
>> >>> async
>> >>> >> job-2135 complete on message bus
>> >>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake
>> up
>> >>> jobs
>> >>> >> related to job-2135
>> >>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>> Update db
>> >>> >> status for job-2135
>> >>> >> 2022-01-04 00:55:59,490 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake
>> up
>> >>> jobs
>> >>> >> joined with job-2135 and disjoin all subjobs created from job- 2135
>> >>> >> 2022-01-04 00:55:59,494 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Done
>> >>> >> executing
>> >>> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd for
>> >>> >> job-2135
>> >>> >> 2022-01-04 00:55:59,494 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Remove
>> >>> >> job-2135 from job monitoring
>> >>> >>
>> >>> >> As far as i understand there is a problem with the "service user
>> >>> data".
>> >>> >> That's correct as i created my own offerings for VPC and the
>> >>> Tier-Networks
>> >>> >> which don't offer the service.
>> >>> >> How ever now i have the situation that i can't dele the static nat
>> >>> via the
>> >>> >> GUI and the public ip is stuck in the state "released", is not
>> >>> availeable
>> >>> >> for further usage and the static NAT is still shown as configured.
>> >>> >>
>> >>> >> Maybe someone can point me on how to release the IP properly? May i
>> >>> need
>> >>> >> to
>> >>> >> work directly in the DB to clean this "mess" ?
>> >>> >> Any suggestion is appreciated.
>> >>> >>
>> >>> >> With regards,
>> >>> >> Chris
>> >>> >>
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Daan
>> >>> >
>> >>>
>> >>
>>
>
Re: Public IP is stucked in state "released" after usage for static nat
Posted by Wei ZHOU <us...@gmail.com>.
Hi Chris,
You can try to update the network/vpc tier to an offering with Userdata and
then disable static nat.
May I ask why you decided not to use Userdata ?
-Wei
On Tue, 4 Jan 2022 at 14:35, Vash_X@gmx.de <Va...@gmx.de> wrote:
> As i will need to fix this - i guess somewhat during the reset of the week
> are there any ideas on how to "remove" this from CS? (BackUP Jobs will be
> started for DB and i wouldn't have the chance to perform a binarylog
> rollback / recovery)
>
> The "clean" way i guess would be a DB Rollback to a point prior to
> enableing static NAT.
> Taking a look into the DB itself, i only found one entry in the table
> "user_ip_address": Entry with rulestate / state "Releasing" for the ip is
> present. one_to_one_nat = 1 and the dnat_vmip is present.
> Does anyone has some experience on cleaning this up directly in db?
>
> Regards,
> Chris
>
> Am Di., 4. Jan. 2022 um 13:16 Uhr schrieb Vash_X@gmx.de <Va...@gmx.de>:
>
> > Hi Wei,
> >
> > of course! :-)
> >
> > https://github.com/apache/cloudstack/issues/5824
> >
> > Regards,
> > Chris
> >
> > Am Di., 4. Jan. 2022 um 11:32 Uhr schrieb Wei ZHOU <
> ustcweizhou@gmail.com
> > >:
> >
> >> Hi Chris,
> >>
> >> This seems to be a bug. Can you file an issue on
> >> https://github.com/apache/cloudstack/issues ?
> >>
> >> -Wei
> >>
> >> On Tue, 4 Jan 2022 at 11:16, Vash_X@gmx.de <Va...@gmx.de> wrote:
> >>
> >>> Hi Dan,
> >>>
> >>> Sorry for the misunderstanding.
> >>> The VPC was created using a "custome" service offering which is not
> >>> using /
> >>> offering "service user data" . The same goes for the NW-Offering which
> is
> >>> used for creation of the tiers. There is no service user data in this
> >>> VPC.
> >>>
> >>> The default / out-of-the-box VPC offering has the service "user data"
> >>> included. Same goes for the default nework offering for VPC Tiers. So
> >>> this
> >>> service should be supported at all from my understanding.
> >>> That's why this error seems a bit "bogus" to me. I was understanding
> >>> somethin like "static nat can't be disabled due to missing service
> "user
> >>> data" for the network".
> >>>
> >>> Deleting and building the whole thing from scratch will be the last
> >>> resort,
> >>> yes. Even taking the risk that it won't be deletable as the allocated
> >>> ressourcces can't be released properly at the moment, e.g. using the
> >>> cloudstack.api.command.user.nat.DisableStaticNatCmd.
> >>> Seems like i found a configuration which doesn't work well but can be
> >>> created via the GUI. Even if i don't understand the connection between
> >>> the
> >>> service user data and the service stativ nat.
> >>>
> >>>
> >>>
> >>> Daan Hoogland <da...@gmail.com> schrieb am Di., 4. Jan. 2022,
> >>> 09:21:
> >>>
> >>> > Chris, it looks to me like you should delete the VPC and then check
> if
> >>> all
> >>> > IPs are available before creating a new one.
> >>> >
> >>> > As for the user data service, the message already says that user data
> >>> is
> >>> > not supported, so why create another offering?
> >>> >
> >>> > On Tue, Jan 4, 2022 at 2:50 AM Vash_X@gmx.de <Va...@gmx.de> wrote:
> >>> >
> >>> >> hi everyone,
> >>> >>
> >>> >> sadly i am encountering some problems in my CS 4.16 set-up.
> >>> >>
> >>> >> today i wanted to take a look into static-nat for the routers of my
> >>> vpc.
> >>> >> Sadly i missed, that the VM i wanted to assaign to the static public
> >>> ip
> >>> >> had
> >>> >> was already targeted by an port forwarding.
> >>> >> As you can imagine i got an error that static can't be applied for
> >>> this VM
> >>> >> / default interface.
> >>> >>
> >>> >> Thats when strange things happend.
> >>> >> The static NAT configuration was successfully applied.
> >>> >> I wanted to revert everything so i tried to delete the static NAT
> And
> >>> i
> >>> >> got
> >>> >> the following error:
> >>> >>
> >>> >> 2022-01-04 00:55:58,394 DEBUG [o.a.c.n.t.AdvancedNetworkTopology]
> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
> >>> (logid:90c06d44)
> >>> >> APPLYING VPC IP RULES
> >>> >> 2022-01-04 00:55:58,399 DEBUG [o.a.c.n.t.BasicNetworkTopology]
> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
> >>> (logid:90c06d44)
> >>> >> Applying vpc ip association in network Ntwk[246|Guest|30]
> >>> >> 2022-01-04 00:55:58,442 DEBUG [c.c.a.t.Request]
> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
> >>> (logid:90c06d44)
> >>> >> Seq 5-4862198747700030042: Sending { Cmd , MgmtId: 90520740759984,
> >>> via:
> >>> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 100001,
> >>> >>
> >>> >>
> >>>
> [{"com.cloud.agent.api.routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":"5","publicIp":"10.129.0.37","sourceNat":"true","add":"false","oneToOneNat":"true","firstIP":"true","broadcastUri":"vlan://2990","vlanGateway":"10.129.0.1","vlanNetmask":"255.255.255.128","vifMacAddress":"1e:00:59:00:00:b2","networkRate":"-1","trafficType":"Public","networkName":"br3_CS_Pub","newNic":"false","isPrivateGateway":"false"}],"accessDetails":{"
> >>> >> router.name
> >>> >>
> >>>
> ":"r-62-VM","router.guest.ip":"10.129.0.42","network.public.last.ip":"false","router.ip":"169.254.1.55","zone.network.type":"Advanced"},"wait":"0","bypassHostMaintenance":"false"}}]
> >>> >> }
> >>> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
> >>> >> (AgentManager-Handler-3:null) (logid:) Seq 5-4862198747700030042:
> >>> >> Processing: { Ans: , MgmtId: 90520740759984, via: 5, Ver: v1,
> Flags:
> >>> 0,
> >>> >> [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null -
> >>> success:
> >>> >> Creating file in VR, with ip: 169.254.1.55, file:
> >>> >> ip_associations.json.167dc5ba-c89a-48c8-9b99-81b375e59691","null -
> >>> >> success:
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> Error: Nexthop has invalid gateway.
> >>> >> "],"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
> >>> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
> >>> (logid:90c06d44)
> >>> >> Seq 5-4862198747700030042: Received: { Ans: , MgmtId:
> 90520740759984,
> >>> >> via:
> >>> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 0, { GroupAnswer }
> }
> >>> >> 2022-01-04 00:55:59,486 ERROR [c.c.a.ApiAsyncJobDispatcher]
> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
> >>> Unexpected
> >>> >> exception while executing
> >>> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd
> >>> >> com.cloud.exception.UnsupportedServiceException: Service UserData is
> >>> not
> >>> >> supported in the network id=246
> >>> >> at
> >>> >>
> >>> >>
> >>>
> com.cloud.network.dao.NetworkServiceMapDaoImpl.getProviderForServiceInNetwork(NetworkServiceMapDaoImpl.java:126)
> >>> >> at jdk.internal.reflect.GeneratedMethodAccessor1754.invoke(Unknown
> >>> Source)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
> >>> >> at com.sun.proxy.$Proxy61.getProviderForServiceInNetwork(Unknown
> >>> Source)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> com.cloud.network.NetworkModelImpl.getUserDataUpdateProvider(NetworkModelImpl.java:902)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> com.cloud.network.rules.RulesManagerImpl.applyUserData(RulesManagerImpl.java:633)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> com.cloud.network.rules.RulesManagerImpl.revokeAllPFAndStaticNatRulesForIp(RulesManagerImpl.java:1147)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1328)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1290)
> >>> >> at
> >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> >>> >> Method)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
> >>> >> at com.sun.proxy.$Proxy122.disableStaticNat(Unknown Source)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd.execute(DisableStaticNatCmd.java:88)
> >>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
> >>> >> at
> >>> >>
> >>>
> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> >>> >> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> >>> >> at
> >>> >>
> >>> >>
> >>>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >>> >> at java.base/java.lang.Thread.run(Thread.java:829)
> >>> >> 2022-01-04 00:55:59,488 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
> Complete
> >>> >> async
> >>> >> job-2135, jobStatus: FAILED, resultCode: 530, result:
> >>> >>
> >>> >>
> >>>
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Service
> >>> >> UserData is not supported in the network id=246"}
> >>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Publish
> >>> async
> >>> >> job-2135 complete on message bus
> >>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up
> >>> jobs
> >>> >> related to job-2135
> >>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Update
> db
> >>> >> status for job-2135
> >>> >> 2022-01-04 00:55:59,490 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up
> >>> jobs
> >>> >> joined with job-2135 and disjoin all subjobs created from job- 2135
> >>> >> 2022-01-04 00:55:59,494 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Done
> >>> >> executing
> >>> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd for
> >>> >> job-2135
> >>> >> 2022-01-04 00:55:59,494 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> >>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Remove
> >>> >> job-2135 from job monitoring
> >>> >>
> >>> >> As far as i understand there is a problem with the "service user
> >>> data".
> >>> >> That's correct as i created my own offerings for VPC and the
> >>> Tier-Networks
> >>> >> which don't offer the service.
> >>> >> How ever now i have the situation that i can't dele the static nat
> >>> via the
> >>> >> GUI and the public ip is stuck in the state "released", is not
> >>> availeable
> >>> >> for further usage and the static NAT is still shown as configured.
> >>> >>
> >>> >> Maybe someone can point me on how to release the IP properly? May i
> >>> need
> >>> >> to
> >>> >> work directly in the DB to clean this "mess" ?
> >>> >> Any suggestion is appreciated.
> >>> >>
> >>> >> With regards,
> >>> >> Chris
> >>> >>
> >>> >
> >>> >
> >>> > --
> >>> > Daan
> >>> >
> >>>
> >>
>
Re: Public IP is stucked in state "released" after usage for static nat
Posted by "Vash_X@gmx.de" <Va...@gmx.de>.
As i will need to fix this - i guess somewhat during the reset of the week
are there any ideas on how to "remove" this from CS? (BackUP Jobs will be
started for DB and i wouldn't have the chance to perform a binarylog
rollback / recovery)
The "clean" way i guess would be a DB Rollback to a point prior to
enableing static NAT.
Taking a look into the DB itself, i only found one entry in the table
"user_ip_address": Entry with rulestate / state "Releasing" for the ip is
present. one_to_one_nat = 1 and the dnat_vmip is present.
Does anyone has some experience on cleaning this up directly in db?
Regards,
Chris
Am Di., 4. Jan. 2022 um 13:16 Uhr schrieb Vash_X@gmx.de <Va...@gmx.de>:
> Hi Wei,
>
> of course! :-)
>
> https://github.com/apache/cloudstack/issues/5824
>
> Regards,
> Chris
>
> Am Di., 4. Jan. 2022 um 11:32 Uhr schrieb Wei ZHOU <ustcweizhou@gmail.com
> >:
>
>> Hi Chris,
>>
>> This seems to be a bug. Can you file an issue on
>> https://github.com/apache/cloudstack/issues ?
>>
>> -Wei
>>
>> On Tue, 4 Jan 2022 at 11:16, Vash_X@gmx.de <Va...@gmx.de> wrote:
>>
>>> Hi Dan,
>>>
>>> Sorry for the misunderstanding.
>>> The VPC was created using a "custome" service offering which is not
>>> using /
>>> offering "service user data" . The same goes for the NW-Offering which is
>>> used for creation of the tiers. There is no service user data in this
>>> VPC.
>>>
>>> The default / out-of-the-box VPC offering has the service "user data"
>>> included. Same goes for the default nework offering for VPC Tiers. So
>>> this
>>> service should be supported at all from my understanding.
>>> That's why this error seems a bit "bogus" to me. I was understanding
>>> somethin like "static nat can't be disabled due to missing service "user
>>> data" for the network".
>>>
>>> Deleting and building the whole thing from scratch will be the last
>>> resort,
>>> yes. Even taking the risk that it won't be deletable as the allocated
>>> ressourcces can't be released properly at the moment, e.g. using the
>>> cloudstack.api.command.user.nat.DisableStaticNatCmd.
>>> Seems like i found a configuration which doesn't work well but can be
>>> created via the GUI. Even if i don't understand the connection between
>>> the
>>> service user data and the service stativ nat.
>>>
>>>
>>>
>>> Daan Hoogland <da...@gmail.com> schrieb am Di., 4. Jan. 2022,
>>> 09:21:
>>>
>>> > Chris, it looks to me like you should delete the VPC and then check if
>>> all
>>> > IPs are available before creating a new one.
>>> >
>>> > As for the user data service, the message already says that user data
>>> is
>>> > not supported, so why create another offering?
>>> >
>>> > On Tue, Jan 4, 2022 at 2:50 AM Vash_X@gmx.de <Va...@gmx.de> wrote:
>>> >
>>> >> hi everyone,
>>> >>
>>> >> sadly i am encountering some problems in my CS 4.16 set-up.
>>> >>
>>> >> today i wanted to take a look into static-nat for the routers of my
>>> vpc.
>>> >> Sadly i missed, that the VM i wanted to assaign to the static public
>>> ip
>>> >> had
>>> >> was already targeted by an port forwarding.
>>> >> As you can imagine i got an error that static can't be applied for
>>> this VM
>>> >> / default interface.
>>> >>
>>> >> Thats when strange things happend.
>>> >> The static NAT configuration was successfully applied.
>>> >> I wanted to revert everything so i tried to delete the static NAT And
>>> i
>>> >> got
>>> >> the following error:
>>> >>
>>> >> 2022-01-04 00:55:58,394 DEBUG [o.a.c.n.t.AdvancedNetworkTopology]
>>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>>> (logid:90c06d44)
>>> >> APPLYING VPC IP RULES
>>> >> 2022-01-04 00:55:58,399 DEBUG [o.a.c.n.t.BasicNetworkTopology]
>>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>>> (logid:90c06d44)
>>> >> Applying vpc ip association in network Ntwk[246|Guest|30]
>>> >> 2022-01-04 00:55:58,442 DEBUG [c.c.a.t.Request]
>>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>>> (logid:90c06d44)
>>> >> Seq 5-4862198747700030042: Sending { Cmd , MgmtId: 90520740759984,
>>> via:
>>> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 100001,
>>> >>
>>> >>
>>> [{"com.cloud.agent.api.routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":"5","publicIp":"10.129.0.37","sourceNat":"true","add":"false","oneToOneNat":"true","firstIP":"true","broadcastUri":"vlan://2990","vlanGateway":"10.129.0.1","vlanNetmask":"255.255.255.128","vifMacAddress":"1e:00:59:00:00:b2","networkRate":"-1","trafficType":"Public","networkName":"br3_CS_Pub","newNic":"false","isPrivateGateway":"false"}],"accessDetails":{"
>>> >> router.name
>>> >>
>>> ":"r-62-VM","router.guest.ip":"10.129.0.42","network.public.last.ip":"false","router.ip":"169.254.1.55","zone.network.type":"Advanced"},"wait":"0","bypassHostMaintenance":"false"}}]
>>> >> }
>>> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
>>> >> (AgentManager-Handler-3:null) (logid:) Seq 5-4862198747700030042:
>>> >> Processing: { Ans: , MgmtId: 90520740759984, via: 5, Ver: v1, Flags:
>>> 0,
>>> >> [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null -
>>> success:
>>> >> Creating file in VR, with ip: 169.254.1.55, file:
>>> >> ip_associations.json.167dc5ba-c89a-48c8-9b99-81b375e59691","null -
>>> >> success:
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> Error: Nexthop has invalid gateway.
>>> >> "],"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
>>> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
>>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>>> (logid:90c06d44)
>>> >> Seq 5-4862198747700030042: Received: { Ans: , MgmtId: 90520740759984,
>>> >> via:
>>> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 0, { GroupAnswer } }
>>> >> 2022-01-04 00:55:59,486 ERROR [c.c.a.ApiAsyncJobDispatcher]
>>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44)
>>> Unexpected
>>> >> exception while executing
>>> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd
>>> >> com.cloud.exception.UnsupportedServiceException: Service UserData is
>>> not
>>> >> supported in the network id=246
>>> >> at
>>> >>
>>> >>
>>> com.cloud.network.dao.NetworkServiceMapDaoImpl.getProviderForServiceInNetwork(NetworkServiceMapDaoImpl.java:126)
>>> >> at jdk.internal.reflect.GeneratedMethodAccessor1754.invoke(Unknown
>>> Source)
>>> >> at
>>> >>
>>> >>
>>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>>> >> at
>>> >>
>>> >>
>>> com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
>>> >> at com.sun.proxy.$Proxy61.getProviderForServiceInNetwork(Unknown
>>> Source)
>>> >> at
>>> >>
>>> >>
>>> com.cloud.network.NetworkModelImpl.getUserDataUpdateProvider(NetworkModelImpl.java:902)
>>> >> at
>>> >>
>>> >>
>>> com.cloud.network.rules.RulesManagerImpl.applyUserData(RulesManagerImpl.java:633)
>>> >> at
>>> >>
>>> >>
>>> com.cloud.network.rules.RulesManagerImpl.revokeAllPFAndStaticNatRulesForIp(RulesManagerImpl.java:1147)
>>> >> at
>>> >>
>>> >>
>>> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1328)
>>> >> at
>>> >>
>>> >>
>>> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1290)
>>> >> at
>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> >> Method)
>>> >> at
>>> >>
>>> >>
>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>> >> at
>>> >>
>>> >>
>>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>>> >> at
>>> >>
>>> >>
>>> org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>>> >> at
>>> >>
>>> >>
>>> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>>> >> at
>>> >>
>>> >>
>>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
>>> >> at com.sun.proxy.$Proxy122.disableStaticNat(Unknown Source)
>>> >> at
>>> >>
>>> >>
>>> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd.execute(DisableStaticNatCmd.java:88)
>>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
>>> >> at
>>> >>
>>> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
>>> >> at
>>> >>
>>> >>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
>>> >> at
>>> >>
>>> >>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
>>> >> at
>>> >>
>>> >>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
>>> >> at
>>> >>
>>> >>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
>>> >> at
>>> >>
>>> >>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
>>> >> at
>>> >>
>>> >>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
>>> >> at
>>> >>
>>> >>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
>>> >> at
>>> >>
>>> >>
>>> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>>> >> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>>> >> at
>>> >>
>>> >>
>>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>>> >> at
>>> >>
>>> >>
>>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>>> >> at java.base/java.lang.Thread.run(Thread.java:829)
>>> >> 2022-01-04 00:55:59,488 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Complete
>>> >> async
>>> >> job-2135, jobStatus: FAILED, resultCode: 530, result:
>>> >>
>>> >>
>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Service
>>> >> UserData is not supported in the network id=246"}
>>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Publish
>>> async
>>> >> job-2135 complete on message bus
>>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up
>>> jobs
>>> >> related to job-2135
>>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Update db
>>> >> status for job-2135
>>> >> 2022-01-04 00:55:59,490 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up
>>> jobs
>>> >> joined with job-2135 and disjoin all subjobs created from job- 2135
>>> >> 2022-01-04 00:55:59,494 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Done
>>> >> executing
>>> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd for
>>> >> job-2135
>>> >> 2022-01-04 00:55:59,494 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Remove
>>> >> job-2135 from job monitoring
>>> >>
>>> >> As far as i understand there is a problem with the "service user
>>> data".
>>> >> That's correct as i created my own offerings for VPC and the
>>> Tier-Networks
>>> >> which don't offer the service.
>>> >> How ever now i have the situation that i can't dele the static nat
>>> via the
>>> >> GUI and the public ip is stuck in the state "released", is not
>>> availeable
>>> >> for further usage and the static NAT is still shown as configured.
>>> >>
>>> >> Maybe someone can point me on how to release the IP properly? May i
>>> need
>>> >> to
>>> >> work directly in the DB to clean this "mess" ?
>>> >> Any suggestion is appreciated.
>>> >>
>>> >> With regards,
>>> >> Chris
>>> >>
>>> >
>>> >
>>> > --
>>> > Daan
>>> >
>>>
>>
Re: Public IP is stucked in state "released" after usage for static nat
Posted by "Vash_X@gmx.de" <Va...@gmx.de>.
Hi Wei,
of course! :-)
https://github.com/apache/cloudstack/issues/5824
Regards,
Chris
Am Di., 4. Jan. 2022 um 11:32 Uhr schrieb Wei ZHOU <us...@gmail.com>:
> Hi Chris,
>
> This seems to be a bug. Can you file an issue on
> https://github.com/apache/cloudstack/issues ?
>
> -Wei
>
> On Tue, 4 Jan 2022 at 11:16, Vash_X@gmx.de <Va...@gmx.de> wrote:
>
>> Hi Dan,
>>
>> Sorry for the misunderstanding.
>> The VPC was created using a "custome" service offering which is not using
>> /
>> offering "service user data" . The same goes for the NW-Offering which is
>> used for creation of the tiers. There is no service user data in this VPC.
>>
>> The default / out-of-the-box VPC offering has the service "user data"
>> included. Same goes for the default nework offering for VPC Tiers. So this
>> service should be supported at all from my understanding.
>> That's why this error seems a bit "bogus" to me. I was understanding
>> somethin like "static nat can't be disabled due to missing service "user
>> data" for the network".
>>
>> Deleting and building the whole thing from scratch will be the last
>> resort,
>> yes. Even taking the risk that it won't be deletable as the allocated
>> ressourcces can't be released properly at the moment, e.g. using the
>> cloudstack.api.command.user.nat.DisableStaticNatCmd.
>> Seems like i found a configuration which doesn't work well but can be
>> created via the GUI. Even if i don't understand the connection between the
>> service user data and the service stativ nat.
>>
>>
>>
>> Daan Hoogland <da...@gmail.com> schrieb am Di., 4. Jan. 2022,
>> 09:21:
>>
>> > Chris, it looks to me like you should delete the VPC and then check if
>> all
>> > IPs are available before creating a new one.
>> >
>> > As for the user data service, the message already says that user data is
>> > not supported, so why create another offering?
>> >
>> > On Tue, Jan 4, 2022 at 2:50 AM Vash_X@gmx.de <Va...@gmx.de> wrote:
>> >
>> >> hi everyone,
>> >>
>> >> sadly i am encountering some problems in my CS 4.16 set-up.
>> >>
>> >> today i wanted to take a look into static-nat for the routers of my
>> vpc.
>> >> Sadly i missed, that the VM i wanted to assaign to the static public ip
>> >> had
>> >> was already targeted by an port forwarding.
>> >> As you can imagine i got an error that static can't be applied for
>> this VM
>> >> / default interface.
>> >>
>> >> Thats when strange things happend.
>> >> The static NAT configuration was successfully applied.
>> >> I wanted to revert everything so i tried to delete the static NAT And i
>> >> got
>> >> the following error:
>> >>
>> >> 2022-01-04 00:55:58,394 DEBUG [o.a.c.n.t.AdvancedNetworkTopology]
>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>> (logid:90c06d44)
>> >> APPLYING VPC IP RULES
>> >> 2022-01-04 00:55:58,399 DEBUG [o.a.c.n.t.BasicNetworkTopology]
>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>> (logid:90c06d44)
>> >> Applying vpc ip association in network Ntwk[246|Guest|30]
>> >> 2022-01-04 00:55:58,442 DEBUG [c.c.a.t.Request]
>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>> (logid:90c06d44)
>> >> Seq 5-4862198747700030042: Sending { Cmd , MgmtId: 90520740759984,
>> via:
>> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 100001,
>> >>
>> >>
>> [{"com.cloud.agent.api.routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":"5","publicIp":"10.129.0.37","sourceNat":"true","add":"false","oneToOneNat":"true","firstIP":"true","broadcastUri":"vlan://2990","vlanGateway":"10.129.0.1","vlanNetmask":"255.255.255.128","vifMacAddress":"1e:00:59:00:00:b2","networkRate":"-1","trafficType":"Public","networkName":"br3_CS_Pub","newNic":"false","isPrivateGateway":"false"}],"accessDetails":{"
>> >> router.name
>> >>
>> ":"r-62-VM","router.guest.ip":"10.129.0.42","network.public.last.ip":"false","router.ip":"169.254.1.55","zone.network.type":"Advanced"},"wait":"0","bypassHostMaintenance":"false"}}]
>> >> }
>> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
>> >> (AgentManager-Handler-3:null) (logid:) Seq 5-4862198747700030042:
>> >> Processing: { Ans: , MgmtId: 90520740759984, via: 5, Ver: v1, Flags:
>> 0,
>> >> [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null -
>> success:
>> >> Creating file in VR, with ip: 169.254.1.55, file:
>> >> ip_associations.json.167dc5ba-c89a-48c8-9b99-81b375e59691","null -
>> >> success:
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> Error: Nexthop has invalid gateway.
>> >> "],"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
>> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
>> (logid:90c06d44)
>> >> Seq 5-4862198747700030042: Received: { Ans: , MgmtId: 90520740759984,
>> >> via:
>> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 0, { GroupAnswer } }
>> >> 2022-01-04 00:55:59,486 ERROR [c.c.a.ApiAsyncJobDispatcher]
>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Unexpected
>> >> exception while executing
>> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd
>> >> com.cloud.exception.UnsupportedServiceException: Service UserData is
>> not
>> >> supported in the network id=246
>> >> at
>> >>
>> >>
>> com.cloud.network.dao.NetworkServiceMapDaoImpl.getProviderForServiceInNetwork(NetworkServiceMapDaoImpl.java:126)
>> >> at jdk.internal.reflect.GeneratedMethodAccessor1754.invoke(Unknown
>> Source)
>> >> at
>> >>
>> >>
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>> >> at
>> >>
>> >>
>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
>> >> at
>> >>
>> >>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
>> >> at
>> >>
>> >>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>> >> at
>> >>
>> >>
>> com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
>> >> at
>> >>
>> >>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>> >> at
>> >>
>> >>
>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
>> >> at
>> >>
>> >>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>> >> at
>> >>
>> >>
>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
>> >> at com.sun.proxy.$Proxy61.getProviderForServiceInNetwork(Unknown
>> Source)
>> >> at
>> >>
>> >>
>> com.cloud.network.NetworkModelImpl.getUserDataUpdateProvider(NetworkModelImpl.java:902)
>> >> at
>> >>
>> >>
>> com.cloud.network.rules.RulesManagerImpl.applyUserData(RulesManagerImpl.java:633)
>> >> at
>> >>
>> >>
>> com.cloud.network.rules.RulesManagerImpl.revokeAllPFAndStaticNatRulesForIp(RulesManagerImpl.java:1147)
>> >> at
>> >>
>> >>
>> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1328)
>> >> at
>> >>
>> >>
>> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1290)
>> >> at
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
>> >> Method)
>> >> at
>> >>
>> >>
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> >> at
>> >>
>> >>
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>> >> at
>> >>
>> >>
>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
>> >> at
>> >>
>> >>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
>> >> at
>> >>
>> >>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>> >> at
>> >>
>> >>
>> org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
>> >> at
>> >>
>> >>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>> >> at
>> >>
>> >>
>> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
>> >> at
>> >>
>> >>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>> >> at
>> >>
>> >>
>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
>> >> at
>> >>
>> >>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>> >> at
>> >>
>> >>
>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
>> >> at com.sun.proxy.$Proxy122.disableStaticNat(Unknown Source)
>> >> at
>> >>
>> >>
>> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd.execute(DisableStaticNatCmd.java:88)
>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
>> >> at
>> >>
>> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
>> >> at
>> >>
>> >>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
>> >> at
>> >>
>> >>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
>> >> at
>> >>
>> >>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
>> >> at
>> >>
>> >>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
>> >> at
>> >>
>> >>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
>> >> at
>> >>
>> >>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
>> >> at
>> >>
>> >>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
>> >> at
>> >>
>> >>
>> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>> >> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>> >> at
>> >>
>> >>
>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>> >> at
>> >>
>> >>
>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>> >> at java.base/java.lang.Thread.run(Thread.java:829)
>> >> 2022-01-04 00:55:59,488 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Complete
>> >> async
>> >> job-2135, jobStatus: FAILED, resultCode: 530, result:
>> >>
>> >>
>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Service
>> >> UserData is not supported in the network id=246"}
>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Publish
>> async
>> >> job-2135 complete on message bus
>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up
>> jobs
>> >> related to job-2135
>> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Update db
>> >> status for job-2135
>> >> 2022-01-04 00:55:59,490 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up
>> jobs
>> >> joined with job-2135 and disjoin all subjobs created from job- 2135
>> >> 2022-01-04 00:55:59,494 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Done
>> >> executing
>> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd for
>> >> job-2135
>> >> 2022-01-04 00:55:59,494 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Remove
>> >> job-2135 from job monitoring
>> >>
>> >> As far as i understand there is a problem with the "service user data".
>> >> That's correct as i created my own offerings for VPC and the
>> Tier-Networks
>> >> which don't offer the service.
>> >> How ever now i have the situation that i can't dele the static nat via
>> the
>> >> GUI and the public ip is stuck in the state "released", is not
>> availeable
>> >> for further usage and the static NAT is still shown as configured.
>> >>
>> >> Maybe someone can point me on how to release the IP properly? May i
>> need
>> >> to
>> >> work directly in the DB to clean this "mess" ?
>> >> Any suggestion is appreciated.
>> >>
>> >> With regards,
>> >> Chris
>> >>
>> >
>> >
>> > --
>> > Daan
>> >
>>
>
Re: Public IP is stucked in state "released" after usage for static nat
Posted by Wei ZHOU <us...@gmail.com>.
Hi Chris,
This seems to be a bug. Can you file an issue on
https://github.com/apache/cloudstack/issues ?
-Wei
On Tue, 4 Jan 2022 at 11:16, Vash_X@gmx.de <Va...@gmx.de> wrote:
> Hi Dan,
>
> Sorry for the misunderstanding.
> The VPC was created using a "custome" service offering which is not using /
> offering "service user data" . The same goes for the NW-Offering which is
> used for creation of the tiers. There is no service user data in this VPC.
>
> The default / out-of-the-box VPC offering has the service "user data"
> included. Same goes for the default nework offering for VPC Tiers. So this
> service should be supported at all from my understanding.
> That's why this error seems a bit "bogus" to me. I was understanding
> somethin like "static nat can't be disabled due to missing service "user
> data" for the network".
>
> Deleting and building the whole thing from scratch will be the last resort,
> yes. Even taking the risk that it won't be deletable as the allocated
> ressourcces can't be released properly at the moment, e.g. using the
> cloudstack.api.command.user.nat.DisableStaticNatCmd.
> Seems like i found a configuration which doesn't work well but can be
> created via the GUI. Even if i don't understand the connection between the
> service user data and the service stativ nat.
>
>
>
> Daan Hoogland <da...@gmail.com> schrieb am Di., 4. Jan. 2022,
> 09:21:
>
> > Chris, it looks to me like you should delete the VPC and then check if
> all
> > IPs are available before creating a new one.
> >
> > As for the user data service, the message already says that user data is
> > not supported, so why create another offering?
> >
> > On Tue, Jan 4, 2022 at 2:50 AM Vash_X@gmx.de <Va...@gmx.de> wrote:
> >
> >> hi everyone,
> >>
> >> sadly i am encountering some problems in my CS 4.16 set-up.
> >>
> >> today i wanted to take a look into static-nat for the routers of my vpc.
> >> Sadly i missed, that the VM i wanted to assaign to the static public ip
> >> had
> >> was already targeted by an port forwarding.
> >> As you can imagine i got an error that static can't be applied for this
> VM
> >> / default interface.
> >>
> >> Thats when strange things happend.
> >> The static NAT configuration was successfully applied.
> >> I wanted to revert everything so i tried to delete the static NAT And i
> >> got
> >> the following error:
> >>
> >> 2022-01-04 00:55:58,394 DEBUG [o.a.c.n.t.AdvancedNetworkTopology]
> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
> (logid:90c06d44)
> >> APPLYING VPC IP RULES
> >> 2022-01-04 00:55:58,399 DEBUG [o.a.c.n.t.BasicNetworkTopology]
> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
> (logid:90c06d44)
> >> Applying vpc ip association in network Ntwk[246|Guest|30]
> >> 2022-01-04 00:55:58,442 DEBUG [c.c.a.t.Request]
> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
> (logid:90c06d44)
> >> Seq 5-4862198747700030042: Sending { Cmd , MgmtId: 90520740759984, via:
> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 100001,
> >>
> >>
> [{"com.cloud.agent.api.routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":"5","publicIp":"10.129.0.37","sourceNat":"true","add":"false","oneToOneNat":"true","firstIP":"true","broadcastUri":"vlan://2990","vlanGateway":"10.129.0.1","vlanNetmask":"255.255.255.128","vifMacAddress":"1e:00:59:00:00:b2","networkRate":"-1","trafficType":"Public","networkName":"br3_CS_Pub","newNic":"false","isPrivateGateway":"false"}],"accessDetails":{"
> >> router.name
> >>
> ":"r-62-VM","router.guest.ip":"10.129.0.42","network.public.last.ip":"false","router.ip":"169.254.1.55","zone.network.type":"Advanced"},"wait":"0","bypassHostMaintenance":"false"}}]
> >> }
> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
> >> (AgentManager-Handler-3:null) (logid:) Seq 5-4862198747700030042:
> >> Processing: { Ans: , MgmtId: 90520740759984, via: 5, Ver: v1, Flags: 0,
> >> [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null - success:
> >> Creating file in VR, with ip: 169.254.1.55, file:
> >> ip_associations.json.167dc5ba-c89a-48c8-9b99-81b375e59691","null -
> >> success:
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> Error: Nexthop has invalid gateway.
> >> "],"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
> >> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
> >> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74)
> (logid:90c06d44)
> >> Seq 5-4862198747700030042: Received: { Ans: , MgmtId: 90520740759984,
> >> via:
> >> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 0, { GroupAnswer } }
> >> 2022-01-04 00:55:59,486 ERROR [c.c.a.ApiAsyncJobDispatcher]
> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Unexpected
> >> exception while executing
> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd
> >> com.cloud.exception.UnsupportedServiceException: Service UserData is not
> >> supported in the network id=246
> >> at
> >>
> >>
> com.cloud.network.dao.NetworkServiceMapDaoImpl.getProviderForServiceInNetwork(NetworkServiceMapDaoImpl.java:126)
> >> at jdk.internal.reflect.GeneratedMethodAccessor1754.invoke(Unknown
> Source)
> >> at
> >>
> >>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >> at
> >>
> >>
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
> >> at
> >>
> >>
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
> >> at
> >>
> >>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> >> at
> >>
> >>
> com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
> >> at
> >>
> >>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
> >> at
> >>
> >>
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
> >> at
> >>
> >>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> >> at
> >>
> >>
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
> >> at com.sun.proxy.$Proxy61.getProviderForServiceInNetwork(Unknown Source)
> >> at
> >>
> >>
> com.cloud.network.NetworkModelImpl.getUserDataUpdateProvider(NetworkModelImpl.java:902)
> >> at
> >>
> >>
> com.cloud.network.rules.RulesManagerImpl.applyUserData(RulesManagerImpl.java:633)
> >> at
> >>
> >>
> com.cloud.network.rules.RulesManagerImpl.revokeAllPFAndStaticNatRulesForIp(RulesManagerImpl.java:1147)
> >> at
> >>
> >>
> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1328)
> >> at
> >>
> >>
> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1290)
> >> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> >> Method)
> >> at
> >>
> >>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >> at
> >>
> >>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> >> at
> >>
> >>
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
> >> at
> >>
> >>
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
> >> at
> >>
> >>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> >> at
> >>
> >>
> org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
> >> at
> >>
> >>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
> >> at
> >>
> >>
> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
> >> at
> >>
> >>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
> >> at
> >>
> >>
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
> >> at
> >>
> >>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> >> at
> >>
> >>
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
> >> at com.sun.proxy.$Proxy122.disableStaticNat(Unknown Source)
> >> at
> >>
> >>
> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd.execute(DisableStaticNatCmd.java:88)
> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
> >> at
> >>
> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
> >> at
> >>
> >>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
> >> at
> >>
> >>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
> >> at
> >>
> >>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
> >> at
> >>
> >>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
> >> at
> >>
> >>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
> >> at
> >>
> >>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
> >> at
> >>
> >>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
> >> at
> >>
> >>
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> >> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> >> at
> >>
> >>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> >> at
> >>
> >>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> >> at java.base/java.lang.Thread.run(Thread.java:829)
> >> 2022-01-04 00:55:59,488 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Complete
> >> async
> >> job-2135, jobStatus: FAILED, resultCode: 530, result:
> >>
> >>
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Service
> >> UserData is not supported in the network id=246"}
> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Publish
> async
> >> job-2135 complete on message bus
> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up
> jobs
> >> related to job-2135
> >> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Update db
> >> status for job-2135
> >> 2022-01-04 00:55:59,490 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up
> jobs
> >> joined with job-2135 and disjoin all subjobs created from job- 2135
> >> 2022-01-04 00:55:59,494 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Done
> >> executing
> >> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd for
> >> job-2135
> >> 2022-01-04 00:55:59,494 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> >> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Remove
> >> job-2135 from job monitoring
> >>
> >> As far as i understand there is a problem with the "service user data".
> >> That's correct as i created my own offerings for VPC and the
> Tier-Networks
> >> which don't offer the service.
> >> How ever now i have the situation that i can't dele the static nat via
> the
> >> GUI and the public ip is stuck in the state "released", is not
> availeable
> >> for further usage and the static NAT is still shown as configured.
> >>
> >> Maybe someone can point me on how to release the IP properly? May i need
> >> to
> >> work directly in the DB to clean this "mess" ?
> >> Any suggestion is appreciated.
> >>
> >> With regards,
> >> Chris
> >>
> >
> >
> > --
> > Daan
> >
>
Re: Public IP is stucked in state "released" after usage for static nat
Posted by "Vash_X@gmx.de" <Va...@gmx.de>.
Hi Dan,
Sorry for the misunderstanding.
The VPC was created using a "custome" service offering which is not using /
offering "service user data" . The same goes for the NW-Offering which is
used for creation of the tiers. There is no service user data in this VPC.
The default / out-of-the-box VPC offering has the service "user data"
included. Same goes for the default nework offering for VPC Tiers. So this
service should be supported at all from my understanding.
That's why this error seems a bit "bogus" to me. I was understanding
somethin like "static nat can't be disabled due to missing service "user
data" for the network".
Deleting and building the whole thing from scratch will be the last resort,
yes. Even taking the risk that it won't be deletable as the allocated
ressourcces can't be released properly at the moment, e.g. using the
cloudstack.api.command.user.nat.DisableStaticNatCmd.
Seems like i found a configuration which doesn't work well but can be
created via the GUI. Even if i don't understand the connection between the
service user data and the service stativ nat.
Daan Hoogland <da...@gmail.com> schrieb am Di., 4. Jan. 2022, 09:21:
> Chris, it looks to me like you should delete the VPC and then check if all
> IPs are available before creating a new one.
>
> As for the user data service, the message already says that user data is
> not supported, so why create another offering?
>
> On Tue, Jan 4, 2022 at 2:50 AM Vash_X@gmx.de <Va...@gmx.de> wrote:
>
>> hi everyone,
>>
>> sadly i am encountering some problems in my CS 4.16 set-up.
>>
>> today i wanted to take a look into static-nat for the routers of my vpc.
>> Sadly i missed, that the VM i wanted to assaign to the static public ip
>> had
>> was already targeted by an port forwarding.
>> As you can imagine i got an error that static can't be applied for this VM
>> / default interface.
>>
>> Thats when strange things happend.
>> The static NAT configuration was successfully applied.
>> I wanted to revert everything so i tried to delete the static NAT And i
>> got
>> the following error:
>>
>> 2022-01-04 00:55:58,394 DEBUG [o.a.c.n.t.AdvancedNetworkTopology]
>> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74) (logid:90c06d44)
>> APPLYING VPC IP RULES
>> 2022-01-04 00:55:58,399 DEBUG [o.a.c.n.t.BasicNetworkTopology]
>> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74) (logid:90c06d44)
>> Applying vpc ip association in network Ntwk[246|Guest|30]
>> 2022-01-04 00:55:58,442 DEBUG [c.c.a.t.Request]
>> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74) (logid:90c06d44)
>> Seq 5-4862198747700030042: Sending { Cmd , MgmtId: 90520740759984, via:
>> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 100001,
>>
>> [{"com.cloud.agent.api.routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":"5","publicIp":"10.129.0.37","sourceNat":"true","add":"false","oneToOneNat":"true","firstIP":"true","broadcastUri":"vlan://2990","vlanGateway":"10.129.0.1","vlanNetmask":"255.255.255.128","vifMacAddress":"1e:00:59:00:00:b2","networkRate":"-1","trafficType":"Public","networkName":"br3_CS_Pub","newNic":"false","isPrivateGateway":"false"}],"accessDetails":{"
>> router.name
>> ":"r-62-VM","router.guest.ip":"10.129.0.42","network.public.last.ip":"false","router.ip":"169.254.1.55","zone.network.type":"Advanced"},"wait":"0","bypassHostMaintenance":"false"}}]
>> }
>> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
>> (AgentManager-Handler-3:null) (logid:) Seq 5-4862198747700030042:
>> Processing: { Ans: , MgmtId: 90520740759984, via: 5, Ver: v1, Flags: 0,
>> [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null - success:
>> Creating file in VR, with ip: 169.254.1.55, file:
>> ip_associations.json.167dc5ba-c89a-48c8-9b99-81b375e59691","null -
>> success:
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> Error: Nexthop has invalid gateway.
>> "],"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
>> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
>> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74) (logid:90c06d44)
>> Seq 5-4862198747700030042: Received: { Ans: , MgmtId: 90520740759984,
>> via:
>> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 0, { GroupAnswer } }
>> 2022-01-04 00:55:59,486 ERROR [c.c.a.ApiAsyncJobDispatcher]
>> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Unexpected
>> exception while executing
>> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd
>> com.cloud.exception.UnsupportedServiceException: Service UserData is not
>> supported in the network id=246
>> at
>>
>> com.cloud.network.dao.NetworkServiceMapDaoImpl.getProviderForServiceInNetwork(NetworkServiceMapDaoImpl.java:126)
>> at jdk.internal.reflect.GeneratedMethodAccessor1754.invoke(Unknown Source)
>> at
>>
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>> at
>>
>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
>> at
>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
>> at
>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>> at
>>
>> com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
>> at
>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>> at
>>
>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
>> at
>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>> at
>>
>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
>> at com.sun.proxy.$Proxy61.getProviderForServiceInNetwork(Unknown Source)
>> at
>>
>> com.cloud.network.NetworkModelImpl.getUserDataUpdateProvider(NetworkModelImpl.java:902)
>> at
>>
>> com.cloud.network.rules.RulesManagerImpl.applyUserData(RulesManagerImpl.java:633)
>> at
>>
>> com.cloud.network.rules.RulesManagerImpl.revokeAllPFAndStaticNatRulesForIp(RulesManagerImpl.java:1147)
>> at
>>
>> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1328)
>> at
>>
>> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1290)
>> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>> at
>>
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at
>>
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>> at
>>
>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
>> at
>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
>> at
>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>> at
>>
>> org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
>> at
>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>> at
>>
>> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
>> at
>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
>> at
>>
>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
>> at
>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>> at
>>
>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
>> at com.sun.proxy.$Proxy122.disableStaticNat(Unknown Source)
>> at
>>
>> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd.execute(DisableStaticNatCmd.java:88)
>> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
>> at
>> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
>> at
>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
>> at
>>
>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
>> at
>>
>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
>> at
>>
>> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>> at
>>
>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>> at
>>
>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>> at java.base/java.lang.Thread.run(Thread.java:829)
>> 2022-01-04 00:55:59,488 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Complete
>> async
>> job-2135, jobStatus: FAILED, resultCode: 530, result:
>>
>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Service
>> UserData is not supported in the network id=246"}
>> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Publish async
>> job-2135 complete on message bus
>> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up jobs
>> related to job-2135
>> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Update db
>> status for job-2135
>> 2022-01-04 00:55:59,490 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up jobs
>> joined with job-2135 and disjoin all subjobs created from job- 2135
>> 2022-01-04 00:55:59,494 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Done
>> executing
>> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd for
>> job-2135
>> 2022-01-04 00:55:59,494 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Remove
>> job-2135 from job monitoring
>>
>> As far as i understand there is a problem with the "service user data".
>> That's correct as i created my own offerings for VPC and the Tier-Networks
>> which don't offer the service.
>> How ever now i have the situation that i can't dele the static nat via the
>> GUI and the public ip is stuck in the state "released", is not availeable
>> for further usage and the static NAT is still shown as configured.
>>
>> Maybe someone can point me on how to release the IP properly? May i need
>> to
>> work directly in the DB to clean this "mess" ?
>> Any suggestion is appreciated.
>>
>> With regards,
>> Chris
>>
>
>
> --
> Daan
>
Re: Public IP is stucked in state "released" after usage for static nat
Posted by Daan Hoogland <da...@gmail.com>.
Chris, it looks to me like you should delete the VPC and then check if all
IPs are available before creating a new one.
As for the user data service, the message already says that user data is
not supported, so why create another offering?
On Tue, Jan 4, 2022 at 2:50 AM Vash_X@gmx.de <Va...@gmx.de> wrote:
> hi everyone,
>
> sadly i am encountering some problems in my CS 4.16 set-up.
>
> today i wanted to take a look into static-nat for the routers of my vpc.
> Sadly i missed, that the VM i wanted to assaign to the static public ip had
> was already targeted by an port forwarding.
> As you can imagine i got an error that static can't be applied for this VM
> / default interface.
>
> Thats when strange things happend.
> The static NAT configuration was successfully applied.
> I wanted to revert everything so i tried to delete the static NAT And i got
> the following error:
>
> 2022-01-04 00:55:58,394 DEBUG [o.a.c.n.t.AdvancedNetworkTopology]
> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74) (logid:90c06d44)
> APPLYING VPC IP RULES
> 2022-01-04 00:55:58,399 DEBUG [o.a.c.n.t.BasicNetworkTopology]
> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74) (logid:90c06d44)
> Applying vpc ip association in network Ntwk[246|Guest|30]
> 2022-01-04 00:55:58,442 DEBUG [c.c.a.t.Request]
> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74) (logid:90c06d44)
> Seq 5-4862198747700030042: Sending { Cmd , MgmtId: 90520740759984, via:
> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 100001,
>
> [{"com.cloud.agent.api.routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":"5","publicIp":"10.129.0.37","sourceNat":"true","add":"false","oneToOneNat":"true","firstIP":"true","broadcastUri":"vlan://2990","vlanGateway":"10.129.0.1","vlanNetmask":"255.255.255.128","vifMacAddress":"1e:00:59:00:00:b2","networkRate":"-1","trafficType":"Public","networkName":"br3_CS_Pub","newNic":"false","isPrivateGateway":"false"}],"accessDetails":{"
> router.name
> ":"r-62-VM","router.guest.ip":"10.129.0.42","network.public.last.ip":"false","router.ip":"169.254.1.55","zone.network.type":"Advanced"},"wait":"0","bypassHostMaintenance":"false"}}]
> }
> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-3:null) (logid:) Seq 5-4862198747700030042:
> Processing: { Ans: , MgmtId: 90520740759984, via: 5, Ver: v1, Flags: 0,
> [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null - success:
> Creating file in VR, with ip: 169.254.1.55, file:
> ip_associations.json.167dc5ba-c89a-48c8-9b99-81b375e59691","null - success:
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> Error: Nexthop has invalid gateway.
> "],"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
> 2022-01-04 00:55:59,476 DEBUG [c.c.a.t.Request]
> (API-Job-Executor-98:ctx-90ac5cad job-2135 ctx-2eb15c74) (logid:90c06d44)
> Seq 5-4862198747700030042: Received: { Ans: , MgmtId: 90520740759984, via:
> 5(srv-4-comp.dnd-digital.infra), Ver: v1, Flags: 0, { GroupAnswer } }
> 2022-01-04 00:55:59,486 ERROR [c.c.a.ApiAsyncJobDispatcher]
> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Unexpected
> exception while executing
> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd
> com.cloud.exception.UnsupportedServiceException: Service UserData is not
> supported in the network id=246
> at
>
> com.cloud.network.dao.NetworkServiceMapDaoImpl.getProviderForServiceInNetwork(NetworkServiceMapDaoImpl.java:126)
> at jdk.internal.reflect.GeneratedMethodAccessor1754.invoke(Unknown Source)
> at
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
>
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
> at
>
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
> at
>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
>
> com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
> at
>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
> at
>
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
> at
>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at
>
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
> at com.sun.proxy.$Proxy61.getProviderForServiceInNetwork(Unknown Source)
> at
>
> com.cloud.network.NetworkModelImpl.getUserDataUpdateProvider(NetworkModelImpl.java:902)
> at
>
> com.cloud.network.rules.RulesManagerImpl.applyUserData(RulesManagerImpl.java:633)
> at
>
> com.cloud.network.rules.RulesManagerImpl.revokeAllPFAndStaticNatRulesForIp(RulesManagerImpl.java:1147)
> at
>
> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1328)
> at
>
> com.cloud.network.rules.RulesManagerImpl.disableStaticNat(RulesManagerImpl.java:1290)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
>
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
> at
>
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
> at
>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
>
> org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
> at
>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
> at
>
> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
> at
>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
> at
>
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
> at
>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at
>
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
> at com.sun.proxy.$Proxy122.disableStaticNat(Unknown Source)
> at
>
> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd.execute(DisableStaticNatCmd.java:88)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
> at
> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
> at
>
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
> at
>
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-01-04 00:55:59,488 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Complete async
> job-2135, jobStatus: FAILED, resultCode: 530, result:
>
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Service
> UserData is not supported in the network id=246"}
> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Publish async
> job-2135 complete on message bus
> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up jobs
> related to job-2135
> 2022-01-04 00:55:59,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Update db
> status for job-2135
> 2022-01-04 00:55:59,490 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Wake up jobs
> joined with job-2135 and disjoin all subjobs created from job- 2135
> 2022-01-04 00:55:59,494 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Done executing
> org.apache.cloudstack.api.command.user.nat.DisableStaticNatCmd for job-2135
> 2022-01-04 00:55:59,494 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (API-Job-Executor-98:ctx-90ac5cad job-2135) (logid:90c06d44) Remove
> job-2135 from job monitoring
>
> As far as i understand there is a problem with the "service user data".
> That's correct as i created my own offerings for VPC and the Tier-Networks
> which don't offer the service.
> How ever now i have the situation that i can't dele the static nat via the
> GUI and the public ip is stuck in the state "released", is not availeable
> for further usage and the static NAT is still shown as configured.
>
> Maybe someone can point me on how to release the IP properly? May i need to
> work directly in the DB to clean this "mess" ?
> Any suggestion is appreciated.
>
> With regards,
> Chris
>
--
Daan