You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Michael Lukzak <mi...@vp.pl> on 2014/08/07 21:17:59 UTC
Re[2]: Urgent help needed. Can't start Virtual Router.
Hi,
We are usning xenserver from Citrix.
I found some clue. In console to server on this host I see that nfs has error "Stale NFS file handle".
How to remount NFS? Maybe this is a problem?
But why other VR are started and few of them not... ;(
Best regards,
Michael
> Hi Michael,
> if it helps, I would restart the agent on the host machine where VR lives.
> it depends on hypervisor but on KVM, following command: service
> cloudstack-agent restart
> Thanks,
> Motty
> On Thu, Aug 7, 2014 at 11:52 AM, Michael Lukzak <mi...@vp.pl> wrote:
>> Hi,
>> One of our host are died (after very high load).
>> Host has a lot of Virtual Routers running.
>> We are using advanced networking.
>> Only way to bring machine alive was hard reset.
>> After this CloudStack decied to start all of VirtualRouterts,
>> when server was back online.
>> Most of VR are started, but couple don't.
>> Cloudstack want to start this VM, but can't ;(
>> Here are log from this action.
>> It's looks like that CS are looking for VR host on Xen host, but he
>> couldn't find.
>> Please help how to repair this situation.
>> Restart network with clean up options ending with the same resoult.
>> Is there any option to recreate VR?
>> I tried to remove VR from Infrastructure -> Virtual Router, but there is
>> still problem
>> with create new one.
>> Here are logs from "restart network with clean up options"
>> WARN [network.element.VpcVirtualRouterElement]
>> (Job-Executor-151:job-6526) Network Ntwk[200|Public|1] is not associated
>> with any VPC
>> WARN [network.element.VpcVirtualRouterElement]
>> (Job-Executor-151:job-6526) Network Ntwk[338|Guest|8] is not associated
>> with any VPC
>> WARN [network.element.VpcVirtualRouterElement]
>> (Job-Executor-151:job-6526) Network Ntwk[202|Control|3] is not associated
>> with any VPC
>> ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-151:job-6526)
>> Failed to start instance VM[DomainRouter|r-1015-VM]
>> java.lang.NullPointerException
>> at
>> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
>> at
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
>> at
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
>> at
>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
>> at
>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1924)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1902)
>> at
>> com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:175)
>> at
>> com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1518)
>> at
>> com.cloud.network.NetworkManagerImpl.restartNetwork(NetworkManagerImpl.java:2473)
>> at
>> com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1381)
>> at
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>> at
>> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
>> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
>> at
>> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:722)
>> INFO [xen.resource.CitrixResourceBase] (DirectAgent-16:) VM does not
>> exist on XenServer0d1d5015-4061-4554-96de-406155449296
>> WARN [cloud.network.NetworkManagerImpl] (Job-Executor-151:job-6526)
>> Failed to implement network Ntwk[338|Guest|8] elements and resources as a
>> part of network restart due to
>> com.cloud.exception.AgentUnavailableException: Resource [Host:9] is
>> unreachable: Host 9: Unable to start instance due to null
>> at
>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:841)
>> at
>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1924)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1902)
>> at
>> com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:175)
>> at
>> com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1518)
>> at
>> com.cloud.network.NetworkManagerImpl.restartNetwork(NetworkManagerImpl.java:2473)
>> at
>> com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1381)
>> at
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>> at
>> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
>> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
>> at
>> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:722)
>> Caused by: java.lang.NullPointerException
>> at
>> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
>> at
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
>> at
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
>> at
>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
>> ... 23 more
>> WARN [cloud.network.NetworkServiceImpl] (Job-Executor-151:job-6526)
>> Network id=338 failed to restart.
>> WARN [cloud.async.AsyncJobManagerImpl] (Job-Executor-151:job-6526) Unable
>> to unregister active job 6526 from JMX monitoring
>> Here are log when I want to start VR (his state is stopeed, I want to
>> start and got this)
>> WARN [network.element.VpcVirtualRouterElement]
>> (Job-Executor-153:job-6528) Network Ntwk[200|Public|1] is not associated
>> with any VPC
>> WARN [network.element.VpcVirtualRouterElement]
>> (Job-Executor-153:job-6528) Network Ntwk[338|Guest|8] is not associated
>> with any VPC
>> WARN [network.element.VpcVirtualRouterElement]
>> (Job-Executor-153:job-6528) Network Ntwk[202|Control|3] is not associated
>> with any VPC
>> ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-153:job-6528)
>> Failed to start instance VM[DomainRouter|r-1015-VM]
>> java.lang.NullPointerException
>> at
>> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
>> at
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
>> at
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
>> at
>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
>> at
>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouter(VirtualNetworkApplianceManagerImpl.java:2874)
>> at
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouter(VirtualNetworkApplianceManagerImpl.java:2824)
>> at
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>> at
>> org.apache.cloudstack.api.command.admin.router.StartRouterCmd.execute(StartRouterCmd.java:103)
>> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
>> at
>> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:722)
>> Best Regards,
>> Michael
--
Pozdrowienia,
Michael Lukzak
Re: Re[4]: Urgent help needed. Can't start Virtual Router.
Posted by Andrei Mikhailovsky <an...@arhont.com>.
Michael,
When you Restart the Network and tick the box that says Clean Up, the ACS will automatically remove the old VR and recreate a new one from the template.
Regarding your Stale NFS - I would investigate this and fix it prior to doing anything else. I've had cases with stale nfs where the client would have no issues accessing some images, but would not see other images. From my experience, this tend to be nfs client specific and usually doesn't effect other clients.
Andrei
--
Andrei Mikhailovsky
Director
Arhont Information Security
Web: http://www.arhont.com
http://www.wi-foo.com
Tel: +44 (0)870 4431337
Fax: +44 (0)208 429 3111
PGP: Key ID - 0x2B3438DE
PGP: Server - keyserver.pgp.com
DISCLAIMER
The information contained in this email is intended only for the use of the person(s) to whom it is addressed and may be confidential or contain legally privileged information. If you are not the intended recipient you are hereby notified that any perusal, use, distribution, copying or disclosure is strictly prohibited. If you have received this email in error please immediately advise us by return email at andrei@arhont.com and delete and purge the email and any attachments without making a copy.
----- Original Message -----
From: "Michael Lukzak" <mi...@vp.pl>
To: users@cloudstack.apache.org
Sent: Thursday, 7 August, 2014 8:43:06 PM
Subject: Re[4]: Urgent help needed. Can't start Virtual Router.
I removed this host from CS, but still problem is.
How I can "rebuild VR" from zero?
It's looks like CS want to start VM (witch really is a VirtualRouter) that not exists on the host.
INFO [xen.resource.CitrixResourceBase] (DirectAgent-148:) VM does not exist on XenServer11681b28-5f38-4f20-aaae-0c9ed1e9ecc7
Or other way. How I can bind new network to VM?
Ony way is modify VM parameter in database?
Michael
> depending on your setup, but what I would do move all your Instances from
> host machine where the VR lives and reboot. Again, that's what I would do.
> you can try this link for XenServer 5
> http://blogs.citrix.com/2009/01/26/how-to-configure-an-nfs-based-iso-sr-using-local-storage-on-a-xenserver5-host/
> Thanks,
> On Thu, Aug 7, 2014 at 12:17 PM, Michael Lukzak <mi...@vp.pl> wrote:
>> Hi,
>> We are usning xenserver from Citrix.
>> I found some clue. In console to server on this host I see that nfs has
>> error "Stale NFS file handle".
>> How to remount NFS? Maybe this is a problem?
>> But why other VR are started and few of them not... ;(
>> Best regards,
>> Michael
>> > Hi Michael,
>> > if it helps, I would restart the agent on the host machine where VR
>> lives.
>> > it depends on hypervisor but on KVM, following command: service
>> > cloudstack-agent restart
>> > Thanks,
>> > Motty
>> > On Thu, Aug 7, 2014 at 11:52 AM, Michael Lukzak <mi...@vp.pl> wrote:
>> >> Hi,
>> >> One of our host are died (after very high load).
>> >> Host has a lot of Virtual Routers running.
>> >> We are using advanced networking.
>> >> Only way to bring machine alive was hard reset.
>> >> After this CloudStack decied to start all of VirtualRouterts,
>> >> when server was back online.
>> >> Most of VR are started, but couple don't.
>> >> Cloudstack want to start this VM, but can't ;(
>> >> Here are log from this action.
>> >> It's looks like that CS are looking for VR host on Xen host, but he
>> >> couldn't find.
>> >> Please help how to repair this situation.
>> >> Restart network with clean up options ending with the same resoult.
>> >> Is there any option to recreate VR?
>> >> I tried to remove VR from Infrastructure -> Virtual Router, but there is
>> >> still problem
>> >> with create new one.
>> >> Here are logs from "restart network with clean up options"
>> >> WARN [network.element.VpcVirtualRouterElement]
>> >> (Job-Executor-151:job-6526) Network Ntwk[200|Public|1] is not associated
>> >> with any VPC
>> >> WARN [network.element.VpcVirtualRouterElement]
>> >> (Job-Executor-151:job-6526) Network Ntwk[338|Guest|8] is not associated
>> >> with any VPC
>> >> WARN [network.element.VpcVirtualRouterElement]
>> >> (Job-Executor-151:job-6526) Network Ntwk[202|Control|3] is not
>> associated
>> >> with any VPC
>> >> ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-151:job-6526)
>> >> Failed to start instance VM[DomainRouter|r-1015-VM]
>> >> java.lang.NullPointerException
>> >> at
>> >>
>> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
>> >> at
>> >>
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
>> >> at
>> >>
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1924)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1902)
>> >> at
>> >>
>> com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:175)
>> >> at
>> >>
>> com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1518)
>> >> at
>> >>
>> com.cloud.network.NetworkManagerImpl.restartNetwork(NetworkManagerImpl.java:2473)
>> >> at
>> >>
>> com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1381)
>> >> at
>> >>
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>> >> at
>> >>
>> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
>> >> at
>> >> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> >> at
>> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> >> at java.lang.Thread.run(Thread.java:722)
>> >> INFO [xen.resource.CitrixResourceBase] (DirectAgent-16:) VM does not
>> >> exist on XenServer0d1d5015-4061-4554-96de-406155449296
>> >> WARN [cloud.network.NetworkManagerImpl] (Job-Executor-151:job-6526)
>> >> Failed to implement network Ntwk[338|Guest|8] elements and resources as
>> a
>> >> part of network restart due to
>> >> com.cloud.exception.AgentUnavailableException: Resource [Host:9] is
>> >> unreachable: Host 9: Unable to start instance due to null
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:841)
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1924)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1902)
>> >> at
>> >>
>> com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:175)
>> >> at
>> >>
>> com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1518)
>> >> at
>> >>
>> com.cloud.network.NetworkManagerImpl.restartNetwork(NetworkManagerImpl.java:2473)
>> >> at
>> >>
>> com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1381)
>> >> at
>> >>
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>> >> at
>> >>
>> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
>> >> at
>> >> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> >> at
>> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> >> at java.lang.Thread.run(Thread.java:722)
>> >> Caused by: java.lang.NullPointerException
>> >> at
>> >>
>> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
>> >> at
>> >>
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
>> >> at
>> >>
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
>> >> ... 23 more
>> >> WARN [cloud.network.NetworkServiceImpl] (Job-Executor-151:job-6526)
>> >> Network id=338 failed to restart.
>> >> WARN [cloud.async.AsyncJobManagerImpl] (Job-Executor-151:job-6526)
>> Unable
>> >> to unregister active job 6526 from JMX monitoring
>> >> Here are log when I want to start VR (his state is stopeed, I want to
>> >> start and got this)
>> >> WARN [network.element.VpcVirtualRouterElement]
>> >> (Job-Executor-153:job-6528) Network Ntwk[200|Public|1] is not associated
>> >> with any VPC
>> >> WARN [network.element.VpcVirtualRouterElement]
>> >> (Job-Executor-153:job-6528) Network Ntwk[338|Guest|8] is not associated
>> >> with any VPC
>> >> WARN [network.element.VpcVirtualRouterElement]
>> >> (Job-Executor-153:job-6528) Network Ntwk[202|Control|3] is not
>> associated
>> >> with any VPC
>> >> ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-153:job-6528)
>> >> Failed to start instance VM[DomainRouter|r-1015-VM]
>> >> java.lang.NullPointerException
>> >> at
>> >>
>> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
>> >> at
>> >>
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
>> >> at
>> >>
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouter(VirtualNetworkApplianceManagerImpl.java:2874)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouter(VirtualNetworkApplianceManagerImpl.java:2824)
>> >> at
>> >>
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>> >> at
>> >>
>> org.apache.cloudstack.api.command.admin.router.StartRouterCmd.execute(StartRouterCmd.java:103)
>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
>> >> at
>> >> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> >> at
>> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> >> at java.lang.Thread.run(Thread.java:722)
>> >> Best Regards,
>> >> Michael
>> --
>> Pozdrowienia,
>> Michael Lukzak
--
Pozdrowienia,
Michael Lukzak
Re: Re[4]: Urgent help needed. Can't start Virtual Router.
Posted by motty cruz <mo...@gmail.com>.
in ACS 4.2.1 if you go to "Network" tab, click on the network you're having
issues with, hit the "restart" button click on "clean network" it will
remove current router and rebuilt a new one. please make sure to click on
"restart" Network no on the X
hope this helps!
On Thu, Aug 7, 2014 at 12:43 PM, Michael Lukzak <mi...@vp.pl> wrote:
> I removed this host from CS, but still problem is.
>
> How I can "rebuild VR" from zero?
> It's looks like CS want to start VM (witch really is a VirtualRouter) that
> not exists on the host.
>
> INFO [xen.resource.CitrixResourceBase] (DirectAgent-148:) VM does not
> exist on XenServer11681b28-5f38-4f20-aaae-0c9ed1e9ecc7
>
> Or other way. How I can bind new network to VM?
> Ony way is modify VM parameter in database?
>
> Michael
>
>
>
>
>
> > depending on your setup, but what I would do move all your Instances from
> > host machine where the VR lives and reboot. Again, that's what I would
> do.
>
> > you can try this link for XenServer 5
> >
> http://blogs.citrix.com/2009/01/26/how-to-configure-an-nfs-based-iso-sr-using-local-storage-on-a-xenserver5-host/
>
> > Thanks,
>
>
> > On Thu, Aug 7, 2014 at 12:17 PM, Michael Lukzak <mi...@vp.pl> wrote:
>
> >> Hi,
>
> >> We are usning xenserver from Citrix.
>
> >> I found some clue. In console to server on this host I see that nfs has
> >> error "Stale NFS file handle".
>
> >> How to remount NFS? Maybe this is a problem?
> >> But why other VR are started and few of them not... ;(
>
> >> Best regards,
> >> Michael
>
>
>
> >> > Hi Michael,
> >> > if it helps, I would restart the agent on the host machine where VR
> >> lives.
>
> >> > it depends on hypervisor but on KVM, following command: service
> >> > cloudstack-agent restart
>
>
> >> > Thanks,
> >> > Motty
>
>
> >> > On Thu, Aug 7, 2014 at 11:52 AM, Michael Lukzak <mi...@vp.pl> wrote:
>
> >> >> Hi,
>
> >> >> One of our host are died (after very high load).
> >> >> Host has a lot of Virtual Routers running.
>
> >> >> We are using advanced networking.
>
> >> >> Only way to bring machine alive was hard reset.
>
> >> >> After this CloudStack decied to start all of VirtualRouterts,
> >> >> when server was back online.
>
> >> >> Most of VR are started, but couple don't.
> >> >> Cloudstack want to start this VM, but can't ;(
> >> >> Here are log from this action.
>
> >> >> It's looks like that CS are looking for VR host on Xen host, but he
> >> >> couldn't find.
>
> >> >> Please help how to repair this situation.
> >> >> Restart network with clean up options ending with the same resoult.
>
> >> >> Is there any option to recreate VR?
> >> >> I tried to remove VR from Infrastructure -> Virtual Router, but
> there is
> >> >> still problem
> >> >> with create new one.
>
>
>
> >> >> Here are logs from "restart network with clean up options"
> >> >> WARN [network.element.VpcVirtualRouterElement]
> >> >> (Job-Executor-151:job-6526) Network Ntwk[200|Public|1] is not
> associated
> >> >> with any VPC
> >> >> WARN [network.element.VpcVirtualRouterElement]
> >> >> (Job-Executor-151:job-6526) Network Ntwk[338|Guest|8] is not
> associated
> >> >> with any VPC
> >> >> WARN [network.element.VpcVirtualRouterElement]
> >> >> (Job-Executor-151:job-6526) Network Ntwk[202|Control|3] is not
> >> associated
> >> >> with any VPC
> >> >> ERROR [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-151:job-6526)
> >> >> Failed to start instance VM[DomainRouter|r-1015-VM]
> >> >> java.lang.NullPointerException
> >> >> at
> >> >>
> >>
> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
> >> >> at
> >> >>
> >>
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
> >> >> at
> >> >>
> >>
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1924)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1902)
> >> >> at
> >> >>
> >>
> com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:175)
> >> >> at
> >> >>
> >>
> com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1518)
> >> >> at
> >> >>
> >>
> com.cloud.network.NetworkManagerImpl.restartNetwork(NetworkManagerImpl.java:2473)
> >> >> at
> >> >>
> >>
> com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1381)
> >> >> at
> >> >>
> >>
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> >> >> at
> >> >>
> >>
> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
> >> >> at
> com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
> >> >> at
> >> >>
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
> >> >> at
> >> >>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >> >> at
> >> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> >> >> at
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >> >> at
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >> >> at java.lang.Thread.run(Thread.java:722)
> >> >> INFO [xen.resource.CitrixResourceBase] (DirectAgent-16:) VM does not
> >> >> exist on XenServer0d1d5015-4061-4554-96de-406155449296
> >> >> WARN [cloud.network.NetworkManagerImpl] (Job-Executor-151:job-6526)
> >> >> Failed to implement network Ntwk[338|Guest|8] elements and resources
> as
> >> a
> >> >> part of network restart due to
> >> >> com.cloud.exception.AgentUnavailableException: Resource [Host:9] is
> >> >> unreachable: Host 9: Unable to start instance due to null
> >> >> at
> >> >>
> >>
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:841)
> >> >> at
> >> >>
> >>
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1924)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1902)
> >> >> at
> >> >>
> >>
> com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:175)
> >> >> at
> >> >>
> >>
> com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1518)
> >> >> at
> >> >>
> >>
> com.cloud.network.NetworkManagerImpl.restartNetwork(NetworkManagerImpl.java:2473)
> >> >> at
> >> >>
> >>
> com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1381)
> >> >> at
> >> >>
> >>
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> >> >> at
> >> >>
> >>
> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
> >> >> at
> com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
> >> >> at
> >> >>
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
> >> >> at
> >> >>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >> >> at
> >> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> >> >> at
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >> >> at
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >> >> at java.lang.Thread.run(Thread.java:722)
> >> >> Caused by: java.lang.NullPointerException
> >> >> at
> >> >>
> >>
> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
> >> >> at
> >> >>
> >>
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
> >> >> ... 23 more
> >> >> WARN [cloud.network.NetworkServiceImpl] (Job-Executor-151:job-6526)
> >> >> Network id=338 failed to restart.
> >> >> WARN [cloud.async.AsyncJobManagerImpl] (Job-Executor-151:job-6526)
> >> Unable
> >> >> to unregister active job 6526 from JMX monitoring
>
>
> >> >> Here are log when I want to start VR (his state is stopeed, I want to
> >> >> start and got this)
>
> >> >> WARN [network.element.VpcVirtualRouterElement]
> >> >> (Job-Executor-153:job-6528) Network Ntwk[200|Public|1] is not
> associated
> >> >> with any VPC
> >> >> WARN [network.element.VpcVirtualRouterElement]
> >> >> (Job-Executor-153:job-6528) Network Ntwk[338|Guest|8] is not
> associated
> >> >> with any VPC
> >> >> WARN [network.element.VpcVirtualRouterElement]
> >> >> (Job-Executor-153:job-6528) Network Ntwk[202|Control|3] is not
> >> associated
> >> >> with any VPC
> >> >> ERROR [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-153:job-6528)
> >> >> Failed to start instance VM[DomainRouter|r-1015-VM]
> >> >> java.lang.NullPointerException
> >> >> at
> >> >>
> >>
> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
> >> >> at
> >> >>
> >>
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
> >> >> at
> >> >>
> >>
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouter(VirtualNetworkApplianceManagerImpl.java:2874)
> >> >> at
> >> >>
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouter(VirtualNetworkApplianceManagerImpl.java:2824)
> >> >> at
> >> >>
> >>
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> >> >> at
> >> >>
> >>
> org.apache.cloudstack.api.command.admin.router.StartRouterCmd.execute(StartRouterCmd.java:103)
> >> >> at
> com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
> >> >> at
> >> >>
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
> >> >> at
> >> >>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >> >> at
> >> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> >> >> at
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >> >> at
> >> >>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >> >> at java.lang.Thread.run(Thread.java:722)
>
>
> >> >> Best Regards,
> >> >> Michael
>
>
>
> >> --
> >> Pozdrowienia,
> >> Michael Lukzak
>
>
>
> --
> Pozdrowienia,
> Michael Lukzak
Re[4]: Urgent help needed. Can't start Virtual Router.
Posted by Michael Lukzak <mi...@vp.pl>.
I removed this host from CS, but still problem is.
How I can "rebuild VR" from zero?
It's looks like CS want to start VM (witch really is a VirtualRouter) that not exists on the host.
INFO [xen.resource.CitrixResourceBase] (DirectAgent-148:) VM does not exist on XenServer11681b28-5f38-4f20-aaae-0c9ed1e9ecc7
Or other way. How I can bind new network to VM?
Ony way is modify VM parameter in database?
Michael
> depending on your setup, but what I would do move all your Instances from
> host machine where the VR lives and reboot. Again, that's what I would do.
> you can try this link for XenServer 5
> http://blogs.citrix.com/2009/01/26/how-to-configure-an-nfs-based-iso-sr-using-local-storage-on-a-xenserver5-host/
> Thanks,
> On Thu, Aug 7, 2014 at 12:17 PM, Michael Lukzak <mi...@vp.pl> wrote:
>> Hi,
>> We are usning xenserver from Citrix.
>> I found some clue. In console to server on this host I see that nfs has
>> error "Stale NFS file handle".
>> How to remount NFS? Maybe this is a problem?
>> But why other VR are started and few of them not... ;(
>> Best regards,
>> Michael
>> > Hi Michael,
>> > if it helps, I would restart the agent on the host machine where VR
>> lives.
>> > it depends on hypervisor but on KVM, following command: service
>> > cloudstack-agent restart
>> > Thanks,
>> > Motty
>> > On Thu, Aug 7, 2014 at 11:52 AM, Michael Lukzak <mi...@vp.pl> wrote:
>> >> Hi,
>> >> One of our host are died (after very high load).
>> >> Host has a lot of Virtual Routers running.
>> >> We are using advanced networking.
>> >> Only way to bring machine alive was hard reset.
>> >> After this CloudStack decied to start all of VirtualRouterts,
>> >> when server was back online.
>> >> Most of VR are started, but couple don't.
>> >> Cloudstack want to start this VM, but can't ;(
>> >> Here are log from this action.
>> >> It's looks like that CS are looking for VR host on Xen host, but he
>> >> couldn't find.
>> >> Please help how to repair this situation.
>> >> Restart network with clean up options ending with the same resoult.
>> >> Is there any option to recreate VR?
>> >> I tried to remove VR from Infrastructure -> Virtual Router, but there is
>> >> still problem
>> >> with create new one.
>> >> Here are logs from "restart network with clean up options"
>> >> WARN [network.element.VpcVirtualRouterElement]
>> >> (Job-Executor-151:job-6526) Network Ntwk[200|Public|1] is not associated
>> >> with any VPC
>> >> WARN [network.element.VpcVirtualRouterElement]
>> >> (Job-Executor-151:job-6526) Network Ntwk[338|Guest|8] is not associated
>> >> with any VPC
>> >> WARN [network.element.VpcVirtualRouterElement]
>> >> (Job-Executor-151:job-6526) Network Ntwk[202|Control|3] is not
>> associated
>> >> with any VPC
>> >> ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-151:job-6526)
>> >> Failed to start instance VM[DomainRouter|r-1015-VM]
>> >> java.lang.NullPointerException
>> >> at
>> >>
>> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
>> >> at
>> >>
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
>> >> at
>> >>
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1924)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1902)
>> >> at
>> >>
>> com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:175)
>> >> at
>> >>
>> com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1518)
>> >> at
>> >>
>> com.cloud.network.NetworkManagerImpl.restartNetwork(NetworkManagerImpl.java:2473)
>> >> at
>> >>
>> com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1381)
>> >> at
>> >>
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>> >> at
>> >>
>> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
>> >> at
>> >> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> >> at
>> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> >> at java.lang.Thread.run(Thread.java:722)
>> >> INFO [xen.resource.CitrixResourceBase] (DirectAgent-16:) VM does not
>> >> exist on XenServer0d1d5015-4061-4554-96de-406155449296
>> >> WARN [cloud.network.NetworkManagerImpl] (Job-Executor-151:job-6526)
>> >> Failed to implement network Ntwk[338|Guest|8] elements and resources as
>> a
>> >> part of network restart due to
>> >> com.cloud.exception.AgentUnavailableException: Resource [Host:9] is
>> >> unreachable: Host 9: Unable to start instance due to null
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:841)
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1924)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1902)
>> >> at
>> >>
>> com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:175)
>> >> at
>> >>
>> com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1518)
>> >> at
>> >>
>> com.cloud.network.NetworkManagerImpl.restartNetwork(NetworkManagerImpl.java:2473)
>> >> at
>> >>
>> com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1381)
>> >> at
>> >>
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>> >> at
>> >>
>> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
>> >> at
>> >> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> >> at
>> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> >> at java.lang.Thread.run(Thread.java:722)
>> >> Caused by: java.lang.NullPointerException
>> >> at
>> >>
>> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
>> >> at
>> >>
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
>> >> at
>> >>
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
>> >> ... 23 more
>> >> WARN [cloud.network.NetworkServiceImpl] (Job-Executor-151:job-6526)
>> >> Network id=338 failed to restart.
>> >> WARN [cloud.async.AsyncJobManagerImpl] (Job-Executor-151:job-6526)
>> Unable
>> >> to unregister active job 6526 from JMX monitoring
>> >> Here are log when I want to start VR (his state is stopeed, I want to
>> >> start and got this)
>> >> WARN [network.element.VpcVirtualRouterElement]
>> >> (Job-Executor-153:job-6528) Network Ntwk[200|Public|1] is not associated
>> >> with any VPC
>> >> WARN [network.element.VpcVirtualRouterElement]
>> >> (Job-Executor-153:job-6528) Network Ntwk[338|Guest|8] is not associated
>> >> with any VPC
>> >> WARN [network.element.VpcVirtualRouterElement]
>> >> (Job-Executor-153:job-6528) Network Ntwk[202|Control|3] is not
>> associated
>> >> with any VPC
>> >> ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-153:job-6528)
>> >> Failed to start instance VM[DomainRouter|r-1015-VM]
>> >> java.lang.NullPointerException
>> >> at
>> >>
>> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
>> >> at
>> >>
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
>> >> at
>> >>
>> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
>> >> at
>> >>
>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouter(VirtualNetworkApplianceManagerImpl.java:2874)
>> >> at
>> >>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouter(VirtualNetworkApplianceManagerImpl.java:2824)
>> >> at
>> >>
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>> >> at
>> >>
>> org.apache.cloudstack.api.command.admin.router.StartRouterCmd.execute(StartRouterCmd.java:103)
>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
>> >> at
>> >> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
>> >> at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> >> at
>> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> >> at
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> >> at java.lang.Thread.run(Thread.java:722)
>> >> Best Regards,
>> >> Michael
>> --
>> Pozdrowienia,
>> Michael Lukzak
--
Pozdrowienia,
Michael Lukzak
Re: Re[2]: Urgent help needed. Can't start Virtual Router.
Posted by motty cruz <mo...@gmail.com>.
depending on your setup, but what I would do move all your Instances from
host machine where the VR lives and reboot. Again, that's what I would do.
you can try this link for XenServer 5
http://blogs.citrix.com/2009/01/26/how-to-configure-an-nfs-based-iso-sr-using-local-storage-on-a-xenserver5-host/
Thanks,
On Thu, Aug 7, 2014 at 12:17 PM, Michael Lukzak <mi...@vp.pl> wrote:
> Hi,
>
> We are usning xenserver from Citrix.
>
> I found some clue. In console to server on this host I see that nfs has
> error "Stale NFS file handle".
>
> How to remount NFS? Maybe this is a problem?
> But why other VR are started and few of them not... ;(
>
> Best regards,
> Michael
>
>
>
> > Hi Michael,
> > if it helps, I would restart the agent on the host machine where VR
> lives.
>
> > it depends on hypervisor but on KVM, following command: service
> > cloudstack-agent restart
>
>
> > Thanks,
> > Motty
>
>
> > On Thu, Aug 7, 2014 at 11:52 AM, Michael Lukzak <mi...@vp.pl> wrote:
>
> >> Hi,
>
> >> One of our host are died (after very high load).
> >> Host has a lot of Virtual Routers running.
>
> >> We are using advanced networking.
>
> >> Only way to bring machine alive was hard reset.
>
> >> After this CloudStack decied to start all of VirtualRouterts,
> >> when server was back online.
>
> >> Most of VR are started, but couple don't.
> >> Cloudstack want to start this VM, but can't ;(
> >> Here are log from this action.
>
> >> It's looks like that CS are looking for VR host on Xen host, but he
> >> couldn't find.
>
> >> Please help how to repair this situation.
> >> Restart network with clean up options ending with the same resoult.
>
> >> Is there any option to recreate VR?
> >> I tried to remove VR from Infrastructure -> Virtual Router, but there is
> >> still problem
> >> with create new one.
>
>
>
> >> Here are logs from "restart network with clean up options"
> >> WARN [network.element.VpcVirtualRouterElement]
> >> (Job-Executor-151:job-6526) Network Ntwk[200|Public|1] is not associated
> >> with any VPC
> >> WARN [network.element.VpcVirtualRouterElement]
> >> (Job-Executor-151:job-6526) Network Ntwk[338|Guest|8] is not associated
> >> with any VPC
> >> WARN [network.element.VpcVirtualRouterElement]
> >> (Job-Executor-151:job-6526) Network Ntwk[202|Control|3] is not
> associated
> >> with any VPC
> >> ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-151:job-6526)
> >> Failed to start instance VM[DomainRouter|r-1015-VM]
> >> java.lang.NullPointerException
> >> at
> >>
> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
> >> at
> >>
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
> >> at
> >>
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
> >> at
> >>
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
> >> at
> >>
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1924)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1902)
> >> at
> >>
> com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:175)
> >> at
> >>
> com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1518)
> >> at
> >>
> com.cloud.network.NetworkManagerImpl.restartNetwork(NetworkManagerImpl.java:2473)
> >> at
> >>
> com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1381)
> >> at
> >>
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> >> at
> >>
> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
> >> at
> >> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
> >> at
> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >> at
> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >> at java.lang.Thread.run(Thread.java:722)
> >> INFO [xen.resource.CitrixResourceBase] (DirectAgent-16:) VM does not
> >> exist on XenServer0d1d5015-4061-4554-96de-406155449296
> >> WARN [cloud.network.NetworkManagerImpl] (Job-Executor-151:job-6526)
> >> Failed to implement network Ntwk[338|Guest|8] elements and resources as
> a
> >> part of network restart due to
> >> com.cloud.exception.AgentUnavailableException: Resource [Host:9] is
> >> unreachable: Host 9: Unable to start instance due to null
> >> at
> >>
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:841)
> >> at
> >>
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1924)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1902)
> >> at
> >>
> com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:175)
> >> at
> >>
> com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1518)
> >> at
> >>
> com.cloud.network.NetworkManagerImpl.restartNetwork(NetworkManagerImpl.java:2473)
> >> at
> >>
> com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1381)
> >> at
> >>
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> >> at
> >>
> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
> >> at
> >> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
> >> at
> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >> at
> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >> at java.lang.Thread.run(Thread.java:722)
> >> Caused by: java.lang.NullPointerException
> >> at
> >>
> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
> >> at
> >>
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
> >> at
> >>
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
> >> at
> >>
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
> >> ... 23 more
> >> WARN [cloud.network.NetworkServiceImpl] (Job-Executor-151:job-6526)
> >> Network id=338 failed to restart.
> >> WARN [cloud.async.AsyncJobManagerImpl] (Job-Executor-151:job-6526)
> Unable
> >> to unregister active job 6526 from JMX monitoring
>
>
> >> Here are log when I want to start VR (his state is stopeed, I want to
> >> start and got this)
>
> >> WARN [network.element.VpcVirtualRouterElement]
> >> (Job-Executor-153:job-6528) Network Ntwk[200|Public|1] is not associated
> >> with any VPC
> >> WARN [network.element.VpcVirtualRouterElement]
> >> (Job-Executor-153:job-6528) Network Ntwk[338|Guest|8] is not associated
> >> with any VPC
> >> WARN [network.element.VpcVirtualRouterElement]
> >> (Job-Executor-153:job-6528) Network Ntwk[202|Control|3] is not
> associated
> >> with any VPC
> >> ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-153:job-6528)
> >> Failed to start instance VM[DomainRouter|r-1015-VM]
> >> java.lang.NullPointerException
> >> at
> >>
> com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346)
> >> at
> >>
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241)
> >> at
> >>
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205)
> >> at
> >>
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763)
> >> at
> >>
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouter(VirtualNetworkApplianceManagerImpl.java:2874)
> >> at
> >>
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouter(VirtualNetworkApplianceManagerImpl.java:2824)
> >> at
> >>
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> >> at
> >>
> org.apache.cloudstack.api.command.admin.router.StartRouterCmd.execute(StartRouterCmd.java:103)
> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
> >> at
> >> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
> >> at
> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >> at
> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >> at java.lang.Thread.run(Thread.java:722)
>
>
> >> Best Regards,
> >> Michael
>
>
>
> --
> Pozdrowienia,
> Michael Lukzak