You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Mohamed Infaz <in...@cse.mrt.ac.lk> on 2015/01/01 09:33:00 UTC
Re: Unable to start VM due to concurrent operation
Hi Somesh,
Yes i was successful in stopping both the system vm's and starting them
back i also could access them through the console that cloudstack provides.
May be the reason for that code snippet is that i didn't register the
template but i ran this script to install the system vm's.
1. /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
\
2. -m /srv/images/secondary \
3. -u http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
\
4. -h kvm \
5. -F
this was on the docs. How about upgrading my cloudstack version to 4.4.
This whole thing happened in 4.3.
Thank you.
On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com> wrote:
> You mean the system VMs for Zone 1 are running? Are they showing as
> connected in the DB?
>
> If they are, can you stop these VMs and see if they start okay.
> ________________________________________
> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
> Sent: Tuesday, December 30, 2014 21:26
> To: users@cloudstack.apache.org
> Cc: dev@cloudstack.apache.org
> Subject: Re: Unable to start VM due to concurrent operation
>
> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the template i
> used.
>
>
> On 31 December 2014 at 07:49, Mohamed Infaz <in...@cse.mrt.ac.lk>
> wrote:
>
> > Hi Somesh,
> >
> > Thank you for the reply. Well both of my system vm's are running and it
> is
> > downloading the iso image. And i was able to ping them. The system vm
> > template i am using is for the version 4.3 cloudstack.
> >
> > Thank you.
> >
> > On 30 December 2014 at 23:59, Somesh Naidu <So...@citrix.com>
> > wrote:
> >
> >> Mohamed,
> >>
> >> The log snippet you have shared doesn't seem to be relevant to the
> >> concurrent operation exception (can't see that exception stack).
> >>
> >> The message that did catch my attention though is,
> >> -
> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> center
> >> 1, wait until it is ready to launch secondary storage vm
> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> storage
> >> VM yet
> >> -
> >>
> >> That's pointing to missing system VM template. Please verify that you
> >> have correctly provisioned the secondary storage with the correct
> system VM
> >> template (as per the docs).
> >>
> >> -Somesh
> >>
> >> -----Original Message-----
> >> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
> >> Sent: Tuesday, December 30, 2014 11:03 AM
> >> To: users@cloudstack.apache.org
> >> Cc: dev@cloudstack.apache.org
> >> Subject: Unable to start VM due to concurrent operation
> >>
> >> Hi All,
> >>
> >> I have successfully deployed cloudstack 4.3 with two hosts and the setup
> >> runs the management server on another physical host. I had issues with
> >> downloading an ISO but finally i was able to download an ISO image.
> When i
> >> want to create an instance i get the following error message saying
> >> "Unable
> >> to start VM due to concurrent operation" i did some searching on the
> topic
> >> and it said version 4.3 system template solved the issue. These are some
> >> of
> >> the MS logs that i collected what could be the issue?
> >>
> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> center
> >> 1, wait until it is ready to launch secondary storage vm
> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> >> storage
> >> VM yet
> >> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-9:null) Ping from 4
> >> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq 3-91695: {
> Cmd
> >> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695: {
> Ans: ,
> >> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >> 2014-12-31 05:25:15,239 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
> >> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats collector
> >> is
> >> running...
> >> 2014-12-31 05:25:15,362 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
> >> 2014-12-31 05:25:15,366 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update status.
> >> 2014-12-31 05:25:15,367 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update RvR
> >> status.
> >> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being called
> at
> >> 2014-12-30 23:55:15 GMT
> >> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
> >> 2014-12-30 23:55:15 GMT
> >> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-14:null) Ping from 3
> >> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-10:null) Ping from 2
> >> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
> >> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on ping:
> >> [1]
> >> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
> (AgentMonitor-1:ctx-f10d32f0)
> >> Ping timeout for host 1, do invstigation
> >> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
> disconnected
> >> with event PingTimeout
> >> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
> >> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
> >> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
> MgmtId:
> >> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
> >> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: { Ans: ,
> >> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
> >>
> >>
> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
> >> is alive","wait":0}}] }
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: { Ans: ,
> >> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, { CheckHealthAnswer
> >> } }
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) Details from executing class
> >> com.cloud.agent.api.CheckHealthCommand: resource is alive
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
> checkHeathCommand,
> >> reporting that agent is Up
> >> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
> >> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and running
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
> >> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state = Enabled,
> >> Agent
> >> event = Ping, Host id = 1, name = virtualops-h4]
> >> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
> >> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1; name =
> >> virtualops-h4; old status = Up; event = Ping; new status = Up; old
> update
> >> count = 34; new update count = 35]
> >> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq 3-91697: {
> Cmd
> >> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697: {
> Ans: ,
> >> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-1:ctx-2acfc
> >>
> >> Thank you.
> >>
> >
> >
>
Re: Unable to start VM due to concurrent operation
Posted by prapul cool <pr...@yahoo.com.INVALID>.
Hi,
please once check the secondary storage path for download vm is right or nor?
Thanks,Prapul Sriram
On Thursday, 1 January 2015 2:03 PM, Mohamed Infaz <in...@cse.mrt.ac.lk> wrote:
Hi Somesh,
Yes i was successful in stopping both the system vm's and starting them
back i also could access them through the console that cloudstack provides.
May be the reason for that code snippet is that i didn't register the
template but i ran this script to install the system vm's.
1. /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
\
2. -m /srv/images/secondary \
3. -u http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
\
4. -h kvm \
5. -F
this was on the docs. How about upgrading my cloudstack version to 4.4.
This whole thing happened in 4.3.
Thank you.
On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com> wrote:
> You mean the system VMs for Zone 1 are running? Are they showing as
> connected in the DB?
>
> If they are, can you stop these VMs and see if they start okay.
> ________________________________________
> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
> Sent: Tuesday, December 30, 2014 21:26
> To: users@cloudstack.apache.org
> Cc: dev@cloudstack.apache.org
> Subject: Re: Unable to start VM due to concurrent operation
>
> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the template i
> used.
>
>
> On 31 December 2014 at 07:49, Mohamed Infaz <in...@cse.mrt.ac.lk>
> wrote:
>
> > Hi Somesh,
> >
> > Thank you for the reply. Well both of my system vm's are running and it
> is
> > downloading the iso image. And i was able to ping them. The system vm
> > template i am using is for the version 4.3 cloudstack.
> >
> > Thank you.
> >
> > On 30 December 2014 at 23:59, Somesh Naidu <So...@citrix.com>
> > wrote:
> >
> >> Mohamed,
> >>
> >> The log snippet you have shared doesn't seem to be relevant to the
> >> concurrent operation exception (can't see that exception stack).
> >>
> >> The message that did catch my attention though is,
> >> -
> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> center
> >> 1, wait until it is ready to launch secondary storage vm
> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> storage
> >> VM yet
> >> -
> >>
> >> That's pointing to missing system VM template. Please verify that you
> >> have correctly provisioned the secondary storage with the correct
> system VM
> >> template (as per the docs).
> >>
> >> -Somesh
> >>
> >> -----Original Message-----
> >> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
> >> Sent: Tuesday, December 30, 2014 11:03 AM
> >> To: users@cloudstack.apache.org
> >> Cc: dev@cloudstack.apache.org
> >> Subject: Unable to start VM due to concurrent operation
> >>
> >> Hi All,
> >>
> >> I have successfully deployed cloudstack 4.3 with two hosts and the setup
> >> runs the management server on another physical host. I had issues with
> >> downloading an ISO but finally i was able to download an ISO image.
> When i
> >> want to create an instance i get the following error message saying
> >> "Unable
> >> to start VM due to concurrent operation" i did some searching on the
> topic
> >> and it said version 4.3 system template solved the issue. These are some
> >> of
> >> the MS logs that i collected what could be the issue?
> >>
> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> center
> >> 1, wait until it is ready to launch secondary storage vm
> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> >> storage
> >> VM yet
> >> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-9:null) Ping from 4
> >> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq 3-91695: {
> Cmd
> >> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695: {
> Ans: ,
> >> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >> 2014-12-31 05:25:15,239 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
> >> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats collector
> >> is
> >> running...
> >> 2014-12-31 05:25:15,362 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
> >> 2014-12-31 05:25:15,366 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update status.
> >> 2014-12-31 05:25:15,367 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update RvR
> >> status.
> >> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being called
> at
> >> 2014-12-30 23:55:15 GMT
> >> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
> >> 2014-12-30 23:55:15 GMT
> >> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-14:null) Ping from 3
> >> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-10:null) Ping from 2
> >> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
> >> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on ping:
> >> [1]
> >> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
> (AgentMonitor-1:ctx-f10d32f0)
> >> Ping timeout for host 1, do invstigation
> >> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
> disconnected
> >> with event PingTimeout
> >> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
> >> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
> >> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
> MgmtId:
> >> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
> >> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: { Ans: ,
> >> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
> >>
> >>
> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
> >> is alive","wait":0}}] }
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: { Ans: ,
> >> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, { CheckHealthAnswer
> >> } }
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) Details from executing class
> >> com.cloud.agent.api.CheckHealthCommand: resource is alive
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
> checkHeathCommand,
> >> reporting that agent is Up
> >> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
> >> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and running
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
> >> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state = Enabled,
> >> Agent
> >> event = Ping, Host id = 1, name = virtualops-h4]
> >> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
> >> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1; name =
> >> virtualops-h4; old status = Up; event = Ping; new status = Up; old
> update
> >> count = 34; new update count = 35]
> >> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq 3-91697: {
> Cmd
> >> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697: {
> Ans: ,
> >> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-1:ctx-2acfc
> >>
> >> Thank you.
> >>
> >
> >
>
Re: Unable to start VM due to concurrent operation
Posted by prapul cool <pr...@yahoo.com.INVALID>.
Hi,
please once check the secondary storage path for download vm is right or nor?
Thanks,Prapul Sriram
On Thursday, 1 January 2015 2:03 PM, Mohamed Infaz <in...@cse.mrt.ac.lk> wrote:
Hi Somesh,
Yes i was successful in stopping both the system vm's and starting them
back i also could access them through the console that cloudstack provides.
May be the reason for that code snippet is that i didn't register the
template but i ran this script to install the system vm's.
1. /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
\
2. -m /srv/images/secondary \
3. -u http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
\
4. -h kvm \
5. -F
this was on the docs. How about upgrading my cloudstack version to 4.4.
This whole thing happened in 4.3.
Thank you.
On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com> wrote:
> You mean the system VMs for Zone 1 are running? Are they showing as
> connected in the DB?
>
> If they are, can you stop these VMs and see if they start okay.
> ________________________________________
> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
> Sent: Tuesday, December 30, 2014 21:26
> To: users@cloudstack.apache.org
> Cc: dev@cloudstack.apache.org
> Subject: Re: Unable to start VM due to concurrent operation
>
> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the template i
> used.
>
>
> On 31 December 2014 at 07:49, Mohamed Infaz <in...@cse.mrt.ac.lk>
> wrote:
>
> > Hi Somesh,
> >
> > Thank you for the reply. Well both of my system vm's are running and it
> is
> > downloading the iso image. And i was able to ping them. The system vm
> > template i am using is for the version 4.3 cloudstack.
> >
> > Thank you.
> >
> > On 30 December 2014 at 23:59, Somesh Naidu <So...@citrix.com>
> > wrote:
> >
> >> Mohamed,
> >>
> >> The log snippet you have shared doesn't seem to be relevant to the
> >> concurrent operation exception (can't see that exception stack).
> >>
> >> The message that did catch my attention though is,
> >> -
> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> center
> >> 1, wait until it is ready to launch secondary storage vm
> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> storage
> >> VM yet
> >> -
> >>
> >> That's pointing to missing system VM template. Please verify that you
> >> have correctly provisioned the secondary storage with the correct
> system VM
> >> template (as per the docs).
> >>
> >> -Somesh
> >>
> >> -----Original Message-----
> >> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
> >> Sent: Tuesday, December 30, 2014 11:03 AM
> >> To: users@cloudstack.apache.org
> >> Cc: dev@cloudstack.apache.org
> >> Subject: Unable to start VM due to concurrent operation
> >>
> >> Hi All,
> >>
> >> I have successfully deployed cloudstack 4.3 with two hosts and the setup
> >> runs the management server on another physical host. I had issues with
> >> downloading an ISO but finally i was able to download an ISO image.
> When i
> >> want to create an instance i get the following error message saying
> >> "Unable
> >> to start VM due to concurrent operation" i did some searching on the
> topic
> >> and it said version 4.3 system template solved the issue. These are some
> >> of
> >> the MS logs that i collected what could be the issue?
> >>
> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> center
> >> 1, wait until it is ready to launch secondary storage vm
> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> >> storage
> >> VM yet
> >> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-9:null) Ping from 4
> >> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq 3-91695: {
> Cmd
> >> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695: {
> Ans: ,
> >> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >> 2014-12-31 05:25:15,239 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
> >> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats collector
> >> is
> >> running...
> >> 2014-12-31 05:25:15,362 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
> >> 2014-12-31 05:25:15,366 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update status.
> >> 2014-12-31 05:25:15,367 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update RvR
> >> status.
> >> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being called
> at
> >> 2014-12-30 23:55:15 GMT
> >> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
> >> 2014-12-30 23:55:15 GMT
> >> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-14:null) Ping from 3
> >> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-10:null) Ping from 2
> >> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
> >> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on ping:
> >> [1]
> >> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
> (AgentMonitor-1:ctx-f10d32f0)
> >> Ping timeout for host 1, do invstigation
> >> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
> disconnected
> >> with event PingTimeout
> >> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
> >> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
> >> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
> MgmtId:
> >> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
> >> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: { Ans: ,
> >> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
> >>
> >>
> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
> >> is alive","wait":0}}] }
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: { Ans: ,
> >> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, { CheckHealthAnswer
> >> } }
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) Details from executing class
> >> com.cloud.agent.api.CheckHealthCommand: resource is alive
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
> checkHeathCommand,
> >> reporting that agent is Up
> >> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
> >> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and running
> >> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
> >> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state = Enabled,
> >> Agent
> >> event = Ping, Host id = 1, name = virtualops-h4]
> >> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
> >> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1; name =
> >> virtualops-h4; old status = Up; event = Ping; new status = Up; old
> update
> >> count = 34; new update count = 35]
> >> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq 3-91697: {
> Cmd
> >> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >> \"connections\": []\n}","wait":0}}] }
> >> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697: {
> Ans: ,
> >> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-1:ctx-2acfc
> >>
> >> Thank you.
> >>
> >
> >
>
Re: Unable to start VM due to concurrent operation
Posted by Mohamed Infaz <in...@cse.mrt.ac.lk>.
hi,
And the status of the template i am trying to create the vm from is fully
downloaded. It's status is ready.
Thank you.
On 2 January 2015 at 21:04, Mohamed Infaz <in...@cse.mrt.ac.lk> wrote:
> Hi,
>
> This is the screen shot. It stops around 96-98%
>
> On 2 January 2015 at 20:58, Somesh Naidu <So...@citrix.com> wrote:
>
>> What's the status of template are you trying to create a VM from? Please
>> share either DB output or UI screenshot.
>>
>> -----Original Message-----
>> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
>> Sent: Friday, January 02, 2015 7:03 AM
>> To: users@cloudstack.apache.org
>> Cc: dev@cloudstack.apache.org
>> Subject: Re: Unable to start VM due to concurrent operation
>>
>> Hi all,
>>
>> I did redeploy cloudstack and i used the
>>
>> >> 1. /usr/share/cloudstack-common/
>> scripts/storage/secondary/cloud-install-sys-tmplt \
>> >> 2. -m /srv/images/secondary \
>> >> 3. -u
>>
>> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
>> \
>> >> 4. -h kvm \
>> >> 5. -F
>>
>> The following script to deploy system vm's. After some time they started
>> working! Now when i try to deploy and vm
>> i am getting the following error message.
>> "com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
>> unreachable: Host 1: Unable to start instance due to Template 3 has not
>> been completely downloaded to zone 1"
>>
>> I did reregister the latest system vm template and i was able to
>> successfully download it. But there are two inbuilt templates where it
>> says
>> in the status that the connection was time out. And one of these in built
>> template is an in built old system vm template. How can i solve this issue
>> now there is a system vm running properly in the environment and i have
>> the
>> latest template registered but it seems so cloudstack searches for those
>> two templates which it couldn't download. Because of this i still can't
>> create a virtual machine.
>>
>> Thank you.
>>
>> On 2 January 2015 at 08:08, Abhinandan Prateek <
>> abhinandan.prateek@shapeblue.com> wrote:
>>
>> > The sec store has some issues, you can login into KVM host and check if
>> > the sec storage mount point is visible and the said folder is there.
>> Check
>> > if ssvm is running fine.
>> > It is also possible that the sec store is fine but the template that is
>> > being used to create VM is in inconsistent state, then you need to
>> remove
>> > and re-register the template.
>> >
>> > -abhi
>> >
>> > > On 01-Jan-2015, at 8:35 pm, Mohamed Infaz <in...@cse.mrt.ac.lk>
>> > wrote:
>> > >
>> > > Hi,
>> > > I updated my cloudstack deployment to 4.4.2 to 4.3.1. Now when we try
>> to
>> > > deploy a VM i am getting following error
>> > >
>> > > message. com.cloud.exception.StorageUnavailableException: Resource
>> > > [StoragePool:2] is unreachable: Unable to create
>> > > Vol[30|vm=58|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
>> > > org.libvirt.LibvirtException: internal error: Child process
>> (/bin/mount
>> > > 10.8.100.201:/mnt/secondary/template/tmpl/2/205
>> > > /mnt/8c6fa424-cbc7-3421-8f05-d69a43ffa15c) unexpected exit status 32:
>> > > mount.nfs: mounting 10.8.100.201:/mnt/secondary/template/tmpl/2/205
>> > failed,
>> > > reason given by server: No such file or directory
>> > >
>> > >
>> > >
>> > > Thank you.
>> > >
>> > >
>> > > On 1 January 2015 at 14:03, Mohamed Infaz <in...@cse.mrt.ac.lk>
>> > wrote:
>> > >
>> > >> Hi Somesh,
>> > >>
>> > >> Yes i was successful in stopping both the system vm's and starting
>> them
>> > >> back i also could access them through the console that cloudstack
>> > provides.
>> > >> May be the reason for that code snippet is that i didn't register the
>> > >> template but i ran this script to install the system vm's.
>> > >>
>> > >>
>> > >> 1.
>> >
>> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
>> > \
>> > >> 2. -m /srv/images/secondary \
>> > >> 3. -u
>> >
>> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
>> > \
>> > >> 4. -h kvm \
>> > >> 5. -F
>> > >>
>> > >> this was on the docs. How about upgrading my cloudstack version to
>> 4.4.
>> > >> This whole thing happened in 4.3.
>> > >>
>> > >> Thank you.
>> > >>
>> > >> On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com>
>> > wrote:
>> > >>
>> > >>> You mean the system VMs for Zone 1 are running? Are they showing as
>> > >>> connected in the DB?
>> > >>>
>> > >>> If they are, can you stop these VMs and see if they start okay.
>> > >>> ________________________________________
>> > >>> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
>> > >>> Sent: Tuesday, December 30, 2014 21:26
>> > >>> To: users@cloudstack.apache.org
>> > >>> Cc: dev@cloudstack.apache.org
>> > >>> Subject: Re: Unable to start VM due to concurrent operation
>> > >>>
>> > >>> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the
>> > template i
>> > >>> used.
>> > >>>
>> > >>>
>> > >>> On 31 December 2014 at 07:49, Mohamed Infaz <infaz.10@cse.mrt.ac.lk
>> >
>> > >>> wrote:
>> > >>>
>> > >>>> Hi Somesh,
>> > >>>>
>> > >>>> Thank you for the reply. Well both of my system vm's are running
>> and
>> > it
>> > >>> is
>> > >>>> downloading the iso image. And i was able to ping them. The system
>> vm
>> > >>>> template i am using is for the version 4.3 cloudstack.
>> > >>>>
>> > >>>> Thank you.
>> > >>>>
>> > >>>> On 30 December 2014 at 23:59, Somesh Naidu <
>> Somesh.Naidu@citrix.com>
>> > >>>> wrote:
>> > >>>>
>> > >>>>> Mohamed,
>> > >>>>>
>> > >>>>> The log snippet you have shared doesn't seem to be relevant to the
>> > >>>>> concurrent operation exception (can't see that exception stack).
>> > >>>>>
>> > >>>>> The message that did catch my attention though is,
>> > >>>>> -
>> > >>>>> 2014-12-31 05:25:07,972 DEBUG
>> [c.c.s.s.SecondaryStorageManagerImpl]
>> > >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at
>> data
>> > >>> center
>> > >>>>> 1, wait until it is ready to launch secondary storage vm
>> > >>>>> 2014-12-31 05:25:07,972 DEBUG
>> [c.c.s.s.SecondaryStorageManagerImpl]
>> > >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch
>> secondary
>> > >>> storage
>> > >>>>> VM yet
>> > >>>>> -
>> > >>>>>
>> > >>>>> That's pointing to missing system VM template. Please verify that
>> you
>> > >>>>> have correctly provisioned the secondary storage with the correct
>> > >>> system VM
>> > >>>>> template (as per the docs).
>> > >>>>>
>> > >>>>> -Somesh
>> > >>>>>
>> > >>>>> -----Original Message-----
>> > >>>>> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
>> > >>>>> Sent: Tuesday, December 30, 2014 11:03 AM
>> > >>>>> To: users@cloudstack.apache.org
>> > >>>>> Cc: dev@cloudstack.apache.org
>> > >>>>> Subject: Unable to start VM due to concurrent operation
>> > >>>>>
>> > >>>>> Hi All,
>> > >>>>>
>> > >>>>> I have successfully deployed cloudstack 4.3 with two hosts and the
>> > >>> setup
>> > >>>>> runs the management server on another physical host. I had issues
>> > with
>> > >>>>> downloading an ISO but finally i was able to download an ISO
>> image.
>> > >>> When i
>> > >>>>> want to create an instance i get the following error message
>> saying
>> > >>>>> "Unable
>> > >>>>> to start VM due to concurrent operation" i did some searching on
>> the
>> > >>> topic
>> > >>>>> and it said version 4.3 system template solved the issue. These
>> are
>> > >>> some
>> > >>>>> of
>> > >>>>> the MS logs that i collected what could be the issue?
>> > >>>>>
>> > >>>>> 2014-12-31 05:25:07,972 DEBUG
>> [c.c.s.s.SecondaryStorageManagerImpl]
>> > >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at
>> data
>> > >>> center
>> > >>>>> 1, wait until it is ready to launch secondary storage vm
>> > >>>>> 2014-12-31 05:25:07,972 DEBUG
>> [c.c.s.s.SecondaryStorageManagerImpl]
>> > >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch
>> secondary
>> > >>>>> storage
>> > >>>>> VM yet
>> > >>>>> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-9:null) Ping from 4
>> > >>>>> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq
>> 3-91695:
>> > {
>> > >>> Cmd
>> > >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >>>>>
>> > >>>>>
>> > >>>
>> >
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
>> > >>>>> \"connections\": []\n}","wait":0}}] }
>> > >>>>> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695:
>> {
>> > >>> Ans: ,
>> > >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
>> > >>>>>
>> > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
>> > >>>>> 2014-12-31 05:25:15,239 DEBUG
>> [c.c.n.ExternalDeviceUsageManagerImpl]
>> > >>>>> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats
>> > >>> collector
>> > >>>>> is
>> > >>>>> running...
>> > >>>>> 2014-12-31 05:25:15,362 DEBUG
>> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >>>>> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
>> > >>>>> 2014-12-31 05:25:15,366 DEBUG
>> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update
>> > status.
>> > >>>>> 2014-12-31 05:25:15,367 DEBUG
>> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update
>> RvR
>> > >>>>> status.
>> > >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> > >>>>> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being
>> > >>> called at
>> > >>>>> 2014-12-30 23:55:15 GMT
>> > >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> > >>>>> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
>> > >>>>> 2014-12-30 23:55:15 GMT
>> > >>>>> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-14:null) Ping from 3
>> > >>>>> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-10:null) Ping from 2
>> > >>>>> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on
>> > >>> ping:
>> > >>>>> [1]
>> > >>>>> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
>> > >>> (AgentMonitor-1:ctx-f10d32f0)
>> > >>>>> Ping timeout for host 1, do invstigation
>> > >>>>> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
>> > >>> disconnected
>> > >>>>> with event PingTimeout
>> > >>>>> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
>> > >>>>> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
>> > >>> MgmtId:
>> > >>>>> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
>> > >>>>> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
>> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
>> > >>>>> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: {
>> Ans: ,
>> > >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
>> > >>>>>
>> > >>>>>
>> > >>>
>> >
>> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
>> > >>>>> is alive","wait":0}}] }
>> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: {
>> Ans: ,
>> > >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, {
>> > >>> CheckHealthAnswer
>> > >>>>> } }
>> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Details from executing class
>> > >>>>> com.cloud.agent.api.CheckHealthCommand: resource is alive
>> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
>> > >>> checkHeathCommand,
>> > >>>>> reporting that agent is Up
>> > >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
>> > >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and
>> > >>> running
>> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state =
>> Enabled,
>> > >>>>> Agent
>> > >>>>> event = Ping, Host id = 1, name = virtualops-h4]
>> > >>>>> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1;
>> name =
>> > >>>>> virtualops-h4; old status = Up; event = Ping; new status = Up; old
>> > >>> update
>> > >>>>> count = 34; new update count = 35]
>> > >>>>> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq
>> 3-91697:
>> > {
>> > >>> Cmd
>> > >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >>>>>
>> > >>>>>
>> > >>>
>> >
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
>> > >>>>> \"connections\": []\n}","wait":0}}] }
>> > >>>>> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697:
>> {
>> > >>> Ans: ,
>> > >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
>> > >>>>>
>> > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
>> > >>>>> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
>> > >>>>> (StatsCollector-1:ctx-2acfc
>> > >>>>>
>> > >>>>> Thank you.
>> > >>>>>
>> > >>>>
>> > >>>>
>> > >>>
>> > >>
>> > >>
>> >
>> > Find out more about ShapeBlue and our range of CloudStack related
>> services
>> >
>> > IaaS Cloud Design & Build<
>> > http://shapeblue.com/iaas-cloud-design-and-build//>
>> > CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/
>> >
>> > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>> > CloudStack Software Engineering<
>> > http://shapeblue.com/cloudstack-software-engineering/>
>> > CloudStack Infrastructure Support<
>> > http://shapeblue.com/cloudstack-infrastructure-support/>
>> > CloudStack Bootcamp Training Courses<
>> > http://shapeblue.com/cloudstack-training/>
>> >
>> > This email and any attachments to it may be confidential and are
>> intended
>> > solely for the use of the individual to whom it is addressed. Any views
>> or
>> > opinions expressed are solely those of the author and do not necessarily
>> > represent those of Shape Blue Ltd or related companies. If you are not
>> the
>> > intended recipient of this email, you must neither take any action based
>> > upon its contents, nor copy or show it to anyone. Please contact the
>> sender
>> > if you believe you have received this email in error. Shape Blue Ltd is
>> a
>> > company incorporated in England & Wales. ShapeBlue Services India LLP
>> is a
>> > company incorporated in India and is operated under license from Shape
>> Blue
>> > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
>> Brasil
>> > and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd
>> is
>> > a company registered by The Republic of South Africa and is traded under
>> > license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>> >
>>
>
>
Re: Unable to start VM due to concurrent operation
Posted by Mohamed Infaz <in...@cse.mrt.ac.lk>.
hi,
And the status of the template i am trying to create the vm from is fully
downloaded. It's status is ready.
Thank you.
On 2 January 2015 at 21:04, Mohamed Infaz <in...@cse.mrt.ac.lk> wrote:
> Hi,
>
> This is the screen shot. It stops around 96-98%
>
> On 2 January 2015 at 20:58, Somesh Naidu <So...@citrix.com> wrote:
>
>> What's the status of template are you trying to create a VM from? Please
>> share either DB output or UI screenshot.
>>
>> -----Original Message-----
>> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
>> Sent: Friday, January 02, 2015 7:03 AM
>> To: users@cloudstack.apache.org
>> Cc: dev@cloudstack.apache.org
>> Subject: Re: Unable to start VM due to concurrent operation
>>
>> Hi all,
>>
>> I did redeploy cloudstack and i used the
>>
>> >> 1. /usr/share/cloudstack-common/
>> scripts/storage/secondary/cloud-install-sys-tmplt \
>> >> 2. -m /srv/images/secondary \
>> >> 3. -u
>>
>> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
>> \
>> >> 4. -h kvm \
>> >> 5. -F
>>
>> The following script to deploy system vm's. After some time they started
>> working! Now when i try to deploy and vm
>> i am getting the following error message.
>> "com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
>> unreachable: Host 1: Unable to start instance due to Template 3 has not
>> been completely downloaded to zone 1"
>>
>> I did reregister the latest system vm template and i was able to
>> successfully download it. But there are two inbuilt templates where it
>> says
>> in the status that the connection was time out. And one of these in built
>> template is an in built old system vm template. How can i solve this issue
>> now there is a system vm running properly in the environment and i have
>> the
>> latest template registered but it seems so cloudstack searches for those
>> two templates which it couldn't download. Because of this i still can't
>> create a virtual machine.
>>
>> Thank you.
>>
>> On 2 January 2015 at 08:08, Abhinandan Prateek <
>> abhinandan.prateek@shapeblue.com> wrote:
>>
>> > The sec store has some issues, you can login into KVM host and check if
>> > the sec storage mount point is visible and the said folder is there.
>> Check
>> > if ssvm is running fine.
>> > It is also possible that the sec store is fine but the template that is
>> > being used to create VM is in inconsistent state, then you need to
>> remove
>> > and re-register the template.
>> >
>> > -abhi
>> >
>> > > On 01-Jan-2015, at 8:35 pm, Mohamed Infaz <in...@cse.mrt.ac.lk>
>> > wrote:
>> > >
>> > > Hi,
>> > > I updated my cloudstack deployment to 4.4.2 to 4.3.1. Now when we try
>> to
>> > > deploy a VM i am getting following error
>> > >
>> > > message. com.cloud.exception.StorageUnavailableException: Resource
>> > > [StoragePool:2] is unreachable: Unable to create
>> > > Vol[30|vm=58|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
>> > > org.libvirt.LibvirtException: internal error: Child process
>> (/bin/mount
>> > > 10.8.100.201:/mnt/secondary/template/tmpl/2/205
>> > > /mnt/8c6fa424-cbc7-3421-8f05-d69a43ffa15c) unexpected exit status 32:
>> > > mount.nfs: mounting 10.8.100.201:/mnt/secondary/template/tmpl/2/205
>> > failed,
>> > > reason given by server: No such file or directory
>> > >
>> > >
>> > >
>> > > Thank you.
>> > >
>> > >
>> > > On 1 January 2015 at 14:03, Mohamed Infaz <in...@cse.mrt.ac.lk>
>> > wrote:
>> > >
>> > >> Hi Somesh,
>> > >>
>> > >> Yes i was successful in stopping both the system vm's and starting
>> them
>> > >> back i also could access them through the console that cloudstack
>> > provides.
>> > >> May be the reason for that code snippet is that i didn't register the
>> > >> template but i ran this script to install the system vm's.
>> > >>
>> > >>
>> > >> 1.
>> >
>> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
>> > \
>> > >> 2. -m /srv/images/secondary \
>> > >> 3. -u
>> >
>> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
>> > \
>> > >> 4. -h kvm \
>> > >> 5. -F
>> > >>
>> > >> this was on the docs. How about upgrading my cloudstack version to
>> 4.4.
>> > >> This whole thing happened in 4.3.
>> > >>
>> > >> Thank you.
>> > >>
>> > >> On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com>
>> > wrote:
>> > >>
>> > >>> You mean the system VMs for Zone 1 are running? Are they showing as
>> > >>> connected in the DB?
>> > >>>
>> > >>> If they are, can you stop these VMs and see if they start okay.
>> > >>> ________________________________________
>> > >>> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
>> > >>> Sent: Tuesday, December 30, 2014 21:26
>> > >>> To: users@cloudstack.apache.org
>> > >>> Cc: dev@cloudstack.apache.org
>> > >>> Subject: Re: Unable to start VM due to concurrent operation
>> > >>>
>> > >>> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the
>> > template i
>> > >>> used.
>> > >>>
>> > >>>
>> > >>> On 31 December 2014 at 07:49, Mohamed Infaz <infaz.10@cse.mrt.ac.lk
>> >
>> > >>> wrote:
>> > >>>
>> > >>>> Hi Somesh,
>> > >>>>
>> > >>>> Thank you for the reply. Well both of my system vm's are running
>> and
>> > it
>> > >>> is
>> > >>>> downloading the iso image. And i was able to ping them. The system
>> vm
>> > >>>> template i am using is for the version 4.3 cloudstack.
>> > >>>>
>> > >>>> Thank you.
>> > >>>>
>> > >>>> On 30 December 2014 at 23:59, Somesh Naidu <
>> Somesh.Naidu@citrix.com>
>> > >>>> wrote:
>> > >>>>
>> > >>>>> Mohamed,
>> > >>>>>
>> > >>>>> The log snippet you have shared doesn't seem to be relevant to the
>> > >>>>> concurrent operation exception (can't see that exception stack).
>> > >>>>>
>> > >>>>> The message that did catch my attention though is,
>> > >>>>> -
>> > >>>>> 2014-12-31 05:25:07,972 DEBUG
>> [c.c.s.s.SecondaryStorageManagerImpl]
>> > >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at
>> data
>> > >>> center
>> > >>>>> 1, wait until it is ready to launch secondary storage vm
>> > >>>>> 2014-12-31 05:25:07,972 DEBUG
>> [c.c.s.s.SecondaryStorageManagerImpl]
>> > >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch
>> secondary
>> > >>> storage
>> > >>>>> VM yet
>> > >>>>> -
>> > >>>>>
>> > >>>>> That's pointing to missing system VM template. Please verify that
>> you
>> > >>>>> have correctly provisioned the secondary storage with the correct
>> > >>> system VM
>> > >>>>> template (as per the docs).
>> > >>>>>
>> > >>>>> -Somesh
>> > >>>>>
>> > >>>>> -----Original Message-----
>> > >>>>> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
>> > >>>>> Sent: Tuesday, December 30, 2014 11:03 AM
>> > >>>>> To: users@cloudstack.apache.org
>> > >>>>> Cc: dev@cloudstack.apache.org
>> > >>>>> Subject: Unable to start VM due to concurrent operation
>> > >>>>>
>> > >>>>> Hi All,
>> > >>>>>
>> > >>>>> I have successfully deployed cloudstack 4.3 with two hosts and the
>> > >>> setup
>> > >>>>> runs the management server on another physical host. I had issues
>> > with
>> > >>>>> downloading an ISO but finally i was able to download an ISO
>> image.
>> > >>> When i
>> > >>>>> want to create an instance i get the following error message
>> saying
>> > >>>>> "Unable
>> > >>>>> to start VM due to concurrent operation" i did some searching on
>> the
>> > >>> topic
>> > >>>>> and it said version 4.3 system template solved the issue. These
>> are
>> > >>> some
>> > >>>>> of
>> > >>>>> the MS logs that i collected what could be the issue?
>> > >>>>>
>> > >>>>> 2014-12-31 05:25:07,972 DEBUG
>> [c.c.s.s.SecondaryStorageManagerImpl]
>> > >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at
>> data
>> > >>> center
>> > >>>>> 1, wait until it is ready to launch secondary storage vm
>> > >>>>> 2014-12-31 05:25:07,972 DEBUG
>> [c.c.s.s.SecondaryStorageManagerImpl]
>> > >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch
>> secondary
>> > >>>>> storage
>> > >>>>> VM yet
>> > >>>>> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-9:null) Ping from 4
>> > >>>>> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq
>> 3-91695:
>> > {
>> > >>> Cmd
>> > >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >>>>>
>> > >>>>>
>> > >>>
>> >
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
>> > >>>>> \"connections\": []\n}","wait":0}}] }
>> > >>>>> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695:
>> {
>> > >>> Ans: ,
>> > >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
>> > >>>>>
>> > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
>> > >>>>> 2014-12-31 05:25:15,239 DEBUG
>> [c.c.n.ExternalDeviceUsageManagerImpl]
>> > >>>>> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats
>> > >>> collector
>> > >>>>> is
>> > >>>>> running...
>> > >>>>> 2014-12-31 05:25:15,362 DEBUG
>> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >>>>> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
>> > >>>>> 2014-12-31 05:25:15,366 DEBUG
>> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update
>> > status.
>> > >>>>> 2014-12-31 05:25:15,367 DEBUG
>> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update
>> RvR
>> > >>>>> status.
>> > >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> > >>>>> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being
>> > >>> called at
>> > >>>>> 2014-12-30 23:55:15 GMT
>> > >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> > >>>>> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
>> > >>>>> 2014-12-30 23:55:15 GMT
>> > >>>>> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-14:null) Ping from 3
>> > >>>>> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-10:null) Ping from 2
>> > >>>>> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on
>> > >>> ping:
>> > >>>>> [1]
>> > >>>>> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
>> > >>> (AgentMonitor-1:ctx-f10d32f0)
>> > >>>>> Ping timeout for host 1, do invstigation
>> > >>>>> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
>> > >>> disconnected
>> > >>>>> with event PingTimeout
>> > >>>>> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
>> > >>>>> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
>> > >>> MgmtId:
>> > >>>>> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
>> > >>>>> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
>> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
>> > >>>>> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: {
>> Ans: ,
>> > >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
>> > >>>>>
>> > >>>>>
>> > >>>
>> >
>> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
>> > >>>>> is alive","wait":0}}] }
>> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: {
>> Ans: ,
>> > >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, {
>> > >>> CheckHealthAnswer
>> > >>>>> } }
>> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Details from executing class
>> > >>>>> com.cloud.agent.api.CheckHealthCommand: resource is alive
>> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
>> > >>> checkHeathCommand,
>> > >>>>> reporting that agent is Up
>> > >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
>> > >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and
>> > >>> running
>> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state =
>> Enabled,
>> > >>>>> Agent
>> > >>>>> event = Ping, Host id = 1, name = virtualops-h4]
>> > >>>>> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
>> > >>>>> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1;
>> name =
>> > >>>>> virtualops-h4; old status = Up; event = Ping; new status = Up; old
>> > >>> update
>> > >>>>> count = 34; new update count = 35]
>> > >>>>> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq
>> 3-91697:
>> > {
>> > >>> Cmd
>> > >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> > >>>>>
>> > >>>>>
>> > >>>
>> >
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
>> > >>>>> \"connections\": []\n}","wait":0}}] }
>> > >>>>> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
>> > >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697:
>> {
>> > >>> Ans: ,
>> > >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
>> > >>>>>
>> > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
>> > >>>>> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
>> > >>>>> (StatsCollector-1:ctx-2acfc
>> > >>>>>
>> > >>>>> Thank you.
>> > >>>>>
>> > >>>>
>> > >>>>
>> > >>>
>> > >>
>> > >>
>> >
>> > Find out more about ShapeBlue and our range of CloudStack related
>> services
>> >
>> > IaaS Cloud Design & Build<
>> > http://shapeblue.com/iaas-cloud-design-and-build//>
>> > CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/
>> >
>> > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>> > CloudStack Software Engineering<
>> > http://shapeblue.com/cloudstack-software-engineering/>
>> > CloudStack Infrastructure Support<
>> > http://shapeblue.com/cloudstack-infrastructure-support/>
>> > CloudStack Bootcamp Training Courses<
>> > http://shapeblue.com/cloudstack-training/>
>> >
>> > This email and any attachments to it may be confidential and are
>> intended
>> > solely for the use of the individual to whom it is addressed. Any views
>> or
>> > opinions expressed are solely those of the author and do not necessarily
>> > represent those of Shape Blue Ltd or related companies. If you are not
>> the
>> > intended recipient of this email, you must neither take any action based
>> > upon its contents, nor copy or show it to anyone. Please contact the
>> sender
>> > if you believe you have received this email in error. Shape Blue Ltd is
>> a
>> > company incorporated in England & Wales. ShapeBlue Services India LLP
>> is a
>> > company incorporated in India and is operated under license from Shape
>> Blue
>> > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
>> Brasil
>> > and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd
>> is
>> > a company registered by The Republic of South Africa and is traded under
>> > license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>> >
>>
>
>
Re: Unable to start VM due to concurrent operation
Posted by Mohamed Infaz <in...@cse.mrt.ac.lk>.
Hi,
This is the screen shot. It stops around 96-98%
On 2 January 2015 at 20:58, Somesh Naidu <So...@citrix.com> wrote:
> What's the status of template are you trying to create a VM from? Please
> share either DB output or UI screenshot.
>
> -----Original Message-----
> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
> Sent: Friday, January 02, 2015 7:03 AM
> To: users@cloudstack.apache.org
> Cc: dev@cloudstack.apache.org
> Subject: Re: Unable to start VM due to concurrent operation
>
> Hi all,
>
> I did redeploy cloudstack and i used the
>
> >> 1. /usr/share/cloudstack-common/
> scripts/storage/secondary/cloud-install-sys-tmplt \
> >> 2. -m /srv/images/secondary \
> >> 3. -u
>
> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
> \
> >> 4. -h kvm \
> >> 5. -F
>
> The following script to deploy system vm's. After some time they started
> working! Now when i try to deploy and vm
> i am getting the following error message.
> "com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
> unreachable: Host 1: Unable to start instance due to Template 3 has not
> been completely downloaded to zone 1"
>
> I did reregister the latest system vm template and i was able to
> successfully download it. But there are two inbuilt templates where it says
> in the status that the connection was time out. And one of these in built
> template is an in built old system vm template. How can i solve this issue
> now there is a system vm running properly in the environment and i have the
> latest template registered but it seems so cloudstack searches for those
> two templates which it couldn't download. Because of this i still can't
> create a virtual machine.
>
> Thank you.
>
> On 2 January 2015 at 08:08, Abhinandan Prateek <
> abhinandan.prateek@shapeblue.com> wrote:
>
> > The sec store has some issues, you can login into KVM host and check if
> > the sec storage mount point is visible and the said folder is there.
> Check
> > if ssvm is running fine.
> > It is also possible that the sec store is fine but the template that is
> > being used to create VM is in inconsistent state, then you need to remove
> > and re-register the template.
> >
> > -abhi
> >
> > > On 01-Jan-2015, at 8:35 pm, Mohamed Infaz <in...@cse.mrt.ac.lk>
> > wrote:
> > >
> > > Hi,
> > > I updated my cloudstack deployment to 4.4.2 to 4.3.1. Now when we try
> to
> > > deploy a VM i am getting following error
> > >
> > > message. com.cloud.exception.StorageUnavailableException: Resource
> > > [StoragePool:2] is unreachable: Unable to create
> > > Vol[30|vm=58|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
> > > org.libvirt.LibvirtException: internal error: Child process (/bin/mount
> > > 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> > > /mnt/8c6fa424-cbc7-3421-8f05-d69a43ffa15c) unexpected exit status 32:
> > > mount.nfs: mounting 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> > failed,
> > > reason given by server: No such file or directory
> > >
> > >
> > >
> > > Thank you.
> > >
> > >
> > > On 1 January 2015 at 14:03, Mohamed Infaz <in...@cse.mrt.ac.lk>
> > wrote:
> > >
> > >> Hi Somesh,
> > >>
> > >> Yes i was successful in stopping both the system vm's and starting
> them
> > >> back i also could access them through the console that cloudstack
> > provides.
> > >> May be the reason for that code snippet is that i didn't register the
> > >> template but i ran this script to install the system vm's.
> > >>
> > >>
> > >> 1.
> >
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> > \
> > >> 2. -m /srv/images/secondary \
> > >> 3. -u
> >
> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
> > \
> > >> 4. -h kvm \
> > >> 5. -F
> > >>
> > >> this was on the docs. How about upgrading my cloudstack version to
> 4.4.
> > >> This whole thing happened in 4.3.
> > >>
> > >> Thank you.
> > >>
> > >> On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com>
> > wrote:
> > >>
> > >>> You mean the system VMs for Zone 1 are running? Are they showing as
> > >>> connected in the DB?
> > >>>
> > >>> If they are, can you stop these VMs and see if they start okay.
> > >>> ________________________________________
> > >>> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
> > >>> Sent: Tuesday, December 30, 2014 21:26
> > >>> To: users@cloudstack.apache.org
> > >>> Cc: dev@cloudstack.apache.org
> > >>> Subject: Re: Unable to start VM due to concurrent operation
> > >>>
> > >>> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the
> > template i
> > >>> used.
> > >>>
> > >>>
> > >>> On 31 December 2014 at 07:49, Mohamed Infaz <in...@cse.mrt.ac.lk>
> > >>> wrote:
> > >>>
> > >>>> Hi Somesh,
> > >>>>
> > >>>> Thank you for the reply. Well both of my system vm's are running and
> > it
> > >>> is
> > >>>> downloading the iso image. And i was able to ping them. The system
> vm
> > >>>> template i am using is for the version 4.3 cloudstack.
> > >>>>
> > >>>> Thank you.
> > >>>>
> > >>>> On 30 December 2014 at 23:59, Somesh Naidu <Somesh.Naidu@citrix.com
> >
> > >>>> wrote:
> > >>>>
> > >>>>> Mohamed,
> > >>>>>
> > >>>>> The log snippet you have shared doesn't seem to be relevant to the
> > >>>>> concurrent operation exception (can't see that exception stack).
> > >>>>>
> > >>>>> The message that did catch my attention though is,
> > >>>>> -
> > >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> > >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> > >>> center
> > >>>>> 1, wait until it is ready to launch secondary storage vm
> > >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> > >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> > >>> storage
> > >>>>> VM yet
> > >>>>> -
> > >>>>>
> > >>>>> That's pointing to missing system VM template. Please verify that
> you
> > >>>>> have correctly provisioned the secondary storage with the correct
> > >>> system VM
> > >>>>> template (as per the docs).
> > >>>>>
> > >>>>> -Somesh
> > >>>>>
> > >>>>> -----Original Message-----
> > >>>>> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
> > >>>>> Sent: Tuesday, December 30, 2014 11:03 AM
> > >>>>> To: users@cloudstack.apache.org
> > >>>>> Cc: dev@cloudstack.apache.org
> > >>>>> Subject: Unable to start VM due to concurrent operation
> > >>>>>
> > >>>>> Hi All,
> > >>>>>
> > >>>>> I have successfully deployed cloudstack 4.3 with two hosts and the
> > >>> setup
> > >>>>> runs the management server on another physical host. I had issues
> > with
> > >>>>> downloading an ISO but finally i was able to download an ISO image.
> > >>> When i
> > >>>>> want to create an instance i get the following error message saying
> > >>>>> "Unable
> > >>>>> to start VM due to concurrent operation" i did some searching on
> the
> > >>> topic
> > >>>>> and it said version 4.3 system template solved the issue. These are
> > >>> some
> > >>>>> of
> > >>>>> the MS logs that i collected what could be the issue?
> > >>>>>
> > >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> > >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> > >>> center
> > >>>>> 1, wait until it is ready to launch secondary storage vm
> > >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> > >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> > >>>>> storage
> > >>>>> VM yet
> > >>>>> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-9:null) Ping from 4
> > >>>>> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq 3-91695:
> > {
> > >>> Cmd
> > >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >>>>>
> > >>>>>
> > >>>
> >
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> > >>>>> \"connections\": []\n}","wait":0}}] }
> > >>>>> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695: {
> > >>> Ans: ,
> > >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> > >>>>>
> > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> > >>>>> 2014-12-31 05:25:15,239 DEBUG
> [c.c.n.ExternalDeviceUsageManagerImpl]
> > >>>>> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats
> > >>> collector
> > >>>>> is
> > >>>>> running...
> > >>>>> 2014-12-31 05:25:15,362 DEBUG
> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >>>>> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
> > >>>>> 2014-12-31 05:25:15,366 DEBUG
> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update
> > status.
> > >>>>> 2014-12-31 05:25:15,367 DEBUG
> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update RvR
> > >>>>> status.
> > >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> > >>>>> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being
> > >>> called at
> > >>>>> 2014-12-30 23:55:15 GMT
> > >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> > >>>>> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
> > >>>>> 2014-12-30 23:55:15 GMT
> > >>>>> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-14:null) Ping from 3
> > >>>>> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-10:null) Ping from 2
> > >>>>> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on
> > >>> ping:
> > >>>>> [1]
> > >>>>> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
> > >>> (AgentMonitor-1:ctx-f10d32f0)
> > >>>>> Ping timeout for host 1, do invstigation
> > >>>>> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
> > >>> disconnected
> > >>>>> with event PingTimeout
> > >>>>> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
> > >>>>> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
> > >>> MgmtId:
> > >>>>> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
> > >>>>> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> > >>>>> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: { Ans:
> ,
> > >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
> > >>>>>
> > >>>>>
> > >>>
> >
> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
> > >>>>> is alive","wait":0}}] }
> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: { Ans:
> ,
> > >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, {
> > >>> CheckHealthAnswer
> > >>>>> } }
> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Details from executing class
> > >>>>> com.cloud.agent.api.CheckHealthCommand: resource is alive
> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
> > >>> checkHeathCommand,
> > >>>>> reporting that agent is Up
> > >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
> > >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and
> > >>> running
> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state =
> Enabled,
> > >>>>> Agent
> > >>>>> event = Ping, Host id = 1, name = virtualops-h4]
> > >>>>> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1; name
> =
> > >>>>> virtualops-h4; old status = Up; event = Ping; new status = Up; old
> > >>> update
> > >>>>> count = 34; new update count = 35]
> > >>>>> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq 3-91697:
> > {
> > >>> Cmd
> > >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >>>>>
> > >>>>>
> > >>>
> >
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> > >>>>> \"connections\": []\n}","wait":0}}] }
> > >>>>> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697: {
> > >>> Ans: ,
> > >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> > >>>>>
> > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> > >>>>> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
> > >>>>> (StatsCollector-1:ctx-2acfc
> > >>>>>
> > >>>>> Thank you.
> > >>>>>
> > >>>>
> > >>>>
> > >>>
> > >>
> > >>
> >
> > Find out more about ShapeBlue and our range of CloudStack related
> services
> >
> > IaaS Cloud Design & Build<
> > http://shapeblue.com/iaas-cloud-design-and-build//>
> > CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> > CloudStack Software Engineering<
> > http://shapeblue.com/cloudstack-software-engineering/>
> > CloudStack Infrastructure Support<
> > http://shapeblue.com/cloudstack-infrastructure-support/>
> > CloudStack Bootcamp Training Courses<
> > http://shapeblue.com/cloudstack-training/>
> >
> > This email and any attachments to it may be confidential and are intended
> > solely for the use of the individual to whom it is addressed. Any views
> or
> > opinions expressed are solely those of the author and do not necessarily
> > represent those of Shape Blue Ltd or related companies. If you are not
> the
> > intended recipient of this email, you must neither take any action based
> > upon its contents, nor copy or show it to anyone. Please contact the
> sender
> > if you believe you have received this email in error. Shape Blue Ltd is a
> > company incorporated in England & Wales. ShapeBlue Services India LLP is
> a
> > company incorporated in India and is operated under license from Shape
> Blue
> > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> Brasil
> > and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd
> is
> > a company registered by The Republic of South Africa and is traded under
> > license from Shape Blue Ltd. ShapeBlue is a registered trademark.
> >
>
Re: Unable to start VM due to concurrent operation
Posted by Mohamed Infaz <in...@cse.mrt.ac.lk>.
Hi,
This is the screen shot. It stops around 96-98%
On 2 January 2015 at 20:58, Somesh Naidu <So...@citrix.com> wrote:
> What's the status of template are you trying to create a VM from? Please
> share either DB output or UI screenshot.
>
> -----Original Message-----
> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
> Sent: Friday, January 02, 2015 7:03 AM
> To: users@cloudstack.apache.org
> Cc: dev@cloudstack.apache.org
> Subject: Re: Unable to start VM due to concurrent operation
>
> Hi all,
>
> I did redeploy cloudstack and i used the
>
> >> 1. /usr/share/cloudstack-common/
> scripts/storage/secondary/cloud-install-sys-tmplt \
> >> 2. -m /srv/images/secondary \
> >> 3. -u
>
> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
> \
> >> 4. -h kvm \
> >> 5. -F
>
> The following script to deploy system vm's. After some time they started
> working! Now when i try to deploy and vm
> i am getting the following error message.
> "com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
> unreachable: Host 1: Unable to start instance due to Template 3 has not
> been completely downloaded to zone 1"
>
> I did reregister the latest system vm template and i was able to
> successfully download it. But there are two inbuilt templates where it says
> in the status that the connection was time out. And one of these in built
> template is an in built old system vm template. How can i solve this issue
> now there is a system vm running properly in the environment and i have the
> latest template registered but it seems so cloudstack searches for those
> two templates which it couldn't download. Because of this i still can't
> create a virtual machine.
>
> Thank you.
>
> On 2 January 2015 at 08:08, Abhinandan Prateek <
> abhinandan.prateek@shapeblue.com> wrote:
>
> > The sec store has some issues, you can login into KVM host and check if
> > the sec storage mount point is visible and the said folder is there.
> Check
> > if ssvm is running fine.
> > It is also possible that the sec store is fine but the template that is
> > being used to create VM is in inconsistent state, then you need to remove
> > and re-register the template.
> >
> > -abhi
> >
> > > On 01-Jan-2015, at 8:35 pm, Mohamed Infaz <in...@cse.mrt.ac.lk>
> > wrote:
> > >
> > > Hi,
> > > I updated my cloudstack deployment to 4.4.2 to 4.3.1. Now when we try
> to
> > > deploy a VM i am getting following error
> > >
> > > message. com.cloud.exception.StorageUnavailableException: Resource
> > > [StoragePool:2] is unreachable: Unable to create
> > > Vol[30|vm=58|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
> > > org.libvirt.LibvirtException: internal error: Child process (/bin/mount
> > > 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> > > /mnt/8c6fa424-cbc7-3421-8f05-d69a43ffa15c) unexpected exit status 32:
> > > mount.nfs: mounting 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> > failed,
> > > reason given by server: No such file or directory
> > >
> > >
> > >
> > > Thank you.
> > >
> > >
> > > On 1 January 2015 at 14:03, Mohamed Infaz <in...@cse.mrt.ac.lk>
> > wrote:
> > >
> > >> Hi Somesh,
> > >>
> > >> Yes i was successful in stopping both the system vm's and starting
> them
> > >> back i also could access them through the console that cloudstack
> > provides.
> > >> May be the reason for that code snippet is that i didn't register the
> > >> template but i ran this script to install the system vm's.
> > >>
> > >>
> > >> 1.
> >
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> > \
> > >> 2. -m /srv/images/secondary \
> > >> 3. -u
> >
> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
> > \
> > >> 4. -h kvm \
> > >> 5. -F
> > >>
> > >> this was on the docs. How about upgrading my cloudstack version to
> 4.4.
> > >> This whole thing happened in 4.3.
> > >>
> > >> Thank you.
> > >>
> > >> On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com>
> > wrote:
> > >>
> > >>> You mean the system VMs for Zone 1 are running? Are they showing as
> > >>> connected in the DB?
> > >>>
> > >>> If they are, can you stop these VMs and see if they start okay.
> > >>> ________________________________________
> > >>> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
> > >>> Sent: Tuesday, December 30, 2014 21:26
> > >>> To: users@cloudstack.apache.org
> > >>> Cc: dev@cloudstack.apache.org
> > >>> Subject: Re: Unable to start VM due to concurrent operation
> > >>>
> > >>> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the
> > template i
> > >>> used.
> > >>>
> > >>>
> > >>> On 31 December 2014 at 07:49, Mohamed Infaz <in...@cse.mrt.ac.lk>
> > >>> wrote:
> > >>>
> > >>>> Hi Somesh,
> > >>>>
> > >>>> Thank you for the reply. Well both of my system vm's are running and
> > it
> > >>> is
> > >>>> downloading the iso image. And i was able to ping them. The system
> vm
> > >>>> template i am using is for the version 4.3 cloudstack.
> > >>>>
> > >>>> Thank you.
> > >>>>
> > >>>> On 30 December 2014 at 23:59, Somesh Naidu <Somesh.Naidu@citrix.com
> >
> > >>>> wrote:
> > >>>>
> > >>>>> Mohamed,
> > >>>>>
> > >>>>> The log snippet you have shared doesn't seem to be relevant to the
> > >>>>> concurrent operation exception (can't see that exception stack).
> > >>>>>
> > >>>>> The message that did catch my attention though is,
> > >>>>> -
> > >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> > >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> > >>> center
> > >>>>> 1, wait until it is ready to launch secondary storage vm
> > >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> > >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> > >>> storage
> > >>>>> VM yet
> > >>>>> -
> > >>>>>
> > >>>>> That's pointing to missing system VM template. Please verify that
> you
> > >>>>> have correctly provisioned the secondary storage with the correct
> > >>> system VM
> > >>>>> template (as per the docs).
> > >>>>>
> > >>>>> -Somesh
> > >>>>>
> > >>>>> -----Original Message-----
> > >>>>> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
> > >>>>> Sent: Tuesday, December 30, 2014 11:03 AM
> > >>>>> To: users@cloudstack.apache.org
> > >>>>> Cc: dev@cloudstack.apache.org
> > >>>>> Subject: Unable to start VM due to concurrent operation
> > >>>>>
> > >>>>> Hi All,
> > >>>>>
> > >>>>> I have successfully deployed cloudstack 4.3 with two hosts and the
> > >>> setup
> > >>>>> runs the management server on another physical host. I had issues
> > with
> > >>>>> downloading an ISO but finally i was able to download an ISO image.
> > >>> When i
> > >>>>> want to create an instance i get the following error message saying
> > >>>>> "Unable
> > >>>>> to start VM due to concurrent operation" i did some searching on
> the
> > >>> topic
> > >>>>> and it said version 4.3 system template solved the issue. These are
> > >>> some
> > >>>>> of
> > >>>>> the MS logs that i collected what could be the issue?
> > >>>>>
> > >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> > >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> > >>> center
> > >>>>> 1, wait until it is ready to launch secondary storage vm
> > >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> > >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> > >>>>> storage
> > >>>>> VM yet
> > >>>>> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-9:null) Ping from 4
> > >>>>> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq 3-91695:
> > {
> > >>> Cmd
> > >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >>>>>
> > >>>>>
> > >>>
> >
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> > >>>>> \"connections\": []\n}","wait":0}}] }
> > >>>>> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695: {
> > >>> Ans: ,
> > >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> > >>>>>
> > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> > >>>>> 2014-12-31 05:25:15,239 DEBUG
> [c.c.n.ExternalDeviceUsageManagerImpl]
> > >>>>> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats
> > >>> collector
> > >>>>> is
> > >>>>> running...
> > >>>>> 2014-12-31 05:25:15,362 DEBUG
> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >>>>> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
> > >>>>> 2014-12-31 05:25:15,366 DEBUG
> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update
> > status.
> > >>>>> 2014-12-31 05:25:15,367 DEBUG
> > >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update RvR
> > >>>>> status.
> > >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> > >>>>> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being
> > >>> called at
> > >>>>> 2014-12-30 23:55:15 GMT
> > >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> > >>>>> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
> > >>>>> 2014-12-30 23:55:15 GMT
> > >>>>> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-14:null) Ping from 3
> > >>>>> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-10:null) Ping from 2
> > >>>>> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on
> > >>> ping:
> > >>>>> [1]
> > >>>>> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
> > >>> (AgentMonitor-1:ctx-f10d32f0)
> > >>>>> Ping timeout for host 1, do invstigation
> > >>>>> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
> > >>> disconnected
> > >>>>> with event PingTimeout
> > >>>>> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
> > >>>>> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
> > >>> MgmtId:
> > >>>>> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
> > >>>>> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> > >>>>> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: { Ans:
> ,
> > >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
> > >>>>>
> > >>>>>
> > >>>
> >
> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
> > >>>>> is alive","wait":0}}] }
> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: { Ans:
> ,
> > >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, {
> > >>> CheckHealthAnswer
> > >>>>> } }
> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Details from executing class
> > >>>>> com.cloud.agent.api.CheckHealthCommand: resource is alive
> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
> > >>> checkHeathCommand,
> > >>>>> reporting that agent is Up
> > >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
> > >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and
> > >>> running
> > >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state =
> Enabled,
> > >>>>> Agent
> > >>>>> event = Ping, Host id = 1, name = virtualops-h4]
> > >>>>> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
> > >>>>> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1; name
> =
> > >>>>> virtualops-h4; old status = Up; event = Ping; new status = Up; old
> > >>> update
> > >>>>> count = 34; new update count = 35]
> > >>>>> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq 3-91697:
> > {
> > >>> Cmd
> > >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> > >>>>>
> > >>>>>
> > >>>
> >
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> > >>>>> \"connections\": []\n}","wait":0}}] }
> > >>>>> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
> > >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697: {
> > >>> Ans: ,
> > >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> > >>>>>
> > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> > >>>>> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
> > >>>>> (StatsCollector-1:ctx-2acfc
> > >>>>>
> > >>>>> Thank you.
> > >>>>>
> > >>>>
> > >>>>
> > >>>
> > >>
> > >>
> >
> > Find out more about ShapeBlue and our range of CloudStack related
> services
> >
> > IaaS Cloud Design & Build<
> > http://shapeblue.com/iaas-cloud-design-and-build//>
> > CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> > CloudStack Software Engineering<
> > http://shapeblue.com/cloudstack-software-engineering/>
> > CloudStack Infrastructure Support<
> > http://shapeblue.com/cloudstack-infrastructure-support/>
> > CloudStack Bootcamp Training Courses<
> > http://shapeblue.com/cloudstack-training/>
> >
> > This email and any attachments to it may be confidential and are intended
> > solely for the use of the individual to whom it is addressed. Any views
> or
> > opinions expressed are solely those of the author and do not necessarily
> > represent those of Shape Blue Ltd or related companies. If you are not
> the
> > intended recipient of this email, you must neither take any action based
> > upon its contents, nor copy or show it to anyone. Please contact the
> sender
> > if you believe you have received this email in error. Shape Blue Ltd is a
> > company incorporated in England & Wales. ShapeBlue Services India LLP is
> a
> > company incorporated in India and is operated under license from Shape
> Blue
> > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> Brasil
> > and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd
> is
> > a company registered by The Republic of South Africa and is traded under
> > license from Shape Blue Ltd. ShapeBlue is a registered trademark.
> >
>
RE: Unable to start VM due to concurrent operation
Posted by Somesh Naidu <So...@citrix.com>.
What's the status of template are you trying to create a VM from? Please share either DB output or UI screenshot.
-----Original Message-----
From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
Sent: Friday, January 02, 2015 7:03 AM
To: users@cloudstack.apache.org
Cc: dev@cloudstack.apache.org
Subject: Re: Unable to start VM due to concurrent operation
Hi all,
I did redeploy cloudstack and i used the
>> 1. /usr/share/cloudstack-common/
scripts/storage/secondary/cloud-install-sys-tmplt \
>> 2. -m /srv/images/secondary \
>> 3. -u
http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
\
>> 4. -h kvm \
>> 5. -F
The following script to deploy system vm's. After some time they started
working! Now when i try to deploy and vm
i am getting the following error message.
"com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
unreachable: Host 1: Unable to start instance due to Template 3 has not
been completely downloaded to zone 1"
I did reregister the latest system vm template and i was able to
successfully download it. But there are two inbuilt templates where it says
in the status that the connection was time out. And one of these in built
template is an in built old system vm template. How can i solve this issue
now there is a system vm running properly in the environment and i have the
latest template registered but it seems so cloudstack searches for those
two templates which it couldn't download. Because of this i still can't
create a virtual machine.
Thank you.
On 2 January 2015 at 08:08, Abhinandan Prateek <
abhinandan.prateek@shapeblue.com> wrote:
> The sec store has some issues, you can login into KVM host and check if
> the sec storage mount point is visible and the said folder is there. Check
> if ssvm is running fine.
> It is also possible that the sec store is fine but the template that is
> being used to create VM is in inconsistent state, then you need to remove
> and re-register the template.
>
> -abhi
>
> > On 01-Jan-2015, at 8:35 pm, Mohamed Infaz <in...@cse.mrt.ac.lk>
> wrote:
> >
> > Hi,
> > I updated my cloudstack deployment to 4.4.2 to 4.3.1. Now when we try to
> > deploy a VM i am getting following error
> >
> > message. com.cloud.exception.StorageUnavailableException: Resource
> > [StoragePool:2] is unreachable: Unable to create
> > Vol[30|vm=58|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
> > org.libvirt.LibvirtException: internal error: Child process (/bin/mount
> > 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> > /mnt/8c6fa424-cbc7-3421-8f05-d69a43ffa15c) unexpected exit status 32:
> > mount.nfs: mounting 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> failed,
> > reason given by server: No such file or directory
> >
> >
> >
> > Thank you.
> >
> >
> > On 1 January 2015 at 14:03, Mohamed Infaz <in...@cse.mrt.ac.lk>
> wrote:
> >
> >> Hi Somesh,
> >>
> >> Yes i was successful in stopping both the system vm's and starting them
> >> back i also could access them through the console that cloudstack
> provides.
> >> May be the reason for that code snippet is that i didn't register the
> >> template but i ran this script to install the system vm's.
> >>
> >>
> >> 1.
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> \
> >> 2. -m /srv/images/secondary \
> >> 3. -u
> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
> \
> >> 4. -h kvm \
> >> 5. -F
> >>
> >> this was on the docs. How about upgrading my cloudstack version to 4.4.
> >> This whole thing happened in 4.3.
> >>
> >> Thank you.
> >>
> >> On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com>
> wrote:
> >>
> >>> You mean the system VMs for Zone 1 are running? Are they showing as
> >>> connected in the DB?
> >>>
> >>> If they are, can you stop these VMs and see if they start okay.
> >>> ________________________________________
> >>> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
> >>> Sent: Tuesday, December 30, 2014 21:26
> >>> To: users@cloudstack.apache.org
> >>> Cc: dev@cloudstack.apache.org
> >>> Subject: Re: Unable to start VM due to concurrent operation
> >>>
> >>> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the
> template i
> >>> used.
> >>>
> >>>
> >>> On 31 December 2014 at 07:49, Mohamed Infaz <in...@cse.mrt.ac.lk>
> >>> wrote:
> >>>
> >>>> Hi Somesh,
> >>>>
> >>>> Thank you for the reply. Well both of my system vm's are running and
> it
> >>> is
> >>>> downloading the iso image. And i was able to ping them. The system vm
> >>>> template i am using is for the version 4.3 cloudstack.
> >>>>
> >>>> Thank you.
> >>>>
> >>>> On 30 December 2014 at 23:59, Somesh Naidu <So...@citrix.com>
> >>>> wrote:
> >>>>
> >>>>> Mohamed,
> >>>>>
> >>>>> The log snippet you have shared doesn't seem to be relevant to the
> >>>>> concurrent operation exception (can't see that exception stack).
> >>>>>
> >>>>> The message that did catch my attention though is,
> >>>>> -
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> >>> center
> >>>>> 1, wait until it is ready to launch secondary storage vm
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> >>> storage
> >>>>> VM yet
> >>>>> -
> >>>>>
> >>>>> That's pointing to missing system VM template. Please verify that you
> >>>>> have correctly provisioned the secondary storage with the correct
> >>> system VM
> >>>>> template (as per the docs).
> >>>>>
> >>>>> -Somesh
> >>>>>
> >>>>> -----Original Message-----
> >>>>> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
> >>>>> Sent: Tuesday, December 30, 2014 11:03 AM
> >>>>> To: users@cloudstack.apache.org
> >>>>> Cc: dev@cloudstack.apache.org
> >>>>> Subject: Unable to start VM due to concurrent operation
> >>>>>
> >>>>> Hi All,
> >>>>>
> >>>>> I have successfully deployed cloudstack 4.3 with two hosts and the
> >>> setup
> >>>>> runs the management server on another physical host. I had issues
> with
> >>>>> downloading an ISO but finally i was able to download an ISO image.
> >>> When i
> >>>>> want to create an instance i get the following error message saying
> >>>>> "Unable
> >>>>> to start VM due to concurrent operation" i did some searching on the
> >>> topic
> >>>>> and it said version 4.3 system template solved the issue. These are
> >>> some
> >>>>> of
> >>>>> the MS logs that i collected what could be the issue?
> >>>>>
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> >>> center
> >>>>> 1, wait until it is ready to launch secondary storage vm
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> >>>>> storage
> >>>>> VM yet
> >>>>> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-9:null) Ping from 4
> >>>>> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq 3-91695:
> {
> >>> Cmd
> >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>>>>
> >>>>>
> >>>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >>>>> \"connections\": []\n}","wait":0}}] }
> >>>>> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695: {
> >>> Ans: ,
> >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >>>>>
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >>>>> 2014-12-31 05:25:15,239 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
> >>>>> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats
> >>> collector
> >>>>> is
> >>>>> running...
> >>>>> 2014-12-31 05:25:15,362 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>>>> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
> >>>>> 2014-12-31 05:25:15,366 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update
> status.
> >>>>> 2014-12-31 05:25:15,367 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update RvR
> >>>>> status.
> >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >>>>> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being
> >>> called at
> >>>>> 2014-12-30 23:55:15 GMT
> >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >>>>> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
> >>>>> 2014-12-30 23:55:15 GMT
> >>>>> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-14:null) Ping from 3
> >>>>> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-10:null) Ping from 2
> >>>>> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on
> >>> ping:
> >>>>> [1]
> >>>>> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
> >>> (AgentMonitor-1:ctx-f10d32f0)
> >>>>> Ping timeout for host 1, do invstigation
> >>>>> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
> >>> disconnected
> >>>>> with event PingTimeout
> >>>>> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
> >>>>> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
> >>> MgmtId:
> >>>>> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
> >>>>> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >>>>> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: { Ans: ,
> >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
> >>>>>
> >>>>>
> >>>
> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
> >>>>> is alive","wait":0}}] }
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: { Ans: ,
> >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, {
> >>> CheckHealthAnswer
> >>>>> } }
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Details from executing class
> >>>>> com.cloud.agent.api.CheckHealthCommand: resource is alive
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
> >>> checkHeathCommand,
> >>>>> reporting that agent is Up
> >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
> >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and
> >>> running
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state = Enabled,
> >>>>> Agent
> >>>>> event = Ping, Host id = 1, name = virtualops-h4]
> >>>>> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1; name =
> >>>>> virtualops-h4; old status = Up; event = Ping; new status = Up; old
> >>> update
> >>>>> count = 34; new update count = 35]
> >>>>> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq 3-91697:
> {
> >>> Cmd
> >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>>>>
> >>>>>
> >>>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >>>>> \"connections\": []\n}","wait":0}}] }
> >>>>> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697: {
> >>> Ans: ,
> >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >>>>>
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >>>>> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
> >>>>> (StatsCollector-1:ctx-2acfc
> >>>>>
> >>>>> Thank you.
> >>>>>
> >>>>
> >>>>
> >>>
> >>
> >>
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Software Engineering<
> http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
> a company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>
RE: Unable to start VM due to concurrent operation
Posted by Somesh Naidu <So...@citrix.com>.
What's the status of template are you trying to create a VM from? Please share either DB output or UI screenshot.
-----Original Message-----
From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
Sent: Friday, January 02, 2015 7:03 AM
To: users@cloudstack.apache.org
Cc: dev@cloudstack.apache.org
Subject: Re: Unable to start VM due to concurrent operation
Hi all,
I did redeploy cloudstack and i used the
>> 1. /usr/share/cloudstack-common/
scripts/storage/secondary/cloud-install-sys-tmplt \
>> 2. -m /srv/images/secondary \
>> 3. -u
http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
\
>> 4. -h kvm \
>> 5. -F
The following script to deploy system vm's. After some time they started
working! Now when i try to deploy and vm
i am getting the following error message.
"com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
unreachable: Host 1: Unable to start instance due to Template 3 has not
been completely downloaded to zone 1"
I did reregister the latest system vm template and i was able to
successfully download it. But there are two inbuilt templates where it says
in the status that the connection was time out. And one of these in built
template is an in built old system vm template. How can i solve this issue
now there is a system vm running properly in the environment and i have the
latest template registered but it seems so cloudstack searches for those
two templates which it couldn't download. Because of this i still can't
create a virtual machine.
Thank you.
On 2 January 2015 at 08:08, Abhinandan Prateek <
abhinandan.prateek@shapeblue.com> wrote:
> The sec store has some issues, you can login into KVM host and check if
> the sec storage mount point is visible and the said folder is there. Check
> if ssvm is running fine.
> It is also possible that the sec store is fine but the template that is
> being used to create VM is in inconsistent state, then you need to remove
> and re-register the template.
>
> -abhi
>
> > On 01-Jan-2015, at 8:35 pm, Mohamed Infaz <in...@cse.mrt.ac.lk>
> wrote:
> >
> > Hi,
> > I updated my cloudstack deployment to 4.4.2 to 4.3.1. Now when we try to
> > deploy a VM i am getting following error
> >
> > message. com.cloud.exception.StorageUnavailableException: Resource
> > [StoragePool:2] is unreachable: Unable to create
> > Vol[30|vm=58|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
> > org.libvirt.LibvirtException: internal error: Child process (/bin/mount
> > 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> > /mnt/8c6fa424-cbc7-3421-8f05-d69a43ffa15c) unexpected exit status 32:
> > mount.nfs: mounting 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> failed,
> > reason given by server: No such file or directory
> >
> >
> >
> > Thank you.
> >
> >
> > On 1 January 2015 at 14:03, Mohamed Infaz <in...@cse.mrt.ac.lk>
> wrote:
> >
> >> Hi Somesh,
> >>
> >> Yes i was successful in stopping both the system vm's and starting them
> >> back i also could access them through the console that cloudstack
> provides.
> >> May be the reason for that code snippet is that i didn't register the
> >> template but i ran this script to install the system vm's.
> >>
> >>
> >> 1.
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> \
> >> 2. -m /srv/images/secondary \
> >> 3. -u
> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
> \
> >> 4. -h kvm \
> >> 5. -F
> >>
> >> this was on the docs. How about upgrading my cloudstack version to 4.4.
> >> This whole thing happened in 4.3.
> >>
> >> Thank you.
> >>
> >> On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com>
> wrote:
> >>
> >>> You mean the system VMs for Zone 1 are running? Are they showing as
> >>> connected in the DB?
> >>>
> >>> If they are, can you stop these VMs and see if they start okay.
> >>> ________________________________________
> >>> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
> >>> Sent: Tuesday, December 30, 2014 21:26
> >>> To: users@cloudstack.apache.org
> >>> Cc: dev@cloudstack.apache.org
> >>> Subject: Re: Unable to start VM due to concurrent operation
> >>>
> >>> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the
> template i
> >>> used.
> >>>
> >>>
> >>> On 31 December 2014 at 07:49, Mohamed Infaz <in...@cse.mrt.ac.lk>
> >>> wrote:
> >>>
> >>>> Hi Somesh,
> >>>>
> >>>> Thank you for the reply. Well both of my system vm's are running and
> it
> >>> is
> >>>> downloading the iso image. And i was able to ping them. The system vm
> >>>> template i am using is for the version 4.3 cloudstack.
> >>>>
> >>>> Thank you.
> >>>>
> >>>> On 30 December 2014 at 23:59, Somesh Naidu <So...@citrix.com>
> >>>> wrote:
> >>>>
> >>>>> Mohamed,
> >>>>>
> >>>>> The log snippet you have shared doesn't seem to be relevant to the
> >>>>> concurrent operation exception (can't see that exception stack).
> >>>>>
> >>>>> The message that did catch my attention though is,
> >>>>> -
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> >>> center
> >>>>> 1, wait until it is ready to launch secondary storage vm
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> >>> storage
> >>>>> VM yet
> >>>>> -
> >>>>>
> >>>>> That's pointing to missing system VM template. Please verify that you
> >>>>> have correctly provisioned the secondary storage with the correct
> >>> system VM
> >>>>> template (as per the docs).
> >>>>>
> >>>>> -Somesh
> >>>>>
> >>>>> -----Original Message-----
> >>>>> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
> >>>>> Sent: Tuesday, December 30, 2014 11:03 AM
> >>>>> To: users@cloudstack.apache.org
> >>>>> Cc: dev@cloudstack.apache.org
> >>>>> Subject: Unable to start VM due to concurrent operation
> >>>>>
> >>>>> Hi All,
> >>>>>
> >>>>> I have successfully deployed cloudstack 4.3 with two hosts and the
> >>> setup
> >>>>> runs the management server on another physical host. I had issues
> with
> >>>>> downloading an ISO but finally i was able to download an ISO image.
> >>> When i
> >>>>> want to create an instance i get the following error message saying
> >>>>> "Unable
> >>>>> to start VM due to concurrent operation" i did some searching on the
> >>> topic
> >>>>> and it said version 4.3 system template solved the issue. These are
> >>> some
> >>>>> of
> >>>>> the MS logs that i collected what could be the issue?
> >>>>>
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> >>> center
> >>>>> 1, wait until it is ready to launch secondary storage vm
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> >>>>> storage
> >>>>> VM yet
> >>>>> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-9:null) Ping from 4
> >>>>> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq 3-91695:
> {
> >>> Cmd
> >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>>>>
> >>>>>
> >>>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >>>>> \"connections\": []\n}","wait":0}}] }
> >>>>> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695: {
> >>> Ans: ,
> >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >>>>>
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >>>>> 2014-12-31 05:25:15,239 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
> >>>>> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats
> >>> collector
> >>>>> is
> >>>>> running...
> >>>>> 2014-12-31 05:25:15,362 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>>>> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
> >>>>> 2014-12-31 05:25:15,366 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update
> status.
> >>>>> 2014-12-31 05:25:15,367 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update RvR
> >>>>> status.
> >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >>>>> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being
> >>> called at
> >>>>> 2014-12-30 23:55:15 GMT
> >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >>>>> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
> >>>>> 2014-12-30 23:55:15 GMT
> >>>>> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-14:null) Ping from 3
> >>>>> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-10:null) Ping from 2
> >>>>> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on
> >>> ping:
> >>>>> [1]
> >>>>> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
> >>> (AgentMonitor-1:ctx-f10d32f0)
> >>>>> Ping timeout for host 1, do invstigation
> >>>>> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
> >>> disconnected
> >>>>> with event PingTimeout
> >>>>> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
> >>>>> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
> >>> MgmtId:
> >>>>> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
> >>>>> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >>>>> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: { Ans: ,
> >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
> >>>>>
> >>>>>
> >>>
> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
> >>>>> is alive","wait":0}}] }
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: { Ans: ,
> >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, {
> >>> CheckHealthAnswer
> >>>>> } }
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Details from executing class
> >>>>> com.cloud.agent.api.CheckHealthCommand: resource is alive
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
> >>> checkHeathCommand,
> >>>>> reporting that agent is Up
> >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
> >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and
> >>> running
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state = Enabled,
> >>>>> Agent
> >>>>> event = Ping, Host id = 1, name = virtualops-h4]
> >>>>> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1; name =
> >>>>> virtualops-h4; old status = Up; event = Ping; new status = Up; old
> >>> update
> >>>>> count = 34; new update count = 35]
> >>>>> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq 3-91697:
> {
> >>> Cmd
> >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>>>>
> >>>>>
> >>>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >>>>> \"connections\": []\n}","wait":0}}] }
> >>>>> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697: {
> >>> Ans: ,
> >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >>>>>
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >>>>> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
> >>>>> (StatsCollector-1:ctx-2acfc
> >>>>>
> >>>>> Thank you.
> >>>>>
> >>>>
> >>>>
> >>>
> >>
> >>
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Software Engineering<
> http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
> a company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>
Re: Unable to start VM due to concurrent operation
Posted by Mohamed Infaz <in...@cse.mrt.ac.lk>.
Hi all,
I did redeploy cloudstack and i used the
>> 1. /usr/share/cloudstack-common/
scripts/storage/secondary/cloud-install-sys-tmplt \
>> 2. -m /srv/images/secondary \
>> 3. -u
http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
\
>> 4. -h kvm \
>> 5. -F
The following script to deploy system vm's. After some time they started
working! Now when i try to deploy and vm
i am getting the following error message.
"com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
unreachable: Host 1: Unable to start instance due to Template 3 has not
been completely downloaded to zone 1"
I did reregister the latest system vm template and i was able to
successfully download it. But there are two inbuilt templates where it says
in the status that the connection was time out. And one of these in built
template is an in built old system vm template. How can i solve this issue
now there is a system vm running properly in the environment and i have the
latest template registered but it seems so cloudstack searches for those
two templates which it couldn't download. Because of this i still can't
create a virtual machine.
Thank you.
On 2 January 2015 at 08:08, Abhinandan Prateek <
abhinandan.prateek@shapeblue.com> wrote:
> The sec store has some issues, you can login into KVM host and check if
> the sec storage mount point is visible and the said folder is there. Check
> if ssvm is running fine.
> It is also possible that the sec store is fine but the template that is
> being used to create VM is in inconsistent state, then you need to remove
> and re-register the template.
>
> -abhi
>
> > On 01-Jan-2015, at 8:35 pm, Mohamed Infaz <in...@cse.mrt.ac.lk>
> wrote:
> >
> > Hi,
> > I updated my cloudstack deployment to 4.4.2 to 4.3.1. Now when we try to
> > deploy a VM i am getting following error
> >
> > message. com.cloud.exception.StorageUnavailableException: Resource
> > [StoragePool:2] is unreachable: Unable to create
> > Vol[30|vm=58|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
> > org.libvirt.LibvirtException: internal error: Child process (/bin/mount
> > 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> > /mnt/8c6fa424-cbc7-3421-8f05-d69a43ffa15c) unexpected exit status 32:
> > mount.nfs: mounting 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> failed,
> > reason given by server: No such file or directory
> >
> >
> >
> > Thank you.
> >
> >
> > On 1 January 2015 at 14:03, Mohamed Infaz <in...@cse.mrt.ac.lk>
> wrote:
> >
> >> Hi Somesh,
> >>
> >> Yes i was successful in stopping both the system vm's and starting them
> >> back i also could access them through the console that cloudstack
> provides.
> >> May be the reason for that code snippet is that i didn't register the
> >> template but i ran this script to install the system vm's.
> >>
> >>
> >> 1.
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> \
> >> 2. -m /srv/images/secondary \
> >> 3. -u
> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
> \
> >> 4. -h kvm \
> >> 5. -F
> >>
> >> this was on the docs. How about upgrading my cloudstack version to 4.4.
> >> This whole thing happened in 4.3.
> >>
> >> Thank you.
> >>
> >> On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com>
> wrote:
> >>
> >>> You mean the system VMs for Zone 1 are running? Are they showing as
> >>> connected in the DB?
> >>>
> >>> If they are, can you stop these VMs and see if they start okay.
> >>> ________________________________________
> >>> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
> >>> Sent: Tuesday, December 30, 2014 21:26
> >>> To: users@cloudstack.apache.org
> >>> Cc: dev@cloudstack.apache.org
> >>> Subject: Re: Unable to start VM due to concurrent operation
> >>>
> >>> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the
> template i
> >>> used.
> >>>
> >>>
> >>> On 31 December 2014 at 07:49, Mohamed Infaz <in...@cse.mrt.ac.lk>
> >>> wrote:
> >>>
> >>>> Hi Somesh,
> >>>>
> >>>> Thank you for the reply. Well both of my system vm's are running and
> it
> >>> is
> >>>> downloading the iso image. And i was able to ping them. The system vm
> >>>> template i am using is for the version 4.3 cloudstack.
> >>>>
> >>>> Thank you.
> >>>>
> >>>> On 30 December 2014 at 23:59, Somesh Naidu <So...@citrix.com>
> >>>> wrote:
> >>>>
> >>>>> Mohamed,
> >>>>>
> >>>>> The log snippet you have shared doesn't seem to be relevant to the
> >>>>> concurrent operation exception (can't see that exception stack).
> >>>>>
> >>>>> The message that did catch my attention though is,
> >>>>> -
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> >>> center
> >>>>> 1, wait until it is ready to launch secondary storage vm
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> >>> storage
> >>>>> VM yet
> >>>>> -
> >>>>>
> >>>>> That's pointing to missing system VM template. Please verify that you
> >>>>> have correctly provisioned the secondary storage with the correct
> >>> system VM
> >>>>> template (as per the docs).
> >>>>>
> >>>>> -Somesh
> >>>>>
> >>>>> -----Original Message-----
> >>>>> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
> >>>>> Sent: Tuesday, December 30, 2014 11:03 AM
> >>>>> To: users@cloudstack.apache.org
> >>>>> Cc: dev@cloudstack.apache.org
> >>>>> Subject: Unable to start VM due to concurrent operation
> >>>>>
> >>>>> Hi All,
> >>>>>
> >>>>> I have successfully deployed cloudstack 4.3 with two hosts and the
> >>> setup
> >>>>> runs the management server on another physical host. I had issues
> with
> >>>>> downloading an ISO but finally i was able to download an ISO image.
> >>> When i
> >>>>> want to create an instance i get the following error message saying
> >>>>> "Unable
> >>>>> to start VM due to concurrent operation" i did some searching on the
> >>> topic
> >>>>> and it said version 4.3 system template solved the issue. These are
> >>> some
> >>>>> of
> >>>>> the MS logs that i collected what could be the issue?
> >>>>>
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> >>> center
> >>>>> 1, wait until it is ready to launch secondary storage vm
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> >>>>> storage
> >>>>> VM yet
> >>>>> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-9:null) Ping from 4
> >>>>> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq 3-91695:
> {
> >>> Cmd
> >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>>>>
> >>>>>
> >>>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >>>>> \"connections\": []\n}","wait":0}}] }
> >>>>> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695: {
> >>> Ans: ,
> >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >>>>>
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >>>>> 2014-12-31 05:25:15,239 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
> >>>>> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats
> >>> collector
> >>>>> is
> >>>>> running...
> >>>>> 2014-12-31 05:25:15,362 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>>>> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
> >>>>> 2014-12-31 05:25:15,366 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update
> status.
> >>>>> 2014-12-31 05:25:15,367 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update RvR
> >>>>> status.
> >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >>>>> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being
> >>> called at
> >>>>> 2014-12-30 23:55:15 GMT
> >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >>>>> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
> >>>>> 2014-12-30 23:55:15 GMT
> >>>>> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-14:null) Ping from 3
> >>>>> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-10:null) Ping from 2
> >>>>> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on
> >>> ping:
> >>>>> [1]
> >>>>> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
> >>> (AgentMonitor-1:ctx-f10d32f0)
> >>>>> Ping timeout for host 1, do invstigation
> >>>>> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
> >>> disconnected
> >>>>> with event PingTimeout
> >>>>> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
> >>>>> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
> >>> MgmtId:
> >>>>> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
> >>>>> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >>>>> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: { Ans: ,
> >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
> >>>>>
> >>>>>
> >>>
> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
> >>>>> is alive","wait":0}}] }
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: { Ans: ,
> >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, {
> >>> CheckHealthAnswer
> >>>>> } }
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Details from executing class
> >>>>> com.cloud.agent.api.CheckHealthCommand: resource is alive
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
> >>> checkHeathCommand,
> >>>>> reporting that agent is Up
> >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
> >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and
> >>> running
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state = Enabled,
> >>>>> Agent
> >>>>> event = Ping, Host id = 1, name = virtualops-h4]
> >>>>> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1; name =
> >>>>> virtualops-h4; old status = Up; event = Ping; new status = Up; old
> >>> update
> >>>>> count = 34; new update count = 35]
> >>>>> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq 3-91697:
> {
> >>> Cmd
> >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>>>>
> >>>>>
> >>>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >>>>> \"connections\": []\n}","wait":0}}] }
> >>>>> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697: {
> >>> Ans: ,
> >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >>>>>
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >>>>> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
> >>>>> (StatsCollector-1:ctx-2acfc
> >>>>>
> >>>>> Thank you.
> >>>>>
> >>>>
> >>>>
> >>>
> >>
> >>
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Software Engineering<
> http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
> a company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>
Re: Unable to start VM due to concurrent operation
Posted by Mohamed Infaz <in...@cse.mrt.ac.lk>.
Hi all,
I did redeploy cloudstack and i used the
>> 1. /usr/share/cloudstack-common/
scripts/storage/secondary/cloud-install-sys-tmplt \
>> 2. -m /srv/images/secondary \
>> 3. -u
http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
\
>> 4. -h kvm \
>> 5. -F
The following script to deploy system vm's. After some time they started
working! Now when i try to deploy and vm
i am getting the following error message.
"com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
unreachable: Host 1: Unable to start instance due to Template 3 has not
been completely downloaded to zone 1"
I did reregister the latest system vm template and i was able to
successfully download it. But there are two inbuilt templates where it says
in the status that the connection was time out. And one of these in built
template is an in built old system vm template. How can i solve this issue
now there is a system vm running properly in the environment and i have the
latest template registered but it seems so cloudstack searches for those
two templates which it couldn't download. Because of this i still can't
create a virtual machine.
Thank you.
On 2 January 2015 at 08:08, Abhinandan Prateek <
abhinandan.prateek@shapeblue.com> wrote:
> The sec store has some issues, you can login into KVM host and check if
> the sec storage mount point is visible and the said folder is there. Check
> if ssvm is running fine.
> It is also possible that the sec store is fine but the template that is
> being used to create VM is in inconsistent state, then you need to remove
> and re-register the template.
>
> -abhi
>
> > On 01-Jan-2015, at 8:35 pm, Mohamed Infaz <in...@cse.mrt.ac.lk>
> wrote:
> >
> > Hi,
> > I updated my cloudstack deployment to 4.4.2 to 4.3.1. Now when we try to
> > deploy a VM i am getting following error
> >
> > message. com.cloud.exception.StorageUnavailableException: Resource
> > [StoragePool:2] is unreachable: Unable to create
> > Vol[30|vm=58|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
> > org.libvirt.LibvirtException: internal error: Child process (/bin/mount
> > 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> > /mnt/8c6fa424-cbc7-3421-8f05-d69a43ffa15c) unexpected exit status 32:
> > mount.nfs: mounting 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> failed,
> > reason given by server: No such file or directory
> >
> >
> >
> > Thank you.
> >
> >
> > On 1 January 2015 at 14:03, Mohamed Infaz <in...@cse.mrt.ac.lk>
> wrote:
> >
> >> Hi Somesh,
> >>
> >> Yes i was successful in stopping both the system vm's and starting them
> >> back i also could access them through the console that cloudstack
> provides.
> >> May be the reason for that code snippet is that i didn't register the
> >> template but i ran this script to install the system vm's.
> >>
> >>
> >> 1.
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> \
> >> 2. -m /srv/images/secondary \
> >> 3. -u
> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
> \
> >> 4. -h kvm \
> >> 5. -F
> >>
> >> this was on the docs. How about upgrading my cloudstack version to 4.4.
> >> This whole thing happened in 4.3.
> >>
> >> Thank you.
> >>
> >> On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com>
> wrote:
> >>
> >>> You mean the system VMs for Zone 1 are running? Are they showing as
> >>> connected in the DB?
> >>>
> >>> If they are, can you stop these VMs and see if they start okay.
> >>> ________________________________________
> >>> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
> >>> Sent: Tuesday, December 30, 2014 21:26
> >>> To: users@cloudstack.apache.org
> >>> Cc: dev@cloudstack.apache.org
> >>> Subject: Re: Unable to start VM due to concurrent operation
> >>>
> >>> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the
> template i
> >>> used.
> >>>
> >>>
> >>> On 31 December 2014 at 07:49, Mohamed Infaz <in...@cse.mrt.ac.lk>
> >>> wrote:
> >>>
> >>>> Hi Somesh,
> >>>>
> >>>> Thank you for the reply. Well both of my system vm's are running and
> it
> >>> is
> >>>> downloading the iso image. And i was able to ping them. The system vm
> >>>> template i am using is for the version 4.3 cloudstack.
> >>>>
> >>>> Thank you.
> >>>>
> >>>> On 30 December 2014 at 23:59, Somesh Naidu <So...@citrix.com>
> >>>> wrote:
> >>>>
> >>>>> Mohamed,
> >>>>>
> >>>>> The log snippet you have shared doesn't seem to be relevant to the
> >>>>> concurrent operation exception (can't see that exception stack).
> >>>>>
> >>>>> The message that did catch my attention though is,
> >>>>> -
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> >>> center
> >>>>> 1, wait until it is ready to launch secondary storage vm
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> >>> storage
> >>>>> VM yet
> >>>>> -
> >>>>>
> >>>>> That's pointing to missing system VM template. Please verify that you
> >>>>> have correctly provisioned the secondary storage with the correct
> >>> system VM
> >>>>> template (as per the docs).
> >>>>>
> >>>>> -Somesh
> >>>>>
> >>>>> -----Original Message-----
> >>>>> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
> >>>>> Sent: Tuesday, December 30, 2014 11:03 AM
> >>>>> To: users@cloudstack.apache.org
> >>>>> Cc: dev@cloudstack.apache.org
> >>>>> Subject: Unable to start VM due to concurrent operation
> >>>>>
> >>>>> Hi All,
> >>>>>
> >>>>> I have successfully deployed cloudstack 4.3 with two hosts and the
> >>> setup
> >>>>> runs the management server on another physical host. I had issues
> with
> >>>>> downloading an ISO but finally i was able to download an ISO image.
> >>> When i
> >>>>> want to create an instance i get the following error message saying
> >>>>> "Unable
> >>>>> to start VM due to concurrent operation" i did some searching on the
> >>> topic
> >>>>> and it said version 4.3 system template solved the issue. These are
> >>> some
> >>>>> of
> >>>>> the MS logs that i collected what could be the issue?
> >>>>>
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
> >>> center
> >>>>> 1, wait until it is ready to launch secondary storage vm
> >>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
> >>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
> >>>>> storage
> >>>>> VM yet
> >>>>> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-9:null) Ping from 4
> >>>>> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq 3-91695:
> {
> >>> Cmd
> >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>>>>
> >>>>>
> >>>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >>>>> \"connections\": []\n}","wait":0}}] }
> >>>>> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695: {
> >>> Ans: ,
> >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >>>>>
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >>>>> 2014-12-31 05:25:15,239 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
> >>>>> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats
> >>> collector
> >>>>> is
> >>>>> running...
> >>>>> 2014-12-31 05:25:15,362 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>>>> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
> >>>>> 2014-12-31 05:25:15,366 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update
> status.
> >>>>> 2014-12-31 05:25:15,367 DEBUG
> >>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update RvR
> >>>>> status.
> >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >>>>> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being
> >>> called at
> >>>>> 2014-12-30 23:55:15 GMT
> >>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
> >>>>> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
> >>>>> 2014-12-30 23:55:15 GMT
> >>>>> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-14:null) Ping from 3
> >>>>> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-10:null) Ping from 2
> >>>>> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on
> >>> ping:
> >>>>> [1]
> >>>>> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
> >>> (AgentMonitor-1:ctx-f10d32f0)
> >>>>> Ping timeout for host 1, do invstigation
> >>>>> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
> >>> disconnected
> >>>>> with event PingTimeout
> >>>>> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
> >>>>> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
> >>> MgmtId:
> >>>>> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
> >>>>> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >>>>> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: { Ans: ,
> >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
> >>>>>
> >>>>>
> >>>
> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
> >>>>> is alive","wait":0}}] }
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: { Ans: ,
> >>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, {
> >>> CheckHealthAnswer
> >>>>> } }
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Details from executing class
> >>>>> com.cloud.agent.api.CheckHealthCommand: resource is alive
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
> >>> checkHeathCommand,
> >>>>> reporting that agent is Up
> >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
> >>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and
> >>> running
> >>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state = Enabled,
> >>>>> Agent
> >>>>> event = Ping, Host id = 1, name = virtualops-h4]
> >>>>> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
> >>>>> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1; name =
> >>>>> virtualops-h4; old status = Up; event = Ping; new status = Up; old
> >>> update
> >>>>> count = 34; new update count = 35]
> >>>>> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq 3-91697:
> {
> >>> Cmd
> >>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> >>>>>
> >>>>>
> >>>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
> >>>>> \"connections\": []\n}","wait":0}}] }
> >>>>> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
> >>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697: {
> >>> Ans: ,
> >>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
> >>>>>
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> >>>>> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
> >>>>> (StatsCollector-1:ctx-2acfc
> >>>>>
> >>>>> Thank you.
> >>>>>
> >>>>
> >>>>
> >>>
> >>
> >>
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Software Engineering<
> http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
> a company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>
Re: Unable to start VM due to concurrent operation
Posted by Abhinandan Prateek <ab...@shapeblue.com>.
The sec store has some issues, you can login into KVM host and check if the sec storage mount point is visible and the said folder is there. Check if ssvm is running fine.
It is also possible that the sec store is fine but the template that is being used to create VM is in inconsistent state, then you need to remove and re-register the template.
-abhi
> On 01-Jan-2015, at 8:35 pm, Mohamed Infaz <in...@cse.mrt.ac.lk> wrote:
>
> Hi,
> I updated my cloudstack deployment to 4.4.2 to 4.3.1. Now when we try to
> deploy a VM i am getting following error
>
> message. com.cloud.exception.StorageUnavailableException: Resource
> [StoragePool:2] is unreachable: Unable to create
> Vol[30|vm=58|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
> org.libvirt.LibvirtException: internal error: Child process (/bin/mount
> 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> /mnt/8c6fa424-cbc7-3421-8f05-d69a43ffa15c) unexpected exit status 32:
> mount.nfs: mounting 10.8.100.201:/mnt/secondary/template/tmpl/2/205 failed,
> reason given by server: No such file or directory
>
>
>
> Thank you.
>
>
> On 1 January 2015 at 14:03, Mohamed Infaz <in...@cse.mrt.ac.lk> wrote:
>
>> Hi Somesh,
>>
>> Yes i was successful in stopping both the system vm's and starting them
>> back i also could access them through the console that cloudstack provides.
>> May be the reason for that code snippet is that i didn't register the
>> template but i ran this script to install the system vm's.
>>
>>
>> 1. /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt \
>> 2. -m /srv/images/secondary \
>> 3. -u http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2 \
>> 4. -h kvm \
>> 5. -F
>>
>> this was on the docs. How about upgrading my cloudstack version to 4.4.
>> This whole thing happened in 4.3.
>>
>> Thank you.
>>
>> On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com> wrote:
>>
>>> You mean the system VMs for Zone 1 are running? Are they showing as
>>> connected in the DB?
>>>
>>> If they are, can you stop these VMs and see if they start okay.
>>> ________________________________________
>>> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
>>> Sent: Tuesday, December 30, 2014 21:26
>>> To: users@cloudstack.apache.org
>>> Cc: dev@cloudstack.apache.org
>>> Subject: Re: Unable to start VM due to concurrent operation
>>>
>>> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the template i
>>> used.
>>>
>>>
>>> On 31 December 2014 at 07:49, Mohamed Infaz <in...@cse.mrt.ac.lk>
>>> wrote:
>>>
>>>> Hi Somesh,
>>>>
>>>> Thank you for the reply. Well both of my system vm's are running and it
>>> is
>>>> downloading the iso image. And i was able to ping them. The system vm
>>>> template i am using is for the version 4.3 cloudstack.
>>>>
>>>> Thank you.
>>>>
>>>> On 30 December 2014 at 23:59, Somesh Naidu <So...@citrix.com>
>>>> wrote:
>>>>
>>>>> Mohamed,
>>>>>
>>>>> The log snippet you have shared doesn't seem to be relevant to the
>>>>> concurrent operation exception (can't see that exception stack).
>>>>>
>>>>> The message that did catch my attention though is,
>>>>> -
>>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
>>> center
>>>>> 1, wait until it is ready to launch secondary storage vm
>>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
>>> storage
>>>>> VM yet
>>>>> -
>>>>>
>>>>> That's pointing to missing system VM template. Please verify that you
>>>>> have correctly provisioned the secondary storage with the correct
>>> system VM
>>>>> template (as per the docs).
>>>>>
>>>>> -Somesh
>>>>>
>>>>> -----Original Message-----
>>>>> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
>>>>> Sent: Tuesday, December 30, 2014 11:03 AM
>>>>> To: users@cloudstack.apache.org
>>>>> Cc: dev@cloudstack.apache.org
>>>>> Subject: Unable to start VM due to concurrent operation
>>>>>
>>>>> Hi All,
>>>>>
>>>>> I have successfully deployed cloudstack 4.3 with two hosts and the
>>> setup
>>>>> runs the management server on another physical host. I had issues with
>>>>> downloading an ISO but finally i was able to download an ISO image.
>>> When i
>>>>> want to create an instance i get the following error message saying
>>>>> "Unable
>>>>> to start VM due to concurrent operation" i did some searching on the
>>> topic
>>>>> and it said version 4.3 system template solved the issue. These are
>>> some
>>>>> of
>>>>> the MS logs that i collected what could be the issue?
>>>>>
>>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
>>> center
>>>>> 1, wait until it is ready to launch secondary storage vm
>>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
>>>>> storage
>>>>> VM yet
>>>>> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-9:null) Ping from 4
>>>>> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq 3-91695: {
>>> Cmd
>>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>>>>>
>>>>>
>>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
>>>>> \"connections\": []\n}","wait":0}}] }
>>>>> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695: {
>>> Ans: ,
>>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
>>>>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
>>>>> 2014-12-31 05:25:15,239 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
>>>>> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats
>>> collector
>>>>> is
>>>>> running...
>>>>> 2014-12-31 05:25:15,362 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>>>> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
>>>>> 2014-12-31 05:25:15,366 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update status.
>>>>> 2014-12-31 05:25:15,367 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update RvR
>>>>> status.
>>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>>>>> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being
>>> called at
>>>>> 2014-12-30 23:55:15 GMT
>>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>>>>> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
>>>>> 2014-12-30 23:55:15 GMT
>>>>> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-14:null) Ping from 3
>>>>> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-10:null) Ping from 2
>>>>> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
>>>>> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on
>>> ping:
>>>>> [1]
>>>>> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
>>> (AgentMonitor-1:ctx-f10d32f0)
>>>>> Ping timeout for host 1, do invstigation
>>>>> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
>>>>> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
>>> disconnected
>>>>> with event PingTimeout
>>>>> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
>>>>> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
>>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
>>> MgmtId:
>>>>> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
>>>>> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
>>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
>>>>> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: { Ans: ,
>>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
>>>>>
>>>>>
>>> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
>>>>> is alive","wait":0}}] }
>>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
>>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: { Ans: ,
>>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, {
>>> CheckHealthAnswer
>>>>> } }
>>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentTaskPool-14:ctx-43a85937) Details from executing class
>>>>> com.cloud.agent.api.CheckHealthCommand: resource is alive
>>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
>>> checkHeathCommand,
>>>>> reporting that agent is Up
>>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
>>>>> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
>>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
>>>>> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and
>>> running
>>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
>>>>> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state = Enabled,
>>>>> Agent
>>>>> event = Ping, Host id = 1, name = virtualops-h4]
>>>>> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
>>>>> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1; name =
>>>>> virtualops-h4; old status = Up; event = Ping; new status = Up; old
>>> update
>>>>> count = 34; new update count = 35]
>>>>> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq 3-91697: {
>>> Cmd
>>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>>>>>
>>>>>
>>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
>>>>> \"connections\": []\n}","wait":0}}] }
>>>>> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697: {
>>> Ans: ,
>>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
>>>>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
>>>>> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
>>>>> (StatsCollector-1:ctx-2acfc
>>>>>
>>>>> Thank you.
>>>>>
>>>>
>>>>
>>>
>>
>>
Find out more about ShapeBlue and our range of CloudStack related services
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: Unable to start VM due to concurrent operation
Posted by Abhinandan Prateek <ab...@shapeblue.com>.
The sec store has some issues, you can login into KVM host and check if the sec storage mount point is visible and the said folder is there. Check if ssvm is running fine.
It is also possible that the sec store is fine but the template that is being used to create VM is in inconsistent state, then you need to remove and re-register the template.
-abhi
> On 01-Jan-2015, at 8:35 pm, Mohamed Infaz <in...@cse.mrt.ac.lk> wrote:
>
> Hi,
> I updated my cloudstack deployment to 4.4.2 to 4.3.1. Now when we try to
> deploy a VM i am getting following error
>
> message. com.cloud.exception.StorageUnavailableException: Resource
> [StoragePool:2] is unreachable: Unable to create
> Vol[30|vm=58|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
> org.libvirt.LibvirtException: internal error: Child process (/bin/mount
> 10.8.100.201:/mnt/secondary/template/tmpl/2/205
> /mnt/8c6fa424-cbc7-3421-8f05-d69a43ffa15c) unexpected exit status 32:
> mount.nfs: mounting 10.8.100.201:/mnt/secondary/template/tmpl/2/205 failed,
> reason given by server: No such file or directory
>
>
>
> Thank you.
>
>
> On 1 January 2015 at 14:03, Mohamed Infaz <in...@cse.mrt.ac.lk> wrote:
>
>> Hi Somesh,
>>
>> Yes i was successful in stopping both the system vm's and starting them
>> back i also could access them through the console that cloudstack provides.
>> May be the reason for that code snippet is that i didn't register the
>> template but i ran this script to install the system vm's.
>>
>>
>> 1. /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt \
>> 2. -m /srv/images/secondary \
>> 3. -u http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2 \
>> 4. -h kvm \
>> 5. -F
>>
>> this was on the docs. How about upgrading my cloudstack version to 4.4.
>> This whole thing happened in 4.3.
>>
>> Thank you.
>>
>> On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com> wrote:
>>
>>> You mean the system VMs for Zone 1 are running? Are they showing as
>>> connected in the DB?
>>>
>>> If they are, can you stop these VMs and see if they start okay.
>>> ________________________________________
>>> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
>>> Sent: Tuesday, December 30, 2014 21:26
>>> To: users@cloudstack.apache.org
>>> Cc: dev@cloudstack.apache.org
>>> Subject: Re: Unable to start VM due to concurrent operation
>>>
>>> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the template i
>>> used.
>>>
>>>
>>> On 31 December 2014 at 07:49, Mohamed Infaz <in...@cse.mrt.ac.lk>
>>> wrote:
>>>
>>>> Hi Somesh,
>>>>
>>>> Thank you for the reply. Well both of my system vm's are running and it
>>> is
>>>> downloading the iso image. And i was able to ping them. The system vm
>>>> template i am using is for the version 4.3 cloudstack.
>>>>
>>>> Thank you.
>>>>
>>>> On 30 December 2014 at 23:59, Somesh Naidu <So...@citrix.com>
>>>> wrote:
>>>>
>>>>> Mohamed,
>>>>>
>>>>> The log snippet you have shared doesn't seem to be relevant to the
>>>>> concurrent operation exception (can't see that exception stack).
>>>>>
>>>>> The message that did catch my attention though is,
>>>>> -
>>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
>>> center
>>>>> 1, wait until it is ready to launch secondary storage vm
>>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
>>> storage
>>>>> VM yet
>>>>> -
>>>>>
>>>>> That's pointing to missing system VM template. Please verify that you
>>>>> have correctly provisioned the secondary storage with the correct
>>> system VM
>>>>> template (as per the docs).
>>>>>
>>>>> -Somesh
>>>>>
>>>>> -----Original Message-----
>>>>> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
>>>>> Sent: Tuesday, December 30, 2014 11:03 AM
>>>>> To: users@cloudstack.apache.org
>>>>> Cc: dev@cloudstack.apache.org
>>>>> Subject: Unable to start VM due to concurrent operation
>>>>>
>>>>> Hi All,
>>>>>
>>>>> I have successfully deployed cloudstack 4.3 with two hosts and the
>>> setup
>>>>> runs the management server on another physical host. I had issues with
>>>>> downloading an ISO but finally i was able to download an ISO image.
>>> When i
>>>>> want to create an instance i get the following error message saying
>>>>> "Unable
>>>>> to start VM due to concurrent operation" i did some searching on the
>>> topic
>>>>> and it said version 4.3 system template solved the issue. These are
>>> some
>>>>> of
>>>>> the MS logs that i collected what could be the issue?
>>>>>
>>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>>>>> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
>>> center
>>>>> 1, wait until it is ready to launch secondary storage vm
>>>>> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>>>>> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
>>>>> storage
>>>>> VM yet
>>>>> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-9:null) Ping from 4
>>>>> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq 3-91695: {
>>> Cmd
>>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>>>>>
>>>>>
>>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
>>>>> \"connections\": []\n}","wait":0}}] }
>>>>> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695: {
>>> Ans: ,
>>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
>>>>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
>>>>> 2014-12-31 05:25:15,239 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
>>>>> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats
>>> collector
>>>>> is
>>>>> running...
>>>>> 2014-12-31 05:25:15,362 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>>>> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
>>>>> 2014-12-31 05:25:15,366 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update status.
>>>>> 2014-12-31 05:25:15,367 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>>>> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update RvR
>>>>> status.
>>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>>>>> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being
>>> called at
>>>>> 2014-12-30 23:55:15 GMT
>>>>> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>>>>> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
>>>>> 2014-12-30 23:55:15 GMT
>>>>> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-14:null) Ping from 3
>>>>> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-10:null) Ping from 2
>>>>> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
>>>>> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on
>>> ping:
>>>>> [1]
>>>>> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
>>> (AgentMonitor-1:ctx-f10d32f0)
>>>>> Ping timeout for host 1, do invstigation
>>>>> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
>>>>> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
>>> disconnected
>>>>> with event PingTimeout
>>>>> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
>>>>> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
>>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
>>> MgmtId:
>>>>> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
>>>>> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
>>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
>>>>> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: { Ans: ,
>>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
>>>>>
>>>>>
>>> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
>>>>> is alive","wait":0}}] }
>>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
>>>>> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: { Ans: ,
>>>>> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, {
>>> CheckHealthAnswer
>>>>> } }
>>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentTaskPool-14:ctx-43a85937) Details from executing class
>>>>> com.cloud.agent.api.CheckHealthCommand: resource is alive
>>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
>>> checkHeathCommand,
>>>>> reporting that agent is Up
>>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
>>>>> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
>>>>> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
>>>>> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and
>>> running
>>>>> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
>>>>> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state = Enabled,
>>>>> Agent
>>>>> event = Ping, Host id = 1, name = virtualops-h4]
>>>>> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
>>>>> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1; name =
>>>>> virtualops-h4; old status = Up; event = Ping; new status = Up; old
>>> update
>>>>> count = 34; new update count = 35]
>>>>> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq 3-91697: {
>>> Cmd
>>>>> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>>>>>
>>>>>
>>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
>>>>> \"connections\": []\n}","wait":0}}] }
>>>>> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697: {
>>> Ans: ,
>>>>> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
>>>>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
>>>>> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
>>>>> (StatsCollector-1:ctx-2acfc
>>>>>
>>>>> Thank you.
>>>>>
>>>>
>>>>
>>>
>>
>>
Find out more about ShapeBlue and our range of CloudStack related services
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: Unable to start VM due to concurrent operation
Posted by Mohamed Infaz <in...@cse.mrt.ac.lk>.
Hi,
I updated my cloudstack deployment to 4.4.2 to 4.3.1. Now when we try to
deploy a VM i am getting following error
message. com.cloud.exception.StorageUnavailableException: Resource
[StoragePool:2] is unreachable: Unable to create
Vol[30|vm=58|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
org.libvirt.LibvirtException: internal error: Child process (/bin/mount
10.8.100.201:/mnt/secondary/template/tmpl/2/205
/mnt/8c6fa424-cbc7-3421-8f05-d69a43ffa15c) unexpected exit status 32:
mount.nfs: mounting 10.8.100.201:/mnt/secondary/template/tmpl/2/205 failed,
reason given by server: No such file or directory
Thank you.
On 1 January 2015 at 14:03, Mohamed Infaz <in...@cse.mrt.ac.lk> wrote:
> Hi Somesh,
>
> Yes i was successful in stopping both the system vm's and starting them
> back i also could access them through the console that cloudstack provides.
> May be the reason for that code snippet is that i didn't register the
> template but i ran this script to install the system vm's.
>
>
> 1. /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt \
> 2. -m /srv/images/secondary \
> 3. -u http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2 \
> 4. -h kvm \
> 5. -F
>
> this was on the docs. How about upgrading my cloudstack version to 4.4.
> This whole thing happened in 4.3.
>
> Thank you.
>
> On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com> wrote:
>
>> You mean the system VMs for Zone 1 are running? Are they showing as
>> connected in the DB?
>>
>> If they are, can you stop these VMs and see if they start okay.
>> ________________________________________
>> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
>> Sent: Tuesday, December 30, 2014 21:26
>> To: users@cloudstack.apache.org
>> Cc: dev@cloudstack.apache.org
>> Subject: Re: Unable to start VM due to concurrent operation
>>
>> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the template i
>> used.
>>
>>
>> On 31 December 2014 at 07:49, Mohamed Infaz <in...@cse.mrt.ac.lk>
>> wrote:
>>
>> > Hi Somesh,
>> >
>> > Thank you for the reply. Well both of my system vm's are running and it
>> is
>> > downloading the iso image. And i was able to ping them. The system vm
>> > template i am using is for the version 4.3 cloudstack.
>> >
>> > Thank you.
>> >
>> > On 30 December 2014 at 23:59, Somesh Naidu <So...@citrix.com>
>> > wrote:
>> >
>> >> Mohamed,
>> >>
>> >> The log snippet you have shared doesn't seem to be relevant to the
>> >> concurrent operation exception (can't see that exception stack).
>> >>
>> >> The message that did catch my attention though is,
>> >> -
>> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>> >> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
>> center
>> >> 1, wait until it is ready to launch secondary storage vm
>> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>> >> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
>> storage
>> >> VM yet
>> >> -
>> >>
>> >> That's pointing to missing system VM template. Please verify that you
>> >> have correctly provisioned the secondary storage with the correct
>> system VM
>> >> template (as per the docs).
>> >>
>> >> -Somesh
>> >>
>> >> -----Original Message-----
>> >> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
>> >> Sent: Tuesday, December 30, 2014 11:03 AM
>> >> To: users@cloudstack.apache.org
>> >> Cc: dev@cloudstack.apache.org
>> >> Subject: Unable to start VM due to concurrent operation
>> >>
>> >> Hi All,
>> >>
>> >> I have successfully deployed cloudstack 4.3 with two hosts and the
>> setup
>> >> runs the management server on another physical host. I had issues with
>> >> downloading an ISO but finally i was able to download an ISO image.
>> When i
>> >> want to create an instance i get the following error message saying
>> >> "Unable
>> >> to start VM due to concurrent operation" i did some searching on the
>> topic
>> >> and it said version 4.3 system template solved the issue. These are
>> some
>> >> of
>> >> the MS logs that i collected what could be the issue?
>> >>
>> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>> >> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
>> center
>> >> 1, wait until it is ready to launch secondary storage vm
>> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>> >> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
>> >> storage
>> >> VM yet
>> >> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-9:null) Ping from 4
>> >> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq 3-91695: {
>> Cmd
>> >> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> >>
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
>> >> \"connections\": []\n}","wait":0}}] }
>> >> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695: {
>> Ans: ,
>> >> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
>> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
>> >> 2014-12-31 05:25:15,239 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
>> >> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats
>> collector
>> >> is
>> >> running...
>> >> 2014-12-31 05:25:15,362 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
>> >> 2014-12-31 05:25:15,366 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update status.
>> >> 2014-12-31 05:25:15,367 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update RvR
>> >> status.
>> >> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> >> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being
>> called at
>> >> 2014-12-30 23:55:15 GMT
>> >> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> >> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
>> >> 2014-12-30 23:55:15 GMT
>> >> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-14:null) Ping from 3
>> >> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-10:null) Ping from 2
>> >> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
>> >> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on
>> ping:
>> >> [1]
>> >> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
>> (AgentMonitor-1:ctx-f10d32f0)
>> >> Ping timeout for host 1, do invstigation
>> >> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
>> >> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
>> disconnected
>> >> with event PingTimeout
>> >> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
>> >> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
>> >> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
>> MgmtId:
>> >> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
>> >> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
>> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
>> >> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: { Ans: ,
>> >> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
>> >>
>> >>
>> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
>> >> is alive","wait":0}}] }
>> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
>> >> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: { Ans: ,
>> >> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, {
>> CheckHealthAnswer
>> >> } }
>> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentTaskPool-14:ctx-43a85937) Details from executing class
>> >> com.cloud.agent.api.CheckHealthCommand: resource is alive
>> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
>> checkHeathCommand,
>> >> reporting that agent is Up
>> >> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
>> >> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
>> >> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
>> >> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and
>> running
>> >> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
>> >> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state = Enabled,
>> >> Agent
>> >> event = Ping, Host id = 1, name = virtualops-h4]
>> >> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
>> >> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1; name =
>> >> virtualops-h4; old status = Up; event = Ping; new status = Up; old
>> update
>> >> count = 34; new update count = 35]
>> >> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq 3-91697: {
>> Cmd
>> >> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> >>
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
>> >> \"connections\": []\n}","wait":0}}] }
>> >> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697: {
>> Ans: ,
>> >> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
>> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
>> >> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
>> >> (StatsCollector-1:ctx-2acfc
>> >>
>> >> Thank you.
>> >>
>> >
>> >
>>
>
>
Re: Unable to start VM due to concurrent operation
Posted by Mohamed Infaz <in...@cse.mrt.ac.lk>.
Hi,
I updated my cloudstack deployment to 4.4.2 to 4.3.1. Now when we try to
deploy a VM i am getting following error
message. com.cloud.exception.StorageUnavailableException: Resource
[StoragePool:2] is unreachable: Unable to create
Vol[30|vm=58|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
org.libvirt.LibvirtException: internal error: Child process (/bin/mount
10.8.100.201:/mnt/secondary/template/tmpl/2/205
/mnt/8c6fa424-cbc7-3421-8f05-d69a43ffa15c) unexpected exit status 32:
mount.nfs: mounting 10.8.100.201:/mnt/secondary/template/tmpl/2/205 failed,
reason given by server: No such file or directory
Thank you.
On 1 January 2015 at 14:03, Mohamed Infaz <in...@cse.mrt.ac.lk> wrote:
> Hi Somesh,
>
> Yes i was successful in stopping both the system vm's and starting them
> back i also could access them through the console that cloudstack provides.
> May be the reason for that code snippet is that i didn't register the
> template but i ran this script to install the system vm's.
>
>
> 1. /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt \
> 2. -m /srv/images/secondary \
> 3. -u http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2 \
> 4. -h kvm \
> 5. -F
>
> this was on the docs. How about upgrading my cloudstack version to 4.4.
> This whole thing happened in 4.3.
>
> Thank you.
>
> On 1 January 2015 at 01:39, Somesh Naidu <So...@citrix.com> wrote:
>
>> You mean the system VMs for Zone 1 are running? Are they showing as
>> connected in the DB?
>>
>> If they are, can you stop these VMs and see if they start okay.
>> ________________________________________
>> From: Mohamed Infaz [infaz.10@cse.mrt.ac.lk]
>> Sent: Tuesday, December 30, 2014 21:26
>> To: users@cloudstack.apache.org
>> Cc: dev@cloudstack.apache.org
>> Subject: Re: Unable to start VM due to concurrent operation
>>
>> systemvm64template-2014-01-14-master-kvm.qcow2.bz2 This is the template i
>> used.
>>
>>
>> On 31 December 2014 at 07:49, Mohamed Infaz <in...@cse.mrt.ac.lk>
>> wrote:
>>
>> > Hi Somesh,
>> >
>> > Thank you for the reply. Well both of my system vm's are running and it
>> is
>> > downloading the iso image. And i was able to ping them. The system vm
>> > template i am using is for the version 4.3 cloudstack.
>> >
>> > Thank you.
>> >
>> > On 30 December 2014 at 23:59, Somesh Naidu <So...@citrix.com>
>> > wrote:
>> >
>> >> Mohamed,
>> >>
>> >> The log snippet you have shared doesn't seem to be relevant to the
>> >> concurrent operation exception (can't see that exception stack).
>> >>
>> >> The message that did catch my attention though is,
>> >> -
>> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>> >> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
>> center
>> >> 1, wait until it is ready to launch secondary storage vm
>> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>> >> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
>> storage
>> >> VM yet
>> >> -
>> >>
>> >> That's pointing to missing system VM template. Please verify that you
>> >> have correctly provisioned the secondary storage with the correct
>> system VM
>> >> template (as per the docs).
>> >>
>> >> -Somesh
>> >>
>> >> -----Original Message-----
>> >> From: Mohamed Infaz [mailto:infaz.10@cse.mrt.ac.lk]
>> >> Sent: Tuesday, December 30, 2014 11:03 AM
>> >> To: users@cloudstack.apache.org
>> >> Cc: dev@cloudstack.apache.org
>> >> Subject: Unable to start VM due to concurrent operation
>> >>
>> >> Hi All,
>> >>
>> >> I have successfully deployed cloudstack 4.3 with two hosts and the
>> setup
>> >> runs the management server on another physical host. I had issues with
>> >> downloading an ISO but finally i was able to download an ISO image.
>> When i
>> >> want to create an instance i get the following error message saying
>> >> "Unable
>> >> to start VM due to concurrent operation" i did some searching on the
>> topic
>> >> and it said version 4.3 system template solved the issue. These are
>> some
>> >> of
>> >> the MS logs that i collected what could be the issue?
>> >>
>> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>> >> (secstorage-1:ctx-463d2f57) System vm template is not ready at data
>> center
>> >> 1, wait until it is ready to launch secondary storage vm
>> >> 2014-12-31 05:25:07,972 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
>> >> (secstorage-1:ctx-463d2f57) Zone 1 is not ready to launch secondary
>> >> storage
>> >> VM yet
>> >> 2014-12-31 05:25:12,425 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-9:null) Ping from 4
>> >> 2014-12-31 05:25:12,834 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-5:null) SeqA 3-91695: Processing Seq 3-91695: {
>> Cmd
>> >> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> >>
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
>> >> \"connections\": []\n}","wait":0}}] }
>> >> 2014-12-31 05:25:12,920 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-5:null) SeqA 3-91695: Sending Seq 3-91695: {
>> Ans: ,
>> >> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
>> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
>> >> 2014-12-31 05:25:15,239 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl]
>> >> (ExternalNetworkMonitor-1:ctx-998a86df) External devices stats
>> collector
>> >> is
>> >> running...
>> >> 2014-12-31 05:25:15,362 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> (RouterMonitor-1:ctx-dfdc73ce) Found 0 running routers.
>> >> 2014-12-31 05:25:15,366 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> (RouterStatusMonitor-1:ctx-79d58896) Found 0 routers to update status.
>> >> 2014-12-31 05:25:15,367 DEBUG
>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> (RouterStatusMonitor-1:ctx-79d58896) Found 0 networks to update RvR
>> >> status.
>> >> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> >> (SnapshotPollTask:ctx-ae761f6b) Snapshot scheduler.poll is being
>> called at
>> >> 2014-12-30 23:55:15 GMT
>> >> 2014-12-31 05:25:15,393 DEBUG [c.c.s.s.SnapshotSchedulerImpl]
>> >> (SnapshotPollTask:ctx-ae761f6b) Got 0 snapshots to be executed at
>> >> 2014-12-30 23:55:15 GMT
>> >> 2014-12-31 05:25:18,535 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-14:null) Ping from 3
>> >> 2014-12-31 05:25:18,698 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-10:null) Ping from 2
>> >> 2014-12-31 05:25:19,834 INFO [c.c.a.m.AgentManagerImpl]
>> >> (AgentMonitor-1:ctx-f10d32f0) Found the following agents behind on
>> ping:
>> >> [1]
>> >> 2014-12-31 05:25:19,835 DEBUG [c.c.h.Status]
>> (AgentMonitor-1:ctx-f10d32f0)
>> >> Ping timeout for host 1, do invstigation
>> >> 2014-12-31 05:25:19,837 INFO [c.c.a.m.AgentManagerImpl]
>> >> (AgentTaskPool-14:ctx-43a85937) Investigating why host 1 has
>> disconnected
>> >> with event PingTimeout
>> >> 2014-12-31 05:25:19,837 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentTaskPool-14:ctx-43a85937) checking if agent (1) is alive
>> >> 2014-12-31 05:25:19,839 DEBUG [c.c.a.t.Request]
>> >> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Sending { Cmd ,
>> MgmtId:
>> >> 248795600505608, via: 1(virtualops-h4), Ver: v1, Flags: 100011,
>> >> [{"com.cloud.agent.api.CheckHealthCommand":{"wait":50}}] }
>> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
>> >> (AgentManager-Handler-7:null) Seq 1-449186863: Processing: { Ans: ,
>> >> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10,
>> >>
>> >>
>> [{"com.cloud.agent.api.CheckHealthAnswer":{"result":true,"details":"resource
>> >> is alive","wait":0}}] }
>> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.t.Request]
>> >> (AgentTaskPool-14:ctx-43a85937) Seq 1-449186863: Received: { Ans: ,
>> >> MgmtId: 248795600505608, via: 1, Ver: v1, Flags: 10, {
>> CheckHealthAnswer
>> >> } }
>> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentTaskPool-14:ctx-43a85937) Details from executing class
>> >> com.cloud.agent.api.CheckHealthCommand: resource is alive
>> >> 2014-12-31 05:25:19,844 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentTaskPool-14:ctx-43a85937) agent (1) responded to
>> checkHeathCommand,
>> >> reporting that agent is Up
>> >> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
>> >> (AgentTaskPool-14:ctx-43a85937) The state determined is Up
>> >> 2014-12-31 05:25:19,844 INFO [c.c.a.m.AgentManagerImpl]
>> >> (AgentTaskPool-14:ctx-43a85937) Agent is determined to be up and
>> running
>> >> 2014-12-31 05:25:19,844 DEBUG [c.c.h.Status]
>> >> (AgentTaskPool-14:ctx-43a85937) Transition:[Resource state = Enabled,
>> >> Agent
>> >> event = Ping, Host id = 1, name = virtualops-h4]
>> >> 2014-12-31 05:25:19,929 DEBUG [c.c.h.Status]
>> >> (AgentTaskPool-14:ctx-43a85937) Agent status update: [id = 1; name =
>> >> virtualops-h4; old status = Up; event = Ping; new status = Up; old
>> update
>> >> count = 34; new update count = 35]
>> >> 2014-12-31 05:25:22,873 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-2:null) SeqA 3-91697: Processing Seq 3-91697: {
>> Cmd
>> >> , MgmtId: -1, via: 3, Ver: v1, Flags: 11,
>> >>
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
>> >> \"connections\": []\n}","wait":0}}] }
>> >> 2014-12-31 05:25:22,955 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (AgentManager-Handler-2:null) SeqA 3-91697: Sending Seq 3-91697: {
>> Ans: ,
>> >> MgmtId: 248795600505608, via: 3, Ver: v1, Flags: 100010,
>> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
>> >> 2014-12-31 05:25:25,002 DEBUG [c.c.s.StatsCollector]
>> >> (StatsCollector-1:ctx-2acfc
>> >>
>> >> Thank you.
>> >>
>> >
>> >
>>
>
>